← Blogs

Replacing regex with agents: what we learned tagging transaction data

How we moved company tagging at YipitData from analyst-written regex to an agent pipeline, what failed on the way, and when I would still reach for a rule.

For years, the limit on how many companies we could tag was the number of analysts we had. Every new company that we wanted to tag meant someone writing and testing regex rules, so coverage grew at the speed of headcount.

In 2025 I led the work to change that. This is what the old system looked like, what I tried that did not work, what did, and how I now decide between a rule and an agent. I also covered the pipeline in a Data + AI Summit talk with my colleague Anup Segu.

The data and the job

At YipitData we process petabytes of data. Much of it is licensed from strategic data partners, and it covers credit card transactions, receipts, invoices, healthcare, and more. We also have first-party data from our own products, SpendHound and Edison Mail.

Almost all of it is unstructured in some way, and we organize almost everything around companies. A transaction row rarely says which company it belongs to in a clean, consistent way. The name may be abbreviated, misspelled, or shared by several businesses. Before our product team and customers can use the data, someone has to tag each row with the right company.

How regex worked, and where it stopped

Our approach was to train analysts to write regex rules that say which company or entity to extract. We built a repository where they could push those rules, and we added them to our pipelines so the extraction happened as early as possible.

It worked, and it was understandable. A rule is easy to read, easy to test, and easy to explain to a customer. But it had three costs.

  • It scaled with people. Analysts maintained more than 500 regex statements. More companies meant more rules and more review. We were limited by what an analyst could do in a day.
  • It was slow to run. Regex is not well optimized inside Spark, and large rule sets became a performance bottleneck in the pipeline.
  • It plateaued. Rules kept getting more complex without getting much more accurate. Accuracy from regex and similar methods stalled at around 70%. In the talk we put it this way: we were trying to solve a reasoning problem by mapping substrings.

The last point was the important one. A rule can find a string. It cannot tell you which of three companies that string most likely refers to.

What did not work

When GPT-class models became practical, we explored removing the manual bottleneck. I tried a number of options, including frontier and open-source models, and I used Databricks’ Agent Bricks to run experiments. Several obvious ideas failed first.

  • Asking an LLM to write the regex. It made rules faster to produce, but it led to the same problem at a larger size. We spent more time maintaining rules than tagging data.
  • Zero-shot prompting. A model asked to name the company for a row gave confident answers that were sometimes wrong. A personal expense might be tagged as Chase Bank because the word looked familiar.
  • Lexical search alone. It reduced hallucination but missed variations in how names were written, so recall dropped.
  • Semantic search alone. It handled variation better, but it did not know our business rules and had trouble with names that collide.

What worked

The version that worked gave the model less to decide and more to look at. It has three steps.

  1. Extract. Remove noise from the raw text and pull out candidate keywords, products, and company names.
  2. Retrieve. Search an ontology of known companies, combining lexical and semantic search, and return five to ten candidates for each row.
  3. Choose. Ask the model to pick the best candidate from that list, like a multiple-choice question, and apply our business rules. For example, ChatGPT should roll up to OpenAI.

Choosing from a short list limits hallucination, because the model cannot invent a company that is not in the ontology. It also moves the hard part of the problem from the prompt to the context we give the model.

That was the main thing I learned. A model is only as good as the data you give it. Most of my time went into deciding what to retrieve, what to show the model, and how to define the right answer, not into the model itself.

Each stage writes its output to a Delta table. That gives us a record of what every step decided, and it lets us restart a stage without redoing the whole run. The ontology sits in Postgres with vector search for the semantic part.

What changed

After integrating AI, company coverage grew 20x, from about 3,000 companies to more than 60,000 in one quarter. In one core use case, tagging accuracy was ranging from 92 to 95% out of the box for our tier 1 companies. Processing one million records dropped from about 24 hours to about one hour.

The biggest change for me was the shape of the work. Adding a company is now mostly a matter of adding it to the ontology and checking the results, not writing and maintaining a new set of rules.

When I would still use regex

Regex is not the villain here. I would still use it in these cases.

  • The format is stable and well defined. A reference number with a known prefix does not need a model.
  • The answer must be fully explainable. A rule can be read line by line.
  • The cost of a model call is not justified. If a rule solves 95% of a job cheaply, start there.

I would reach for an agent or a model when the same input can have several plausible answers and choosing one depends on context. Even then, I keep deterministic checks around it. The output must match the schema, the chosen company must exist in the ontology, and rows without enough evidence should go to review.

The most useful habit was building an evaluation set early, with human review. Without it, there was no way to tell whether a change helped. That set is also how I decided which tasks to give to the model and which to leave with a rule.

Sources