You have 2,000 lines of exported text and you need every email address out of it. Or you need to change one field name throughout. These are the same operation, pattern matching over text, approached from opposite ends, and they share one way of going wrong.
The substring trap
The default behaviour of find and replace is substring matching. Replacing cat with dog turns category into dogegory, concatenate into condogenate and location into lodogion.
This is the single most common way a bulk edit goes wrong, and it is silent. The replacement succeeds, the count looks plausible, and the damage is scattered through the file.
The fix is word boundaries. In regex, \b matches the position between a word character and a non-word character:
\bcat\b matches: cat, "cat", (cat)
does not match: category, concatenate
Most find and replace tools offer a "whole word" checkbox that does exactly this. Use it by default and turn it off only when you mean to, rather than the other way round.
Capture groups are the useful part
Plain replacement swaps one fixed string for another. Capture groups let you rearrange what you matched, which is where the real time saving is.
Find: (\w+)@(\w+)\.com
Replace: $2 contact: $1
Parentheses capture, and $1, $2 refer back to them. This turns reformatting tasks that look like manual work into one operation: swapping date components, reordering columns, converting one log format to another.
A regex tester with live highlighting is worth using before running anything destructive, because you can see what matches while you build the pattern rather than discovering it afterwards.
Extraction is find without replace
Pulling every email, URL or phone number out of a block of text is the same matching operation, collecting results instead of substituting them. The patterns are well understood and each has a known limitation worth knowing.
Emails. No regex validates an email address correctly. The actual specification permits quoted strings, comments and characters almost nobody uses, and a pattern that accepts all of it also accepts nonsense. Practical extraction uses a deliberately loose pattern and accepts that some results will not be deliverable. The only real validation is sending mail to it.
URLs. The hard part is the end, not the start. A URL at the end of a sentence collides with the full stop, and trailing brackets are ambiguous because parentheses are legal inside URLs. Extractors apply heuristics, so check the last character of anything you extract from prose.
Phone numbers. The loosest of all, because formatting varies by country, and a long digit sequence might be an order number, an account reference or a date.
The honest framing for all of these: extraction finds candidates. Verification is a separate step and usually not a text-processing problem.
Greedy matching, the other classic
Quantifiers are greedy by default, meaning they match as much as possible and then backtrack.
<.+> on <b>bold</b> matches the whole string
<.+?> on <b>bold</b> matches <b>
The ? makes it lazy. This one character accounts for a large share of patterns that "work on my test string" and then swallow entire lines of real data. When a pattern matches far more than expected, greediness is the first thing to check.
And the standing advice: do not parse HTML with regex. Extracting all URLs from a page's text is fine. Trying to match nested tags is not, because HTML is not a regular language and the pattern will be wrong on input you have not seen.
Before any bulk edit
- Search first, and read the match count.
- Test the pattern on the awkward cases, not the obvious ones.
- Turn on whole-word matching unless you have a reason not to.
- Keep the original.
Test a pattern on a small sample in the regex tester before running it across a whole document with find and replace, because a pattern that is almost right does its damage silently.