Regular expressions in Python — the re module

Pattern syntax itself (covered in the JavaScript domain's regex lesson) is nearly universal — what's Python-specific is the re module's API: match vs search vs findall vs finditer, raw strings, named groups, and compiling a pattern once instead of re-parsing it every call.

Intermediate

4 min read

Raw strings: why regex patterns should almost always use `r"..."`

"\d+"    # SyntaxWarning in modern Python — \d isn't a recognized string escape
r"\d+"    # a raw string — backslashes are passed through literally, untouched

Regex syntax leans heavily on backslashes (\d, \w, \s, \b), but Python's own string literals also treat backslash as an escape character (\n, \t). Without the r prefix, Python tries to interpret \d as a string escape first, doesn't recognize it, and (in modern versions) warns about it — the pattern usually still happens to work by accident, but relying on that is fragile. Every regex pattern in Python should be a raw string, full stop — it's the difference between "backslash means what the regex engine expects" and "backslash means whatever Python's string parser decides first."

`match` vs `search` vs `findall` vs `finditer`: four ways to look, four different jobs

import re
 
text = "Order #4471 shipped, Order #9902 pending"
 
re.match(r"Order #\d+", text)       # matches ONLY at the very start of the string
re.search(r"Order #\d+", text)       # finds the FIRST match anywhere in the string
re.findall(r"Order #\d+", text)      # ['Order #4471', 'Order #9902'] — ALL matches, as strings
re.finditer(r"Order #\d+", text)     # a lazy ITERATOR of match objects — memory-efficient for huge text

These four are the source of most early re confusion. match only ever looks at the beginning of the string — a pattern that would match text starting at position 5 returns None from match, even though search on the exact same pattern and string would find it. findall is the most commonly reached-for function, but it returns plain strings (or tuples, if the pattern has groups) — it throws away the Match object entirely, which means no access to .start(), .end(), or named groups. finditer keeps that information by yielding full Match objects lazily, one at a time.

Groups: capturing and naming parts of a match

m = re.search(r"Order #(\d+) shipped", text)
m.group(0)      # 'Order #4471 shipped' — the entire match
m.group(1)       # '4471'                — the first parenthesized group
 
m = re.search(r"Order #(?P<order_id>\d+)", text)
m.group("order_id")     # '4471' — accessed by NAME instead of a fragile position number
m.groupdict()             # {'order_id': '4471'}

Parentheses (...) capture a portion of the match for separate retrieval — .group(1), .group(2), and so on by position. Named groups ((?P<name>...)) are Python-specific syntax that let a group be retrieved by name via .group("name") instead of counting parentheses — this matters in any pattern with more than two or three groups, where "which numbered group was that again" becomes a real source of bugs whenever the pattern gets edited and the group order shifts.

Compiling a pattern once, instead of re-parsing it every call

# Called in a loop, re.search re-parses the SAME pattern string every single time
for line in lines:
    if re.search(r"^ERROR: (.+)$", line):
        ...
 
# Compiled once, reused — the pattern is parsed a single time up front
ERROR_PATTERN = re.compile(r"^ERROR: (.+)$")
for line in lines:
    if ERROR_PATTERN.search(line):
        ...

Every top-level re.search(pattern, text) call re-parses pattern from scratch internally (Python does cache a small number of recently used patterns, but it's not guaranteed, and it's not free). re.compile() parses the pattern exactly once into a reusable Pattern object, whose .search()/.match()/.findall() methods take only the text argument from then on. For any pattern used inside a loop or called repeatedly — validating rows from a file, matching every line of a log — compiling it once outside the loop is both clearer and measurably faster.

`re.VERBOSE`: making a dense pattern readable

PHONE = re.compile(r"""
    \(?(\d{3})\)?    # area code, optionally in parentheses
    [-.\s]?           # separator: dash, dot, or space
    (\d{3})            # first three digits
    [-.\s]?
    (\d{4})            # last four digits
""", re.VERBOSE)

Dense patterns like phone-number or email validators become unreadable as one unbroken line. re.VERBOSE (or re.X) tells the engine to ignore whitespace and #-comments inside the pattern itself, letting a complex pattern be laid out and commented like ordinary code — a literal space or # still has to be escaped (\ or \#) to be matched literally under this mode.

Regular expressions in Python — the re module

Raw strings: why regex patterns should almost always use r"..."

match vs search vs findall vs finditer: four ways to look, four different jobs

Groups: capturing and naming parts of a match

Compiling a pattern once, instead of re-parsing it every call

re.VERBOSE: making a dense pattern readable

Further reading

Raw strings: why regex patterns should almost always use `r"..."`

`match` vs `search` vs `findall` vs `finditer`: four ways to look, four different jobs

`re.VERBOSE`: making a dense pattern readable