itertools and functools — the standard library's real power tools
Both modules solve the same underlying kind of problem — patterns that come up constantly when working with iterables and functions — with implementations that are more memory-efficient and more correct than the hand-rolled version most people write from scratch.
3 min read
itertools.chain: iterating over several iterables as if they were one
from itertools import chain
list1 = [1, 2, 3]
list2 = [4, 5, 6]
for item in chain(list1, list2): # walks through BOTH, without concatenating into a new list first
print(item) # 1, 2, 3, 4, 5, 6
# vs the naive version — actually builds a NEW combined list in memory first
for item in list1 + list2:
print(item)chain produces a single iterator that walks through multiple iterables in sequence, without ever building a combined list in memory — for large iterables, this is a real, structural memory saving over list1 + list2, which has to allocate an entirely new list holding every element from both before the loop even starts. This is the same "process incrementally instead of materializing everything at once" idea this platform's Node.js domain covered for streams, applied here to Python iterables specifically.
itertools.groupby: grouping consecutive items — a real, common gotcha
from itertools import groupby
data = [1, 1, 2, 2, 1, 1] # NOTE: 1 appears in two separate, non-adjacent groups
for key, group in groupby(data):
print(key, list(group))
# 1 [1, 1]
# 2 [2, 2]
# 1 [1, 1] <- a SEPARATE group, not merged with the first "1" group
# The fix, if what's actually wanted is ALL 1s together regardless of position:
sorted_data = sorted(data)
for key, group in groupby(sorted_data):
print(key, list(group)) # 1 [1, 1, 1, 1] / 2 [2, 2]groupby only groups consecutive equal elements — it doesn't scan the whole iterable looking for every occurrence of each key the way a dictionary-based grouping would. This is a genuinely common, real gotcha: groupby on unsorted data produces multiple separate groups for the same key if that key's occurrences aren't adjacent, which is rarely what's actually intended — sorting the data first (by the same key groupby will use) is almost always required before groupby produces the "all items with this key together" result people usually expect.
functools.reduce: combining every element into a single result
from functools import reduce
numbers = [1, 2, 3, 4]
total = reduce(lambda acc, n: acc + n, numbers, 0) # 10 — same idea as JS's Array.reduce, covered in this platform's JS domain
product = reduce(lambda acc, n: acc * n, numbers, 1) # 24reduce combines every element of an iterable into a single accumulated result, applying a function that takes the accumulator-so-far and the next element — the exact same general concept this platform's JavaScript Fundamentals domain covered for Array.prototype.reduce, just as a standalone function rather than a method, since Python's built-in list type doesn't have a .reduce() method of its own. Python's built-in sum() covers the common addition case directly; reduce is the general tool for combining logic that isn't already a built-in.
functools.lru_cache: memoization, with zero manual cache-management code
from functools import lru_cache
@lru_cache(maxsize=128) # caches up to 128 distinct (argument-combination → result) pairs
def fibonacci(n):
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2) # each call is now CACHED — repeat calls with the same n are instant
fibonacci(35) # without caching: exponentially slow. With @lru_cache: fast, each unique n computed once@lru_cache wraps a function so that calling it again with the exact same arguments returns the cached result instantly instead of re-running the function body — for a function like naive recursive Fibonacci, which recomputes the same values an enormous number of times without caching, this turns an exponential-time function into a linear-time one with a single decorator line, no manual cache dictionary or cache-checking logic needed. maxsize bounds how many distinct argument combinations get cached before older entries are evicted (LRU — Least Recently Used), the same real eviction concern this platform's Testing & QA domain's memory-leaks-adjacent coverage touched on for unbounded caches.
Further reading
Check your understanding
A quick comprehension check — not tracked, not graded, just for you.
1. Why is itertools.chain(list1, list2) more memory-efficient than list1 + list2 for large lists?
2. Why does itertools.groupby() on unsorted data often produce unexpected results?
3. What does @lru_cache actually do to a function like naive recursive Fibonacci?