# Introducing codeflash-agent: 90% infra cost cut at Unstructured

**Codeflash × Unstructured · April 2026**
**24 PRs merged across 5 repos · 7 weeks · zero regressions**

<!-- ============================================================
     SHARED STYLE (applies to all figures unless overridden)
     ============================================================
     Style: modern tech-blog announcement illustration. Clean, editorial,
     high-contrast. Dark charcoal/near-black background (#0E0F12), warm
     off-white foreground text (#F4EFE6), one accent color: a saturated
     amber/yellow (#F6C948) used sparingly for the hero number or a
     single highlight element. Secondary accent: muted teal (#3FA7A0)
     for neutral data. Typography: crisp geometric sans-serif for labels
     (feel of Inter / Söhne), monospaced for numbers (feel of JetBrains
     Mono). No drop shadows, no 3D, no gradients on the background,
     no stock-illustration people. Flat vector look. Plenty of negative
     space. Subtle 1px lines, no heavy strokes. No emojis. No clip-art
     icons — if an icon is needed, use a simple geometric glyph.
     Aspect ratio 16:9 unless specified otherwise.
     ============================================================ -->

<!-- FIG 1 — HERO METRIC STRIP
     ChatGPT image-gen prompt (copy-paste):

     Create a wide 16:9 hero banner for an engineering blog post announcement.
     Dark charcoal background (#0E0F12). Four equally spaced metric tiles
     in a single horizontal row, separated by thin vertical divider lines.
     No borders around the tiles themselves. Top-left corner: a small
     lockup reading "Codeflash × Unstructured" in a refined geometric
     sans-serif, off-white.

     Tile 1 (leftmost, largest visual weight): the number "−90%" in very
     large monospaced amber yellow (#F6C948), with a small off-white
     caption below reading "Core-product infra cost" and a second line
     in monospaced off-white reading "$10K → $1.1K / month".

     Tile 2: "−52%" in large monospaced off-white, caption "Peak memory
     per pod", second line "4,651 MB → 2,227 MB" in monospaced off-white.

     Tile 3: "32 GB → 4 GB" in large monospaced off-white, caption
     "K8s memory allocation per pod".

     Tile 4: "5 → 46" in large monospaced off-white, caption "Pods per
     D48s_v5 node".

     Only the −90% tile uses the amber accent; the other three are
     off-white on charcoal for hierarchy. Plenty of vertical negative
     space above and below. Flat vector look, no shadows, no gradients.
     Editorial, understated, confident. -->

> [FIG 1 — HERO METRIC STRIP placeholder]

Codeflash cut the infra bill on Unstructured's core document-processing pipeline by **90%**. Same hardware, 9.2× the workload, 24 reviewed PRs across five repos, zero regressions.

Unstructured processes billions of documents a month. Their **core-product** pipeline turns raw PDFs, images, and office files into clean, structured elements (titles, paragraphs, tables, figures) that downstream LLM and RAG systems can actually use. It ran on `Standard_D48s_v5` nodes, RAM-bound at 5 pods per node. Seven weeks later, the numbers above are the production result; latency improved 12.9% as a side effect, and latency was never even the objective.

Company-wide performance engineering on this scale used to take a dedicated team a full quarter. This one ran autonomously on **codeflash-agent**, and it is the first public case study of what the agent does.

A 90% number invites a fair question: what was the starting point hiding? For a fast-moving ML company, the answer is rarely negligence. It is the rational trade off a production team makes under pressure. An OOM kill on an important workload pages someone, costs customer trust, and is loud; a wasted gigabyte of RAM is silent. Root-causing a memory spike in a pipeline that touches PDFium, PIL, ONNX, PaddleOCR, spaCy, pytesseract, and a half-dozen internal subsystems is genuinely hard work, and defensive over-allocation is the cheap, reliable answer that keeps the pager quiet. Most production ML infra is shaped exactly this way. As Crag Wolfe, Unstructured's Chief Architect, put it plainly:

> *"We knew we could do better, but we didn't have the bandwidth. Functionality improvements were ranked higher on the list. And we'd occasionally see OOMs, which pushed us to keep memory allocations higher than we wanted."*
> — Crag Wolfe, Chief Architect, Unstructured

The codeflash-agent is a general-purpose performance optimizer that finds and fixes the root causes behind those defensive tradeoffs, on any objective the team can measure: memory, latency, throughput, cold-start, cost.

---

## What a 90% reduction actually looks like

The same hardware now holds **9.2× the workload**.

<!-- FIG 2 — POD DENSITY VISUAL
     ChatGPT image-gen prompt (copy-paste):

     Create a 16:9 editorial diagram for a tech blog on the same shared
     dark charcoal background (#0E0F12). Two identical outlined rectangles
     sit side by side, each labeled at the top in small off-white
     geometric sans-serif: "D48s_v5 node — 48 vCPU / 384 GB". A thin
     vertical divider line runs down the middle of the canvas between
     them.

     LEFT rectangle ("Before"): contains 5 tall, fat, stacked rounded
     blocks representing pods, filling most of the node's height. The
     blocks are rendered in a muted desaturated red-orange, flat fill,
     no gradient. Small monospaced label on the node: "5 pods · 32 GB
     request each · RAM-bound".

     RIGHT rectangle ("After"): the same outer node outline, but packed
     densely with 46 thin, short, rounded blocks arranged in a neat grid
     — visibly 9× the density. The blocks are rendered in the amber
     accent (#F6C948), flat fill. Small monospaced label on the node:
     "46 pods · 4 GB request each · CPU-bound".

     Below both nodes, centered, a single line of off-white caption:
     "Same hardware. Same workload. 9.2× the pod density."

     Flat vector, no shadows, no 3D, plenty of negative space. -->

> [FIG 2 — POD DENSITY VISUAL placeholder]

- **9.2× pod density** on existing infrastructure. No new nodes to provision.
- **41 previously-idle vCPUs per node** freed for throughput or adjacent workloads.
- **12.9% end-to-end latency reduction** on the production path (50.8s → 44.3s on a 10-page scanned document).

For an infra or platform team, the story is not "we saved money." It is "RAM stopped being the thing we architect around."

**The unit economics are simple: `D48s_v5` nodes goes from ~$10,000/month to ~$1,100/month.** That is about **$107K/year** recovered per 10 nodes, **$1M/year** per ~100 nodes, **$10M/year** per ~1,000 nodes. The saving scales linearly with cluster size, and the 41 vCPUs per node that used to sit idle next to a RAM-bound pod are free capacity for the next workload, not a rebate you have to negotiate.

> *"Operationally, the cost-performance profile of our running instances is in a different place. We're seeing fewer tail events where you might hit an OOM, and faster scale-out because we're not bottlenecked on how many pods can fit at once."*
> — Crag Wolfe, Chief Architect, Unstructured

---

## Inside the 7 weeks at Unstructured

Here's what actually happened. Four stacked bottlenecks, each invisible until the one above it was fixed.

<!-- FIG 6 — ITERATIVE DISCOVERY LAYERS
     ChatGPT image-gen prompt (copy-paste):

     Create a 16:9 editorial diagram on the shared dark charcoal
     background (#0E0F12) showing four nested, concentric rounded-square
     layers — like peeling back an onion, viewed from above. Each layer
     is a flat outlined band in a muted off-white stroke, slightly
     thicker at each inward step. The outermost layer is the largest;
     each inner layer sits inside the previous one with clear negative
     space between them.

     Label each layer with a small monospaced tag on the left edge and
     an off-white geometric sans-serif title on the right:
       Outer  — "LAYER 1  ·  Worker pool over-provisioning"
       Second — "LAYER 2  ·  Per-request PIL image churn (24 MB/req)"
       Third  — "LAYER 3  ·  O(N²) PDF character-patching hotspot"
       Inner  — "LAYER 4  ·  Redundant PNG ↔ bitmap round-trips"
     The innermost layer is filled with the amber accent (#F6C948);
     the other three are unfilled with off-white strokes only.

     Centered below the diagram, a single italic off-white caption line:
     "Each layer was invisible until the one above it was fixed."

     Flat vector, no shadows, no 3D, generous negative space, editorial
     understated feel. -->

> [FIG 6 — ITERATIVE DISCOVERY LAYERS placeholder]

### Layer 1: `os.cpu_count()` was lying

**Finding:** `os.cpu_count()` returned the host's 48 CPUs on a pod with a 1-CPU CFS quota. The OCR worker pool spawned 4 workers per pod, each loading the full ONNX model set. **4× the memory for zero parallelism benefit.**
**Fix:** three-tier container-aware detection (cgroup v2 → `sched_getaffinity` → `os.cpu_count()`, minimum of all three).
**Result:** worker pool collapses to one on 1-CPU pods. Memory drops by gigabytes overnight.

A reasonable engineer could have stopped there. The ticket said "32 GB → smaller-number," and smaller-number had arrived. But the agent loop kept going.

### Layer 2: 24 MB of per-request creep, hidden in the old baseline's noise

**Finding:** a per-request memory creep of **24 MB/req** in PIL image preprocessing, previously masked by worker-pool noise.
**Fix:** reorder the allocation so resizes happen on the cheap PIL representation before the expensive numpy conversion; release page buffers as soon as OCR completes.
**Result:** **24 MB/req → 17 MB/req.**

### Layer 3: An O(N²) hotspot that had lived in the code for years

**Finding:** an O(N²) hotspot in [`_patch_current_chars_with_render_mode`](https://github.com/Unstructured-IO/unstructured/pull/4266) that re-scanned the full character list on every PDF text operator — surfaced once memory noise cleared.
**Fix:** rewrite as single-pass, tracking a `last_patched` index.
**Result:** **7.98× median speedup on a 2,000-char page, 14.08× on 5,000 chars.** Superlinear, which is the signature of an O(N²)→O(N) change.

### Layer 4: Three PNG round-trips on one image, in one request

**Finding:** pdfium was rendering pages as PNG; the downstream consumer immediately decompressed to raw bitmap; pytesseract then wrote *another* PNG temp file to hand to the tesseract CLI. Three unnecessary compression round-trips on the same image, in a single request.
**Fix:** switch pdfium render and pytesseract temp format to BMP; where possible, pass the on-disk path directly to tesseract so the image never round-trips back through Python at all.
**Result:** redundant CPU work eliminated from the hottest request path.

---

**No human profiling pass would have found all four in seven weeks — and a point optimizer would have stopped at one. The compounding is the product.**

---

## Selected public optimizations

<!-- FIG 7 — PR SCOREBOARD
     ChatGPT image-gen prompt (copy-paste):

     Create a wide 16:9 thin banner on the shared dark charcoal background
     (#0E0F12). A single horizontal row of five equally sized card tiles,
     separated by thin vertical divider lines. No borders around the cards.

     Each card has a large monospaced number on top and a small off-white
     geometric sans-serif caption below. Left to right:

       Card 1: "24"  — caption "PRs merged"
       Card 2: "5+"  — caption "PRs in review"
       Card 3: "5"   — caption "Repositories"
       Card 4: "354" — caption "Tests passing"
       Card 5: "0"   — caption "Regressions"

     Only the "0 regressions" card uses the amber accent color (#F6C948)
     on its number; the other four are off-white (#F4EFE6) on charcoal.
     Plenty of vertical negative space above and below the row. No icons,
     no shadows, no gradients. Editorial, confident, flat vector. -->

> [FIG 7 — PR SCOREBOARD placeholder]

Every change was delivered as a reviewable PR with benchmark numbers attached. A sample from the open-source repos:

| PR | Change | Metric | Delta |
|---|---|---|---|
| [unstructured#4266](https://github.com/Unstructured-IO/unstructured/pull/4266) | O(N²) → O(N) PDF character-patching | Speedup | **7.98× / 14.08×** |
| [unstructured#4314](https://github.com/Unstructured-IO/unstructured/pull/4314) | `standardize_quotes`: per-char regex → `str.translate` | Speedup | **2.24×** |
| [unstructured#4282](https://github.com/Unstructured-IO/unstructured/pull/4282) | Custom `lazyproperty` → `functools.cached_property` | Correctness + speedup | **26 props fixed** |
| [unstructured#4296](https://github.com/Unstructured-IO/unstructured/pull/4296) | spaCy loaded without `ner` / `lemmatizer` / `attribute_ruler` | Peak memory | **−26% / −23%** |
| [unstructured#4295](https://github.com/Unstructured-IO/unstructured/pull/4295) | PaddleOCR `rec_batch_num` 6 → 1 (CPU has no batch parallelism) | Peak memory | **−83%** |
| [unstructured-inference#498](https://github.com/Unstructured-IO/unstructured-inference/pull/498) | PDF page render + save moved inside the loop | Peak memory | **O(N) → O(1)** |
| [unstructured-inference#496](https://github.com/Unstructured-IO/unstructured-inference/pull/496) | Free large buffers before ONNX `session.run()` | Peak memory | **−35%** |
| [unstructured-inference#497](https://github.com/Unstructured-IO/unstructured-inference/pull/497) | `np.ones(shape) * 114` → `np.full(shape, 114)` | Allocations | **−50%** |

Every PR ships with benchmark numbers, a verification report, and rationale. Links go to the live diffs. Five more PRs are in review as of publication; we'll update this post as they land.

### 24 PRs. 354 tests green. Zero regressions.

Every PR ships with a Codeflash verification report. Behavior on the changed code path is compared against the original across the existing test suite plus generated test cases the agent writes alongside the change — return values, side effects, and exception shapes checked byte-for-byte. Nothing reaches a reviewer until behavior is provably unchanged. That is how complex changes landed in a production system across multiple repos with no regressions. Correctness is not the trade you make for the speedup; it is the precondition for the PR existing at all.

> *"What stood out is real demonstrable progress, not theoretical. A PR comes in, we measure RSS before and after on a running system, and the improvement is 2× or 3×. You don't have to run the simulation yourself."*
> — Crag Wolfe, Chief Architect, Unstructured

---

## Meet codeflash-agent

Everything above came out of a single loop.

**codeflash-agent is an autonomous performance engineer.** It runs inside a customer's codebase with a global objective (memory, latency, throughput, cold-start, cost) and executes the full performance-engineering loop that a senior engineer would run manually, except it runs it continuously for days and weeks, in parallel, and without tiring.

<!-- FIG 3 — AGENT LOOP DIAGRAM
     ChatGPT image-gen prompt (copy-paste):

     Create a 1:1 square diagram on the shared dark charcoal background
     (#0E0F12) showing a circular flow with seven nodes arranged evenly
     around a perfect circle. Each node is a small rounded rectangle
     with a thin off-white outline (no fill) and a short monospaced
     label inside. Arrows between nodes point clockwise; the arrows
     are thin, single-stroke, in the amber accent (#F6C948).

     Node labels, starting at 12 o'clock and moving clockwise:
       1. "Analyze codebase"
       2. "Build real-world benchmarks"
       3. "Profile for bottleneck"
       4. "Explore strategies"
       5. "Prove correctness"
       6. "Prove speedup"
       7. "Refine to expert quality"

     Dead center of the circle: a small off-white geometric sans-serif
     block reading:
         OBJECTIVE
         memory · latency · throughput · cost
     "OBJECTIVE" in small caps, the second line smaller and lighter.

     Flat vector, no shadows, no 3D, generous negative space around
     the whole circle. Minimal, editorial, confident. -->

> [FIG 3 — AGENT LOOP DIAGRAM placeholder]

Under the loop sits a detailed model of expert-level performance engineering: the real tradeoffs between optimization approaches, which bottlenecks justify which tools, when a local win costs you a global one, and, most importantly, when to investigate deeper and continue experimenting instead of stopping at the first plausible fix. That judgment is why the agent found four stacked bottlenecks on core-product instead of one.

This is what separates the agent from a hot-loop rewriter. On core-product it didn't just tune existing code. It identified multi-step data transformations where half the steps were unnecessary, deleted them, and replaced the flow with a better one. That only works if the agent understands what the code is trying to achieve, not just what it's doing.

> *"It wasn't just tweaks to existing functions. Codeflash understood the abstractions. If a process had six steps, it would recognize that two or three were unnecessary and replace the flow with a better one. That requires really deep understanding of what the code is actually doing."*
> — Crag Wolfe, Chief Architect, Unstructured

Each turn of the loop:

1. **Analyzes the codebase** end-to-end to surface optimization opportunities, not just the ones a linter would find.
2. **Builds real-world benchmarks** against representative workloads: production endpoints, real documents, real request shapes. Not toy microbenchmarks.
3. **Profiles** to identify the actual bottleneck for the current objective: CPU, allocator, I/O, contention, fragmentation.
4. **Explores multiple optimization strategies** per bottleneck, not just the first one that looks promising.
5. **Proves correctness** through Codeflash's verification system, which combines the existing test suite with generated test cases and deeply compares the behavior by comparing return values among others.
6. **Proves the speedup** through Codeflash's benchmarking harness: A/B/A pattern, statistical rigor, standalone *and* end-to-end against a noise-controlled baseline.
7. **Refines** the change until it reads like something a senior engineer would have written by hand: idiomatic, reviewable, mergeable.

Then the loop restarts. **The previous bottleneck is gone, and that almost always reveals a new one that was masked underneath.** This is the engine behind the Unstructured results, and it is the property that turns a point optimization into a 90% company-level reduction.

The codeflash-agent can act on an objective which is anything that is objectively measurable. Memory on this engagement; latency, throughput, or cold start on others. The loop is the same.

---

## How a Codeflash engagement runs

Every change arrives as a pull request in your repo. Your engineers review and merge.

<!-- FIG 4 — ENGAGEMENT SWIMLANE
     ChatGPT image-gen prompt (copy-paste):

     Create a 16:9 swimlane diagram on the shared dark charcoal
     background (#0E0F12) with three horizontal lanes stacked vertically,
     separated by thin 1px off-white horizontal divider lines. Each lane
     has a left-side label in small off-white geometric sans-serif,
     uppercase small-caps:

       Top lane:    "CODEFLASH TEAM"
       Middle lane: "CODEFLASH-AGENT"   (amber accent label, #F6C948)
       Bottom lane: "CUSTOMER TEAM"

     Inside each lane, a sequence of rounded-rectangle step tiles with
     thin off-white strokes (no fill) connected by thin horizontal arrows.
     Labels inside each tile are short monospaced text.

       Top lane, left to right:
         "Scope objective" → "Reproduce baselines" → "Review PRs" → "Architectural fit"

       Middle lane, left to right, tiles rendered in amber outline (#F6C948):
         "Analyze" → "Benchmark" → "Profile" → "Optimize" → "Verify" → "Refine"
         A small curved arrow loops back from "Refine" to "Analyze" to
         indicate the loop.

       Bottom lane, left to right:
         "Share context" → "Point at known pain" → "Merge"

     Flat vector, no shadows, no gradients, plenty of horizontal space
     between steps, editorial and minimal. -->

> [FIG 4 — ENGAGEMENT SWIMLANE placeholder]

The flow:

1. **Scope the objective.** What is on the roadmap: pod cost, tail latency, cold start, throughput?
2. **Reproduce baselines.** On a VM the customer can log into. Every number cited in the engagement is measurable from there.
3. **The agent runs.** It explores autonomously inside the customer environment in a sandboxed environment. Codeflash engineers point it at the areas the customer flags as hot.
4. **Codeflash engineers review every change** before it reaches the customer: architectural fit, idiomatic style, merge-readiness. The expert performance engineers also discover any opportunities to optimize the code.
5. **PRs arrive with reproducible benchmark numbers, correctness proof, and a short rationale attached.** Your engineers review and merge, delivering the gains.

> *"Having Codeflash (the subject-matter experts on optimization) run the agent makes more sense than asking our engineers, who are focused on product and features, to develop and run that process themselves."*
> — Crag Wolfe, Chief Architect, Unstructured

---

## The wins compound, they don't decay with time

A 90% reduction is easy to erode. Engineers ship features, AI coding agents generate code against your repo, dependency bumps land on defaults no one profiled, and six months later the savings have drifted back.

**Codeflash runs continuously.** After the engagement, the same agent watches every pull request: finds or generates benchmarks for the changed functions, runs the optimization loop scoped to the diff, and posts a suggested patch if it finds a material win. Same engine, pointed at tomorrow's diffs instead of today's hot path.

New code starts optimal. Regressions get caught in the PR that would have caused them, not six months later when pod memory starts climbing again. **That is what keeps the initial 90% from drifting back to 50%**, whether the diffs come from your engineers, your Claude Code / Cursor / Codex agents, or anything else shipping into your repo.

Signal-to-noise is tuned per team. Some run Codeflash on every PR with a material-win threshold that filters out microsecond-noise suggestions; others pair a lighter per-PR mode with a weekly codebase sweep for the bigger opportunities. Same engine, different cadence.

<!-- FIG 5 — CI-MODE DIAGRAM
     ChatGPT image-gen prompt (copy-paste):

     Create a 16:9 editorial diagram on the shared dark charcoal
     background (#0E0F12) styled like a stylized GitHub pull-request
     timeline, viewed left to right. Draw a single vertical thin
     off-white line on the left side representing the PR conversation
     thread, with five small circular nodes along it, evenly spaced
     top to bottom. From each node, a thin horizontal line extends
     rightward to a labeled rectangular "event card" with an off-white
     thin outline (no fill) containing a short monospaced title and a
     one-line off-white description.

     Event cards, top to bottom:
       1. "Diff opened"
          description: "engineer pushes changes to a PR"
       2. "Benchmarks discovered"
          description: "Codeflash finds or generates benchmarks for changed functions"
       3. "Optimization loop runs"
          description: "agent runs the full loop scoped to the diff"
       4. "Suggested optimization posted"
          description: "material-win patch posted as a PR comment"what 
          (render this card's outline in amber accent #F6C948)
       5. "Author accepts or dismisses"
          description: "engineer stays in control"

     Below the timeline, centered, a single italic off-white caption:
     "Every PR that touches a hot path gets the same treatment
     core-product got in week one."

     Flat vector, no shadows, no screenshots, no real GitHub logos.
     Editorial, confident, minimal. -->

> [FIG 5 — CI-MODE DIAGRAM placeholder]

---

## If the infra bill is a P&L problem

Core-product was not a special case. Container-unaware concurrency, allocator fragmentation, redundant format conversions, lifetime bugs in large intermediates, O(N²) hotspots masked by memory noise: these patterns exist in almost every production Python service at scale. The bottleneck in your codebase is probably different. The agent doesn't care.

**Positive ROI, guaranteed.** Every engagement is scoped so the infra savings exceed what you pay Codeflash. If the numbers don't pencil out on your workload, we'll tell you before you commit, not after.

**If you have a workload you can share with us, we'll tell you what Codeflash would find.** [codeflash.ai/contact](https://codeflash.ai/contact)
