The AI Cost of JavaScript Frameworks: What Nolan Lawson's Benchmark Tells Us
"Rewrite it in X, agents are better at X" has become a common argument in frontend teams, and it almost never comes with numbers. Nolan Lawson's The AI cost of JavaScript frameworks (October 28, 2026) is one of the first attempts to measure it. He had the same coding agent build the same app in 12 frameworks, five times each, and tracked what it cost, how long it took, how much code it produced, and how fast the result loaded. This post summarizes the findings, the caveats that matter, and what they mean for the Aliz tech stack.
The setup
- The app: a small but non-trivial Pokémon browser on top of PokéAPI, with search, type filters, moves, and game data.
- The agent: Claude Opus 5.5 at high effort, told to use each framework's latest version and its official MCP server or agent skills where one exists. It was not told to optimize for performance and did not know Lighthouse would be run.
- The rules: no Tailwind, Bootstrap, or component libraries; shared design tokens; no pre-generated static HTML.
- Verification: a conformance suite that checks the accessibility tree, so each app had to behave the same without being forced into the same markup.
- Measurement: median of five runs per framework for cost (API-equivalent prices reported by Claude Code), wall-clock time, tokens, and non-blank lines of code, plus Lighthouse on the home page with mobile emulation, 4x CPU throttling, and Slow 4G.
The benchmark and all generated apps are public in nolanlawson/agent-framework-benchmark.
The results
Medians per framework, ordered by cost:
| Framework | Cost | Time (min) | Tokens (M) | Lines of code | JS (KiB) | LCP (ms) | TBT (ms) |
|---|---|---|---|---|---|---|---|
| Hono + htmx | $2.67 | 8.6 | 4.26 | 1370 | 18 | 978 | 0 |
| Astro | $2.76 | 8.2 | 4.13 | 1461 | 0 | 810 | 0 |
| Nuxt | $2.82 | 9.0 | 3.85 | 1415 | 84 | 1727 | 40 |
| Vanilla (Vite) | $2.98 | 10.1 | 5.00 | 1492 | 7 | 1667 | 0 |
| Next.js | $3.14 | 10.9 | 4.99 | 1539 | 142 | 2090 | 71 |
| Qwik | $3.27 | 10.7 | 5.64 | 1744 | 40 | 979 | 3 |
| TanStack Start | $3.54 | 10.3 | 7.07 | 1697 | 118 | 2028 | 1 |
| SolidStart | $3.79 | 11.2 | 7.13 | 1630 | 40 | 1204 | 11 |
| React Router | $3.81 | 12.5 | 7.58 | 1564 | 115 | 1729 | 17 |
| SvelteKit | $4.28 | 10.8 | 8.53 | 1611 | 36 | 1130 | 0 |
| Gracile + Lit | $4.65 | 12.1 | 9.04 | 1717 | 8 | 956 | 0 |
| Angular | $4.97 | 17.2 | 9.76 | 2009 | 93 | 1954 | 103 |
Angular's JS, LCP, TBT, and line counts are medians of three apps, because two of its five runs produced servers that rejected non-localhost requests and failed the spec.
Four things stand out.
The cost spread is almost 2x. Hono + htmx cost $2.67 per build, Angular $4.97. Angular also took twice as long (17.2 vs 8.6 minutes) and burned 2.3x the tokens of the cheapest option. Per build that is small money. Multiplied over every feature, bug fix, and refactor an agent does on a codebase for years, it is not.
Code size compounds. Angular's output was 2009 lines against 1370 for Hono + htmx, almost 50% more for the same behavior. More code means more tokens every time an agent reads the project again, so a verbose framework keeps costing after the first build.
Lighthouse quality was mostly a tie. Every app scored 100 on accessibility and performance scores were saturated between 97 and 100. The real differences show up in the raw metrics: Astro shipped zero client JavaScript and had the best LCP (810 ms), while Next.js shipped 142 KiB and had the worst LCP (2090 ms). LCP tracked bundle size closely. Qwik is the interesting exception: a client-routed app that loaded like a multi-page one.
The official AI tooling did not reliably help. Usage of framework MCP servers and skills varied wildly. The Nuxt MCP server was never called in five runs, yet Nuxt was among the cheapest. SvelteKit's MCP was used in every run, yet its agents read the SvelteKit 3 migration guide on every run and landed near the bottom on cost. Lawson suspects some of this tooling currently does more harm than good. Gracile + Lit, with no tooling and little training data, struggled for a different reason: the agent simply did not know it well.
The caveats that matter
Lawson is careful not to declare a winner, and so should we.
- One app type. A content-heavy browser is closer to a Wikipedia page than a dashboard. Lighthouse and Core Web Vitals favor multi-page apps on that kind of page. A highly interactive SPA could tell a different story.
- Small samples, noisy agents. Five runs per framework (three for some Angular metrics), and agents occasionally wandered off task. One Next.js app skipped image lazy-loading, which inflated its LCP spread.
- Home page only, Chromium only. Navigation after first load was not measured.
- API-equivalent prices. The dollar figures are what Claude Code reports, not what a subscription plan actually charges.
- One model. Different models have different training data, and framework rankings could shift with them.
What this means for Aliz projects
Our defaults are React for applications and Astro for content-focused sites, with Angular, Vue, and Svelte on the discouraged list. The benchmark broadly supports that, with nuance.
- Astro for content is a strong default. It was the fastest to build, second cheapest, and shipped zero JavaScript with the best LCP. If a project is mostly content, the evidence points the same way our stack already does.
- React meta-frameworks are mid-pack, not bad. Next.js, TanStack Start, and React Router landed in the middle on cost and at the heavy end on JavaScript. That is the price of a client-side runtime on a page that does not need much of one. For interactive applications, which this benchmark did not test, the trade-off is different.
- Angular is the expensive outlier. It was the slowest, most token-hungry, most verbose, and least reliable option in this test. That is one more data point, not a verdict, but it does not argue for moving it off the discouraged list.
- Do not assume vendor AI tooling helps. If a framework ships an MCP server or skills, check what your agent actually does with them before making them part of the project setup. Our AI-assisted development section covers how we configure agents.
- Treat "agents like it" as a claim that needs evidence. The cheapest options here were the ones with the least framework: Hono + htmx, Astro, and plain Vite. Familiarity in the training data matters, but so does how much code the framework asks the agent to write.
Try it yourself
The most useful part of the post is that it is reproducible. Lawson invites people to fork the benchmark, add frameworks, run more iterations, and try different app types. If your team is about to pick a framework partly on "it works better with agents", a few hours with this harness and your own app shape will tell you more than any opinion thread, including this one.