Skip to main content

TanStack AI Beta: The 'Switzerland of AI Tooling' Goes Multi-Modal

· 3 min read
Gergely Sipos
Gergely Sipos
Frontend Architect

TanStack AI reached beta on June 9, positioning itself as the "Switzerland of AI tooling" — framework-agnostic, provider-agnostic, and fully open-source with no hosted platform required. It's built by the same team behind React Query and TanStack Router, and it brings that same philosophy of composable, type-safe primitives to AI application development.

Why It Matters​

The core value proposition is neutrality at every layer:

  • Protocol-first — built on the AG-UI protocol, so you're never locked into a proprietary wire format.
  • No hosted platform — unlike Vercel AI SDK, there's no deployment platform attached. Bring your own backend.
  • Framework-agnostic client — first-class support for React, Vue, Solid, Svelte, Preact, and vanilla JS.
  • Language-agnostic backend — server implementations in TypeScript, Python, Go, and anything that speaks AG-UI.

Multi-Modal from Day One​

Every modality is first-class and fully typed — not bolted on after the fact:

  • useChat() — text messaging with streaming
  • useGeneration() — streaming structured output (typed JSON)
  • useGenerateImage(), useGenerateAudio(), useGenerateVideo() — media generation
  • useRealtimeChat() — voice via OpenAI Realtime and ElevenLabs
  • All hooks share a consistent API shape: send(), status, output, error

Provider Portability​

Switching providers is a one-line import change — everything else stays the same:

// Switch from OpenAI to Anthropic — only the import changes
- import { openaiText } from '@tanstack/ai-openai'
+ import { anthropicText } from '@tanstack/ai-anthropic'

Supported providers at beta: OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, xAI, ElevenLabs, and fal.ai.

Type Safety That Goes Deep​

This isn't just "TypeScript types for the API surface." Each provider's model options are typed individually — gpt-4o exposes different config knobs than claude-sonnet-4. Provider-specific tools (web search, code execution) are gated at the type level: if you try to pass an OpenAI-only tool to an Anthropic adapter, the compiler catches it before you ever hit a runtime error.

Beyond Chat: Tools, Middleware, and Orchestration​

The SDK covers the full AI application surface area beyond simple chat:

  • Tool calling — isolated server/client implementations with schema validation
  • Lazy tool discovery — tools loaded on demand for token savings in large tool registries
  • Middleware — composable pipeline for logging, caching, and content guarding
  • Host-side MCP — connect to MCP servers directly from your application
  • Experimental orchestration — generator-based workflows with human-in-the-loop checkpoints

Maturity Signals​

Alpha shipped in December 2025; beta landed six months later with a substantially larger surface area. The project runs 265 deterministic E2E tests across 10 providers on every PR — not mocked, actual provider calls. Downloads have reached 7.1M total (~98K weekly). Notably, TanStack AI interoperates with any AG-UI-compliant agent framework, including Microsoft Agent Framework, LangGraph, CrewAI, Mastra, Pydantic AI, and LlamaIndex.

Worth Evaluating​

If your team wants provider flexibility and observable AI pipelines without platform lock-in, TanStack AI is worth a serious look. The docs are thorough, and the team published an honest comparison with Vercel AI SDK that's refreshingly direct about trade-offs in both directions.