Full-Stack AI Observability & Tracing with OpenTelemetry
Debugging distributed AI architectures with console.log is impossible when a request spans React, Node.js middleware, PostgreSQL vector queries, and upstream LLM streams. This article instruments the full Workspace stack with OpenTelemetry GenAI semantic conventions, providing end-to-end distributed traces, token attribution, and latency waterfall debugging across all seven tiers.
Full-Stack AI Architecture: From Streaming Foundations to Resilient Agentic Systems in Node.js & React
Full-Stack AI Observability & Tracing with OpenTelemetry
"The AI is slow" is not an actionable incident report. Is it slow because the HNSW query is taking 400ms? Because the cross-encoder reranker is timing out? Because the LLM provider's P99 spiked? Because the React client is queuing SSE events faster than it can render? Without distributed tracing that spans from the React component through the Fastify gateway, pgvector retrieval, and the streaming LLM call, all of these look identical from the outside: a slow response.
This article instruments the complete Workspace stack with OpenTelemetry following the GenAI semantic conventions: TTFT and ITL as span attributes on LLM calls, per-tenant token attribution for cost analysis, retrieval spans with chunk counts and reranker scores, and a React TTFT measurement hook that feeds the first-token timestamp into a browser span exported to the same trace.
Who this is for: Engineers debugging AI system performance with only server access logs and no visibility into which layer of a multi-stage pipeline is the bottleneck. After this article, a single Grafana trace view shows the exact latency breakdown across every layer.
1. GenAI Semantic Conventions
The OpenTelemetry GenAI Semantic Conventions (spec v0.9+) define standard attribute names for LLM spans, enabling vendor-neutral instrumentation that any observability backend understands.
| Attribute | Value Example |
|---|---|
gen_ai.provider.name |
'anthropic', 'openai' |
gen_ai.operation.name |
'chat', 'text_completion', 'embeddings' |
gen_ai.request.model |
'claude-3-5-sonnet-20241022' |
gen_ai.request.max_tokens |
4096 |
gen_ai.response.model |
'claude-3-5-sonnet-20241022' |
gen_ai.response.finish_reason |
'end_turn', 'max_tokens', 'stop' |
gen_ai.usage.input_tokens |
1284 |
gen_ai.usage.output_tokens |
823 |
gen_ai.workspace.ttft_ms |
1340 (custom) |
gen_ai.workspace.org_id |
'org-uuid' (custom) |
2. OpenTelemetry Setup
3. LLM Call Instrumentation
4. RAG Pipeline Span
5. Browser TTFT Measurement (React)
Summary
| Layer | Span | Key Attributes |
|---|---|---|
| React Client | client.sse.ttft |
client_ttft_ms, server_trace_id |
| Fastify Route | http.server (auto) |
http.status_code, http.route |
| RAG Retrieval | workspace.rag.retrieve |
chunks_returned, top_score, hybrid_search_ms |
| LLM Call | gen_ai.chat |
ttft_ms, p99_itl_ms, input_tokens, output_tokens |
What's Next
The final part of this series — Part 24 — assembles every component from Parts 1–23 into the complete Enterprise Collaborative AI Workspace production deployment: end-to-end system diagram, team conventions, performance benchmarks, and the architecture decision record for each key design choice.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.