tracelab

Upload a CSV and ask questions in plain English. A team of LangGraph agents plans the analysis, writes and runs Python in a sandbox, and streams the whole thing back live — router, planner, parallel analysts, critic, and composer each lighting up on the trace with their token cost ticking up in real time. Most agent demos show you the answer; tracelab shows you the reasoning, the verification, and the receipts.

Its central claim is that LLM output is untrusted by default. Every number an analyst claims is independently re-derived by a critic that never sees the analyst's code — only the question, the dataset profile, and the claim — so agreement actually means something. A deterministic reconciliation function, not the model's judgment, decides whether the two match: integers exactly, floats within a relative epsilon, and statistical claims on direction, significance, and whether the method was appropriate at all. A discrepancy triggers exactly one bounded retry with the critic's findings injected; anything still unresolved ships flagged unverified rather than quietly dropped.

The measurement layer is the other half of the project. Every run is persisted as a span tree with per-agent tokens, cost, and latency, and can be replayed offline — deterministically and for free. A golden eval set of 33 questions is computed from the actual CSVs rather than hand-typed, scored in two tiers (programmatic comparison, plus an LLM judge pinned to a model independent of the config under test), and every run is tagged with a config hash covering the per-role models and a content digest of every agent prompt, so a pass-rate move is attributable rather than merely noticed.