OTELDEBUG.NOCHAOS.IO
back

How to use

oteldebug catches OpenTelemetry data on its way out of your application. You get a short-lived token, stamp your telemetry with it, and every span, log record and metric carrying that stamp shows up on the console in near real time. Nothing else is stored, and everything is wiped when the token expires.

01 generate a debug token

  1. Open the console and hit generate token.
  2. The token is a 32-character hex string, valid for 5 minutes. The countdown is on the console.
  3. The page URL now carries ?token=…: bookmark or share it to look at the same session.
limit 5 tokens per minute per client. If you hit the limit, wait for the Retry-After the API hands back.

02 stamp your telemetry

Add an attribute named oteldebug whose value is the token. Where it goes depends on the signal:

  • traces: a span attribute (the collector filters spans without it).
  • logs: a log record attribute.
  • metrics: a resource attribute.
note the API reads the token from the first span / record / resource of each OTLP export batch. Batches without a valid, unexpired token are discarded at the collector or dropped by the API.

03 send it to the collector

protocolendpoint
otlp/grpclocalhost:4317
otlp/httphttp://localhost:4318/v1/traces · /v1/logs · /v1/metrics

With telemetrygen (from the collector-contrib repo) a full run looks like this:

telemetrygen traces \
  --otlp-endpoint localhost:4317 \
  --otlp-insecure \
  --traces 10 --rate 1 --duration 10s \
  --status-code Ok --span-duration 100ms \
  --telemetry-attributes 'oteldebug="YOUR_TOKEN_HERE"'

Or with plain curl against the OTLP/HTTP endpoint:

curl -sS http://localhost:4318/v1/traces \
  -H 'Content-Type: application/json' \
  -d '{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"checkout"}}]},
       "scopeSpans":[{"spans":[{"traceId":"5b8efff798038103d269b633813fc60c","spanId":"eee19b7ec3c1b174",
       "name":"GET /cart","kind":2,"startTimeUnixNano":"1700000000000000000","endTimeUnixNano":"1700000000100000000",
       "attributes":[{"key":"oteldebug","value":{"stringValue":"YOUR_TOKEN_HERE"}}]}]}]}]}'

04 watch it land

The console polls the API every 5 seconds and shows the newest 100 items of each signal. Expand a row to see the raw OTLP JSON, json copies it, link copies a URL that opens straight on that item. The filter box greps the current tab; press / to focus it.

ttl when the token expires the console stops polling and the API deletes the captured data within a minute. Generate a new token and update your exporter.

how it works

  1. Your app exports OTLP to the OpenTelemetry Collector (gRPC 4317 / HTTP 4318).
  2. A filter processor drops anything without the oteldebug attribute.
  3. What survives is published as OTLP JSON to Kafka (otel-traces, otel-logs, otel-metrics).
  4. The Go API consumes the topics, checks the token is known and unexpired, and stores the payload in PostgreSQL.
  5. This page reads it back through the REST API. A cleanup job purges expired tokens and their data every minute.

api

methodpathnotes
POST/api/tokensnew token; rate-limited (5/min per client)
GET/api/tokens/{token}/statusvalid, expires_at
GET/api/tokens/{token}/tracesnewest 100; 401 once expired
GET/api/tokens/{token}/logssame
GET/api/tokens/{token}/metricssame
GET/healthz · /readyzliveness · readiness (db ping)

local stack

serviceaddress
webhttp://localhost:3000 (or WEB_PORT)
apiproxied at /api by the web container
otlp/grpclocalhost:4317
otlp/httplocalhost:4318
kafka uihttp://localhost:8900