Open source · Apache-2.0 · Go

Your API tells you everything. Decide what it repeats.

OpticTrace is a governance gateway for HTTP APIs. One declarative optic.yaml controls which routes are monitored, which payloads are stored, what gets masked, and which request attributes become Prometheus dimensions — reviewed in your repo like any other code.

Get started brew install dwarka-prasad/tap/optictrace

Terminal demo: a payment request is sent with a real card number, the client receives the real bytes back, and the stored telemetry shows the card, CVV, email and bearer token all replaced with [REDACTED]. — Real output from a running agent. The client receives the original bytes; the card number, CVV, email and bearer token never reach telemetry.

+0.22 µsadded latency, restricted route

13CLI commands

4framework SDKs, one engine

0CGO dependencies

API observability usually forces a bad trade

Log everything and credit card numbers end up in your log pipeline, replicated to three vendors and retained for a year. Log nothing and you are debugging production from status codes alone.

Option A

Capture everything

Fast to set up, and now every downstream system is in PCI scope. The decision was never written down, so nobody can tell you what is being stored.

Option B

Capture nothing

Safe and useless. When something breaks at 3am you have a status code, a duration, and no idea what the client actually sent.

OpticTrace

Declare the difference

Everything is captured unless a rule says otherwise. The rules live in your repo, get code-reviewed, and are testable in CI — so the trade-off is explicit and auditable.

How it works

Two lanes, one tee point

A request travels one path while its telemetry travels another. The traffic lane is a plain reverse proxy. The telemetry lane branches off a bounded copy and is where every governance decision happens.

This is the invariant everything else rests on: live traffic is never modified. A rule that masks $.credit_card.number does not strip the card from the payment request — the payment still works. It strips it from what gets logged, stored and exported.

STAGE 1EvaluateMatch rules, merge into one policy.

STAGE 2AttachWire up buffers — or skip entirely.

STAGE 3ObserveStatus, latency and byte counts.

STAGE 4GovernRestrict, redact, extract meters.

STAGE 5Fan outOne record to every sink.

Stage 2 is where the performance story lives: the policy resolves before any capture machinery attaches, so a route you have told OpticTrace to leave alone allocates nothing.

The whole interface

One file, reviewed like code

Parsing is strict — an unknown key is rejected at load, so a typo like restirct: fails immediately instead of quietly disabling your governance.

optic.yaml

# Everything is captured unless a rule subtracts from it.
rules:
  # Credentials never reach telemetry.
  - name: no-capture-on-auth
    match: { path: "/api/v1/auth/**" }
    restrict: [request_body, response_body, headers]

  - name: redact-payment-secrets
    match: { path: "/api/v1/payments/**" }
    redact:
      headers: [Authorization, X-Api-Key]
      query_params: [api_key]
      json_fields:
        - "$.credit_card.number"   # exact path
        - "$.*.ssn"                # any single key
        - "$.**.card_token"        # any depth
    labels:
      tenant: "header:X-Tenant-ID"  # a Prometheus dimension
    sample: 0.25                # bodies for 25%…
    keep_errors: true          # …but always keep 5xx

  # Prompts stay private; tokens are still counted and billed.
  - name: meter-ai-tokens
    match: { path: "/api/v1/ai/**" }
    restrict: [request_body, response_body]
    meter: { tokens: "$.usage.total_tokens" }

what gets stored

{
  "request_body": {
    "amount": 4200,
    "credit_card": {
      "cvv": "[REDACTED]",
      "number": "[REDACTED]"
    },
    "customer": {
      "email": "[REDACTED]",
      "name": "Ada Lovelace"
    }
  },
  "headers": {
    "Authorization": "[REDACTED]",
    "X-Tenant-Id": "acme"
  },
  "labels": { "tenant": "acme" },
  "matched_rules": ["redact-payment-secrets"]
}

Non-sensitive fields survive, so the record is still worth debugging with. Rules merge top-to-bottom: restrictions only narrow capture, while redactions and labels accumulate.

Built in

A dashboard the binary serves itself

No separate frontend to deploy. The agent compiles the UI in and serves it from its control plane, on a port you can firewall independently of the API it proxies.

Eight tabs, each answering a question someone actually arrives with: what is happening (Overview), which routes (Routes), what happened to this request (Traces), show me that exchange (Inspector), where did this log line come from (Logs), who is using what (Usage), is my policy actually applying (Governance), plus the config it loaded and the agent’s own health.

Traces page with a checkout open: a waterfall on one shared timeline showing the order hop, the four database queries, the insert with an index refresh nested inside it, and a cache lookup under the catalog hop — each operation with its statement, and the customer email inside an interpolated INSERT shown as REDACTED. — Traces, down to the query
One row per request, however many services it touched — then a waterfall on a shared timeline: every hop, **every operation inside each hop**, and the log lines each wrote. Four identical `db.query stock` calls in one request is an N+1 you can see. The `self` figure separates a hop’s own code from the work it delegated, and the interpolated `INSERT` shows a customer email stored `[REDACTED]` — span attributes are governed before storage, exactly like a log line.

Overview page: requests, error rate, P95 and P99, rules firing and log line counts across the top; request volume against errors; succeeded, rejected and failed as a stacked area; a latency panel showing p95 spiking to 300ms through an incident window while the average stays flat; traffic by tenant; log lines by level; a capture and sampling panel reporting 775 of 1,285 bodies stored; top routes; and which rules are firing. — Overview
Golden signals, per-tenant traffic, and two readings you cannot get elsewhere: **p95 drawn against the average**, because the mean hides the requests worth investigating — the spike here is an incident the average barely registers — and **capture & sampling**, the only honest check that a sampling rule is doing what you think. A rule that matched nothing gets called out by name.

Application logs page: counts per level as clickable filters, a search box, level and service pickers, and a table of log lines each linking to the request that wrote it. One debug line reads charging card [REDACTED]. — Application logs
The line is usually what you have — someone pasted it. Every one links back to the request that wrote it.

Usage page showing per-tenant requests, data transferred, compute time, token meters and estimated cost. — Usage & cost
Per-tenant consumption, meters and billing export.

Request Inspector with a payment selected: the card number and CVV show as [REDACTED] in the request body, the matched rules and tenant, region and tier tags are listed, the request trace is drawn, and the application log lines that request wrote appear beneath it. — Request Inspector
Redacted values are highlighted so you can see governance working, the rules responsible are named, and every non-sensitive field is still there — with the request’s own trace and the log lines its handler wrote underneath it.

The provisioned Grafana dashboard showing request rate, error rate, latency percentiles, per-route table and agent health panels. — Grafana, provisioned
`docker compose up` brings the agent, Prometheus and Grafana up together with this dashboard and seven alert rules already loaded — including alerts on the agent's own health, because a monitor that silently stops recording is worse than none.

Getting started

Start from the spec you already have

Writing governance by hand against an API you may not have written means finding out what you missed once traffic flows. A specification already lists the routes and the shape of every payload, so most of the first draft can be derived rather than guessed.

OpenAPI 3.x or Swagger 2.0, YAML or JSON

optictrace init -spec openapi.yaml -out optic.yaml

✓ wrote optic.yaml — 4 route(s), 4 rule(s), 10 field(s) masked

what it derives

  - name: redact-api-v1-payments-charge
    match:
      path: "/api/v1/payments/charge"
    redact:
      query_params: ["api_key"]
      json_fields:
        - "$.**.card.number"   # high · PCI-DSS scope
        - "$.**.card.cvv"      # high · PCI-DSS scope
        - "$.**.customer.email" # medium · personal data

Credential headers come from the document’s securitySchemes — the one thing a specification states outright rather than implies. Routes that look like credential exchanges get metadata-only capture, because there no redaction rule is as reliable as not recording the body at all.

It is a starting point, and the file says so. A spec describes what an API claims; governance has to hold for what it does
A field the document does not model cannot be masked by a rule derived from it — and a field called ref can hold a card number, which is what optictrace scan finds on real traffic and no name heuristic will
Caveats print to stderr, so init -spec x.yaml > optic.yaml still gives you a clean file; it refuses to overwrite an existing policy, and validates what it generated before handing it over

What it did, and why

One request, every hop, and the lines it logged

Every record carries W3C trace context, so services reporting into one store stop being a flat list and become a request tree. The forwarded request carries this hop’s span, which is what makes downstream calls nest under it rather than becoming siblings.

That span is also handed to your application — so the lines it writes while serving a request can be filed under that exact request. Correlation is a fact, not a guess: nothing is matched by timestamp, which under concurrent traffic would file one tenant’s log line inside another tenant’s request.

$ curl '.../api/logs?trace=a8f2890a4536810938c4ec89ddd40872'

storefront  POST /api/v1/orders            200  span=83c03ddd parent=-
  catalog   GET  /api/v1/catalog/SKU-100   200  span=3a2c0d92 parent=83c03ddd
  payments  POST /api/v1/payments/charge   200  span=e0f134d5 parent=83c03ddd
      [debug] charging card [REDACTED] for 129.00
      [info ] charge requested  amount=129.0 order_ref=ord-SKU-100
      [info ] charge captured   amount=129.0

The dashboard draws the same thing as a waterfall — every hop on one timeline, with each hop’s log lines under it — so a request made of two 40ms calls reads as either nearly optimal or 40ms of avoidable waiting, which is a distinction a list of durations cannot make.

And inside one hop. A hop tells you a request took 300ms; it does not tell you that 280 of them were one query. Name an operation — a query, a cache lookup, a call out — and it appears on the same timeline, nested under the hop that ran it. Two things separate this from any other tracing library: the attributes are governed, so a statement that quotes its parameters is redacted before storage rather than cleaned up afterwards; and the breakdown reports count and requests apart, so the per-request multiplier is visible — four thousand calls to one query reads as busy traffic until you know it was a hundred requests.

That [debug] line is a service logging its own card number while someone was debugging — which is how the leak actually happens. Log lines are the highest-risk surface here: a payload is structured and can be redacted by path, but a log line is free text. So they run through the same policy on the way in rather than being stored raw and cleaned up later.

optic.yaml

telemetry:
  app_logs:
    enabled: true
    level_min: info
    max_lines_per_span: 200
    retention_max_age: 168h
    drop_orphans: true
    redact:
      patterns:
        - 'Bearer\s+\S+'
        - '\b\d{13,19}\b'
      fields: [authorization, password]

Erasure covers logs. purge deletes a tenant’s log lines with their records in one transaction — removing the requests but keeping the lines they wrote is not erasure
Drops are counted, never silent. Lines with no request behind them are discarded by default and reported in optictrace_app_logs_dropped_total
Bounded by design. A level floor, a per-span line cap and a byte cap, so one retry loop cannot swamp the store
Storage is optional. ext.AppLogStore is separate from ext.Store, so a third-party driver without it is still a complete driver

Because it owns your traffic history

Things a static tool cannot do

OpticTrace holds a governed record of what your API actually did. That turns several hard questions into simple ones.

optictrace scan

Find what your rules missed

Redaction masks what you name. This catches the field you forgot, using checksums and issuer prefixes rather than guesswork — and prints the rule that fixes it. Samples are always masked.

optictrace check

Break nothing

Answers "is any live client actually using the field I am about to remove?" with usage counts and last-seen times. Exits non-zero in CI.

optictrace spec / sdk

Docs that write themselves

Infers an OpenAPI document from what clients really send, then emits typed TypeScript, Python or Go clients. Redacted fields still contribute their name and type.

optictrace mock

A mock with state

POST /cart then GET /cart returns the item you added. Other routes return schema-conforming data, optionally generated by Claude.

optictrace test

Testable governance

Assert that a route redacts what it should, with no server and no network, so CI proves a refactor did not quietly stop masking.

optictrace replay

Replay, honestly

Re-issues captured traffic against staging and diffs status codes — skipping what governance made unreplayable rather than pretending.

In your workflow

It reviews your pull requests

Every other command is one you have to remember to run. This one runs itself and answers the question a reviewer actually has: does this change make governance weaker?

It evaluates the same captured traffic under the base branch's rules and the pull request's, then reports where they disagree. A rule reordering that silently stops masking a card number is invisible in a text diff and obvious here.

optictrace commented on this pull request

✗ This change weakens governance on 4 points

Each one below was verified by replaying real traffic. If it is deliberate, say so in the PR.

	Route	Change	Requests affected
✗	`POST /api/v1/payments/**`	stops redacting `$.**.credit_card.cvv`	34
✗	`POST /api/v1/payments/**`	stops redacting query param `api_key`	34
⚠	`POST /api/v1/auth/**`	now captures request bodies (was restricted)	34
⚠	`POST /api/v1/payments/**`	drops label `region`	34

By default a pull request fails only for what it changed. Pre-existing findings are reported for context but do not block — failing every PR for a problem someone else introduced is how a bot gets muted, and a muted bot protects nothing.

Measured, not asserted

What it costs

"Low overhead" deserves numbers. These come from go test -bench comparing a bare handler against the same handler wrapped by the interceptor.

Restricted route (capture off)

+0.22 µs

Full capture + redaction

+5.4 µs

…plus Prometheus, custom label

+5.5 µs

Read the absolute numbers

Against a typical API call of 1–100 ms, full capture is 0.005–0.5% of the request. It is not free — the cost is dominated by JSON parse and re-serialise — which is why sample with keep_errors exists for very hot routes.

The design claim holds

Restricting a route really is near-free. The policy resolves before any buffer is attached, so a route you have excluded costs 0.22 µs — and Prometheus observation, even with a custom label dimension, is noise at 0.13 µs.

12th Gen Intel i5-1235U, Go 1.25, -benchtime=2s, parallel. Reproduce with make bench.

Deploy it

Sidecar, middleware, or SDK

All three share one interception code path, so governance behaves identically however you run it.

standalone gateway

# proxies to service.upstream, dashboard on :9095
optictrace run -config optic.yaml

embedded in Go

agent, _ := optictrace.New("optic.yaml")
defer agent.Close()
http.ListenAndServe(":8080", agent.Middleware(mux))

Express

app.use(optictrace({
  configPath: 'optic.yaml',
  agentUrl: 'http://localhost:9095',
}));

FastAPI

app.add_middleware(
    OpticTraceMiddleware,
    config_path="optic.yaml",
    agent_url="http://localhost:9095",
)

# your ordinary logging, filed under the request that wrote it
logging.getLogger().addHandler(
    OpticTraceLogHandler("http://localhost:9095"))

Java · Spring Boot, Quarkus, Jetty

new OpticTraceFilter(
    "optic.yaml", "http://localhost:9095", "checkout");

Logger.getLogger("").addHandler(
    new OpticTraceLogHandler(agentUrl, "checkout"));

SDKs apply governance in-process — raw payloads never cross a process boundary
All four evaluate the same rule engine: the same optic.yaml producing different series depending on which runtime served the request would be worse than no telemetry, so parity is asserted by each SDK's suite against a live agent
With no listen and no upstream the agent runs in collector mode: store, metrics and dashboard, no proxy in the request path
SQLite for a sidecar, Postgres when several agents share history
Export to files, webhooks, OpenTelemetry, or your own executable

Try it in about a minute

The Compose stack brings up the agent, a demo API, Prometheus and Grafana with dashboards and alerts already provisioned — and deliberately leaves one route ungoverned so scan has something to find.

git clone https://github.com/dwarka-prasad/optictrace && cd optictrace
docker compose up --build

# http://localhost:9095  dashboard · metrics · APIs
# http://localhost:3000  Grafana
# http://localhost:9090  Prometheus

Want to see the SDK, traces and application logs working together? examples/python-shop is three FastAPI services making real calls to each other, with a 25-assertion suite that checks the claims on this page against a live stack. examples/springboot-shop does the same on the JVM — one checkout that fans out to a catalog read and a payment charge over real HTTP, so a single request produces three correlated spans.

Prefer just the binary? brew install dwarka-prasad/tap/optictrace covers macOS and Linux on Intel and ARM, go install builds from source, and every release ships checksummed, cosign-signed archives with an SBOM.

Putting it in front of something that already exists? The integration guide has the steps for each route — sidecar on a host, in Docker, in Compose or in Kubernetes; middleware for Express, FastAPI, Java, Go and Gin — plus a rollout order for a service already in production that never has a step where the store holds data you did not intend it to.

Integration guide Read the docs on GitHub Download a release