Skip to content

Set up gateway telemetry Beta ​

If your app reaches LLM providers through a gateway, the gateway can send WildEdge a record of every model call. You don't need to change your app or install an SDK. The gateway exports OpenTelemetry (OTLP) traces, and WildEdge turns each model call into an inference event.

Each event records:

  • the provider and the model that served the call
  • input, output, cached and reasoning tokens, when the gateway reports them
  • latency, and time to first token for streamed calls
  • whether the call succeeded, and the error code when it failed
  • the gateway and its service name, and the model name your app asked the gateway for
  • the cost the gateway reports, when it reports one

When the gateway runs MCP tools, each tool call appears in the trace as a tool span, with the tool name, the MCP server, its duration and whether it failed. Tool arguments and results are never kept.

When a gateway includes prompt and response text in its traces, WildEdge keeps it as the event's input and output summaries. Each gateway's setup says how to leave it out.

WildEdge accepts traces only. Don't send OpenTelemetry metrics or logs to it.

See Troubleshooting guide for guidance on failures, latency, usage, and cost.

LiteLLM ​

LiteLLM proxy 1.106.6, configured with environment variables. Set up LiteLLM

OpenRouter ​

OpenRouter Broadcast, configured in the OpenRouter dashboard. Set up OpenRouter

Bifrost ​

Bifrost v2.5.1, configured with its otel plugin. Set up Bifrost

Query gateway events ​

Gateway events have their own columns, which you can use in SQL on the Analytics page or through MCP. They are empty on events from the SDKs.

ColumnWhat it holds
gatewayThe gateway that reported the call, for example litellm.
service_nameThe service name you set on the gateway.
model_providerThe provider that served the call, for example openai.
requested_model_idThe model name your app sent to the gateway.
request_idOne gateway request, where the gateway reports it.
reported_cost_amountThe cost the gateway reports, as an exact decimal.
reported_cost_currencyThe currency of that cost, for example USD.
reported_cost_is_estimatedtrue when the gateway priced the call itself.

What each gateway puts in these columns is on its setup page.

For example, the cost and token use per requested model over the selected time range:

sql
SELECT
  gateway,
  requested_model_id,
  model_name,
  COUNT(*) AS calls,
  SUM(tokens_in) AS tokens_in,
  SUM(tokens_out) AS tokens_out,
  SUM(reported_cost_amount) AS cost_usd
FROM events
WHERE gateway IS NOT NULL
  AND reported_cost_currency = 'USD'
GROUP BY ALL
ORDER BY cost_usd DESC

Calls without a reported cost have no currency, so the filter leaves them out. Count them separately with reported_cost_amount IS NULL.

Troubleshooting ​

When an export fails, the gateway logs the HTTP status:

StatusMeaningWhat to do
401The secret is missing or wrong.Check the X-Project-Secret header, and that the project key isn't revoked.
404Wrong endpoint.Check the endpoint against your gateway's setup page.
413The export is too large.Send smaller batches, or leave prompt and response text out.
415Unsupported encoding.Send OTLP over HTTP as protobuf or JSON.
429Too many requests for this project key.Check your gateway's retries. If it persists, contact us.
503WildEdge couldn't store the events yet.Check your gateway's retries.

When WildEdge answers 429 or 503, it adds a Retry-After of a second or two. Whether the gateway retries, and for how long, is on its setup page. Events in an export the gateway drops are lost.

Next: investigate ​

Connect your AI agent, or follow Investigate events and traces. If your app also uses an SDK, check the combined setup guide to avoid recording the same call twice.