I have an AI Gateway

Amberflo integrates with third-party AI Gateways. Gateways act as the control point for all model traffic, capturing the metadata needed to understand usage, attribute costs, and govern AI workloads. When connected to Amberflo, a gateway becomes significantly more powerful: its raw traffic is transformed into structured, attributed, real-time usage data that unlocks budgets, guardrails, attribution, multi-source aggregation, and AI governance across your organization.

Amberflo supports connecting multiple gateways to a single Amberflo account. You can run different gateways for different teams, environments, or use cases, and all usage will be unified inside Amberflo.

This guide covers how to connect LiteLLM when it is already deployed in your environment.

If you do not already have LiteLLM running, or if your gateway is not currently supported, follow the guide below to deploy LiteLLM from scratch and integrate it with Amberflo.

This guide assumes you already have LiteLLM deployed and operational. You will:

LiteLLM handles and proxies LLM requests.

Amberflo provides real-time usage metering, attribution, and cost governance.

The Amberflo callback attaches to LiteLLM and pushes meter events directly to the Amberflo ingestion API. Once enabled, model usage appears immediately in the Amberflo AI Control Tower. It is important to note that the meter events are only metadata about the call to the LLM. The prompt and response never leave your VPC.

You must already have:

Note:

LiteLLM can technically run without Postgres, but many required features (virtual keys, teams, routing rules) will not function. Postgres is required for the Amberflo integration.

Amberflo has created a package to collect the necessary metadata and send it to Amberflo.

Your directory should look like this:

Note: Do not modify any callback source files.

Edit your LiteLLM config and add:

If you already have other callbacks, add "amberflo.litellm.callback" as an additional entry.

This tells LiteLLM to load the callback code from the amberflo/ directory.

Amberflo provides the required variables and values for connecting the Amberflo callback to your account. You can find them in the connection wizard. Simply copy them and add them to your existing environment file.

The variables contain:

If you do not currently have an environment file you can create the .env file in the same directory where you run your Docker command and then copy in the values.

Note: Do not commit this file to Git.

Optionally you can update the AFLO_HOSTED_ENV value. The string set as the value will be used to identify the instance of the AI Gateway. Amberflo supports the ability to connect multiple AI Gateways and this will allow you to filter the data based on which AI Gateway instance the data is coming from.

Modify your existing Docker command to include if you were not already using an environment file:

A volume mount for the amberflo callback directory

Example:

Key points:

No additional installation is required.

Stop your current LiteLLM container and redeploy using the updated command.

Make sure to update the placeholder values in the cURL command below.:

Once you've successfully made the API call in the previous step you should log in to Amberflo and check the AI Spend Dashboard's Summary page. You will begin to see your usage and cost show up there.

Events should appear in near real time. It can take up to 2 minutes for the first data to show up. You may need to refresh the page.

Amberflo automatically creates a new Business Unit the first time a virtual key is used.

Mapping:

All future events for that key are attributed to that Business Unit.

You can rename Business Units later if needed. The first time you send data for a particular team you will see only their LiteLLM team ID shown in the Amberflo App. If can take up to 5 minutes for the business unit to be fully created in Amberflo.