Configuring LiteLLM

the following are the steps to set up providers, models, teams and keys for the litellm ai gateway log into the admin ui the ai gateway includes a web interface for managing models, provider credentials, api keys, users, and teams it allows you to configure access control, set usage limits, and monitor real time activity and cost without restarting the service the interface also provides logs to help you manage and audit llm usage across your organization the first time you access the ai gateway you will need the master username and password that you set in the environment file you edited previously they were the values for ui username and ui password go to http //\<vm ip address> 4000/ui if running locally it would be http //localhost 4000/ui enter the username enter the master password click the login button add llm provider(s) you will need to provide llm provider credentials using the web interface llm provider credentials are the authentication details required to connect the ai gateway to your llm providers, such as openai, aws bedrock, azure openai, and others these credentials are stored securely and used by the gateway to route requests to the correct provider adding provider credentials enables you to centralize access management and eliminates the need for individual users or teams to manage their own keys open the models + endpoints tab from the left hand navigation menu click the llm credentials tab click add credential select the provider you want to configure (e g , openai) fill in the required fields credential name a friendly name for this provider instance (e g , openai default) provider select the provider you want to add an api key for from the drop down api key the access token or api key for the provider when you have entered these fields click add credential you can add additional providers by simply repeating the steps above after setting up your providers, the next step is to add models that reference those providers and specify the configuration details for each add llm model name(s) after providers are set up, models determine the specific endpoints the gateway makes available a model configuration links a model name (for example, gpt 5o) to a provider and any associated credentials or parameters this allows multiple providers or variants of the same model to be managed consistently through the gateway open the models + endpoints tab from the left hand navigation menu click on the add model tab choose the provider from the list this will populate the list for the model names click on the list to the right of where it says model name(s) this will be a long list with every model available use the fuzzy search to narrow down the list to find the model you are looking for and click on it you will see the model added in the search bar space this menu is multi select so you can add all of the models from this provider at once you will see the models you’ve selected automatically show up in the model mappings table this will allow you to use a different public name than the model identifier that is used under the hood our recommendation is to create credentials to use here as opposed to entering the api keys as part of the model setup click in the box to the right of where it says existing credentials and select the appropriate credentials from the list when done, click the add model button in the bottom right of the page once a model is set up, it becomes available for use through the ai gateway’s openai compatible api any virtual api key with access to that model can now send requests to it, and the gateway will automatically route the call to the correct provider using the associated credentials set up teams (cost centers) teams in the ai gateway let you group users and api keys under a shared identity for easier access control and cost management teams will map to business units (a k a cost centers) in amberflo you can assign budgets, rate limits, and model permissions at the team level, ensuring consistent policies across multiple users teams are useful for managing access by department, project, or external partner while tracking usage and spend in a centralized way add a team via the web interface open the teams link from the left hand navigation menu click create new team enter a name for your team a team can be used simply as a tool to help organize users the name is the only required field ⚠️ it is extremely important that the name of the team match exactly to the business unit id set in the amberflo app this is how we allocate the costs the allowed character set is alphanumeric characters as well as (underscore) (dash) (point) + (plus) @ (at) it must start and end with an alphanumeric character max 200 characters ⚠️ if that is all you are looking for then you can simply hit the create team button at the bottom right of the window otherwise continue to the team limits section below virtual keys virtual keys are how you will provide access to the resources you make available in the ai gateway virtual keys will also help give you a more granular view into your usage and costs for each use case you should create a new virtual key for example, you should create one for each environment that an application would be deployed in this makes it easy to see the cost breakdown between development and production environments virtual keys can also be created to grant admin access to the ai gateway for example, you can create a virtual key that would provide access to the ai gateway so you can have existing internal tools make changes to the gateway adding virtual keys via the web interface open the virtual keys link from the left hand navigation menu click create new key select you if you want to assign this key to a team you previously created, select the team from the team dropdown if not, you do not need to select anything here select the models you would like this key to have access to if you want this key to have access to all the models you do not need to select them individually you can choose all team models finally, choose the key type is this key only for accessing llms, management apis or both if the option is both, choose default otherwise you can narrow its access to one of the two use cases click create key and you will get a popup that will show you the key value and allow you to copy it you must copy it now because it will not show the key again you will need to create a new one