LiteLLM is moving to Rust Read the latest updates.
Changelog
The main changes in each LiteLLM release. Open a release for its full notes, upgrade steps and every merged PR. Watch BerriAI/litellm on GitHub to hear about new releases.
LiteLLM v1.106.8 fixes transcription cost for some FLAC files
- Transcription cost is correct for some FLAC files that could push a key over its budget.
- Per-second transcriptions bill the
usage.secondsthat the provider reports, so spend matches the provider invoice and unmeasured files no longer log $0.
LiteLLM v1.109.0 adds 11 providers, Codex tool search and daily limits
- LiteLLM adds 11 providers, including ai&, StreamLake, OpenInfer and BigModel, among 311 new model catalog entries.
- Codex discovers MCP tools through tool search, and deployments accept
rpdandtpddaily limits. - Streams that stall, abort or end without a finish reason now fail visibly, so retries, cooldowns and fallbacks apply to them.
- Claude models missing from the cost map now default to
max_tokens: 128000, andDEFAULT_ANTHROPIC_CHAT_MAX_TOKENS=4096keeps the old value.
LiteLLM v1.110.0rc1 adds the Agent Skills Hub and a Rust-only router
- The router always runs on the Rust core, and a router config that the Rust core declines now stops the proxy at startup.
- Spend goes through the
LiteLLM_SpendLogDeliveryledger by default, andgeneral_settings.use_spend_logs_ledger: falserestores the previous spend queues. - The Agent Skills Hub adds skill upload and import, version history and an admin review queue for member submissions.
- Streamed
/v1/responsesno longer ends with[DONE], and raw response ids that the proxy did not issue are rejected by default.
LiteLLM v1.108.5 (LTS) fixes workload identity on OpenAI routes and transcription cost
- Keyless workload identity OpenAI deployments no longer answer 401 on image edits, audio, files, batches and the other routes beyond chat.
- Transcription cost is correct for some FLAC files that could push a key over its budget.
ollama_chat/streams give each parallel tool call its own index, and/v1/messageshonors a deployment'sadditional_drop_params.
LiteLLM v1.104.9 and v1.103.11 speed up request lookups in the Logs view
- After migrations, the proxy builds the spend log indexes that some installs lack, so request lookups in the Logs view stay fast on large tables.
- On v1.103.11, replicas that boot together against one database start up more reliably while a peer runs or recovers migrations.
LiteLLM v1.108.4 (LTS) fixes Bedrock pass-through guardrails and video polls
- The
/bedrockpass-through routes enforce the model'sguardrailConfig, and it replaces any guardrail the caller sends. - Video status and content polls keep their model, so per-model keys work and each poll reaches the account that created the video.
- A base SDK install no longer fails with
No module named 'mcp'whencompletion(),responses()ormessagesget a tool.
LiteLLM v1.108.3 (LTS) fixes Bedrock api_base and large MCP tool results
- Bedrock deployments with
api_base: ""serve requests again, and no longer return 500 or 400. - MCP tool results over 1 MiB no longer fail with
SSE stream ended without a response, and model-levelpost_callguardrails check each response once.
LiteLLM v1.108.2 (LTS) fixes Bedrock service tiers and Claude PDF input
- Bedrock Converse honors the
reserved,autoandstandard_onlyservice tiers and returns 400 for a tier that it does not support. - Base64 PDFs that you send to Claude as
image_urlwork, andopenai/deployments with an emptyapi_keyno longer sendOPENAI_ADMIN_KEY. - Groups that accept unmatched tags keep the request after affinity or cooldown filters.
LiteLLM v1.108.1 (LTS) fixes Bedrock guardrail billing and router cooldowns
- A string fallback target such as
default_fallbacks: gpt-3.5stops the proxy at startup, so write it as a list,default_fallbacks: ["gpt-3.5"]. - Spend and budgets include native Bedrock guardrail traces and the Gemini audio rates for flex, priority and batch.
- The router keeps the only deployment of a group warm under
allowed_fails, and evicted SDK clients close their aiohttp sessions.
LiteLLM v1.106.7 supports custom video providers on /v1/videos
/v1/videosserves providers thatcustom_provider_mapregisters.- The proxy and its Helm charts no longer package the Lens beta, so remove
lensWorker.enabledfrom your Helm values before you upgrade.
LiteLLM v1.108.0 (LTS) adds Gemini 4 Argon, Microsoft 365 Copilot and OpenAI Live
- LiteLLM adds Gemini 4 Argon, the Microsoft 365 Copilot provider, OpenAI Live sessions on
/v1/live/sessionsand Azure PTU capacity that teams share. - The proxy count routes return the provider's error instead of a local estimate, and
token_count_framinginmodel_inforestores a local estimate.
LiteLLM v1.106.6 and v1.105.7 fix the gzip performance regression
- The proxy gzips only dashboard analytics responses and compresses them on a worker thread, so
Accept-Encoding: gzipno longer slows down other requests. - Inference and other endpoints outside the dashboard analytics routes return uncompressed responses, and no setting restores the old gzip behavior.
LiteLLM v1.106.5, v1.105.6, v1.104.8 and v1.103.10 trim failure logs for rejected requests
- Failure logs for rejected requests keep only core request fields,
tagsand guardrail details. - The GCS logger uses the bucket and credentials configured on the proxy.
LiteLLM v1.106.4, v1.105.5, v1.104.7 and v1.103.9 refresh image base layers
- Docker images move to newer base images, so image scans stop flagging brotli 1.2.0 and packages the images never install.
- v1.106.4, v1.105.5 and v1.104.7 no longer sometimes exit on startup with an
ImportErrorwhen the remote model cost map is unreachable.
LiteLLM v1.106.3, v1.105.4, v1.104.6 and v1.103.8 change the image-signing key
- Docker images and Helm charts are signed with the cosign key from commit
e82a3c6, and the previous key no longer verifies them. - Every certificate bundle the SDK, proxy and images trust drops the eMudhra root CA.
- v1.104.6 also fixes spend on failed and batch requests, token counts and tool calls on gpt-5.4+ models.
LiteLLM v1.106.2, v1.105.3 and v1.103.7 fix spend on failed requests and batch cost
- Requests that time out after reaching the provider, client cancellations and each billed retry attempt are priced on their prompt instead of logging $0.
- Completed batches keep their spend through malformed results and temporary accounting failures, and Bedrock inference profiles are priced.
- Azure
gpt-6-astrawithreasoning_effort: noneandgpt-5.3-codexwithminimalnow return 400 unlessdrop_paramsis on. - v1.105.3 also lists per-user auth MCP tools in the catalog again.
LiteLLM v1.104.5 counts Claude tokens on Bedrock Mantle when Bedrock Runtime cannot
- Token counts for newer Claude models on Bedrock come from the Bedrock Mantle endpoint instead of a local estimate that undercounts.
LiteLLM v1.106.1 fixes count_tokens, MCP tool calls and thinking blocks
- Bare MCP tool names on a single-server scope now go to that server, so a cold replica no longer returns
Tool not found. - Vector stores created through the proxy send only the caller's
metadatato the provider, so later metadata updates no longer fail with 400. - Token count responses carry
litellm_count_source, and Anthropic-format count requests for OpenAI and Gemini models match how/v1/messagesconverts them.
LiteLLM v1.105.2 fixes count_tokens, bridged tool calls and thinking blocks
- Chat completions bridged to
/responsesreturn text and tool calls in onechoices[0], so agents that read onlychoices[0]run the tool. - Signed thinking blocks with empty text stay on
/v1/messagesand chat completion requests, so replayed turns keep their reasoning. - The
v1.104.4token count fixes are included, and count responses carrylitellm_count_source.
LiteLLM v1.104.4 fixes count_tokens, team batch access and guardrail settings
- Anthropic-format count requests for OpenAI and Gemini models are converted the same way
/v1/messagesinference converts them. - Team keys can read and cancel their own team's batches and download the finished results.
- The Add Guardrail wizard in the Admin UI lists its providers again.
Older releases are listed in the sidebar.