From 4c4298751f2d5dbdf45bcea5e0b4c4039d3a22f4 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Thu, 22 Oct 2026 12:37:01 +0800 Subject: [PATCH] remove litellm, it got hacked twic --- README.md | 2 -- website/data/category_intros/ai-and-agents.md | 3 --- 2 files changed, 5 deletions(-) diff --git a/README.md b/README.md index 52bb149d..562fd850 100644 --- a/README.md +++ b/README.md @@ -176,8 +176,6 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. - [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models. - [deepinfer](https://github.com/deepseek-ai/DeepInfer) - DeepSeek's production inference engine for DeepSeek models, with MoE expert parallelism. -- LLM Gateways - - [litellm](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format. - Image and Video Generation - [diffusers](https://github.com/huggingface/diffusers) - Pre-trained diffusion models for generating and editing images, audio, and video. - Fine-tuning diff --git a/website/data/category_intros/ai-and-agents.md b/website/data/category_intros/ai-and-agents.md index d614a705..881a724c 100644 --- a/website/data/category_intros/ai-and-agents.md +++ b/website/data/category_intros/ai-and-agents.md @@ -12,7 +12,6 @@ How to choose: - Open models from the Hugging Face Hub: Transformers - Running open models locally: Ollama, llama-cpp-python in your own process, MLX LM on Apple silicon - Serving open models: vLLM or SGLang -- One OpenAI-style API for many model providers: LiteLLM - Image and video generation: Diffusers - Fine-tuning: TRL with PEFT adapters, Unsloth for less GPU memory, Axolotl from YAML - Speech: faster-whisper, FunASR for Chinese, gTTS for text to speech @@ -49,8 +48,6 @@ The Ollama Python library talks to Ollama, which [runs models on your computer]( vLLM runs [batch inference offline or an OpenAI-compatible server](https://docs.vllm.ai/en/latest/getting_started/quickstart/), so apps written for the OpenAI API work unchanged. SGLang is [optimized for agentic workloads, RL rollouts, and large-scale serving](https://github.com/sgl-project/sglang). -LiteLLM [returns every provider's responses in the OpenAI format](https://docs.litellm.ai/docs/) and maps their errors to OpenAI's exception types, so one code path covers them all. Use its Python SDK inside your app, or run its proxy as a self-hosted gateway for your team. - Diffusers centers on the [`DiffusionPipeline`](https://huggingface.co/docs/diffusers/index): a few lines generate an image, video, or audio clip. Start fine-tuning with LoRA or QLoRA rather than training every weight, as both [PEFT](https://huggingface.co/docs/peft/methods/overview) and [Unsloth](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide) advise. TRL trains with methods from SFT to DPO and GRPO, and every TRL trainer takes a [`peft_config`](https://huggingface.co/docs/trl/peft_integration) for PEFT adapters. Axolotl keeps the whole pipeline, from dataset preprocessing to inference, in [one YAML file](https://github.com/axolotl-ai-cloud/axolotl).