remove litellm, it got hacked twic

This commit is contained in:
Vinta Chen
2026-10-22 12:37:01 +08:00
parent f8e618ccae
commit 4c4298751f
2 changed files with 0 additions and 5 deletions
-2
View File
@@ -176,8 +176,6 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents
- [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs.
- [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models.
- [deepinfer](https://github.com/deepseek-ai/DeepInfer) - DeepSeek's production inference engine for DeepSeek models, with MoE expert parallelism.
- LLM Gateways
- [litellm](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format.
- Image and Video Generation
- [diffusers](https://github.com/huggingface/diffusers) - Pre-trained diffusion models for generating and editing images, audio, and video.
- Fine-tuning
@@ -12,7 +12,6 @@ How to choose:
- Open models from the Hugging Face Hub: Transformers
- Running open models locally: Ollama, llama-cpp-python in your own process, MLX LM on Apple silicon
- Serving open models: vLLM or SGLang
- One OpenAI-style API for many model providers: LiteLLM
- Image and video generation: Diffusers
- Fine-tuning: TRL with PEFT adapters, Unsloth for less GPU memory, Axolotl from YAML
- Speech: faster-whisper, FunASR for Chinese, gTTS for text to speech
@@ -49,8 +48,6 @@ The Ollama Python library talks to Ollama, which [runs models on your computer](
vLLM runs [batch inference offline or an OpenAI-compatible server](https://docs.vllm.ai/en/latest/getting_started/quickstart/), so apps written for the OpenAI API work unchanged. SGLang is [optimized for agentic workloads, RL rollouts, and large-scale serving](https://github.com/sgl-project/sglang).
LiteLLM [returns every provider's responses in the OpenAI format](https://docs.litellm.ai/docs/) and maps their errors to OpenAI's exception types, so one code path covers them all. Use its Python SDK inside your app, or run its proxy as a self-hosted gateway for your team.
Diffusers centers on the [`DiffusionPipeline`](https://huggingface.co/docs/diffusers/index): a few lines generate an image, video, or audio clip.
Start fine-tuning with LoRA or QLoRA rather than training every weight, as both [PEFT](https://huggingface.co/docs/peft/methods/overview) and [Unsloth](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide) advise. TRL trains with methods from SFT to DPO and GRPO, and every TRL trainer takes a [`peft_config`](https://huggingface.co/docs/trl/peft_integration) for PEFT adapters. Axolotl keeps the whole pipeline, from dataset preprocessing to inference, in [one YAML file](https://github.com/axolotl-ai-cloud/axolotl).