mirror of
https://github.com/vinta/awesome-python.git
synced 2026-11-08 02:15:12 -06:00
remove litellm, it got hacked twic
This commit is contained in:
@@ -176,8 +176,6 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents
|
||||
- [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs.
|
||||
- [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models.
|
||||
- [deepinfer](https://github.com/deepseek-ai/DeepInfer) - DeepSeek's production inference engine for DeepSeek models, with MoE expert parallelism.
|
||||
- LLM Gateways
|
||||
- [litellm](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format.
|
||||
- Image and Video Generation
|
||||
- [diffusers](https://github.com/huggingface/diffusers) - Pre-trained diffusion models for generating and editing images, audio, and video.
|
||||
- Fine-tuning
|
||||
|
||||
@@ -12,7 +12,6 @@ How to choose:
|
||||
- Open models from the Hugging Face Hub: Transformers
|
||||
- Running open models locally: Ollama, llama-cpp-python in your own process, MLX LM on Apple silicon
|
||||
- Serving open models: vLLM or SGLang
|
||||
- One OpenAI-style API for many model providers: LiteLLM
|
||||
- Image and video generation: Diffusers
|
||||
- Fine-tuning: TRL with PEFT adapters, Unsloth for less GPU memory, Axolotl from YAML
|
||||
- Speech: faster-whisper, FunASR for Chinese, gTTS for text to speech
|
||||
@@ -49,8 +48,6 @@ The Ollama Python library talks to Ollama, which [runs models on your computer](
|
||||
|
||||
vLLM runs [batch inference offline or an OpenAI-compatible server](https://docs.vllm.ai/en/latest/getting_started/quickstart/), so apps written for the OpenAI API work unchanged. SGLang is [optimized for agentic workloads, RL rollouts, and large-scale serving](https://github.com/sgl-project/sglang).
|
||||
|
||||
LiteLLM [returns every provider's responses in the OpenAI format](https://docs.litellm.ai/docs/) and maps their errors to OpenAI's exception types, so one code path covers them all. Use its Python SDK inside your app, or run its proxy as a self-hosted gateway for your team.
|
||||
|
||||
Diffusers centers on the [`DiffusionPipeline`](https://huggingface.co/docs/diffusers/index): a few lines generate an image, video, or audio clip.
|
||||
|
||||
Start fine-tuning with LoRA or QLoRA rather than training every weight, as both [PEFT](https://huggingface.co/docs/peft/methods/overview) and [Unsloth](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide) advise. TRL trains with methods from SFT to DPO and GRPO, and every TRL trainer takes a [`peft_config`](https://huggingface.co/docs/trl/peft_integration) for PEFT adapters. Axolotl keeps the whole pipeline, from dataset preprocessing to inference, in [one YAML file](https://github.com/axolotl-ai-cloud/axolotl).
|
||||
|
||||
Reference in New Issue
Block a user