From 71bd580429666d45bf68e2c976faae7b9a7b6e5b Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:23:29 +0800 Subject: [PATCH 01/23] refactor: split LLM Inference and Serving into Local LLM Inference and LLM Serving Local runners (ollama, llama-cpp-python, mlx-lm) and production serving engines (vllm, sglang) answer different needs, and one use case could not hold both at the cap. The old page redirects to LLM Serving. Co-Authored-By: Claude --- README.md | 5 +++-- website/data/redirects.json | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 73a63a73..0ad0d4c6 100644 --- a/README.md +++ b/README.md @@ -168,10 +168,11 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents - Pre-trained Models - [transformers](https://github.com/huggingface/transformers) - The model-definition framework for pretrained models in text, computer vision, audio, video, and multimodal tasks, for inference and training. - [sentence-transformers](https://github.com/huggingface/sentence-transformers) - Compute embeddings, run retrieval, and rerank results with pretrained and fine-tuned models. -- LLM Inference and Serving +- Local LLM Inference + - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. +- LLM Serving - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. - [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models. - - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. - LLM Gateways - [litellm](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format. - Image and Video Generation diff --git a/website/data/redirects.json b/website/data/redirects.json index 51e5593b..d5dc47fd 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -1,5 +1,6 @@ { "/categories/": "/", + "/categories/ai-and-agents/llm-inference-and-serving/": "/categories/ai-and-agents/llm-serving/", "/categories/ai-and-agents/pre-trained-models-and-inference/": "/categories/ai-and-agents/pre-trained-models/", "/categories/cli-development/cli-development/": "/categories/cli-development/", "/categories/cli-tools/productivity-tools/": "/categories/cli-tools/", From 532f05e4f61b38df23a9df7ee63e3c04a47c9729 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:09 +0800 Subject: [PATCH 02/23] feat: add ollama to AI and Agents > Local LLM Inference Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 0ad0d4c6..9855be08 100644 --- a/README.md +++ b/README.md @@ -169,6 +169,7 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents - [transformers](https://github.com/huggingface/transformers) - The model-definition framework for pretrained models in text, computer vision, audio, video, and multimodal tasks, for inference and training. - [sentence-transformers](https://github.com/huggingface/sentence-transformers) - Compute embeddings, run retrieval, and rerank results with pretrained and fine-tuned models. - Local LLM Inference + - [ollama](https://github.com/ollama/ollama-python) - Python client for Ollama, which downloads and runs open-weight LLMs locally. - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. - LLM Serving - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. From 72469f9d7db0856edbef8cd2074712e9d92e1bbe Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:14 +0800 Subject: [PATCH 03/23] feat: add llama-cpp-python to AI and Agents > Local LLM Inference Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 9855be08..065d7b0b 100644 --- a/README.md +++ b/README.md @@ -170,6 +170,7 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents - [sentence-transformers](https://github.com/huggingface/sentence-transformers) - Compute embeddings, run retrieval, and rerank results with pretrained and fine-tuned models. - Local LLM Inference - [ollama](https://github.com/ollama/ollama-python) - Python client for Ollama, which downloads and runs open-weight LLMs locally. + - [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) - Python bindings for llama.cpp, for running GGUF models locally. - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. - LLM Serving - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. From 349e0d1aeb9e9fed196f4d19c406eff0e8542899 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:19 +0800 Subject: [PATCH 04/23] feat: add imbalanced-learn to Machine Learning > General Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 065d7b0b..40c14308 100644 --- a/README.md +++ b/README.md @@ -210,6 +210,7 @@ _Libraries for Machine Learning. Also see [awesome-machine-learning](https://git - General - [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning with extensive documentation and community support. + - [imbalanced-learn](https://github.com/scikit-learn-contrib/imbalanced-learn) - Resampling techniques for datasets with imbalanced classes, compatible with scikit-learn. - [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for causal and probabilistic reasoning with graphical models. - [feature-engine](https://github.com/feature-engine/feature_engine) - sklearn compatible API with the widest toolset for feature engineering and selection. - Gradient Boosting From 369f75e85208c19cd8a9fb72c651cb77fbfec55c Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:24 +0800 Subject: [PATCH 05/23] feat: add optuna to Machine Learning > Hyperparameter Optimization Hyperparameter optimization had no use case, and optuna is its obvious choice. Co-Authored-By: Claude --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 40c14308..d2343136 100644 --- a/README.md +++ b/README.md @@ -222,6 +222,8 @@ _Libraries for Machine Learning. Also see [awesome-machine-learning](https://git - [statsforecast](https://github.com/Nixtla/statsforecast) - Fast statistical forecasting models such as ARIMA, ETS, and Theta, compiled with numba. - [sktime](https://github.com/sktime/sktime) - A unified scikit-learn-style framework for forecasting and other time-series learning tasks. - [timesfm](https://github.com/google-research/timesfm) - A pretrained foundation model from Google Research for time-series forecasting, with non-commercial default weights. +- Hyperparameter Optimization + - [optuna](https://github.com/optuna/optuna) - A hyperparameter optimization framework with define-by-run search spaces and pruning of unpromising trials. ### Natural Language Processing From 05d9798280561ade5548e30f0a8d17010162a4c0 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:28 +0800 Subject: [PATCH 06/23] feat: add hyperopt to Machine Learning > Hyperparameter Optimization Co-Authored-By: Claude --- README.md | 1 + docs/audit-logs.md | 1 + 2 files changed, 2 insertions(+) diff --git a/README.md b/README.md index d2343136..3203e6b2 100644 --- a/README.md +++ b/README.md @@ -224,6 +224,7 @@ _Libraries for Machine Learning. Also see [awesome-machine-learning](https://git - [timesfm](https://github.com/google-research/timesfm) - A pretrained foundation model from Google Research for time-series forecasting, with non-commercial default weights. - Hyperparameter Optimization - [optuna](https://github.com/optuna/optuna) - A hyperparameter optimization framework with define-by-run search spaces and pruning of unpromising trials. + - [hyperopt](https://github.com/hyperopt/hyperopt) - Distributed asynchronous hyperparameter optimization with Tree-structured Parzen Estimators. ### Natural Language Processing diff --git a/docs/audit-logs.md b/docs/audit-logs.md index 94d295c3..c61a681c 100644 --- a/docs/audit-logs.md +++ b/docs/audit-logs.md @@ -40,6 +40,7 @@ Entries admitted as challengers without adoption-trajectory evidence, by maintai - seleniumbase -- admitted 2026-08-16 (PR #3284) as a Browser Automation challenger. It wraps selenium, the successor trajectory in the use case belongs to playwright, and its distinct value is its stealth/UC mode. Admitted because that stealth value counts toward Browser Automation. - semantica -- kept 2026-10-01 in the AI and Agents audit as a Data Layer challenger at 16,582 downloads/month on ClickPy, with no adoption-trajectory evidence found. Kept by maintainer decision. - jev-ultrafast -- kept 2026-09-24 as a Web Scraping Frameworks challenger, recorded 2026-10-02. Admitted while the repo was under a month old and not on PyPI, so it has no download signal and no adoption-trajectory evidence; its README calls it an MVP and it needs a hosted TypeSafe API key. Kept by maintainer decision. +- hyperopt -- admitted 2026-10-16 as a Hyperparameter Optimization challenger at 1,793,872 downloads/month on ClickPy. Its 0.3.0 release came after a 4.5-year gap and it has no adoption-trajectory evidence. Admitted by maintainer decision. ### Cap Exceptions From c5bb17ad8bee016e6e8cb50a6d75d577951b85df Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:33 +0800 Subject: [PATCH 07/23] feat: add OpenCC to Natural Language Processing > Chinese Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3203e6b2..259c3622 100644 --- a/README.md +++ b/README.md @@ -238,6 +238,7 @@ _Libraries for working with human languages._ - Chinese - [jieba](https://github.com/fxsjy/jieba) - The most popular Chinese text segmentation library. - [pypinyin](https://github.com/mozillazg/python-pinyin) - Convert Chinese hanzi (漢字) to pinyin (拼音). + - [OpenCC](https://github.com/BYVoid/OpenCC) - Conversion between Traditional and Simplified Chinese, with regional variants and phrase-level rules. - [pangu.py](https://github.com/vinta/pangu.py) - Paranoid text spacing. ### Computer Vision From 7e1a9e72dda50db81bf42b0f0258fa25bf79b1a0 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:38 +0800 Subject: [PATCH 08/23] feat: add pgvector to Database Drivers > PostgreSQL Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 259c3622..07104f0c 100644 --- a/README.md +++ b/README.md @@ -449,6 +449,7 @@ _Libraries for connecting and operating databases._ - PostgreSQL - [awesome-postgres](https://github.com/dhamaniasad/awesome-postgres) - [psycopg](https://github.com/psycopg/psycopg) - A PostgreSQL adapter for Python, the successor to psycopg2. - [asyncpg](https://github.com/MagicStack/asyncpg) - A fast PostgreSQL Database Client Library for Python/asyncio. + - [pgvector](https://github.com/pgvector/pgvector-python) - pgvector support for Python, adding vector types and similarity search to psycopg, asyncpg, SQLAlchemy, and Django. - SQLite - [awesome-sqlite](https://github.com/planetopendata/awesome-sqlite) - [sqlite3](https://docs.python.org/3/library/sqlite3.html) - (Python standard library) SQLite interface compliant with DB-API 2.0. - [sqlite-utils](https://github.com/simonw/sqlite-utils) - Python CLI utility and library for manipulating SQLite databases. From 4941ace9d3ea463273c7ac25c8681825048dc44a Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:43 +0800 Subject: [PATCH 09/23] feat: add apsw to Database Drivers > SQLite Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 07104f0c..07da9e58 100644 --- a/README.md +++ b/README.md @@ -452,6 +452,7 @@ _Libraries for connecting and operating databases._ - [pgvector](https://github.com/pgvector/pgvector-python) - pgvector support for Python, adding vector types and similarity search to psycopg, asyncpg, SQLAlchemy, and Django. - SQLite - [awesome-sqlite](https://github.com/planetopendata/awesome-sqlite) - [sqlite3](https://docs.python.org/3/library/sqlite3.html) - (Python standard library) SQLite interface compliant with DB-API 2.0. + - [apsw](https://github.com/rogerbinns/apsw) - Another Python SQLite Wrapper, exposing the complete SQLite C API. - [sqlite-utils](https://github.com/simonw/sqlite-utils) - Python CLI utility and library for manipulating SQLite databases. - ClickHouse - [clickhouse-connect](https://github.com/ClickHouse/clickhouse-connect) - The official ClickHouse client, with SQLAlchemy and Superset connectors. From 99c7e7805086ee8e68ceec91bed23cf124bbc529 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:48 +0800 Subject: [PATCH 10/23] feat: add faiss-cpu to Database > Vector Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 07da9e58..8d978b5f 100644 --- a/README.md +++ b/README.md @@ -474,6 +474,7 @@ _In-process databases usable directly from Python._ - [duckdb](https://github.com/duckdb/duckdb) - An in-process SQL OLAP database management system; optimized for analytics and fast queries, similar to SQLite but for analytical workloads. - [chdb](https://github.com/chdb-io/chdb) - In-process OLAP SQL engine with the full ClickHouse dialect, zero-copy pandas/Arrow interop, and federation to remote ClickHouse clusters via `remoteSecure()`. - Vector + - [faiss-cpu](https://github.com/facebookresearch/faiss) - A library for efficient similarity search and clustering of dense vectors. - [lancedb](https://github.com/lancedb/lancedb) - A developer-friendly embedded retrieval database for multimodal AI. - [chromadb](https://github.com/chroma-core/chroma) - An open-source embedding database for building AI applications with embeddings and semantic search. - [zvec](https://github.com/alibaba/zvec) - A lightweight, in-process vector database that embeds directly into applications. From 90a809d398cd644174376893413f88ce4f811a86 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:53 +0800 Subject: [PATCH 11/23] audit: remove dogpile.cache from Caching Removed: dogpile.cache (about 1.9M downloads/month on ClickPy, 2026-10-16, down from 2.67M a year earlier): falling demand, and its slot goes to django-redis in the next commit. Co-Authored-By: Claude --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 8d978b5f..88592e82 100644 --- a/README.md +++ b/README.md @@ -489,7 +489,6 @@ _Libraries for caching data._ - [diskcache](https://github.com/grantjenks/python-diskcache) - SQLite and file backed cache backend, compatible with Django. - [hishel](https://github.com/karpetrosyan/hishel) - RFC 9111 compliant HTTP caching for clients like httpx and requests and servers like FastAPI, with sync and async support. - [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. -- [dogpile.cache](https://github.com/sqlalchemy/dogpile.cache) - dogpile.cache is a next generation replacement for Beaker made by the same authors. ### Search From f82dbec298544302a0e96e79a7bbdfe85f7df1e3 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:25:58 +0800 Subject: [PATCH 12/23] feat: add django-redis to Caching Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 88592e82..692d3218 100644 --- a/README.md +++ b/README.md @@ -487,6 +487,7 @@ _Libraries for caching data._ - [cachetools](https://github.com/tkem/cachetools) - Extensible memoizing collections and decorators. - [diskcache](https://github.com/grantjenks/python-diskcache) - SQLite and file backed cache backend, compatible with Django. +- [django-redis](https://github.com/jazzband/django-redis) - Full-featured Redis cache backend for Django. - [hishel](https://github.com/karpetrosyan/hishel) - RFC 9111 compliant HTTP caching for clients like httpx and requests and servers like FastAPI, with sync and async support. - [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. From 187b25bb1d83cb1d566d0286e5bf3d618f0ce54c Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:03 +0800 Subject: [PATCH 13/23] audit: remove jev-ultrafast from Web Scraping > Frameworks Removed: jev-ultrafast (no PyPI download signal; not published on PyPI): its README calls it an MVP and it needs a hosted API key, so the challenger slot goes to scrapling in the next commit. This overrides the Challenger Exception recorded 2026-10-02. Co-Authored-By: Claude --- README.md | 1 - docs/audit-logs.md | 1 - website/data/pypi_name_overrides.json | 1 - 3 files changed, 3 deletions(-) diff --git a/README.md b/README.md index 692d3218..3db5071b 100644 --- a/README.md +++ b/README.md @@ -407,7 +407,6 @@ _Libraries to automate web scraping and extract web content._ - [scrapy](https://github.com/scrapy/scrapy) - A fast high-level web crawling and scraping framework. - [crawl4ai](https://github.com/unclecode/crawl4ai) - An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents. - [stagehand](https://github.com/browserbase/stagehand) - A fast and token-efficient browser automation SDK to extract data and perform self-healing actions on web pages. - - [jev-ultrafast](https://github.com/browser-use/jev-ultrafast) - A fast browser agent that picks actions from an indexed table of page elements through TypeSafe's hosted Jev API, using a small LLM only to type text. - Content Extraction - [markdownify](https://github.com/matthewwithanm/python-markdownify) - Convert HTML to Markdown, with customizable tag handling. - [feedparser](https://github.com/kurtmckee/feedparser) - Universal feed parser. diff --git a/docs/audit-logs.md b/docs/audit-logs.md index c61a681c..5aad5c2f 100644 --- a/docs/audit-logs.md +++ b/docs/audit-logs.md @@ -39,7 +39,6 @@ Entries admitted as challengers without adoption-trajectory evidence, by maintai - seleniumbase -- admitted 2026-08-16 (PR #3284) as a Browser Automation challenger. It wraps selenium, the successor trajectory in the use case belongs to playwright, and its distinct value is its stealth/UC mode. Admitted because that stealth value counts toward Browser Automation. - semantica -- kept 2026-10-01 in the AI and Agents audit as a Data Layer challenger at 16,582 downloads/month on ClickPy, with no adoption-trajectory evidence found. Kept by maintainer decision. -- jev-ultrafast -- kept 2026-09-24 as a Web Scraping Frameworks challenger, recorded 2026-10-02. Admitted while the repo was under a month old and not on PyPI, so it has no download signal and no adoption-trajectory evidence; its README calls it an MVP and it needs a hosted TypeSafe API key. Kept by maintainer decision. - hyperopt -- admitted 2026-10-16 as a Hyperparameter Optimization challenger at 1,793,872 downloads/month on ClickPy. Its 0.3.0 release came after a 4.5-year gap and it has no adoption-trajectory evidence. Admitted by maintainer decision. ### Cap Exceptions diff --git a/website/data/pypi_name_overrides.json b/website/data/pypi_name_overrides.json index daf57097..66775d6d 100644 --- a/website/data/pypi_name_overrides.json +++ b/website/data/pypi_name_overrides.json @@ -16,7 +16,6 @@ "google-cloud-python": { "package": null, "reason": "monorepo umbrella; ships as many google-cloud-* packages", "badge": "Multiple on PyPI" }, "graphify": { "package": "graphifyy", "reason": null }, "httpx-url": { "package": null, "reason": "class in the httpx package, not a package" }, - "jev-ultrafast": { "package": null, "reason": "not published on PyPI" }, "jinja": { "package": "jinja2", "reason": "PyPI jinja is Jinja1; current releases publish as jinja2" }, "mem0": { "package": "mem0ai", "reason": null }, "micropython": { "package": null, "reason": "interpreter distributed as firmware builds, not pip" }, From 6084417daf3d6ddd17549b7742d659987a17a4c3 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:08 +0800 Subject: [PATCH 14/23] feat: add scrapling to Web Scraping > Frameworks Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3db5071b..cf81ead3 100644 --- a/README.md +++ b/README.md @@ -407,6 +407,7 @@ _Libraries to automate web scraping and extract web content._ - [scrapy](https://github.com/scrapy/scrapy) - A fast high-level web crawling and scraping framework. - [crawl4ai](https://github.com/unclecode/crawl4ai) - An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents. - [stagehand](https://github.com/browserbase/stagehand) - A fast and token-efficient browser automation SDK to extract data and perform self-healing actions on web pages. + - [scrapling](https://github.com/D4Vinci/Scrapling) - An adaptive scraping framework whose selectors relocate elements after page changes, with stealth fetchers and spiders. - Content Extraction - [markdownify](https://github.com/matthewwithanm/python-markdownify) - Convert HTML to Markdown, with customizable tag handling. - [feedparser](https://github.com/kurtmckee/feedparser) - Universal feed parser. From c9095fc74bd723bf4725173d3323f2adfb21861e Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:13 +0800 Subject: [PATCH 15/23] feat: add ast-grep-cli to Code Analysis > Refactoring Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index cf81ead3..634d47f3 100644 --- a/README.md +++ b/README.md @@ -654,6 +654,7 @@ _Tools of static analysis, linters and code quality checkers. Also see [awesome- - [bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code. - Refactoring - [rope](https://github.com/python-rope/rope) - Rope is a python refactoring library. + - [ast-grep-cli](https://github.com/ast-grep/ast-grep) - A CLI tool for structural code search, linting, and rewriting based on syntax trees. - Type Checkers - [awesome-python-typing](https://github.com/typeddjango/awesome-python-typing) - [mypy](https://github.com/python/mypy) - A static type checker for Python. - [ty](https://github.com/astral-sh/ty) - An extremely fast Python type checker and language server. From d37b45d03767ac89b6a7dd9034e9b307b8dbc64d Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:18 +0800 Subject: [PATCH 16/23] feat: add django-silk to Debugging Tools > Others Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 634d47f3..e168b864 100644 --- a/README.md +++ b/README.md @@ -712,6 +712,7 @@ _Libraries for debugging code._ - [scalene](https://github.com/plasma-umass/scalene) - A high-performance, high-precision CPU, GPU, and memory profiler for Python. - Others - [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) - Display various debug information for Django. + - [django-silk](https://github.com/jazzband/django-silk) - Live profiling and inspection of HTTP requests and database queries for Django. - [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call. - [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to flask. From 9a0040257bbbca70285540016e9a6d41241214e2 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:23 +0800 Subject: [PATCH 17/23] feat: add mise to Environment Management Co-Authored-By: Claude --- README.md | 1 + website/data/pypi_name_overrides.json | 1 + 2 files changed, 2 insertions(+) diff --git a/README.md b/README.md index e168b864..5cea146f 100644 --- a/README.md +++ b/README.md @@ -1082,6 +1082,7 @@ _Libraries for Python version and virtual environment management._ - [virtualenv](https://github.com/pypa/virtualenv) - A tool to create isolated Python environments. - [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. - [pyenv](https://github.com/pyenv/pyenv) - Simple Python version management. +- [mise](https://github.com/jdx/mise) - A polyglot tool version manager that installs Python versions and reads .python-version files. ### Package Management diff --git a/website/data/pypi_name_overrides.json b/website/data/pypi_name_overrides.json index 66775d6d..184197e7 100644 --- a/website/data/pypi_name_overrides.json +++ b/website/data/pypi_name_overrides.json @@ -20,6 +20,7 @@ "mem0": { "package": "mem0ai", "reason": null }, "micropython": { "package": null, "reason": "interpreter distributed as firmware builds, not pip" }, "mimetypes": { "package": null, "reason": "stdlib module" }, + "mise": { "package": null, "reason": "not published on PyPI; PyPI mise is an unrelated package" }, "odoo": { "package": null, "reason": "self-hosted suite installed from odoo.com, not pip" }, "pangu-py": { "package": "pangu", "reason": null }, "playwright-python": { "package": "playwright", "reason": null }, From 3e6e5c1932e670132b3303142ed5380da920c192 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:28 +0800 Subject: [PATCH 18/23] feat: add pypandoc to File Format Processing > File Conversion Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 5cea146f..94197554 100644 --- a/README.md +++ b/README.md @@ -935,6 +935,7 @@ _Libraries for parsing and manipulating specific file formats._ - [tablib](https://github.com/jazzband/tablib) - A module for Tabular Datasets in XLS, CSV, JSON, YAML. - File Conversion - [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown. + - [pypandoc](https://github.com/JessicaTegner/pypandoc) - A thin wrapper for pandoc, the universal document converter. - [docling](https://github.com/docling-project/docling) - Library for converting documents into structured data. - Excel - [openpyxl](https://openpyxl.readthedocs.io/en/stable/) - A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files. From e011e9a1f1acc8e3920bf9087eb5f1424cd532f1 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:33 +0800 Subject: [PATCH 19/23] refactor: split File Manipulation into Paths and File Watching and File Type Detection File type detection is a different job from paths and file watching, and splitting them leaves room for a challenger in each without passing the cap. Co-Authored-By: Claude --- README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 94197554..b0b8ba8a 100644 --- a/README.md +++ b/README.md @@ -963,11 +963,13 @@ _Libraries for parsing and manipulating specific file formats._ _Libraries for file manipulation._ -- [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. -- [pathlib](https://docs.python.org/3/library/pathlib.html) - (Python standard library) A cross-platform, object-oriented path library. -- [watchfiles](https://github.com/samuelcolvin/watchfiles) - Simple, modern and fast file watching and code reload in python. -- [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. -- [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. +- Paths and File Watching + - [pathlib](https://docs.python.org/3/library/pathlib.html) - (Python standard library) A cross-platform, object-oriented path library. + - [watchfiles](https://github.com/samuelcolvin/watchfiles) - Simple, modern and fast file watching and code reload in python. + - [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. +- File Type Detection + - [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. + - [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. **Media** From 3f6ed15ba93b3c2682a5f01d4e22058636ce2a29 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:39 +0800 Subject: [PATCH 20/23] feat: add magika to File Manipulation > File Type Detection Co-Authored-By: Claude --- README.md | 1 + docs/audit-logs.md | 1 + 2 files changed, 2 insertions(+) diff --git a/README.md b/README.md index b0b8ba8a..b38cccb3 100644 --- a/README.md +++ b/README.md @@ -970,6 +970,7 @@ _Libraries for file manipulation._ - File Type Detection - [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. - [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. + - [magika](https://github.com/google/magika) - Detects file content types with a deep-learning model. **Media** diff --git a/docs/audit-logs.md b/docs/audit-logs.md index 5aad5c2f..14c6713e 100644 --- a/docs/audit-logs.md +++ b/docs/audit-logs.md @@ -40,6 +40,7 @@ Entries admitted as challengers without adoption-trajectory evidence, by maintai - seleniumbase -- admitted 2026-08-16 (PR #3284) as a Browser Automation challenger. It wraps selenium, the successor trajectory in the use case belongs to playwright, and its distinct value is its stealth/UC mode. Admitted because that stealth value counts toward Browser Automation. - semantica -- kept 2026-10-01 in the AI and Agents audit as a Data Layer challenger at 16,582 downloads/month on ClickPy, with no adoption-trajectory evidence found. Kept by maintainer decision. - hyperopt -- admitted 2026-10-16 as a Hyperparameter Optimization challenger at 1,793,872 downloads/month on ClickPy. Its 0.3.0 release came after a 4.5-year gap and it has no adoption-trajectory evidence. Admitted by maintainer decision. +- magika -- admitted 2026-10-16 as a File Type Detection challenger at 16,388,380 downloads/month on ClickPy. markitdown requires it, so most of that count is transitive, and no adoption-trajectory evidence was found. Admitted by maintainer decision. ### Cap Exceptions From 7de9a00a67e0f353f12ecbfd9def9464a5a25db9 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:44 +0800 Subject: [PATCH 21/23] feat: add cloudpickle to Serialization > Object Pickling Pickling arbitrary Python objects is a different job from data formats, so Serialization splits into Data Formats and Object Pickling. Co-Authored-By: Claude --- README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index b38cccb3..3d43a265 100644 --- a/README.md +++ b/README.md @@ -504,10 +504,13 @@ _Libraries and software for indexing and performing search queries on data._ _Libraries for serializing complex data types._ -- [msgpack](https://github.com/msgpack/msgpack-python) - MessagePack serializer implementation for Python. -- [orjson](https://github.com/ijl/orjson) - Fast, correct JSON library. -- [marshmallow](https://github.com/marshmallow-code/marshmallow) - A lightweight library for converting complex objects to and from simple Python datatypes. -- [msgspec](https://github.com/msgspec/msgspec) - A fast serialization and validation library with built-in support for JSON, MessagePack, YAML, and TOML. +- Data Formats + - [msgpack](https://github.com/msgpack/msgpack-python) - MessagePack serializer implementation for Python. + - [orjson](https://github.com/ijl/orjson) - Fast, correct JSON library. + - [marshmallow](https://github.com/marshmallow-code/marshmallow) - A lightweight library for converting complex objects to and from simple Python datatypes. + - [msgspec](https://github.com/msgspec/msgspec) - A fast serialization and validation library with built-in support for JSON, MessagePack, YAML, and TOML. +- Object Pickling + - [cloudpickle](https://github.com/cloudpipe/cloudpickle) - Extends pickle to functions, lambdas, and interactively defined classes, for shipping code to remote workers. **Data & Science** From cf2667a14558f5a159f706154d5f36af4b3ca692 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:49 +0800 Subject: [PATCH 22/23] feat: add dill to Serialization > Object Pickling Co-Authored-By: Claude --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3d43a265..2cd7fa64 100644 --- a/README.md +++ b/README.md @@ -511,6 +511,7 @@ _Libraries for serializing complex data types._ - [msgspec](https://github.com/msgspec/msgspec) - A fast serialization and validation library with built-in support for JSON, MessagePack, YAML, and TOML. - Object Pickling - [cloudpickle](https://github.com/cloudpipe/cloudpickle) - Extends pickle to functions, lambdas, and interactively defined classes, for shipping code to remote workers. + - [dill](https://github.com/uqfoundation/dill) - Extends pickle to serialize almost any Python object, including closures and whole interpreter sessions. **Data & Science** From 3322bb3ce75e5788cc08242ecae54f9e5d1ef7dc Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 16 Oct 2026 17:26:54 +0800 Subject: [PATCH 23/23] feat: add betterleaks to Supply Chain Security > Secret Scanning Leaked secrets are a supply chain risk that dependency audits don't cover, so Supply Chain Security splits into Dependency Audits and Secret Scanning. Co-Authored-By: Claude --- README.md | 9 ++++++--- website/data/pypi_name_overrides.json | 1 + 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 2cd7fa64..ee4bd8ff 100644 --- a/README.md +++ b/README.md @@ -1163,10 +1163,13 @@ _Frameworks and tools for penetration testing._ ### Supply Chain Security -_Tools for auditing dependencies against known vulnerabilities._ +_Tools for auditing dependencies against known vulnerabilities and scanning for leaked secrets._ -- [pip-audit](https://github.com/pypa/pip-audit) - Audits Python environments and dependency trees for known vulnerabilities, using the Python Packaging Advisory Database or OSV. -- [uv-audit](https://github.com/astral-sh/uv) - (part of uv) uv's [dependency vulnerability scanning](https://docs.astral.sh/uv/reference/cli/#uv-audit) backed by OSV. +- Dependency Audits + - [pip-audit](https://github.com/pypa/pip-audit) - Audits Python environments and dependency trees for known vulnerabilities, using the Python Packaging Advisory Database or OSV. + - [uv-audit](https://github.com/astral-sh/uv) - (part of uv) uv's [dependency vulnerability scanning](https://docs.astral.sh/uv/reference/cli/#uv-audit) backed by OSV. +- Secret Scanning + - [betterleaks](https://github.com/betterleaks/betterleaks) - Finds leaked secrets in git history, directories, and files, from the original author of gitleaks. ### Web Security diff --git a/website/data/pypi_name_overrides.json b/website/data/pypi_name_overrides.json index 184197e7..0ee6dfa6 100644 --- a/website/data/pypi_name_overrides.json +++ b/website/data/pypi_name_overrides.json @@ -1,6 +1,7 @@ { "autobahn-python": { "package": "autobahn", "reason": null }, "azure-sdk-for-python": { "package": null, "reason": "monorepo umbrella; ships as many azure-* packages, no single package represents it", "badge": "Multiple on PyPI" }, + "betterleaks": { "package": null, "reason": "not published on PyPI" }, "cloud-init": { "package": null, "reason": "installed via distro images, not pip" }, "concurrent-futures": { "package": null, "reason": "stdlib module" }, "conda": { "package": null, "reason": "installed via Miniforge or Anaconda installers; PyPI conda is a stale 2017 upload" },