Merge branch 'audit/catch-all-subcategories'

This commit is contained in:
Vinta Chen
2026-11-08 21:39:21 +08:00
6 changed files with 36 additions and 26 deletions
+13 -13
View File
@@ -561,7 +561,7 @@ _Libraries for visualizing data. Also see [awesome-javascript](https://github.co
- [seaborn](https://github.com/mwaskom/seaborn) - Statistical data visualization using Matplotlib.
- [altair](https://github.com/vega/altair) - Declarative statistical visualization library for Python.
- [bokeh](https://github.com/bokeh/bokeh) - Interactive web plotting for Python.
- Specialized
- Graphs and Maps
- [graphviz](https://github.com/xflr6/graphviz) - Simple Python interface for creating and rendering Graphviz graphs.
- [cartopy](https://github.com/SciTools/cartopy) - A cartographic Python library with matplotlib support.
- [graphify](https://github.com/Graphify-Labs/graphify) - Turns code, SQL schemas, docs, papers, images, or videos into a queryable knowledge graph.
@@ -711,21 +711,21 @@ _Libraries for testing codebases and generating test data. Also see [awesome-pyt
_Libraries for debugging code._
- pdb-like Debugger
- Debuggers
- [debugpy](https://github.com/microsoft/debugpy) - A Debug Adapter Protocol debugger for Python, used by VS Code for remote debugging.
- [ipdb](https://github.com/gotcha/ipdb) - IPython-enabled [pdb](https://docs.python.org/3/library/pdb.html).
- [pudb](https://github.com/inducer/pudb) - A full-screen, console-based Python debugger.
- Tracing
- [viztracer](https://github.com/gaogaotiantian/viztracer) - A low-overhead tool that traces and visualizes Python code execution.
- [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call.
- Profiler
- [py-spy](https://github.com/benfred/py-spy) - A sampling profiler for Python programs. Written in Rust.
- [memray](https://github.com/bloomberg/memray) - Tracks memory allocations in Python code, native extensions, and the interpreter itself.
- [pyinstrument](https://github.com/joerick/pyinstrument) - A statistical wall-clock profiler with low overhead and readable call-tree output.
- [scalene](https://github.com/plasma-umass/scalene) - A high-performance, high-precision CPU, GPU, and memory profiler for Python.
- Others
- [debugpy](https://github.com/microsoft/debugpy) - A Debug Adapter Protocol debugger for Python, used by VS Code for remote debugging.
- Web Framework Toolbars
- [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) - Display various debug information for Django.
- [django-silk](https://github.com/jazzband/django-silk) - Live profiling and inspection of HTTP requests and database queries for Django.
- [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call.
- [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to Flask.
### Build Tools
@@ -776,7 +776,7 @@ _Software and libraries for DevOps._
- [supervisor](https://github.com/Supervisor/supervisor) - Supervisor process control system for UNIX.
- [flower](https://github.com/mher/flower) - A real-time monitor and web admin for Celery task queues.
- [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python.
- Other
- Backup
- [borgbackup](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption.
### Distributed Computing
@@ -844,19 +844,20 @@ _Tools and libraries for packet manipulation and network device automation._
_Libraries for building command-line applications._
- General
- Argument Parsing
- [argparse](https://docs.python.org/3/library/argparse.html) - (Python standard library) Command-line option and argument parsing.
- [click](https://github.com/pallets/click) - A package for creating beautiful command-line interfaces in a composable way.
- [typer](https://github.com/fastapi/typer) - Modern CLI framework that uses Python type hints. Built on Click.
- [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines.
- [cyclopts](https://github.com/BrianPugh/cyclopts) - Build CLIs from type-hinted functions, with docstring-based help.
- Terminal Rendering
- [rich](https://github.com/Textualize/rich) - Python library for rich text and beautiful formatting in the terminal. Also provides a great `RichHandler` log handler.
- [tqdm](https://github.com/tqdm/tqdm) - Fast, extensible progress bar for loops and CLI.
- [colorama](https://github.com/tartley/colorama) - Cross-platform colored terminal text.
- [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python.
- [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of progress bar, with real-time throughput, ETA and very cool animations.
- TUI Frameworks
- [textual](https://github.com/Textualize/textual) - A framework for building interactive user interfaces that run in the terminal and the browser.
- [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines.
- [urwid](https://github.com/urwid/urwid) - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc.
- [asciimatics](https://github.com/peterbrittain/asciimatics) - A package to create full-screen text UIs (from interactive forms to ASCII animations).
@@ -908,15 +909,14 @@ _Libraries for working with graphical user interface applications._
_Libraries for parsing and manipulating plain text._
- Diffs
- [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas.
- Encoding and Unicode
- [charset-normalizer](https://github.com/jawah/charset_normalizer) - Universal character encoding detector, and a dependency of requests.
- [chardet](https://github.com/chardet/chardet) - Python character encoding detector.
- [ftfy](https://github.com/rspeer/python-ftfy) - Makes Unicode text less broken and more consistent automagically.
- Fuzzy Matching
- [rapidfuzz](https://github.com/rapidfuzz/RapidFuzz) - Rapid fuzzy string matching using various string metrics, with a C++ core.
- General
- [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas.
- [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python.
- Internationalization
- [babel](https://github.com/python-babel/babel) - An internationalization library for Python.
- Parser
@@ -945,9 +945,8 @@ _Libraries for working with HTML and XML._
_Libraries for parsing and manipulating specific file formats._
- General
- Binary Formats
- [pyelftools](https://github.com/eliben/pyelftools) - Parsing and analyzing ELF files and DWARF debugging information.
- [tablib](https://github.com/jazzband/tablib) - A module for tabular datasets in XLS, CSV, JSON, YAML.
- File Conversion
- [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown.
- [pypandoc](https://github.com/JessicaTegner/pypandoc) - A thin wrapper for pandoc, the universal document converter.
@@ -973,6 +972,7 @@ _Libraries for parsing and manipulating specific file formats._
- Data Formats
- [tomllib](https://docs.python.org/3/library/tomllib.html) - (Python standard library) Parse TOML files.
- [pyyaml](https://github.com/yaml/pyyaml) - A full-featured YAML framework for Python.
- [tablib](https://github.com/jazzband/tablib) - A module for tabular datasets in XLS, CSV, JSON, YAML.
### File Manipulation
@@ -5,11 +5,12 @@ How to choose:
- Basic command-line app with no dependencies: argparse
- Nested commands, or subcommands loaded lazily: Click
- Options declared once as type-hinted function parameters: Typer
- Interactive prompts and REPLs: prompt_toolkit
- Help text read from your docstrings: cyclopts
- Progress bar for a loop: tqdm, or alive-progress for animated bars
- Colored text, tables, and logs: Rich, or colorama for ANSI colors on Windows
- ASCII-art banners: pyfiglet
- Full-screen app in the terminal or a browser: Textual
- Interactive prompts and REPLs: prompt_toolkit
- Terminal widgets on an event loop you already run: Urwid
- Full-screen forms and ASCII animations: asciimatics
@@ -19,8 +20,6 @@ Click builds a CLI from [commands declared with decorators](https://click.pallet
Typer builds a CLI from your function signatures: you [declare each argument and option once](https://typer.tiangolo.com/), as a type-hinted function parameter. Write those hints with `Annotated`, which its tutorial [recommends wherever possible](https://typer.tiangolo.com/tutorial/arguments/optional/). Its docs say you get [the best results by pairing it with Rich](https://typer.tiangolo.com/tutorial/printing/): Typer structures the commands and options, and Rich displays the output.
prompt_toolkit is for interactive input. It can [replace GNU readline](https://python-prompt-toolkit.readthedocs.io/en/stable/) or build full-screen apps. For a REPL, use a [`PromptSession`](https://python-prompt-toolkit.readthedocs.io/en/stable/pages/asking_for_input.html), which keeps the history for the whole session.
cyclopts also builds a CLI from type-hinted functions, but reads the help text from your docstrings. Its docs call that [the preferred way to document a command](https://cyclopts.readthedocs.io/en/latest/commands.html), and reach for `Annotated` with `Parameter` only when you need more control. It follows Python's own argument binding rules, so a parameter can be passed [by position or as a keyword](https://cyclopts.readthedocs.io/en/latest/vs_typer/positional_or_keyword/README.html). It has its own parser, with no Click underneath. Register commands with `@app.command`, and the function to run when no command matches with [`@app.default`](https://cyclopts.readthedocs.io/en/latest/getting_started.html).
tqdm adds a progress bar to any loop: [wrap the iterable in `tqdm()`](https://github.com/tqdm/tqdm). Import it from `tqdm.auto`, which picks the console bar or the Jupyter widget for you. alive-progress wraps the loop in an [`alive_bar` context manager](https://github.com/rsalmei/alive-progress) instead, with a spinner that speeds up and slows down with your throughput.
@@ -29,8 +28,12 @@ Rich writes colored text, tables, Markdown, and syntax-highlighted code to the t
colorama makes ANSI escape codes work on Windows and does nothing on other platforms. If that's all you need, call [`just_fix_windows_console()`](https://github.com/tartley/colorama).
pyfiglet is [a full port of FIGlet into pure Python](https://github.com/pwaller/pyfiglet): it renders text in ASCII-art fonts, as in `pyfiglet.figlet_format("text", font="slant")`.
Textual apps run in the terminal, and the `textual serve` command from textual-dev [serves them in a browser](https://textual.textualize.io/guide/devtools/). Build one by [subclassing `App`](https://textual.textualize.io/guide/app/), and style its widgets with [CSS](https://textual.textualize.io/guide/CSS/).
prompt_toolkit is for interactive input. It can [replace GNU readline](https://python-prompt-toolkit.readthedocs.io/en/stable/) or build full-screen apps. For a REPL, use a [`PromptSession`](https://python-prompt-toolkit.readthedocs.io/en/stable/pages/asking_for_input.html), which keeps the history for the whole session.
Urwid is a [console widget construction set](https://urwid.org/manual/overview.html) rather than a finished UI library. It runs on [your choice of event loop](https://github.com/urwid/urwid): asyncio, or another one you already use. It's [licensed under the LGPL](https://github.com/urwid/urwid/blob/master/COPYING), while Textual and asciimatics use permissive licenses.
asciimatics does [full-screen text UIs, from interactive forms to ASCII animations](https://github.com/peterbrittain/asciimatics): create a Screen, build a Scene from Effect objects, and let the Screen play it.
@@ -4,24 +4,29 @@ How to choose:
- pdb with tab completion and syntax highlighting: ipdb
- Finding where a running program spends its time, without changing it: py-spy
- Debugging from your editor, or attaching to a running or remote process: debugpy
- A full-screen debugger in your terminal: PuDB
- A timeline of every function call: VizTracer
- Print debugging that labels each value: IceCream
- Memory use and leaks, native extensions included: Memray
- A readable call tree of where your script spends wall-clock time: pyinstrument
- CPU, GPU, and memory use, line by line: Scalene
- SQL queries and request data on Django pages: Django Debug Toolbar
- Every Django request and its SQL queries, recorded to inspect later: Silk
- Print debugging that labels each value: IceCream
- The same toolbar on Flask pages: Flask-DebugToolbar
ipdb gives you IPython's debugger, with [tab completion, syntax highlighting, and better tracebacks](https://github.com/gotcha/ipdb), behind the same interface as pdb. Drop into it with `import ipdb; ipdb.set_trace()`, or call `ipdb.pm()` for post-mortem debugging after an exception.
py-spy is a sampling profiler that shows what a program spends its time on [without restarting it or modifying its code](https://github.com/benfred/py-spy). It runs outside the profiled process, which makes it safe to use on production code. `py-spy record` writes a [flame graph](https://github.com/benfred/py-spy#record) of a process ID or a command you hand it, and `py-spy dump` prints [the call stack where a program is hung](https://github.com/benfred/py-spy#dump).
debugpy implements the [Debug Adapter Protocol](https://github.com/microsoft/debugpy), so an editor that speaks it, like VS Code, drives the debugging session. Start your script with [`python -m debugpy --listen 5678 --wait-for-client myfile.py`](https://github.com/microsoft/debugpy#debugging-a-script-file) and attach from your editor, or [attach to a process that's already running](https://github.com/microsoft/debugpy#attaching-to-a-running-process-by-id) with `--pid`.
PuDB keeps the source, the stack, breakpoints, and variables [all visible at once](https://documen.tician.de/pudb/) in one full-screen terminal view. Start it from your code with `from pudb import set_trace; set_trace()`, or [run a whole script](https://documen.tician.de/pudb/starting.html#starting-the-debugger) with `python -m pudb my-script.py`.
VizTracer records every function's entry and exit time across a whole run, which [helps catch sporadic performance issues](https://viztracer.readthedocs.io/en/latest/). Run your script [with `viztracer my_script.py`](https://viztracer.readthedocs.io/en/latest/basic_usage.html#command-line) instead of `python`, then open the `result.json` it writes with `vizviewer`.
IceCream's `ic()` [prints each expression along with its value](https://github.com/gruns/icecream). With no arguments, it prints [the file, line, and function it runs in](https://github.com/gruns/icecream#inspect-execution). Since `ic()` [returns its arguments](https://github.com/gruns/icecream#return-value), you can wrap an expression in existing code without rewriting the line.
Memray tracks allocations [in Python code, native extension modules, and the interpreter itself](https://bloomberg.github.io/memray/overview.html), to find memory leaks and the code behind high memory use. Profiling [takes two steps](https://bloomberg.github.io/memray/getting_started.html): `memray run` saves the allocations to a file, then `memray flamegraph` builds a report from it.
pyinstrument measures wall-clock time, so time spent [downloading data, reading files, and talking to databases](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#wall-clock-time-not-cpu-time) counts too. Type [`pyinstrument script.py`](https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-python-script) instead of `python script.py`, or wrap a block in `with pyinstrument.profile():`.
@@ -32,8 +37,6 @@ Django Debug Toolbar adds [panels](https://django-debug-toolbar.readthedocs.io/e
Silk [intercepts and stores HTTP requests and database queries](https://github.com/jazzband/django-silk), then shows them in its own UI: each request's time, query count, headers, and bodies. [Its setup](https://github.com/jazzband/django-silk#installation) adds its middleware and app to your settings, includes `silk.urls` in your URLconf, and runs `migrate`. The UI then lives at `/silk/`.
IceCream's `ic()` [prints each expression along with its value](https://github.com/gruns/icecream). With no arguments, it prints [the file, line, and function it runs in](https://github.com/gruns/icecream#inspect-execution). Since `ic()` [returns its arguments](https://github.com/gruns/icecream#return-value), you can wrap an expression in existing code without rewriting the line.
Flask-DebugToolbar is [a port of Django Debug Toolbar](https://github.com/pallets-eco/flask-debugtoolbar) to Flask. Wrap your app in `DebugToolbarExtension(app)`, and the toolbar [shows up in HTML responses while debug mode is on](https://flask-debugtoolbar.readthedocs.io/en/latest/).
Both debuggers also open from the built-in `breakpoint()`: set the [`PYTHONBREAKPOINT` environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONBREAKPOINT) to `ipdb.set_trace` or `pudb.set_trace`. Among profilers, sampling ones like py-spy, pyinstrument, and Scalene [cost much less overhead than tracing ones](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#statistical-profiling-not-tracing), while a tracer like VizTracer records every call and [gives you more information](https://github.com/gaogaotiantian/viztracer#performance).
ipdb and PuDB also open from the built-in `breakpoint()`: set the [`PYTHONBREAKPOINT` environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONBREAKPOINT) to `ipdb.set_trace` or `pudb.set_trace`. Among profilers, sampling ones like py-spy, pyinstrument, and Scalene [cost much less overhead than tracing ones](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#statistical-profiling-not-tracing), while a tracer like VizTracer records every call and [gives you more information](https://github.com/gaogaotiantian/viztracer#performance).
@@ -4,7 +4,6 @@ How to choose:
- Merging, splitting, cropping, or encrypting PDFs: pypdf
- ELF binaries and DWARF debug info: pyelftools
- A table you import or export as CSV, JSON, YAML, or Excel: Tablib
- Documents turned into Markdown for an LLM: MarkItDown, or Docling for complex layouts and scans
- Documents converted between Markdown, Word, LaTeX, and HTML: pypandoc
- Excel: openpyxl to edit a workbook, XlsxWriter to create one with charts and formatting
@@ -15,13 +14,12 @@ How to choose:
- PDFs from HTML and CSS: WeasyPrint
- Markdown to HTML: markdown-it-py for CommonMark, Python-Markdown for its extensions, Mistune for speed
- Config files: tomllib for TOML, PyYAML for YAML
- A table you import or export as CSV, JSON, YAML, or Excel: Tablib
pypdf is a [pure-Python PDF library](https://pypdf.readthedocs.io/en/stable/) for splitting, merging, cropping, and transforming pages, and it adds passwords too. To merge files, [`append()` each one to a `PdfWriter`](https://pypdf.readthedocs.io/en/stable/user/merging-pdfs.html) and write it out. With [no C dependency](https://pypdf.readthedocs.io/en/stable/meta/comparisons.html), it's easy to install, but it doesn't generate new PDFs. It reads text with `page.extract_text()`, while a scanned page needs [OCR software](https://pypdf.readthedocs.io/en/stable/user/extract-text.html).
pyelftools [parses ELF files and DWARF debug info](https://github.com/eliben/pyelftools) in pure Python. Pass an open file to [`ELFFile`](https://github.com/eliben/pyelftools/blob/main/doc/user-guide.md), its main entry point, and stay on the high-level API: its guide says to use the low-level structs at your own peril.
Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format.
MarkItDown converts PDF, Word, Excel, PowerPoint, HTML, and more into Markdown [for LLMs and text analysis](https://github.com/microsoft/markitdown). Its README says it may not be the best option for high-fidelity conversions that people read. Don't pass untrusted input straight to it, and when you only read local files, [call `convert_local()` instead of `convert()`](https://github.com/microsoft/markitdown#security-considerations), which also fetches URLs.
Docling is for documents where layout matters: it [understands page layout, reading order, and tables, and runs OCR](https://docling-project.github.io/docling/) on scanned PDFs. It converts each file into a `DoclingDocument`, which you export to Markdown.
@@ -54,4 +52,6 @@ tomllib comes with Python and parses TOML, but doesn't write it. With untrusted
PyYAML parses and emits YAML. Load it with `yaml.safe_load()`, since [`yaml.load` can call any Python function](https://pyyaml.org/wiki/PyYAMLDocumentation) when the data comes from an untrusted source.
Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format.
Markdown your users write can carry HTML, so give each parser its safe setting. With markdown-it-py, use the [`js-default` preset](https://markdown-it-py.readthedocs.io/en/latest/security.html), which its docs strongly recommend for user content in web apps. Python-Markdown [doesn't sanitize its output](https://python-markdown.github.io/sanitization/), so run a sanitizer over the HTML. With Mistune, build your parser with [`mistune.create_markdown()`](https://mistune.lepture.com/en/latest/guide.html), which escapes HTML, instead of `mistune.html()`, which doesn't.
@@ -6,7 +6,6 @@ How to choose:
- Bytes in an unknown encoding: charset-normalizer
- Fuzzy matching against a long list of strings: RapidFuzz
- Diffs, or close matches with nothing to install: difflib
- ASCII-art banners: pyfiglet
- Translations, plus dates and numbers in each user's locale: Babel
- Syntax highlighting: Pygments
- A grammar written in Python code: pyparsing, or parsy for small languages
@@ -23,8 +22,6 @@ RapidFuzz [does fuzzy string matching with various string metrics](https://rapid
difflib comes with Python and [compares sequences](https://docs.python.org/3/library/difflib.html), mostly lines of text, producing unified, context, and HTML diffs like the ones `diff` and `git diff` print. For a "did you mean" suggestion, [`get_close_matches()`](https://docs.python.org/3/library/difflib.html#difflib.get_close_matches) returns the best "good enough" matches for a word.
pyfiglet is [a full port of FIGlet into pure Python](https://github.com/pwaller/pyfiglet): it renders text in ASCII-art fonts, as in `pyfiglet.figlet_format("text", font="slant")`.
Babel [does two jobs](https://babel.pocoo.org/en/latest/intro.html): it builds gettext message catalogs without the GNU gettext tools for common tasks, and it formats dates, numbers, and locale names from CLDR data. Run catalogs through the `pybabel` command: [`extract` messages, `init` a catalog per language, `update` it, and `compile` it](https://babel.pocoo.org/en/latest/cmdline.html).
Pygments is [a generic syntax highlighter](https://github.com/pygments/pygments) for code hosting, forums, wikis, and other apps that show source code. If you highlight code your users submit, its docs recommend you [stop the Pygments process after a short timeout](https://pygments.org/docs/security/#security-considerations).
+7
View File
@@ -3,14 +3,21 @@
"/categories/ai-and-agents/llm-inference-and-serving/": "/categories/ai-and-agents/llm-serving/",
"/categories/ai-and-agents/pre-trained-models-and-inference/": "/categories/ai-and-agents/pre-trained-models/",
"/categories/cli-development/cli-development/": "/categories/cli-development/",
"/categories/cli-development/general/": "/categories/cli-development/argument-parsing/",
"/categories/cli-tools/productivity-tools/": "/categories/cli-tools/",
"/categories/code-analysis/code-analysis/": "/categories/code-analysis/",
"/categories/code-analysis/code-linters/": "/categories/code-analysis/linters-and-formatters/",
"/categories/code-analysis/type-annotations-generators/": "/categories/code-analysis/",
"/categories/data-analysis/financial-data/": "/categories/data-ingestion-etl/financial-data/",
"/categories/data-visualization/specialized/": "/categories/data-visualization/graphs-and-maps/",
"/categories/debugging-tools/others/": "/categories/debugging-tools/web-framework-toolbars/",
"/categories/debugging-tools/pdb-like-debugger/": "/categories/debugging-tools/debuggers/",
"/categories/devops-tools/other/": "/categories/devops-tools/backup/",
"/categories/distributed-computing/batch-processing/": "/categories/distributed-computing/",
"/categories/file-format-processing/general/": "/categories/file-format-processing/binary-formats/",
"/categories/gui-development/terminal/": "/categories/cli-development/tui-frameworks/",
"/categories/gui-development/wrappers/": "/categories/gui-development/",
"/categories/http-clients/clients/": "/categories/http-clients/",
"/categories/text-processing/general/": "/categories/text-processing/diffs/",
"/categories/web-servers/rpc/": "/categories/web-apis/rpc/"
}