From d10664c5559f9bef41021d67a66395d64bb4d77f Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:51 +0800 Subject: [PATCH 01/10] style: audit Debugging Tools Others mixed three jobs. pdb-like Debugger becomes Debuggers and takes debugpy, a new Web Framework Toolbars holds django-debug-toolbar, django-silk, and flask-debugtoolbar, and icecream joins Tracing. Redirects the two old subcategory URLs. Co-Authored-By: Claude --- README.md | 8 ++++---- website/data/redirects.json | 2 ++ 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 03c32588..c03de67a 100644 --- a/README.md +++ b/README.md @@ -711,21 +711,21 @@ _Libraries for testing codebases and generating test data. Also see [awesome-pyt _Libraries for debugging code._ -- pdb-like Debugger +- Debuggers + - [debugpy](https://github.com/microsoft/debugpy) - A Debug Adapter Protocol debugger for Python, used by VS Code for remote debugging. - [ipdb](https://github.com/gotcha/ipdb) - IPython-enabled [pdb](https://docs.python.org/3/library/pdb.html). - [pudb](https://github.com/inducer/pudb) - A full-screen, console-based Python debugger. - Tracing - [viztracer](https://github.com/gaogaotiantian/viztracer) - A low-overhead tool that traces and visualizes Python code execution. + - [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call. - Profiler - [py-spy](https://github.com/benfred/py-spy) - A sampling profiler for Python programs. Written in Rust. - [memray](https://github.com/bloomberg/memray) - Tracks memory allocations in Python code, native extensions, and the interpreter itself. - [pyinstrument](https://github.com/joerick/pyinstrument) - A statistical wall-clock profiler with low overhead and readable call-tree output. - [scalene](https://github.com/plasma-umass/scalene) - A high-performance, high-precision CPU, GPU, and memory profiler for Python. -- Others - - [debugpy](https://github.com/microsoft/debugpy) - A Debug Adapter Protocol debugger for Python, used by VS Code for remote debugging. +- Web Framework Toolbars - [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) - Display various debug information for Django. - [django-silk](https://github.com/jazzband/django-silk) - Live profiling and inspection of HTTP requests and database queries for Django. - - [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call. - [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to Flask. ### Build Tools diff --git a/website/data/redirects.json b/website/data/redirects.json index d5dc47fd..f28aa83b 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -8,6 +8,8 @@ "/categories/code-analysis/code-linters/": "/categories/code-analysis/linters-and-formatters/", "/categories/code-analysis/type-annotations-generators/": "/categories/code-analysis/", "/categories/data-analysis/financial-data/": "/categories/data-ingestion-etl/financial-data/", + "/categories/debugging-tools/others/": "/categories/debugging-tools/web-framework-toolbars/", + "/categories/debugging-tools/pdb-like-debugger/": "/categories/debugging-tools/debuggers/", "/categories/distributed-computing/batch-processing/": "/categories/distributed-computing/", "/categories/gui-development/terminal/": "/categories/cli-development/tui-frameworks/", "/categories/gui-development/wrappers/": "/categories/gui-development/", From 9789452c33753dcebf6e9cd8fef4a3e3b07c45ea Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:56 +0800 Subject: [PATCH 02/10] style: audit Text Processing General held two unrelated tools. pyfiglet moves to CLI Development > Terminal Rendering as a challenger, and the rest becomes Diffs for difflib. Redirects the old General URL. Co-Authored-By: Claude --- README.md | 6 +++--- website/data/redirects.json | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index c03de67a..c37db14f 100644 --- a/README.md +++ b/README.md @@ -854,6 +854,7 @@ _Libraries for building command-line applications._ - [rich](https://github.com/Textualize/rich) - Python library for rich text and beautiful formatting in the terminal. Also provides a great `RichHandler` log handler. - [tqdm](https://github.com/tqdm/tqdm) - Fast, extensible progress bar for loops and CLI. - [colorama](https://github.com/tartley/colorama) - Cross-platform colored terminal text. + - [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python. - [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of progress bar, with real-time throughput, ETA and very cool animations. - TUI Frameworks - [textual](https://github.com/Textualize/textual) - A framework for building interactive user interfaces that run in the terminal and the browser. @@ -908,15 +909,14 @@ _Libraries for working with graphical user interface applications._ _Libraries for parsing and manipulating plain text._ +- Diffs + - [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas. - Encoding and Unicode - [charset-normalizer](https://github.com/jawah/charset_normalizer) - Universal character encoding detector, and a dependency of requests. - [chardet](https://github.com/chardet/chardet) - Python character encoding detector. - [ftfy](https://github.com/rspeer/python-ftfy) - Makes Unicode text less broken and more consistent automagically. - Fuzzy Matching - [rapidfuzz](https://github.com/rapidfuzz/RapidFuzz) - Rapid fuzzy string matching using various string metrics, with a C++ core. -- General - - [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas. - - [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python. - Internationalization - [babel](https://github.com/python-babel/babel) - An internationalization library for Python. - Parser diff --git a/website/data/redirects.json b/website/data/redirects.json index f28aa83b..99ce7054 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -14,5 +14,6 @@ "/categories/gui-development/terminal/": "/categories/cli-development/tui-frameworks/", "/categories/gui-development/wrappers/": "/categories/gui-development/", "/categories/http-clients/clients/": "/categories/http-clients/", + "/categories/text-processing/general/": "/categories/text-processing/diffs/", "/categories/web-servers/rpc/": "/categories/web-apis/rpc/" } From 7e7713f1b19b43d197b5339ea582355750c14cb3 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:56 +0800 Subject: [PATCH 03/10] style: audit File Format Processing General held two unrelated tools. tablib moves to Data Formats, and General becomes Binary Formats for pyelftools. Redirects the old General URL. Co-Authored-By: Claude --- README.md | 4 ++-- website/data/redirects.json | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c37db14f..0ccffd8b 100644 --- a/README.md +++ b/README.md @@ -945,9 +945,8 @@ _Libraries for working with HTML and XML._ _Libraries for parsing and manipulating specific file formats._ -- General +- Binary Formats - [pyelftools](https://github.com/eliben/pyelftools) - Parsing and analyzing ELF files and DWARF debugging information. - - [tablib](https://github.com/jazzband/tablib) - A module for tabular datasets in XLS, CSV, JSON, YAML. - File Conversion - [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown. - [pypandoc](https://github.com/JessicaTegner/pypandoc) - A thin wrapper for pandoc, the universal document converter. @@ -973,6 +972,7 @@ _Libraries for parsing and manipulating specific file formats._ - Data Formats - [tomllib](https://docs.python.org/3/library/tomllib.html) - (Python standard library) Parse TOML files. - [pyyaml](https://github.com/yaml/pyyaml) - A full-featured YAML framework for Python. + - [tablib](https://github.com/jazzband/tablib) - A module for tabular datasets in XLS, CSV, JSON, YAML. ### File Manipulation diff --git a/website/data/redirects.json b/website/data/redirects.json index 99ce7054..d426f162 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -11,6 +11,7 @@ "/categories/debugging-tools/others/": "/categories/debugging-tools/web-framework-toolbars/", "/categories/debugging-tools/pdb-like-debugger/": "/categories/debugging-tools/debuggers/", "/categories/distributed-computing/batch-processing/": "/categories/distributed-computing/", + "/categories/file-format-processing/general/": "/categories/file-format-processing/binary-formats/", "/categories/gui-development/terminal/": "/categories/cli-development/tui-frameworks/", "/categories/gui-development/wrappers/": "/categories/gui-development/", "/categories/http-clients/clients/": "/categories/http-clients/", From 5710d8cf9b82b127ba2c43345de8f71b4b393001 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:56 +0800 Subject: [PATCH 04/10] style: audit CLI Development General becomes Argument Parsing for argparse, click, typer, and cyclopts, and prompt_toolkit moves to TUI Frameworks after textual. Redirects the old General URL. Co-Authored-By: Claude --- README.md | 4 ++-- website/data/redirects.json | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0ccffd8b..a84c92d1 100644 --- a/README.md +++ b/README.md @@ -844,11 +844,10 @@ _Tools and libraries for packet manipulation and network device automation._ _Libraries for building command-line applications._ -- General +- Argument Parsing - [argparse](https://docs.python.org/3/library/argparse.html) - (Python standard library) Command-line option and argument parsing. - [click](https://github.com/pallets/click) - A package for creating beautiful command-line interfaces in a composable way. - [typer](https://github.com/fastapi/typer) - Modern CLI framework that uses Python type hints. Built on Click. - - [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. - [cyclopts](https://github.com/BrianPugh/cyclopts) - Build CLIs from type-hinted functions, with docstring-based help. - Terminal Rendering - [rich](https://github.com/Textualize/rich) - Python library for rich text and beautiful formatting in the terminal. Also provides a great `RichHandler` log handler. @@ -858,6 +857,7 @@ _Libraries for building command-line applications._ - [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of progress bar, with real-time throughput, ETA and very cool animations. - TUI Frameworks - [textual](https://github.com/Textualize/textual) - A framework for building interactive user interfaces that run in the terminal and the browser. + - [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. - [urwid](https://github.com/urwid/urwid) - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc. - [asciimatics](https://github.com/peterbrittain/asciimatics) - A package to create full-screen text UIs (from interactive forms to ASCII animations). diff --git a/website/data/redirects.json b/website/data/redirects.json index d426f162..fbbcac4d 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -3,6 +3,7 @@ "/categories/ai-and-agents/llm-inference-and-serving/": "/categories/ai-and-agents/llm-serving/", "/categories/ai-and-agents/pre-trained-models-and-inference/": "/categories/ai-and-agents/pre-trained-models/", "/categories/cli-development/cli-development/": "/categories/cli-development/", + "/categories/cli-development/general/": "/categories/cli-development/argument-parsing/", "/categories/cli-tools/productivity-tools/": "/categories/cli-tools/", "/categories/code-analysis/code-analysis/": "/categories/code-analysis/", "/categories/code-analysis/code-linters/": "/categories/code-analysis/linters-and-formatters/", From d4257c44b545c7c14265cc756dff24de1df86655 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:56 +0800 Subject: [PATCH 05/10] style: audit DevOps Tools Other, holding only borgbackup, becomes Backup. Redirects the old URL. Co-Authored-By: Claude --- README.md | 2 +- website/data/redirects.json | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a84c92d1..05a91c28 100644 --- a/README.md +++ b/README.md @@ -776,7 +776,7 @@ _Software and libraries for DevOps._ - [supervisor](https://github.com/Supervisor/supervisor) - Supervisor process control system for UNIX. - [flower](https://github.com/mher/flower) - A real-time monitor and web admin for Celery task queues. - [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python. -- Other +- Backup - [borgbackup](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption. ### Distributed Computing diff --git a/website/data/redirects.json b/website/data/redirects.json index fbbcac4d..79890abe 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -11,6 +11,7 @@ "/categories/data-analysis/financial-data/": "/categories/data-ingestion-etl/financial-data/", "/categories/debugging-tools/others/": "/categories/debugging-tools/web-framework-toolbars/", "/categories/debugging-tools/pdb-like-debugger/": "/categories/debugging-tools/debuggers/", + "/categories/devops-tools/other/": "/categories/devops-tools/backup/", "/categories/distributed-computing/batch-processing/": "/categories/distributed-computing/", "/categories/file-format-processing/general/": "/categories/file-format-processing/binary-formats/", "/categories/gui-development/terminal/": "/categories/cli-development/tui-frameworks/", From 4c0698394697a8dba14e522afe7b8ff0a58fc7ff Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:34:56 +0800 Subject: [PATCH 06/10] style: audit Data Visualization Specialized becomes Graphs and Maps for graphviz, cartopy, and graphify. Redirects the old URL. Co-Authored-By: Claude --- README.md | 2 +- website/data/redirects.json | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 05a91c28..816d3df6 100644 --- a/README.md +++ b/README.md @@ -561,7 +561,7 @@ _Libraries for visualizing data. Also see [awesome-javascript](https://github.co - [seaborn](https://github.com/mwaskom/seaborn) - Statistical data visualization using Matplotlib. - [altair](https://github.com/vega/altair) - Declarative statistical visualization library for Python. - [bokeh](https://github.com/bokeh/bokeh) - Interactive web plotting for Python. -- Specialized +- Graphs and Maps - [graphviz](https://github.com/xflr6/graphviz) - Simple Python interface for creating and rendering Graphviz graphs. - [cartopy](https://github.com/SciTools/cartopy) - A cartographic Python library with matplotlib support. - [graphify](https://github.com/Graphify-Labs/graphify) - Turns code, SQL schemas, docs, papers, images, or videos into a queryable knowledge graph. diff --git a/website/data/redirects.json b/website/data/redirects.json index 79890abe..42746e2d 100644 --- a/website/data/redirects.json +++ b/website/data/redirects.json @@ -9,6 +9,7 @@ "/categories/code-analysis/code-linters/": "/categories/code-analysis/linters-and-formatters/", "/categories/code-analysis/type-annotations-generators/": "/categories/code-analysis/", "/categories/data-analysis/financial-data/": "/categories/data-ingestion-etl/financial-data/", + "/categories/data-visualization/specialized/": "/categories/data-visualization/graphs-and-maps/", "/categories/debugging-tools/others/": "/categories/debugging-tools/web-framework-toolbars/", "/categories/debugging-tools/pdb-like-debugger/": "/categories/debugging-tools/debuggers/", "/categories/devops-tools/other/": "/categories/devops-tools/backup/", From e7ac42296ad6a59d940fc3c38c39acd744a81a63 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:37:16 +0800 Subject: [PATCH 07/10] docs: add debugpy and follow new subcategories in Debugging Tools intro Co-Authored-By: Claude --- website/data/category_intros/debugging-tools.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/website/data/category_intros/debugging-tools.md b/website/data/category_intros/debugging-tools.md index 47bbd0ca..7664ad78 100644 --- a/website/data/category_intros/debugging-tools.md +++ b/website/data/category_intros/debugging-tools.md @@ -4,24 +4,29 @@ How to choose: - pdb with tab completion and syntax highlighting: ipdb - Finding where a running program spends its time, without changing it: py-spy +- Debugging from your editor, or attaching to a running or remote process: debugpy - A full-screen debugger in your terminal: PuDB - A timeline of every function call: VizTracer +- Print debugging that labels each value: IceCream - Memory use and leaks, native extensions included: Memray - A readable call tree of where your script spends wall-clock time: pyinstrument - CPU, GPU, and memory use, line by line: Scalene - SQL queries and request data on Django pages: Django Debug Toolbar - Every Django request and its SQL queries, recorded to inspect later: Silk -- Print debugging that labels each value: IceCream - The same toolbar on Flask pages: Flask-DebugToolbar ipdb gives you IPython's debugger, with [tab completion, syntax highlighting, and better tracebacks](https://github.com/gotcha/ipdb), behind the same interface as pdb. Drop into it with `import ipdb; ipdb.set_trace()`, or call `ipdb.pm()` for post-mortem debugging after an exception. py-spy is a sampling profiler that shows what a program spends its time on [without restarting it or modifying its code](https://github.com/benfred/py-spy). It runs outside the profiled process, which makes it safe to use on production code. `py-spy record` writes a [flame graph](https://github.com/benfred/py-spy#record) of a process ID or a command you hand it, and `py-spy dump` prints [the call stack where a program is hung](https://github.com/benfred/py-spy#dump). +debugpy implements the [Debug Adapter Protocol](https://github.com/microsoft/debugpy), so an editor that speaks it, like VS Code, drives the debugging session. Start your script with [`python -m debugpy --listen 5678 --wait-for-client myfile.py`](https://github.com/microsoft/debugpy#debugging-a-script-file) and attach from your editor, or [attach to a process that's already running](https://github.com/microsoft/debugpy#attaching-to-a-running-process-by-id) with `--pid`. + PuDB keeps the source, the stack, breakpoints, and variables [all visible at once](https://documen.tician.de/pudb/) in one full-screen terminal view. Start it from your code with `from pudb import set_trace; set_trace()`, or [run a whole script](https://documen.tician.de/pudb/starting.html#starting-the-debugger) with `python -m pudb my-script.py`. VizTracer records every function's entry and exit time across a whole run, which [helps catch sporadic performance issues](https://viztracer.readthedocs.io/en/latest/). Run your script [with `viztracer my_script.py`](https://viztracer.readthedocs.io/en/latest/basic_usage.html#command-line) instead of `python`, then open the `result.json` it writes with `vizviewer`. +IceCream's `ic()` [prints each expression along with its value](https://github.com/gruns/icecream). With no arguments, it prints [the file, line, and function it runs in](https://github.com/gruns/icecream#inspect-execution). Since `ic()` [returns its arguments](https://github.com/gruns/icecream#return-value), you can wrap an expression in existing code without rewriting the line. + Memray tracks allocations [in Python code, native extension modules, and the interpreter itself](https://bloomberg.github.io/memray/overview.html), to find memory leaks and the code behind high memory use. Profiling [takes two steps](https://bloomberg.github.io/memray/getting_started.html): `memray run` saves the allocations to a file, then `memray flamegraph` builds a report from it. pyinstrument measures wall-clock time, so time spent [downloading data, reading files, and talking to databases](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#wall-clock-time-not-cpu-time) counts too. Type [`pyinstrument script.py`](https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-python-script) instead of `python script.py`, or wrap a block in `with pyinstrument.profile():`. @@ -32,8 +37,6 @@ Django Debug Toolbar adds [panels](https://django-debug-toolbar.readthedocs.io/e Silk [intercepts and stores HTTP requests and database queries](https://github.com/jazzband/django-silk), then shows them in its own UI: each request's time, query count, headers, and bodies. [Its setup](https://github.com/jazzband/django-silk#installation) adds its middleware and app to your settings, includes `silk.urls` in your URLconf, and runs `migrate`. The UI then lives at `/silk/`. -IceCream's `ic()` [prints each expression along with its value](https://github.com/gruns/icecream). With no arguments, it prints [the file, line, and function it runs in](https://github.com/gruns/icecream#inspect-execution). Since `ic()` [returns its arguments](https://github.com/gruns/icecream#return-value), you can wrap an expression in existing code without rewriting the line. - Flask-DebugToolbar is [a port of Django Debug Toolbar](https://github.com/pallets-eco/flask-debugtoolbar) to Flask. Wrap your app in `DebugToolbarExtension(app)`, and the toolbar [shows up in HTML responses while debug mode is on](https://flask-debugtoolbar.readthedocs.io/en/latest/). -Both debuggers also open from the built-in `breakpoint()`: set the [`PYTHONBREAKPOINT` environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONBREAKPOINT) to `ipdb.set_trace` or `pudb.set_trace`. Among profilers, sampling ones like py-spy, pyinstrument, and Scalene [cost much less overhead than tracing ones](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#statistical-profiling-not-tracing), while a tracer like VizTracer records every call and [gives you more information](https://github.com/gaogaotiantian/viztracer#performance). +ipdb and PuDB also open from the built-in `breakpoint()`: set the [`PYTHONBREAKPOINT` environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONBREAKPOINT) to `ipdb.set_trace` or `pudb.set_trace`. Among profilers, sampling ones like py-spy, pyinstrument, and Scalene [cost much less overhead than tracing ones](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#statistical-profiling-not-tracing), while a tracer like VizTracer records every call and [gives you more information](https://github.com/gaogaotiantian/viztracer#performance). From 5df072e3c2e98f57444f438789ecb97eec9cc88d Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:37:16 +0800 Subject: [PATCH 08/10] docs: drop pyfiglet from Text Processing intro Co-Authored-By: Claude --- website/data/category_intros/text-processing.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/website/data/category_intros/text-processing.md b/website/data/category_intros/text-processing.md index efa8a35a..bb7ac257 100644 --- a/website/data/category_intros/text-processing.md +++ b/website/data/category_intros/text-processing.md @@ -6,7 +6,6 @@ How to choose: - Bytes in an unknown encoding: charset-normalizer - Fuzzy matching against a long list of strings: RapidFuzz - Diffs, or close matches with nothing to install: difflib -- ASCII-art banners: pyfiglet - Translations, plus dates and numbers in each user's locale: Babel - Syntax highlighting: Pygments - A grammar written in Python code: pyparsing, or parsy for small languages @@ -23,8 +22,6 @@ RapidFuzz [does fuzzy string matching with various string metrics](https://rapid difflib comes with Python and [compares sequences](https://docs.python.org/3/library/difflib.html), mostly lines of text, producing unified, context, and HTML diffs like the ones `diff` and `git diff` print. For a "did you mean" suggestion, [`get_close_matches()`](https://docs.python.org/3/library/difflib.html#difflib.get_close_matches) returns the best "good enough" matches for a word. -pyfiglet is [a full port of FIGlet into pure Python](https://github.com/pwaller/pyfiglet): it renders text in ASCII-art fonts, as in `pyfiglet.figlet_format("text", font="slant")`. - Babel [does two jobs](https://babel.pocoo.org/en/latest/intro.html): it builds gettext message catalogs without the GNU gettext tools for common tasks, and it formats dates, numbers, and locale names from CLDR data. Run catalogs through the `pybabel` command: [`extract` messages, `init` a catalog per language, `update` it, and `compile` it](https://babel.pocoo.org/en/latest/cmdline.html). Pygments is [a generic syntax highlighter](https://github.com/pygments/pygments) for code hosting, forums, wikis, and other apps that show source code. If you highlight code your users submit, its docs recommend you [stop the Pygments process after a short timeout](https://pygments.org/docs/security/#security-considerations). From 385cdfa48c9f7a269f297f692e885f7afa8f22df Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:37:16 +0800 Subject: [PATCH 09/10] docs: follow new subcategories in CLI Development intro Co-Authored-By: Claude --- website/data/category_intros/cli-development.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/website/data/category_intros/cli-development.md b/website/data/category_intros/cli-development.md index e8a0a153..3b4e4079 100644 --- a/website/data/category_intros/cli-development.md +++ b/website/data/category_intros/cli-development.md @@ -5,11 +5,12 @@ How to choose: - Basic command-line app with no dependencies: argparse - Nested commands, or subcommands loaded lazily: Click - Options declared once as type-hinted function parameters: Typer -- Interactive prompts and REPLs: prompt_toolkit - Help text read from your docstrings: cyclopts - Progress bar for a loop: tqdm, or alive-progress for animated bars - Colored text, tables, and logs: Rich, or colorama for ANSI colors on Windows +- ASCII-art banners: pyfiglet - Full-screen app in the terminal or a browser: Textual +- Interactive prompts and REPLs: prompt_toolkit - Terminal widgets on an event loop you already run: Urwid - Full-screen forms and ASCII animations: asciimatics @@ -19,8 +20,6 @@ Click builds a CLI from [commands declared with decorators](https://click.pallet Typer builds a CLI from your function signatures: you [declare each argument and option once](https://typer.tiangolo.com/), as a type-hinted function parameter. Write those hints with `Annotated`, which its tutorial [recommends wherever possible](https://typer.tiangolo.com/tutorial/arguments/optional/). Its docs say you get [the best results by pairing it with Rich](https://typer.tiangolo.com/tutorial/printing/): Typer structures the commands and options, and Rich displays the output. -prompt_toolkit is for interactive input. It can [replace GNU readline](https://python-prompt-toolkit.readthedocs.io/en/stable/) or build full-screen apps. For a REPL, use a [`PromptSession`](https://python-prompt-toolkit.readthedocs.io/en/stable/pages/asking_for_input.html), which keeps the history for the whole session. - cyclopts also builds a CLI from type-hinted functions, but reads the help text from your docstrings. Its docs call that [the preferred way to document a command](https://cyclopts.readthedocs.io/en/latest/commands.html), and reach for `Annotated` with `Parameter` only when you need more control. It follows Python's own argument binding rules, so a parameter can be passed [by position or as a keyword](https://cyclopts.readthedocs.io/en/latest/vs_typer/positional_or_keyword/README.html). It has its own parser, with no Click underneath. Register commands with `@app.command`, and the function to run when no command matches with [`@app.default`](https://cyclopts.readthedocs.io/en/latest/getting_started.html). tqdm adds a progress bar to any loop: [wrap the iterable in `tqdm()`](https://github.com/tqdm/tqdm). Import it from `tqdm.auto`, which picks the console bar or the Jupyter widget for you. alive-progress wraps the loop in an [`alive_bar` context manager](https://github.com/rsalmei/alive-progress) instead, with a spinner that speeds up and slows down with your throughput. @@ -29,8 +28,12 @@ Rich writes colored text, tables, Markdown, and syntax-highlighted code to the t colorama makes ANSI escape codes work on Windows and does nothing on other platforms. If that's all you need, call [`just_fix_windows_console()`](https://github.com/tartley/colorama). +pyfiglet is [a full port of FIGlet into pure Python](https://github.com/pwaller/pyfiglet): it renders text in ASCII-art fonts, as in `pyfiglet.figlet_format("text", font="slant")`. + Textual apps run in the terminal, and the `textual serve` command from textual-dev [serves them in a browser](https://textual.textualize.io/guide/devtools/). Build one by [subclassing `App`](https://textual.textualize.io/guide/app/), and style its widgets with [CSS](https://textual.textualize.io/guide/CSS/). +prompt_toolkit is for interactive input. It can [replace GNU readline](https://python-prompt-toolkit.readthedocs.io/en/stable/) or build full-screen apps. For a REPL, use a [`PromptSession`](https://python-prompt-toolkit.readthedocs.io/en/stable/pages/asking_for_input.html), which keeps the history for the whole session. + Urwid is a [console widget construction set](https://urwid.org/manual/overview.html) rather than a finished UI library. It runs on [your choice of event loop](https://github.com/urwid/urwid): asyncio, or another one you already use. It's [licensed under the LGPL](https://github.com/urwid/urwid/blob/master/COPYING), while Textual and asciimatics use permissive licenses. asciimatics does [full-screen text UIs, from interactive forms to ASCII animations](https://github.com/peterbrittain/asciimatics): create a Screen, build a Scene from Effect objects, and let the Screen play it. From 1c89e904af1d5be9e15ed855cd4a58c8c3b29ad7 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 21:37:16 +0800 Subject: [PATCH 10/10] docs: move Tablib next to Data Formats in File Format Processing intro Co-Authored-By: Claude --- website/data/category_intros/file-format-processing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/data/category_intros/file-format-processing.md b/website/data/category_intros/file-format-processing.md index 659160f2..6549b493 100644 --- a/website/data/category_intros/file-format-processing.md +++ b/website/data/category_intros/file-format-processing.md @@ -4,7 +4,6 @@ How to choose: - Merging, splitting, cropping, or encrypting PDFs: pypdf - ELF binaries and DWARF debug info: pyelftools -- A table you import or export as CSV, JSON, YAML, or Excel: Tablib - Documents turned into Markdown for an LLM: MarkItDown, or Docling for complex layouts and scans - Documents converted between Markdown, Word, LaTeX, and HTML: pypandoc - Excel: openpyxl to edit a workbook, XlsxWriter to create one with charts and formatting @@ -15,13 +14,12 @@ How to choose: - PDFs from HTML and CSS: WeasyPrint - Markdown to HTML: markdown-it-py for CommonMark, Python-Markdown for its extensions, Mistune for speed - Config files: tomllib for TOML, PyYAML for YAML +- A table you import or export as CSV, JSON, YAML, or Excel: Tablib pypdf is a [pure-Python PDF library](https://pypdf.readthedocs.io/en/stable/) for splitting, merging, cropping, and transforming pages, and it adds passwords too. To merge files, [`append()` each one to a `PdfWriter`](https://pypdf.readthedocs.io/en/stable/user/merging-pdfs.html) and write it out. With [no C dependency](https://pypdf.readthedocs.io/en/stable/meta/comparisons.html), it's easy to install, but it doesn't generate new PDFs. It reads text with `page.extract_text()`, while a scanned page needs [OCR software](https://pypdf.readthedocs.io/en/stable/user/extract-text.html). pyelftools [parses ELF files and DWARF debug info](https://github.com/eliben/pyelftools) in pure Python. Pass an open file to [`ELFFile`](https://github.com/eliben/pyelftools/blob/main/doc/user-guide.md), its main entry point, and stay on the high-level API: its guide says to use the low-level structs at your own peril. -Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format. - MarkItDown converts PDF, Word, Excel, PowerPoint, HTML, and more into Markdown [for LLMs and text analysis](https://github.com/microsoft/markitdown). Its README says it may not be the best option for high-fidelity conversions that people read. Don't pass untrusted input straight to it, and when you only read local files, [call `convert_local()` instead of `convert()`](https://github.com/microsoft/markitdown#security-considerations), which also fetches URLs. Docling is for documents where layout matters: it [understands page layout, reading order, and tables, and runs OCR](https://docling-project.github.io/docling/) on scanned PDFs. It converts each file into a `DoclingDocument`, which you export to Markdown. @@ -54,4 +52,6 @@ tomllib comes with Python and parses TOML, but doesn't write it. With untrusted PyYAML parses and emits YAML. Load it with `yaml.safe_load()`, since [`yaml.load` can call any Python function](https://pyyaml.org/wiki/PyYAMLDocumentation) when the data comes from an untrusted source. +Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format. + Markdown your users write can carry HTML, so give each parser its safe setting. With markdown-it-py, use the [`js-default` preset](https://markdown-it-py.readthedocs.io/en/latest/security.html), which its docs strongly recommend for user content in web apps. Python-Markdown [doesn't sanitize its output](https://python-markdown.github.io/sanitization/), so run a sanitizer over the HTML. With Mistune, build your parser with [`mistune.create_markdown()`](https://mistune.lepture.com/en/latest/guide.html), which escapes HTML, instead of `mistune.html()`, which doesn't.