mirror of
https://github.com/vinta/awesome-python.git
synced 2026-11-08 02:15:12 -06:00
Merge branch 'vinta:master' into 2Master
This commit is contained in:
Goldensloth420
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: audit-the-list
|
||||
description: Audit README.md sections against the shortlist rules — re-verify every entry's verdict with live data, promote or demote challengers, restructure oversized use cases, prune, and evaluate proposed additions. Fire when the maintainer asks to audit, sweep, prune, re-check, or maintain sections, or asks whether an entry (or a proposed one) deserves its slot.
|
||||
argument-hint: [all or specific sections]
|
||||
---
|
||||
|
||||
# Audit the list
|
||||
|
||||
An Audit re-runs the admission test over live sections of README.md: every entry re-verified against current data, tiers reassessed, structure reconsidered, then the maintainer adjudicates and the result is committed. The rules live in CONTRIBUTING.md (Admission, Evidence, Entry Ordering) and CONTEXT.md (vocabulary) — this skill is the process that applies them, not a second copy of them. Entry changes land only on the maintainer's explicit go.
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Resolve the scope from the arguments. Named sections mean exactly those, whether or not they were audited before (a re-run is how a past verdict gets rechecked). `all` is ambiguous — `AskUserQuestion` whether it means every section or only never-audited ones; prior audits are recorded in git history (`git log --oneline --grep="sweep\|audit"`). With no arguments, AskUserQuestion listing the never-audited Thematic Groups. Batch the work one Thematic Group per sitting. Done when the section list is settled.
|
||||
|
||||
## 2. Evidence
|
||||
|
||||
Fetch live evidence for every entry in scope before judging anything (CLAUDE.md verification rule):
|
||||
|
||||
- **Downloads/month**: `cd website && uv run python fetch_pypi_downloads_via_clickpy.py` — free keyless ClickPy sweep of the full README, sole writer of `data/pypi_downloads.tsv` (rewritten from scratch each run; header row, every row stamped with its `fetched_at` date). A cache whose `fetched_at` is within the last 7 days is current enough for verdicts — skip the sweep; older than that, re-run it (costs ~1s). Cross-checks print to stdout, take explicit names, and never touch the cache: `fetch_pypi_downloads_via_bigquery.py <name> ...` (canonical source, maintainer's own GCP account, `--dry-run` first — the docstring carries the cost constraints; full-README sweeps exceed the free tier, keep name lists small), `fetch_pypi_downloads_via_pepy.py <name> ...` (needs `PEPY_TECH_API_KEY` in repo-root `.env`, throttled to 5 requests/minute), or `https://pypistats.org/api/packages/{name}/recent` paced 8s or slower. pypistats excludes mirror/CI traffic; ClickPy, BigQuery, and pepy include it — never mix sources within one comparison.
|
||||
- **Repo state**: archived flag, last push, created date, stars, description — `gh api repos/{owner}/{repo}`, GitLab API for GitLab-hosted projects.
|
||||
- **PyPI metadata** (`https://pypi.org/pypi/{name}/json`) wherever a name might not be the canonical package — ownership collisions and wrong display names surface here. Wrong-package cache rows are common: the sweep looks up README display names, so when the display name differs from the canonical package, the row silently measures an unrelated squatter or a dead predecessor. When a famous entry shows a count that looks off (too small, or absurdly round), verify identity before citing it, and fetch the canonical package's count via pepy.
|
||||
- **Renaming an entry can silently expose it to the sweep, and no automatic check catches that.** The sweep only queries names matching `PYPI_NAME_RE`, so a display name carrying a space or other non-PyPI shape is skipped outright. Rename it into a PyPI-shaped one and it starts being queried: `uv audit` became `uv-audit`, and PyPI `uv-audit` is an unrelated third-party tool by rocshers, not Astral's subcommand. After any rename, fetch `https://pypi.org/pypi/{new-name}/json` and confirm the package is the project the entry links to; when it is not, add a null entry to `website/data/pypi_name_overrides.json` with the reason. Only one case is guarded automatically: `test_bundled_entries_are_never_queried_on_pypi` requires a null override for every `(part of X)` entry. Two broader checks were measured against the full list and rejected — do not re-propose either. Requiring a package's PyPI metadata to link back to the entry's GitHub repo misses this case completely, since `uv-audit` declares no repo URL at all, as do 26 legitimate entries including numba, selenium and pyglet; it also flags 10 benign org moves and binding splits (gensim, msgspec, pendulum, instructor, duckdb). Flagging entries whose display name differs from their repo name returns 46 hits, every one legitimate, and `uv-build` sits among them with the shape identical to `uv-audit` despite being a real Astral package.
|
||||
|
||||
Done when every entry in scope has downloads (or a stated no-signal reason), repo state, and a confirmed PyPI name.
|
||||
|
||||
## 3. Verdicts
|
||||
|
||||
Draft a verdict and reason for every entry, restructure before cap: decide Splits, mints, and re-homes first — cutting before restructuring destroys entries — then apply the cap tier by tier, including promotions and demotions between obvious choice and challenger. Ground every reason in the fetched evidence; label anything unverifiable as a judgment call. Where fresh evidence contradicts the standing verdict, say so in the reason instead of silently keeping the seed.
|
||||
|
||||
## 4. Review and go
|
||||
|
||||
Run the `preview-verdicts` skill: it generates the interactive review page and defines how the maintainer's feedback JSON comes back and gets processed. Their verdicts are final. Before touching README.md, surface what the feedback implies but does not decide — cap overflows, homeless entries after a Split, tier for a tierless flip — as named assumptions or questions, then get an explicit go.
|
||||
|
||||
## 5. Execute
|
||||
|
||||
One commit per section: body lists each removal with its reason and downloads figure, naming the figure's source and fetch date so a later audit can tell whether a trend compares like with like (ClickPy counts mirror traffic, pypistats does not); restructures, tier moves, and reorders ride the same commit. A restructure that renames or dissolves a category or subcategory adds `"old path": "new path"` to `website/data/redirects.json` in that commit, so the old URL keeps its search ranking; the build fails when a redirect target no longer exists, so a later rename also updates the entries pointing at the renamed path. Format-only outcomes (no removals) are a single style commit. `make test` before every commit. After the last add or rename, re-run the ClickPy sweep, since new names have no cache row and the build would count them without downloads, then `make build`. Generic commit helpers tend to split a section audit into structural and per-subcategory commits — if that happens, squash back to one commit per section. Done when the tree is clean, tests passed before each commit, and the build count reconciles with the adjudicated changes.
|
||||
|
||||
## 6. Record
|
||||
|
||||
A conclusion that outlives the sitting goes into the repo before the audit ends: admission or evidence rules into CONTRIBUTING.md, repo process and environment facts into CLAUDE.md and AGENTS.md (kept in sync), vocabulary into CONTEXT.md. Done when nothing the next audit needs is stranded in conversation.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: preview-verdicts
|
||||
description: Generate the interactive keep/drop/add verdict preview (HTML page with per-row feedback controls) whenever a prune sweep, batch entry edit, or restructure needs maintainer review before touching README.md — and process the feedback JSON the maintainer pastes back.
|
||||
---
|
||||
|
||||
# Verdict preview
|
||||
|
||||
Maintainer review happens through an interactive HTML page: one row per entry with your seeded verdict and reason, Keep/Drop buttons (Add/Drop on proposed new entries) and a reason field for the maintainer, and a **Copy feedback** button that exports only changed or commented rows as JSON. Generate the page, wait for the pasted JSON, then apply it. Entry changes land in README.md only after the review — and only on an explicit go.
|
||||
|
||||
## Generate the preview
|
||||
|
||||
1. Build the `DATA` array. A group is `[section, subcategory, rows]`; a row is `[entry, url, downloads, verdict, reason]`.
|
||||
- `subcategory` may carry a note after ` — ` (rendered muted): use it for proposed splits, re-homes, or anything the maintainer should weigh for the whole group.
|
||||
- `downloads` is PyPI last-month as a comma-formatted string; use `—` when no signal exists (e.g. agent skill packs), `stdlib` for standard-library modules, `fetch failed` when the lookup failed. State the fetch date in the sub-header.
|
||||
- `verdict` is `keep`, `drop`, or `add`, seeded from the current adjudication or dry-run. `add` marks a proposed new entry; its row gets Add/Drop buttons, where Drop means not added. Show a replacement as two rows: the old entry `drop`, the new entry `add`.
|
||||
- `reason` is plain language the maintainer reads cold — no invented shorthand. When fresh evidence contradicts the seeded verdict (a big download count on a drop, a dead repo on a keep), say so in that row's reason instead of silently changing the seed.
|
||||
2. Copy `template.html` (sibling of this file) and replace the placeholders: `__TITLE__` (page title), `__SUB__` (sub-header: scope, seed provenance, fetch date, and the standing instruction to flip/comment then Copy feedback), `__KEY__` (localStorage key), `__DATA__` (the array). `__KEY__` must be unique per review — slug plus date, e.g. `awesome-python-science-2026-09-01` — because saved state under a reused key bleeds a previous review's flips into rows with the same section and entry name.
|
||||
3. Write the page to `tmp/awesome-python-<slug>-preview.html` in the repo root (git-ignored; create the directory if needed), `open` it, and tell the maintainer the path and the return path: flip or comment rows (they highlight yellow), press **Copy feedback**, paste the JSON into the chat. Done when the page is open and the return path is stated.
|
||||
|
||||
## Process the pasted feedback
|
||||
|
||||
Each JSON row is `{section, subcategory, entry, my_verdict, your_verdict, reason}`. On a row seeded `add`, `your_verdict: "drop"` means do not add it. The maintainer's verdict is final — apply it, never re-argue it. An empty reason means the verdict stands unexplained; that is enough.
|
||||
|
||||
Before executing, surface anything the flips imply that the maintainer has not decided: a use case pushed past its cap, an entry left homeless by a proposed split, a request that is already satisfied (a no-op). Ask, then execute on their go. Done when every pasted row is either applied or surfaced back — none silently dropped.
|
||||
@@ -0,0 +1,163 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>__TITLE__</title>
|
||||
<style>
|
||||
:root {
|
||||
--ink: #1c1917; --paper: #faf9f7; --muted: #78716c; --line: #e7e5e4;
|
||||
--keep: #1d4ed8; --keep-bg: #eff4ff; --add: #6d28d9; --add-bg: #f3efff; --drop: #b42318; --drop-bg: #fdf1ef;
|
||||
--changed: #fffbeb; --changed-line: #f5c518;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; background: var(--paper); color: var(--ink); font: 16px/1.5 -apple-system, "SF Pro Text", "Helvetica Neue", Arial, sans-serif; }
|
||||
.wrap { padding: 2rem 2.5rem 6rem; }
|
||||
h1 { font-size: 1.7rem; margin: 0 0 .3rem; }
|
||||
.sub { color: var(--muted); margin: 0 0 1.6rem; font-size: 1rem; }
|
||||
.sub b { color: var(--ink); }
|
||||
.sub code { background: #f0eeec; padding: .1em .35em; border-radius: 4px; font-size: .92em; }
|
||||
h2 { font-size: 1.35rem; margin: 2.2rem 0 .2rem; border-bottom: 2px solid var(--ink); padding-bottom: .3rem; }
|
||||
h3 { font-size: 1.05rem; margin: 1.3rem 0 .4rem; color: var(--ink); }
|
||||
h3 .note { font-weight: 400; color: var(--muted); font-size: .92rem; }
|
||||
table { width: 100%; border-collapse: collapse; }
|
||||
th { text-align: left; font-size: .82rem; text-transform: uppercase; letter-spacing: .04em; color: var(--muted); padding: .35rem .6rem; border-bottom: 1px solid var(--line); }
|
||||
td { padding: .5rem .6rem; border-bottom: 1px solid var(--line); vertical-align: top; }
|
||||
tr.changed td { background: var(--changed); border-bottom-color: var(--changed-line); }
|
||||
td.name { white-space: nowrap; font-weight: 600; }
|
||||
td.name a { color: var(--ink); text-decoration: none; border-bottom: 1px solid #c8c4c0; }
|
||||
td.name a:hover { border-bottom-color: var(--ink); }
|
||||
td.dl { white-space: nowrap; font-variant-numeric: tabular-nums; color: var(--muted); text-align: right; }
|
||||
td.mine { white-space: nowrap; }
|
||||
.badge { display: inline-block; font-size: .78rem; font-weight: 700; letter-spacing: .03em; padding: .12em .5em; border-radius: 4px; }
|
||||
.badge.keep { color: var(--keep); background: var(--keep-bg); }
|
||||
.badge.drop { color: var(--drop); background: var(--drop-bg); }
|
||||
.badge.add { color: var(--add); background: var(--add-bg); }
|
||||
td.reason { color: #44403c; font-size: .95rem; max-width: 34rem; }
|
||||
td.yours { white-space: nowrap; }
|
||||
.seg { display: inline-flex; border: 1px solid var(--line); border-radius: 6px; overflow: hidden; }
|
||||
.seg button { border: 0; background: #fff; padding: .3rem .8rem; font: inherit; font-size: .88rem; cursor: pointer; color: var(--muted); }
|
||||
.seg button + button { border-left: 1px solid var(--line); }
|
||||
.seg button.on.k { background: var(--keep); color: #fff; font-weight: 700; }
|
||||
.seg button.on.a { background: var(--add); color: #fff; font-weight: 700; }
|
||||
.seg button.on.d { background: var(--drop); color: #fff; font-weight: 700; }
|
||||
td.fb input { width: 100%; min-width: 14rem; font: inherit; font-size: .92rem; padding: .3rem .5rem; border: 1px solid var(--line); border-radius: 6px; background: #fff; }
|
||||
td.fb input:focus { outline: 2px solid var(--keep); outline-offset: 0; }
|
||||
.bar { position: fixed; bottom: 0; left: 0; right: 0; background: var(--ink); color: #fafaf9; display: flex; align-items: center; gap: 1.4rem; padding: .7rem 2.5rem; font-size: .95rem; }
|
||||
.bar .counts b { font-variant-numeric: tabular-nums; }
|
||||
.bar button { font: inherit; font-weight: 700; padding: .45rem 1.1rem; border: 0; border-radius: 6px; background: #fafaf9; color: var(--ink); cursor: pointer; }
|
||||
.bar button:hover { background: #e7e5e4; }
|
||||
.bar .hint { color: #a8a29e; }
|
||||
#out { display: none; width: 100%; height: 9rem; margin-top: 1rem; font: 13px/1.4 ui-monospace, monospace; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
<h1>__TITLE__</h1>
|
||||
<p class="sub">__SUB__</p>
|
||||
<div id="app"></div>
|
||||
<textarea id="out" readonly></textarea>
|
||||
</div>
|
||||
<div class="bar">
|
||||
<span class="counts" id="counts"></span>
|
||||
<button onclick="copyFeedback()">Copy feedback</button>
|
||||
<span class="hint" id="hint">Copies only rows you changed or commented, as JSON.</span>
|
||||
</div>
|
||||
<script>
|
||||
const DATA = __DATA__;
|
||||
|
||||
const KEY = "__KEY__";
|
||||
let state = {};
|
||||
try { state = JSON.parse(localStorage.getItem(KEY) || "{}"); } catch (e) {}
|
||||
|
||||
function rowId(section, name) { return section + "::" + name; }
|
||||
|
||||
function render() {
|
||||
const app = document.getElementById("app");
|
||||
let html = "", lastSection = "";
|
||||
for (const [section, subcat, rows] of DATA) {
|
||||
if (section !== lastSection) { html += `<h2>${section}</h2>`; lastSection = section; }
|
||||
const parts = subcat.split(" — ");
|
||||
html += `<h3>${parts[0]}${parts[1] ? ` <span class="note">— ${parts[1]}</span>` : ""}</h3>`;
|
||||
html += `<table><tr><th>Entry</th><th style="text-align:right">Downloads/mo</th><th>My verdict</th><th>My reason</th><th>Yours</th><th>Your reason (optional)</th></tr>`;
|
||||
for (const [name, url, dl, mine, reason] of rows) {
|
||||
const id = rowId(section, name);
|
||||
const s = state[id] || {};
|
||||
const yours = s.verdict || mine;
|
||||
const changed = yours !== mine || (s.feedback || "").trim() !== "";
|
||||
html += `<tr id="${cssId(id)}" class="${changed ? "changed" : ""}">
|
||||
<td class="name"><a href="${url}" target="_blank" rel="noopener">${name}</a></td>
|
||||
<td class="dl">${dl}</td>
|
||||
<td class="mine"><span class="badge ${mine}">${mine.toUpperCase()}</span></td>
|
||||
<td class="reason">${reason}</td>
|
||||
<td class="yours"><span class="seg">
|
||||
${mine === "add"
|
||||
? `<button class="a ${yours === "add" ? "on" : ""}" onclick="setV('${esc(id)}','add')">Add</button>`
|
||||
: `<button class="k ${yours === "keep" ? "on" : ""}" onclick="setV('${esc(id)}','keep')">Keep</button>`}
|
||||
<button class="d ${yours === "drop" ? "on" : ""}" onclick="setV('${esc(id)}','drop')">Drop</button>
|
||||
</span></td>
|
||||
<td class="fb"><input value="${(s.feedback || "").replace(/"/g, """)}" oninput="setF('${esc(id)}', this.value)" placeholder="why…"></td>
|
||||
</tr>`;
|
||||
}
|
||||
html += `</table>`;
|
||||
}
|
||||
app.innerHTML = html;
|
||||
updateCounts();
|
||||
}
|
||||
|
||||
function cssId(id) { return "r-" + id.replace(/[^a-zA-Z0-9]/g, "_"); }
|
||||
function esc(id) { return id.replace(/'/g, "\\'"); }
|
||||
|
||||
function setV(id, v) { state[id] = state[id] || {}; state[id].verdict = v; save(); render(); }
|
||||
function setF(id, f) {
|
||||
state[id] = state[id] || {}; state[id].feedback = f; save();
|
||||
const [section, name] = id.split("::");
|
||||
const mine = findMine(section, name);
|
||||
const yours = state[id].verdict || mine;
|
||||
document.getElementById(cssId(id)).className = (yours !== mine || f.trim() !== "") ? "changed" : "";
|
||||
updateCounts();
|
||||
}
|
||||
function findMine(section, name) {
|
||||
for (const [s, , rows] of DATA) if (s === section) for (const r of rows) if (r[0] === name) return r[3];
|
||||
}
|
||||
function save() { localStorage.setItem(KEY, JSON.stringify(state)); }
|
||||
|
||||
function collect() {
|
||||
const out = [];
|
||||
for (const [section, subcat, rows] of DATA) {
|
||||
for (const [name, , , mine] of rows) {
|
||||
const id = rowId(section, name);
|
||||
const s = state[id] || {};
|
||||
const yours = s.verdict || mine;
|
||||
const fb = (s.feedback || "").trim();
|
||||
if (yours !== mine || fb) out.push({ section, subcategory: subcat.split(" — ")[0], entry: name, my_verdict: mine, your_verdict: yours, reason: fb });
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function updateCounts() {
|
||||
let keeps = 0, adds = 0, drops = 0, changed = 0;
|
||||
for (const [section, , rows] of DATA) for (const [name, , , mine] of rows) {
|
||||
const s = state[rowId(section, name)] || {};
|
||||
const yours = s.verdict || mine;
|
||||
if (yours === "keep") keeps++; else if (yours === "add") adds++; else drops++;
|
||||
if (yours !== mine) changed++;
|
||||
}
|
||||
document.getElementById("counts").innerHTML = `<b>${keeps}</b> keep · <b>${drops}</b> drop · <b>${adds}</b> add | <b>${changed}</b> verdicts flipped`;
|
||||
}
|
||||
|
||||
function copyFeedback() {
|
||||
const fb = collect();
|
||||
const text = fb.length ? JSON.stringify(fb, null, 1) : `(no changes — all verdicts accepted as-is: ${document.title} — ${[...new Set(DATA.map(g => g[0]))].join(", ")})`;
|
||||
const hint = document.getElementById("hint");
|
||||
navigator.clipboard.writeText(text).then(
|
||||
() => { hint.textContent = `Copied ${fb.length ? fb.length + " row(s)" : "acceptance note"} — paste into the chat.`; },
|
||||
() => { const o = document.getElementById("out"); o.style.display = "block"; o.value = text; o.select(); hint.textContent = "Clipboard blocked — select & copy the box above."; }
|
||||
);
|
||||
}
|
||||
|
||||
render();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: review-prs
|
||||
description: Triage open PRs — screen from the diff, delegate the admission judgment to audit-the-list, then merge or close on GitHub. Fire when the maintainer asks to review PRs, process the PR queue, or judge whether a specific PR should be merged.
|
||||
argument-hint: [PR numbers]
|
||||
---
|
||||
|
||||
# Review PRs
|
||||
|
||||
A PR review turns the open queue into terminal states: merged, closed, or explicitly parked. The rules live in CONTRIBUTING.md (Quality Requirements, Admission, Review Process, Automatic Rejection) — this skill is the workflow that applies them, not a second copy. Every PR named in chat is a PR link, `[#<number>](https://github.com/vinta/awesome-python/pull/<number>)`, so the maintainer can open it from the terminal.
|
||||
|
||||
## 1. Fetch
|
||||
|
||||
Arguments name specific PRs; otherwise take the queue: `gh pr list --repo vinta/awesome-python --limit 10 --json number,title,author,url,body,files,mergeable,mergeStateStatus`. Fetch all diffs in parallel: `gh pr diff <number> --repo vinta/awesome-python`.
|
||||
|
||||
Sort the batch: entry additions continue; anything else (typo fixes, website changes, docs) is out of scope — report it under "needs human" and touch nothing, so it resurfaces every run until a human acts. Done when every PR is sorted and has its diff.
|
||||
|
||||
## 2. Screen
|
||||
|
||||
Apply the Automatic Rejection rules that the diff and PR metadata answer without judgment. The recently-closed-duplicate rule needs a lookup: `gh pr list --repo vinta/awesome-python --state closed --search "<project name>" --limit 10`. A screened-out PR goes straight to step 4 as a close, with the rule it broke as its reason.
|
||||
|
||||
Resolve each survivor's target use case against the current README — a diff's context lines show the base the PR was written on, which may have changed since. A merge conflict rides to the Merge arm, which absorbs it. Done when every surviving PR names its target use case.
|
||||
|
||||
## 3. Judge
|
||||
|
||||
Group survivors by target use case — PRs proposing entries for the same use case compete for the same slots, so they ride one invocation. Per group, invoke the `audit-the-list` skill with arguments in this shape: "Judge proposed entries <names, each with its PR number> for the <section — subcategory> use case. Evidence and Verdicts steps only; report the verdicts back. No preview page, no README changes, no commits." The verdict for each PR is merge or close, grounded in the fetched evidence; when the use case is at cap, a merge verdict names the entry that leaves. An entry no current use case fits is a third outcome — a structure question: carry it to Act with evidence attached, and the maintainer decides there (mint the subcategory and merge, close, or leave open). Done when every survivor holds a verdict with a reason.
|
||||
|
||||
## 4. Act
|
||||
|
||||
- **Close**: draft each closing comment — it states the reason and links CONTRIBUTING.md. Per batch of up to 4 closing PRs: print each draft in chat, its PR link as a label above a fenced block holding only the comment bytes, then AskUserQuestion over that batch — arms: close with the draft as printed, close without a comment, keep open. The chat print is the review copy; an option description only points at it. Keep a checklist of which verdicts have been asked; answers often arrive as custom text, and that text is the decision — an edited comment replaces the draft. Then `gh pr close <number> --repo vinta/awesome-python --comment "<comment>"`, or a plain close.
|
||||
- **Merge**: a clean PR merges with `gh pr merge <number> --repo vinta/awesome-python --merge`. A conflicted one merges locally: `git fetch origin pull/<number>/head`, `git merge FETCH_HEAD` with the standard `Merge pull request #<number> from <owner>/<headRef>` message, resolving the conflict by placing the entry correctly — GitHub still marks the PR merged and the contributor keeps credit. Either way, reconcile the section per CONTRIBUTING before pushing: remove the entry the verdict displaced, fix the new entry's display name and Entry Ordering position, `make test`, commit. An add-only diff that displaces is the normal case — the removal is this step's job, not the contributor's.
|
||||
Done when every verdict has been adjudicated by the maintainer and its action executed, with every posted comment byte-identical to a printed draft or to the custom text the maintainer supplied. A kept-open PR resurfaces next run — that is its point.
|
||||
|
||||
## 5. Report
|
||||
|
||||
Summary table: PR link, verdict, action taken, plus the needs-human list. Done when every fetched PR ends in exactly one state — merged with its section reconciled, closed, or left open (kept open, needs-human, or structure question pending).
|
||||
+2
-1
@@ -1 +1,2 @@
|
||||
# These are supported funding model platforms
|
||||
open_collective: awesome-python
|
||||
custom: https://awesome-python.com/sponsorship/
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
---
|
||||
name: Add new library to this list
|
||||
about: ''
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Please open a Pull Request instead.**
|
||||
@@ -1,10 +0,0 @@
|
||||
---
|
||||
name: Add XXX
|
||||
about: Add new item to the list
|
||||
title: ''
|
||||
labels: ''
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Please open a Pull Request instead.**
|
||||
@@ -1,11 +1,26 @@
|
||||
## What is this Python project?
|
||||
## Project
|
||||
|
||||
Describe features.
|
||||
[project-name](url)
|
||||
|
||||
## What's the difference between this Python project and similar ones?
|
||||
## Checklist
|
||||
|
||||
Enumerate comparisons.
|
||||
- [ ] I read [CONTRIBUTING.md](https://github.com/vinta/awesome-python/blob/master/CONTRIBUTING.md) - awesome-python is a shortlist, not a catalog
|
||||
- [ ] One project per PR
|
||||
- [ ] PR title format: `Add project-name`
|
||||
- [ ] Entry format: `- [pypi-name](https://github.com/owner/repo) - Description ending with period.`
|
||||
- [ ] Display name is the PyPI package name
|
||||
- [ ] Placed in an existing use case (new sections and subcategories are maintainer-only)
|
||||
- [ ] Meets all Quality Requirements: active, stable, documented, at least 1 month old
|
||||
|
||||
--
|
||||
## Which Tier
|
||||
|
||||
Anyone who agrees with this pull request could submit an *Approve* review to it.
|
||||
Pick one:
|
||||
|
||||
- [ ] **Obvious choice** - a tool an experienced Python developer would name when asked "what do I use for this?"
|
||||
- [ ] **Challenger** - not yet the obvious choice, but a credible successor to one. Give adoption-trajectory evidence, not popularity alone.
|
||||
|
||||
Explain:
|
||||
|
||||
## Displacement
|
||||
|
||||
If the use case is at its cap, name the entry your project replaces and argue that yours does that entry's job better. One in, one out.
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --group build --locked
|
||||
|
||||
- name: Run tests
|
||||
run: make test
|
||||
|
||||
- name: Build website
|
||||
run: make build
|
||||
@@ -0,0 +1,105 @@
|
||||
name: Deploy Website
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
schedule:
|
||||
- cron: "0 0 * * *"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
if: github.repository == 'vinta/awesome-python'
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: https://awesome-python.com/
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
# build.py reads each page's last change from git history for the sitemap's lastmod
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --group build --locked
|
||||
|
||||
- name: Run tests
|
||||
if: github.event_name == 'schedule'
|
||||
run: make test
|
||||
|
||||
- name: Get date
|
||||
id: date
|
||||
run: echo "today=$(date -u +%Y-%m-%d)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Restore star data cache
|
||||
id: cache-stars
|
||||
uses: actions/cache/restore@v6
|
||||
with:
|
||||
path: website/data/github_stars.json
|
||||
key: github-stars-${{ steps.date.outputs.today }}
|
||||
restore-keys: github-stars-
|
||||
|
||||
- name: Fetch GitHub stars
|
||||
id: fetch-stars
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: make fetch_github_stars
|
||||
|
||||
- name: Save star data cache
|
||||
if: steps.fetch-stars.outcome == 'success'
|
||||
uses: actions/cache/save@v6
|
||||
with:
|
||||
path: website/data/github_stars.json
|
||||
key: github-stars-${{ steps.date.outputs.today }}
|
||||
|
||||
- name: Verify star data exists
|
||||
run: |
|
||||
if [ ! -f website/data/github_stars.json ]; then
|
||||
echo "::error::github_stars.json not found. No cache and fetch failed or was skipped."
|
||||
exit 1
|
||||
fi
|
||||
python -m json.tool website/data/github_stars.json > /dev/null
|
||||
|
||||
- name: Restore PyPI download data cache
|
||||
uses: actions/cache/restore@v6
|
||||
with:
|
||||
path: website/data/pypi_downloads.tsv
|
||||
key: pypi-downloads-${{ steps.date.outputs.today }}
|
||||
restore-keys: pypi-downloads-
|
||||
|
||||
- name: Fetch PyPI downloads
|
||||
id: fetch-downloads
|
||||
continue-on-error: true
|
||||
run: make fetch_pypi_downloads
|
||||
|
||||
- name: Save PyPI download data cache
|
||||
if: steps.fetch-downloads.outcome == 'success'
|
||||
uses: actions/cache/save@v6
|
||||
with:
|
||||
path: website/data/pypi_downloads.tsv
|
||||
key: pypi-downloads-${{ steps.date.outputs.today }}
|
||||
|
||||
- name: Build website
|
||||
run: make build
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@v5
|
||||
with:
|
||||
path: website/output/
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
uses: actions/deploy-pages@v5
|
||||
+16
-4
@@ -1,9 +1,21 @@
|
||||
# secrets
|
||||
.env*
|
||||
|
||||
# macOS
|
||||
.DS_Store
|
||||
|
||||
# python
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.py[co]
|
||||
|
||||
docs/index.md
|
||||
site/
|
||||
# website
|
||||
website/output/
|
||||
website/data/*
|
||||
!website/data/pypi_name_overrides.json
|
||||
!website/data/redirects.json
|
||||
!website/data/category_intros/
|
||||
|
||||
# PyCharm IDE
|
||||
.idea
|
||||
# agents
|
||||
.playwright-cli/
|
||||
tmp/
|
||||
|
||||
-18
@@ -1,18 +0,0 @@
|
||||
language: python
|
||||
|
||||
python:
|
||||
- "3.6"
|
||||
|
||||
script:
|
||||
- python sort.py
|
||||
- cp README.md docs/index.md
|
||||
- mkdocs build
|
||||
|
||||
deploy:
|
||||
provider: pages
|
||||
local-dir: site
|
||||
skip-cleanup: true
|
||||
keep-history: true
|
||||
github-token: $GITHUB_TOKEN
|
||||
on:
|
||||
branch: master
|
||||
@@ -0,0 +1,23 @@
|
||||
# AGENTS.md
|
||||
|
||||
An opinionated guide to the best Python frameworks, libraries, and tools.
|
||||
|
||||
[README.md](README.md) is the single source of content truth; `website/` renders it into the static site: [awesome-python.com](https://awesome-python.com/).
|
||||
|
||||
## Entry Rules
|
||||
|
||||
[CONTRIBUTING.md](CONTRIBUTING.md) holds the admission rules, quality requirements, rejection rules, entry format, and ordering. Apply it whenever adding or removing an entry — direct commits included, not only PR reviews.
|
||||
|
||||
- Every keep/drop reason must be verified against current online data at decision time — download counts, repo activity and archived status, PyPI metadata, project docs. Judging tiers — obvious choice vs challenger — also requires WebSearch evidence (adoption trajectory, community sentiment), not download counts alone. Training-data recollections are not evidence; label anything unverifiable as a judgment call.
|
||||
- Checks stay read-only: judge whether a listed project installs or works from its PyPI metadata (`requires_python`, classifiers, wheel tags) and issue tracker, never by installing or running it, since that runs untrusted code.
|
||||
- A download count is not automatically independent demand. When one listed entry depends on another, check `requires_dist` on PyPI before citing the depended-on entry's count: mkdocs-material hard-depends on mkdocs, so mkdocs' figure exceeded mkdocs-material's by only about one percent and nearly all of it was mkdocs-material pulling it in.
|
||||
- One entry per commit when adding or deleting entries. Exceptions: a prune sweep is one commit per section, its body listing each removal with its reason; format, wording, or categorization changes may be bundled. Cross-section re-homes ride the originating audit's commit (both sides of the move in one diff).
|
||||
- Resources sections are not project entries: out of audit scope, and the website never parses them.
|
||||
- Sponsor placement never influences which projects get listed — see [SPONSORSHIP.md](SPONSORSHIP.md).
|
||||
|
||||
## Website
|
||||
|
||||
- Model the layout on https://www.placestoread.xyz: the whole list on one page, dense rows, a row that expands inline with its content aligned to the Name column, sorting by column headers, full-text search, tags that filter, a divider under the header row instead of a strong top border, a solid dark footer, and minimal decoration. Keep that model (no card grid, modal details, or pagination) unless the maintainer asks, and keep green out of the palette.
|
||||
- `.section-shell` (`--shell-max: 84rem`) is the only width cap: sections, tables, rows, CTAs, and paragraphs stay uncapped, since the maintainer reads on wide screens and prose-width advice like 65-75ch doesn't apply. Remove inner `max-width` rules you come across, and ask with a concrete reason before adding one.
|
||||
- Pick font sizes one step larger than feels right (`--text-base` over `--text-sm`, 1.75rem over 1.5rem for a heading), and shrink an existing size only when the maintainer asks: they asked for bigger type 11 times across 8 sessions and never for smaller.
|
||||
- When you style one link, tag, or label, give its peers the same style in the same change: hero, nav, footer, project, and sponsor links share hover styles, and tag variants build on `.tag`.
|
||||
@@ -0,0 +1,23 @@
|
||||
# CLAUDE.md
|
||||
|
||||
An opinionated guide to the best Python frameworks, libraries, and tools.
|
||||
|
||||
[README.md](README.md) is the single source of content truth; `website/` renders it into the static site: [awesome-python.com](https://awesome-python.com/).
|
||||
|
||||
## Entry Rules
|
||||
|
||||
[CONTRIBUTING.md](CONTRIBUTING.md) holds the admission rules, quality requirements, rejection rules, entry format, and ordering. Apply it whenever adding or removing an entry — direct commits included, not only PR reviews.
|
||||
|
||||
- Every keep/drop reason must be verified against current online data at decision time — download counts, repo activity and archived status, PyPI metadata, project docs. Judging tiers — obvious choice vs challenger — also requires WebSearch evidence (adoption trajectory, community sentiment), not download counts alone. Training-data recollections are not evidence; label anything unverifiable as a judgment call.
|
||||
- Checks stay read-only: judge whether a listed project installs or works from its PyPI metadata (`requires_python`, classifiers, wheel tags) and issue tracker, never by installing or running it, since that runs untrusted code.
|
||||
- A download count is not automatically independent demand. When one listed entry depends on another, check `requires_dist` on PyPI before citing the depended-on entry's count: mkdocs-material hard-depends on mkdocs, so mkdocs' figure exceeded mkdocs-material's by only about one percent and nearly all of it was mkdocs-material pulling it in.
|
||||
- One entry per commit when adding or deleting entries. Exceptions: a prune sweep is one commit per section, its body listing each removal with its reason; format, wording, or categorization changes may be bundled. Cross-section re-homes ride the originating audit's commit (both sides of the move in one diff).
|
||||
- Resources sections are not project entries: out of audit scope, and the website never parses them.
|
||||
- Sponsor placement never influences which projects get listed — see [SPONSORSHIP.md](SPONSORSHIP.md).
|
||||
|
||||
## Website
|
||||
|
||||
- Model the layout on https://www.placestoread.xyz: the whole list on one page, dense rows, a row that expands inline with its content aligned to the Name column, sorting by column headers, full-text search, tags that filter, a divider under the header row instead of a strong top border, a solid dark footer, and minimal decoration. Keep that model (no card grid, modal details, or pagination) unless the maintainer asks, and keep green out of the palette.
|
||||
- `.section-shell` (`--shell-max: 84rem`) is the only width cap: sections, tables, rows, CTAs, and paragraphs stay uncapped, since the maintainer reads on wide screens and prose-width advice like 65-75ch doesn't apply. Remove inner `max-width` rules you come across, and ask with a concrete reason before adding one.
|
||||
- Pick font sizes one step larger than feels right (`--text-base` over `--text-sm`, 1.75rem over 1.5rem for a heading), and shrink an existing size only when the maintainer asks: they asked for bigger type 11 times across 8 sessions and never for smaller.
|
||||
- When you style one link, tag, or label, give its peers the same style in the same change: hero, nav, footer, project, and sponsor links share hover styles, and tag variants build on `.tag`.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our
|
||||
community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, religion, or sexual identity and
|
||||
orientation.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment:
|
||||
|
||||
- Using welcoming and inclusive language
|
||||
- Being respectful of differing viewpoints and experiences
|
||||
- Gracefully accepting constructive criticism
|
||||
- Focusing on what is best for the community
|
||||
- Showing empathy towards other community members
|
||||
|
||||
Examples of unacceptable behavior:
|
||||
|
||||
- The use of sexualized language or imagery
|
||||
- Trolling, insulting or derogatory comments
|
||||
- Public or private harassment
|
||||
- Publishing others' private information without permission
|
||||
- Other conduct which could reasonably be considered inappropriate
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the project maintainers. All complaints will be reviewed and
|
||||
investigated and will result in a response that is deemed necessary and
|
||||
appropriate to the circumstances.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
|
||||
version 2.0.
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Awesome Python Curation
|
||||
|
||||
The editorial domain of awesome-python: which projects the list carries, how they are admitted, and how they are displaced. The list is a curated shortlist of obvious choices, not a catalog of everything good.
|
||||
|
||||
## Language
|
||||
|
||||
### List structure
|
||||
|
||||
**Entry**:
|
||||
A single listed project: `- [name](url) - Description.` The unit that is admitted, displaced, or pruned. Named by PyPI package name when one exists, else by repository name.
|
||||
|
||||
**Sub-item**:
|
||||
An indented awesome-* also-see link under an Entry (awesome-django under django, awesome-pytest under pytest). Not an Entry: it holds no slot, does not count against the Cap, and rides its parent — re-homed or removed with it. Maintainer-only placement, and only for awesome-* lists: a companion project either earns a full Entry in its proper Use Case or is not listed.
|
||||
|
||||
**Bundled Entry**:
|
||||
An Entry for a feature that ships inside a larger library, framework, or tool rather than as its own PyPI package (django.db.models, geodjango, httpx.URL, uv audit). Marked in README.md by a `(part of X)` description prefix, which the website reads to render a "Bundled" badge where a download count would go. It holds a normal slot and is audited like any other Entry; only its download signal is absent.
|
||||
_Avoid_: Pointer entry (reform-era name)
|
||||
|
||||
**Thematic Group**:
|
||||
A bold group line (e.g. "AI & ML", "Web Development") that clusters Sections in the TOC and the Projects body.
|
||||
|
||||
**Section**:
|
||||
A `###` heading in README.md (e.g. "Testing", "AI and Agents"). Sections group entries and live under a Thematic Group.
|
||||
_Avoid_: Category (overloaded — the TOC calls them categories, but rules bind to Use Cases, not Sections)
|
||||
|
||||
**Subcategory**:
|
||||
A named bullet inside a Section with indented entries under it (e.g. "Mock" inside "Testing"). Where present, each Subcategory is one Use Case.
|
||||
|
||||
**Use Case**:
|
||||
The unit of "it" in "one obvious way to do it": one distinct job a reader needs done. Defined by the list's structure — each Subcategory is a Use Case; a flat Section is a single Use Case. Submitters cannot define Use Cases; only the maintainer changes the structure (see Split).
|
||||
_Avoid_: Niche, micro-category
|
||||
|
||||
### Admission
|
||||
|
||||
**Serves Python Developers**:
|
||||
The scope test for what belongs on the list at all: Python developers use it in their Python work. Implementation language and packaging are irrelevant — uv and ty are Rust and belong; a pure-Python library nobody uses in Python work does not.
|
||||
_Avoid_: Python-first, written-in-Python (old requirement — removed)
|
||||
|
||||
**Obvious Choice**:
|
||||
An entry an experienced Python developer would name unprompted when asked "what do I use for [Use Case]?". Certified by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars; judgment overrides the signal's known failure modes (CI-inflated counts, model releases consumed as weights rather than pip installs — extended to any project consumed outside pip: SDK downloads like renpy, deployed services like thumbor — and large-but-specific audiences misread as "niche"). A standard-library module holds a slot only when it is itself the Obvious Choice for the Use Case, not merely relevant to it.
|
||||
_Avoid_: Industry Standard (old lane name), awesome (unfalsifiable)
|
||||
|
||||
**Cap**:
|
||||
The per-Use-Case entry limit: up to 3 Obvious Choices plus up to 2 Challengers, hard maximum 5. A qualitative bar first, a numeric backstop second — and a ceiling, not a floor: a minted Use Case may hold a single entry.
|
||||
|
||||
**Displacement**:
|
||||
The only admission path into a full Use Case: the PR names the entry it replaces and argues the newcomer does that entry's job better. One in, one out.
|
||||
_Avoid_: One-in-one-out (informal alias)
|
||||
|
||||
**Challenger**:
|
||||
An entry (at most two per Use Case) that is not yet the Obvious Choice but is a credible successor to one. Marked by ordering, not description: within a Use Case, Obvious Choices are listed first, then Challengers, each tier by PyPI downloads/month descending (standard-library modules first in the Use Case outright — always Obvious Choices by the stdlib rule — other no-signal entries last in tier, alphabetically). Admission requires adoption-trajectory evidence, not popularity alone.
|
||||
_Avoid_: Rising Star (old lane name), Hidden Gem (old lane name — concept removed entirely)
|
||||
|
||||
**Second Tier**:
|
||||
A Challenger slot holding a demoted incumbent instead of a rising successor — a former Obvious Choice or fading community standard an Audit kept below the current Obvious Choices (clickhouse-driver behind the official client, dogpile.cache and django-cacheops behind cachetools and diskcache, django-haystack in Search). It counts against the two Challenger slots and is marked by position like any Challenger; the adoption-trajectory bar gates only new admissions, not demotions.
|
||||
|
||||
**Override**:
|
||||
A maintainer decision, made in an Audit and explicitly recorded, that keeps an entry or use-case shape past a written limit — the activity line, the stability bar, a cap, or a structure rule. Case-by-case: an Override never loosens the written rule for any other entry, and submitters cannot cite one.
|
||||
|
||||
**Split**:
|
||||
Maintainer-only restructuring of an oversized Use Case into finer Use Cases (new Subcategories), considered before any trimming when the size reflects genuinely distinct jobs. A submitter's entry PR can never perform a Split.
|
||||
|
||||
### Maintenance
|
||||
|
||||
**Audit**:
|
||||
The recurring maintenance pass over one or more Sections: every Entry's verdict re-verified against current evidence, Challengers promoted or demoted, oversized Use Cases Split, drops and admissions re-decided. The maintainer adjudicates through an interactive preview, and entry changes land only on their explicit go. The 2026-08 shortlist-reform sweeps were the first Audits.
|
||||
_Avoid_: Sweep (reform-era name for a Section's first Audit)
|
||||
+123
-15
@@ -1,20 +1,128 @@
|
||||
# Contributing
|
||||
|
||||
Your contributions are always welcome!
|
||||
awesome-python is a shortlist, not a catalog. Each use case lists only its obvious choices, and most rejections mean "the use case is full", not "your project is bad". Read this whole page before opening a PR.
|
||||
|
||||
## Guidelines
|
||||
## Quality Requirements
|
||||
|
||||
* Add one link per Pull Request.
|
||||
* Make sure the PR title is in the format of `Add project-name`.
|
||||
* Write down the reason why the library is awesome.
|
||||
* Add the link: `* [project-name](http://example.com/) - A short description ends with a period.`
|
||||
* Keep descriptions concise and **short**.
|
||||
* Add a section if needed.
|
||||
* Add the section description.
|
||||
* Add the section title to Table of Contents.
|
||||
* Search previous Pull Requests or Issues before making a new one, as yours may be a duplicate.
|
||||
* Don't mention `Python` in the description as it's implied.
|
||||
* Check your spelling and grammar.
|
||||
* Remove any trailing whitespace.
|
||||
All submissions must satisfy **ALL** of these:
|
||||
|
||||
Just a gentle reminder: **Try not to submit your own project. Instead, wait for someone finds it useful and submits it for you.**
|
||||
1. **Serves Python Developers**: Python developers use it in their Python work. Implementation language and packaging are irrelevant — uv and ty are written in Rust, and agent skill packs are markdown, yet all belong; a pure-Python project nobody uses in Python work does not.
|
||||
2. **Active**: Commits within the last 12 months
|
||||
3. **Stable**: Production-ready, not alpha/beta/experimental. A PyPI "Development Status" classifier alone does not decide this; judge by releases, documentation, and production use.
|
||||
4. **Documented**: Clear README with examples and use cases
|
||||
5. **Established**: Repository at least 1 month old
|
||||
|
||||
## Admission
|
||||
|
||||
A **use case** is one distinct job a reader needs done. Use cases are defined by the list's structure: each subcategory is a use case, and a section without subcategories is a single use case. Structure changes — new sections, new subcategories, splitting an oversized use case into finer ones — are made only by the maintainer; an entry PR can never create the subcategory it needs.
|
||||
|
||||
Each use case lists at most:
|
||||
|
||||
- **Up to 3 obvious choices** — tools an experienced Python developer would name when asked "what do I use for this?"
|
||||
- **Up to 2 challengers** — tools that are not yet the obvious choice but are credible successors to one. Admission as a challenger requires adoption-trajectory evidence, not popularity alone.
|
||||
|
||||
Hard maximum: 5 entries per use case. This is a qualitative bar first and a numeric backstop second — most use cases should carry fewer.
|
||||
|
||||
**Overrides**: the maintainer may exceed any limit on this page — the caps, the activity requirement, the stability requirement — for a specific entry or use case by explicit decision. An override is case-by-case; it does not loosen these rules for submissions, and citing one in a PR carries no weight.
|
||||
|
||||
**Displacement**: once a use case is at its cap, the only way in is to name the entry your project replaces and argue that yours does that entry's job better. One in, one out.
|
||||
|
||||
**Dual-listing**: a tool may hold entries in multiple use cases, but only when it is an obvious choice in each. List the full entry in each home with identical lines; never a "see X above" note, since the website only renders list items. Description edits update every copy in the same commit. Each slot is audited on its own: dropping one home keeps the other, and dropping the tool entirely removes all copies in one commit. Dual-listing is a maintainer decision; a PR adding a second home for an existing entry is treated as a duplicate.
|
||||
|
||||
**Standard library**: a standard-library module is listed only where the stdlib is itself the obvious choice for the use case (tomllib yes, unittest no).
|
||||
|
||||
**Python versions**: missing support for the newest Python releases is not by itself grounds for removal while the entry still has a large user base; it can place the entry in the Second Tier instead.
|
||||
|
||||
**Evidence**: admission is decided by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars. Judgment overrides the signal's known failure modes (CI-inflated counts, model releases consumed as weights rather than pip installs, large-but-specific audiences misread as "niche"). The maintainer's decision is final.
|
||||
|
||||
Looking for an exhaustive catalog instead? Follow the awesome-\* lists linked under individual entries (for example awesome-python-testing) — they exist precisely so this list doesn't have to be one.
|
||||
|
||||
## Entry Format Reference
|
||||
|
||||
**Use GitHub repository URLs** whenever possible. Projects linked to a GitHub repo are ranked higher on [awesome-python.com](https://awesome-python.com/).
|
||||
|
||||
### Naming Convention
|
||||
|
||||
Use the **PyPI package name** as the display name so developers can copy it directly to `pip install`. Check the canonical name at `https://pypi.org/pypi/{package}/json`. If the project is not on PyPI, use the GitHub repository name instead.
|
||||
|
||||
### Standard Entry
|
||||
|
||||
```markdown
|
||||
- [pypi-name](https://github.com/owner/repo) - Description ending with period.
|
||||
```
|
||||
|
||||
### Standard Library Module
|
||||
|
||||
```markdown
|
||||
- [module](https://docs.python.org/3/library/module.html) - (Python standard library) Description.
|
||||
```
|
||||
|
||||
### Bundled Feature
|
||||
|
||||
A feature that ships inside a larger library, framework, or tool rather than as its own PyPI package. The `(part of X)` prefix is parsed by the website, which renders a "Bundled" badge in place of a download count, so write it exactly in that form and name the parent as developers say it.
|
||||
|
||||
```markdown
|
||||
- [name](url) - (part of Parent) Description ending with period.
|
||||
```
|
||||
|
||||
### Fork of Another Project
|
||||
|
||||
```markdown
|
||||
- [new-name](https://github.com/owner/new-name) - Description ([original-name](original-url) fork).
|
||||
```
|
||||
|
||||
### Entry with Related Awesome List
|
||||
|
||||
```markdown
|
||||
- [project](https://github.com/owner/project) - Description.
|
||||
- [awesome-project](https://github.com/someone/awesome-project)
|
||||
```
|
||||
|
||||
### Subcategory Format
|
||||
|
||||
```markdown
|
||||
- Subcategory Name
|
||||
- [project](url) - Description.
|
||||
```
|
||||
|
||||
### Entry Ordering
|
||||
|
||||
Within a use case, the obvious choices are listed first, ordered by PyPI downloads per month from high to low; challengers follow, in the same order. Standard-library modules sort first in the use case — built-ins lead, before everything else — alphabetically when there are several. (This never conflicts with tier order: a standard-library module is only listed where it is itself the obvious choice, so it always belongs to the first tier.) Other entries without a download signal (agent skill packs, projects distributed outside PyPI) sort last within their tier, alphabetically. There is no marker in the entry text — position is the marker, so the last entries of a use case may be its challengers.
|
||||
|
||||
## Changing the Structure
|
||||
|
||||
Adding sections or subcategories is maintainer-only (see Admission). For maintainer reference:
|
||||
|
||||
1. Add the section description in italics: `*Libraries for doing X.*`
|
||||
2. Add the section under the appropriate thematic group (e.g., **AI & ML**, **Web**, **Data & Science**)
|
||||
3. Add the section title to the Table of Contents under its group
|
||||
4. Order entries per Entry Ordering above
|
||||
|
||||
A subcategory never repeats its section's name: one named like its section competes with the section page in search results. Name the section's catch-all or main use case General instead.
|
||||
|
||||
## Review Process
|
||||
|
||||
PRs are reviewed by automated tools and maintainers:
|
||||
|
||||
1. **Format Check**: Entry follows the correct format
|
||||
2. **Category Check**: Placed in the appropriate use case
|
||||
3. **Duplicate Check**: Not already listed or previously rejected
|
||||
4. **Activity Check**: Project shows recent activity
|
||||
5. **Admission Check**: Meets the Admission rules above, including Displacement when the use case is at its cap
|
||||
|
||||
Search previous Pull Requests and Issues before submitting, as yours may be a duplicate.
|
||||
|
||||
## Automatic Rejection
|
||||
|
||||
PRs will be **closed** if:
|
||||
|
||||
- Adding multiple projects in one PR
|
||||
- The use case is at its cap and the PR makes no Displacement argument
|
||||
- The PR creates a new section or subcategory and fills it (structure changes are maintainer-only)
|
||||
- Coordinated multi-entry self-promotion: multiple related projects from the same organization or author, across one or several PRs
|
||||
- Duplicate of existing entry or recently-closed PR
|
||||
- Empty or placeholder PR descriptions
|
||||
- Placed under an inappropriate category
|
||||
- Project is archived or abandoned (no commits in 12+ months)
|
||||
- No documentation or unclear use case
|
||||
- Repository less than 1 month old
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
---
|
||||
version: alpha
|
||||
name: awesome-python.com
|
||||
description: Warm editorial Python index. Light cream canvas, brown-red interactive accent, Cormorant Garamond plus Manrope, table-driven single-page reference.
|
||||
---
|
||||
|
||||
# awesome-python.com DESIGN.md
|
||||
|
||||
awesome-python.com is a searchable, filterable index of ~650 curated Python projects. It is a reference tool, not a landing page and not a GitHub README mirror.
|
||||
|
||||
This file follows the [Google Stitch DESIGN.md format](https://stitch.withgoogle.com/docs/design-md/overview/). The source of truth for token values lives in `website/static/style.css`. Color tokens here are written in OKLCH because the project mandates OKLCH over hex, which is a deliberate divergence from the spec's hex-only token requirement.
|
||||
|
||||
## Overview
|
||||
|
||||
Three words: **opinionated, confident, dense**.
|
||||
|
||||
Working Python developers (mid to senior) are the target reader. They write Python daily and arrive with a specific question in mind: "what's a good HTTP client these days", "is there still a maintained ORM for X", "what are people using for task queues now". Secondary readers: polyglot developers evaluating Python's ecosystem, and curious browsers.
|
||||
|
||||
Jobs to be done:
|
||||
|
||||
1. Find a library for a specific need fast (search + tag filter).
|
||||
2. Compare candidates at a glance (stars, last commit, tags, one-line description).
|
||||
3. Confirm a project is alive before clicking through.
|
||||
|
||||
These users skim. They reward density and terse copy. They penalize marketing fluff.
|
||||
|
||||
Voice:
|
||||
|
||||
- Editorial. Every word earns its place.
|
||||
- Confident, not combative. "This is the list" energy, not "check out these cool projects".
|
||||
- No hype. The content is what's interesting.
|
||||
- Calm authority. Closer to a well-edited technical reference (O'Reilly index, The Economist briefing, a good man page) than a blog or product site.
|
||||
|
||||
Emotional goals: trust, efficiency, craft. The reader should feel the list was edited by someone with taste, find what they need in seconds, and notice the typographic care as a signal that the curation is careful too.
|
||||
|
||||
Reference points (stay close to these):
|
||||
|
||||
- **https://www.placestoread.xyz** is the primary visual model for the table, expand row, sorting, and footer. "Like placestoread" means dense single-page list, inline click-to-expand rows that indent under the Name column, sortable headers, minimum decoration. When in doubt about a table or row treatment, check placestoread first.
|
||||
- Magazine reference pages (The Economist, FT Weekend, Monocle).
|
||||
- Field-guide books. Curated, functional, hand-made.
|
||||
- Library card catalogs. Dense tabular information, excellent typography, no decoration for decoration's sake.
|
||||
|
||||
Anti-references (avoid strictly):
|
||||
|
||||
- Generic dark developer-tool look. No cyan on near-black, neon gradients, VSCode-palette dashboards, terminal-green monospace branding.
|
||||
- Other awesome-\* sites. No plain README dumps, bare lists of links, no voice.
|
||||
- SaaS marketing pages. No big metric counters, testimonial cards, feature grids, pricing tiers, or "join 10,000+ developers" social proof bands.
|
||||
|
||||
Design principles:
|
||||
|
||||
1. **The list is the hero.** Hero, sponsor band, and CTA exist, but they must not compete with the table for attention.
|
||||
2. **Density is a feature.** Prefer tables and tight rhythm over giant cards with one fact each. Mid-senior developers want to see more at once.
|
||||
3. **Editorial typography over decoration.** Visual interest comes from the serif/sans pairing, type hierarchy, and whitespace. Not from gradients, shadows, badges, or icon boxes above headings.
|
||||
4. **Warm, not cool.** Neutrals tint toward warm hues (roughly 55 to 80 in OKLCH). Pure grays and cool blues do not belong.
|
||||
5. **One point of view.** No dark mode, no theme picker, no alternate palettes. Consistency signals curation.
|
||||
|
||||
## Colors
|
||||
|
||||
Warm editorial palette. Light theme only (`color-scheme: light`). OKLCH only.
|
||||
|
||||
Each token below shows the OKLCH value (canonical, lives in `style.css`) followed by an approximate hex sRGB equivalent for spec linters and any tool that expects hex.
|
||||
|
||||
Surfaces:
|
||||
|
||||
- `--bg-page` `oklch(96.8% 0.018 80)` ≈ `#FBF3E7`. Cream/ivory canvas, the body floor. The body uses a vertical gradient between `--bg-page-top` `oklch(95.2% 0.018 78)` ≈ `#F7EFE3`, `--bg-page` at 24rem, and `--bg-page-end` `oklch(98.4% 0.01 80)` ≈ `#FCF8F0`, with a soft radial highlight in the top-left corner.
|
||||
- `--bg-paper` `oklch(98.6% 0.01 80)` ≈ `#FEFAF3`. Warm white for the content shell.
|
||||
- `--bg-paper-strong` `oklch(95.7% 0.016 76)` ≈ `#F7F0E5`. Tinted paper for sponsor band, CTA backgrounds, static decoration.
|
||||
- `--hero-bg-start` `oklch(14% 0.03 32)` ≈ `#130503` through `--hero-bg-mid` `oklch(19% 0.035 35)` ≈ `#22120B` to `--hero-bg-end` `oklch(28% 0.05 42)` ≈ `#3D2014`. Dark earthy hero gradient.
|
||||
- `--footer-bg` `oklch(16% 0.025 35)` ≈ `#170906`. Dark warm charcoal footer, part of the same system.
|
||||
|
||||
Ink:
|
||||
|
||||
- `--ink` `oklch(22% 0.02 55)` ≈ `#221812`. Body text.
|
||||
- `--ink-soft` `oklch(38% 0.018 55)` ≈ `#4A4039`. Secondary copy.
|
||||
- `--ink-muted` `oklch(52% 0.02 55)` ≈ `#72665E`. Meta rows, captions, static labels.
|
||||
- `--line` / `--line-strong`. Hairlines and dividers.
|
||||
|
||||
Accent (warm brown-red, reserved for interactive):
|
||||
|
||||
- `--accent` `oklch(58% 0.16 45)` ≈ `#C4530F`. Primary accent.
|
||||
- `--accent-deep` `oklch(44% 0.15 42)` ≈ `#922900`. Link text, hover.
|
||||
- `--accent-soft` `oklch(92% 0.045 55)` ≈ `#FDDDC9`. Tinted background for filter tags.
|
||||
- `--accent-underline` `oklch(58% 0.16 45 / 0.4)` ≈ `#C4530F66`. Subtle text-decoration-color.
|
||||
|
||||
Rules:
|
||||
|
||||
- Use OKLCH for any new color. Not HSL, not hex.
|
||||
- Accent tokens (`--accent`, `--accent-deep`, `--accent-soft`) are reserved for interactive elements. Clickable filter tags (`.tag`) correctly use `--accent-soft` background with `--accent-deep` text. Interactive link states (`.col-name > a:hover`, `.sponsor-link:hover`, `.hero-action-primary`, `.back-to-top`, CTAs) use accent tokens.
|
||||
- Non-interactive elements (inline code, `.source-badge`, static labels, decorative pills) must use ink tokens (`--ink`, `--ink-soft`, `--ink-muted`) on `--bg-paper-strong` or `--bg-paper`, never the accent. `.source-badge` uses `--ink-soft`; `.sponsorship-body code` uses `--ink`. Users should not mistake static decoration for something clickable.
|
||||
- Same role uses the same token everywhere. No one-off inline `color: oklch(...)` buried in a rule.
|
||||
|
||||
Aversions:
|
||||
|
||||
- **No green.** The user rejected it when picking the palette. Warm brown-red, ivory, and dark earthy tones are the established system. Do not introduce green even for success states or ancillary accents.
|
||||
- No cyan, no neon gradients, no pure grays, no cool blues.
|
||||
|
||||
## Typography
|
||||
|
||||
Pairing (do not swap):
|
||||
|
||||
- **Display**: `Cormorant Garamond` (serif, 600 only).
|
||||
- **Body**: `Manrope` (sans, 400 / 600 / 700 / 800).
|
||||
|
||||
Scale:
|
||||
|
||||
| Role | Token | Size | Family | Weight | Notes |
|
||||
| ---------------- | ----------------- | ----------------------------- | ------------------ | --------- | ---------------------------------------------- |
|
||||
| Hero headline | (literal `clamp`) | `clamp(4.5rem, 11vw, 8.5rem)` | Cormorant Garamond | 600 | Magazine-cover scale, single use on the hero |
|
||||
| Body large | `--text-lg` | `1.125rem` | Manrope | 400 | Content-heavy passages |
|
||||
| Body | `--text-base` | `1rem` (16px) | Manrope | 400 | Body floor, do not go smaller |
|
||||
| Meta / secondary | `--text-sm` | `0.95rem` | Manrope | 400 / 600 | Meta rows, secondary copy |
|
||||
| Caption / pill | `--text-xs` | `0.8rem` (12.8px) | Manrope | 600 / 700 | Smallest token, pills, badges, tags, footnotes |
|
||||
|
||||
Hard-won sizing rules (do not relax):
|
||||
|
||||
- **Body type floor is 16px.** Do not go smaller.
|
||||
- **Absolute minimum font size is 12px (`0.75rem`) for ANY text**, including pills, badges, tags, captions, footnotes. Anything smaller hits Chrome's default minimum-font-size floor and renders inconsistently across browsers and user accessibility settings. Use `var(--text-xs)` (`0.8rem`) as the smallest token in code.
|
||||
- **When in doubt, pick one step larger** than what generic scale references suggest. The user has corrected sizes upward 11+ times across 8 sessions. Footer, meta rows, expand content, labels, and headings all trend too small by default. **Never reduce an existing size unprompted.**
|
||||
- Adjacent heading levels differ by at least 0.25rem of rendered size.
|
||||
- Row numbers in the table: left-align, no leading zeros. Zero-padding was tried and rejected.
|
||||
- **Never `text-transform`.** Write the casing in the markup.
|
||||
|
||||
## Layout
|
||||
|
||||
- **Single width cap: `--shell-max: 84rem` (~1344px) applied via `.section-shell`.** This is the ONLY width cap in the project. Widescreen monitors are the default viewing context.
|
||||
- **Do NOT add `max-width`** to sections, cards, table cells, table rows, expanded rows, CTA backgrounds, sponsor descriptions, hero subcopy, paragraphs, or list items. The user has removed narrow inner caps repeatedly (`56ch`, `65-75ch`, etc.). Default is no inner cap.
|
||||
- The "cap line length at ~65-75ch" rule does NOT apply here. Ignore it. Readability at wide widths is carried by vertical rhythm, leading, and the modular type scale instead.
|
||||
- If a width cap is genuinely necessary for a specific element, ask first with a concrete reason before adding it.
|
||||
- Shell padding: `--shell-pad: clamp(1.25rem, 3vw, 2.5rem)`. Symmetric gutters: logo left-gap equals logo right-gap, column paddings match across header and body.
|
||||
- `gap` over child margins in flex and grid.
|
||||
- Logical properties (`margin-inline`, `padding-block`) over physical (`margin-left`, `padding-top`).
|
||||
- `rem` for spacing and type. `px` only for borders and shadows.
|
||||
- CSS custom properties for all colors and repeated values.
|
||||
- Sibling components (card lists, grid items) share identical spacing.
|
||||
- Use flexbox or grid for layout. Avoid floats and absolute positioning except for genuine overlay cases (focus rings, sticky headers).
|
||||
- Never `!important`. Fix specificity instead.
|
||||
|
||||
## Elevation & Depth
|
||||
|
||||
Depth comes from **tonal layers**, not heavy shadows.
|
||||
|
||||
- The page is a quiet warm canvas (`--bg-page`). The content shell is slightly brighter paper (`--bg-paper`). The sponsor band, CTA backgrounds, and inline decorative blocks step up to `--bg-paper-strong`.
|
||||
- The hero and the category guide band are the only places with real atmosphere: warm gradients on a dark earthy ground (`--hero-bg-start` → `--hero-bg-mid` → `--hero-bg-end`). Only the hero adds the subtle grid and slow sheen. The sheen and any other motion respect `prefers-reduced-motion`.
|
||||
- The footer is a single tonal block in `--footer-bg`, no internal gradients.
|
||||
- Two depth treatments are allowed and only these two. The search input combines a 1px inset highlight (`--search-inset`) with a soft warm drop shadow (`--search-shadow`, intensified by `--search-focus-shadow` on focus). The primary CTA button (`.hero-action-primary`) carries a warm drop shadow for press affordance. Both shadows are soft, warm-tinted, and tied to interactive elements. No new drop shadows on cards, panels, rows, or static decoration.
|
||||
- No glassmorphism as default decoration.
|
||||
- No bounce or elastic easing. Real objects decelerate smoothly.
|
||||
|
||||
## Shapes
|
||||
|
||||
The shape language is overwhelmingly **pill on small, zero radius on large**.
|
||||
|
||||
- **Pills** (`border-radius: 999px`) for tags, search, sponsor logo chip, source badges, back-to-top, and primary CTA buttons.
|
||||
- **`0.4rem`** is used in exactly one place: inline `<code>` inside `.sponsorship-body`. Do not introduce a tokenized radius scale. The project does not need one.
|
||||
- Containers use the page surface itself, not rounded panels. When a panel is needed, prefer pill on small chips and zero radius on large surfaces.
|
||||
- **No `border-left` or `border-right` greater than 1px as a colored accent stripe** on cards, list items, callouts, or alerts. Use a different structure.
|
||||
|
||||
## Components
|
||||
|
||||
The component vocabulary is small and table-led. Source of truth: `website/static/style.css`.
|
||||
|
||||
- **Table-driven index** (the hero of the page). Sticky header, sortable columns, click-to-expand rows that indent under the Name column. Modeled on placestoread.xyz. Not a card grid.
|
||||
- **Group rows**. Section, subcategory, and group pages list rows in README order. Section pages add one H2 row per use case; group pages add one per section, linking to that section's page. A note replaces the sort arrow until someone sorts a column, which flattens the list and hides the group rows.
|
||||
- **Filter tags** (`.tag`). `--accent-soft` background with `--accent-deep` text. Pill shape. Hover swaps to `--highlight` background with `--tag-hover-border` border and ink text. Active state uses the warm `--tag-active-start` → `--tag-active-end` gradient with hero-ink text. Tag variants (`tag-group`, `tag-source`) inherit the base `.tag` style today and differ only at narrow widths (`tag-group` hides under 960px). Add a new variant only when a real visual difference is needed.
|
||||
- **Hero**. Magazine-cover headline, dark earthy ground, kicker and proof microcopy, primary CTA button using `--hero-btn-start` / `--hero-btn-end`. Subtle grid plus slow sheen. Respects `prefers-reduced-motion`.
|
||||
- **Jump links**. One per use case in a category hero, plus one for the guide. Underlined like the intro's links, with a trailing ↓. Never tag-styled: tags filter the table or open another page, jump links only scroll.
|
||||
- **Guide band**. Everything in a category intro after its "How to choose" list. It sits below the table on the hero's dark gradient, with its H2 above the text.
|
||||
- **Sponsor band**. Sits in the README header on `--bg-paper-strong`. Editorial layout, not a logo wall. Sponsor links share the global accent treatment.
|
||||
- **CTA**. Warm `--cta-bg`, full-bleed within shell. The button itself uses accent tokens.
|
||||
- **Footer**. Dark warm charcoal, part of the same system. Footer links share the global hover and focus treatment.
|
||||
- **Search**. Pill input with `--search-inset` interior and `--search-focus-ring` focus ring. Focus shadow uses `--search-focus-shadow`.
|
||||
- **Source badge / inline code**. Static decoration on `--bg-paper-strong`. `.source-badge` uses `--ink-soft` text in pill shape; `.sponsorship-body code` uses `--ink` text with the lone `0.4rem` radius. Never the accent.
|
||||
|
||||
Peer-consistency check (run before shipping any visual change):
|
||||
|
||||
- Hover and focus states: if one link type gets a treatment, peer links (hero topbar, footer, project names, sponsor names, expand-meta) share it.
|
||||
- Tag variants inherit the base `.tag` style. Differ only where a real difference is needed.
|
||||
- Typography tiers: labels that play the same role share size, weight, and letter-spacing.
|
||||
- Symmetric gutters: logo left-gap equals logo right-gap, column paddings match across header and body.
|
||||
- Role-based color tokens: same role uses the same token everywhere.
|
||||
|
||||
## Do's and Don'ts
|
||||
|
||||
- **Do** keep the table the focal point. Hero, sponsor band, and CTA must not compete.
|
||||
- **Do** use accent tokens only on interactive elements.
|
||||
- **Do** prefer density over whitespace expansion.
|
||||
- **Do** check peer elements before shipping a visual change.
|
||||
- **Do** use OKLCH for every new color.
|
||||
- **Don't** add inner `max-width` to anything. The shell handles width.
|
||||
- **Don't** introduce green, cyan, neon, pure gray, or cool blue.
|
||||
- **Don't** add a dark mode, theme picker, or alternate palette.
|
||||
- **Don't** use gradient text (`background-clip: text` on gradients). Solid color only.
|
||||
- **Don't** use `!important`. Fix specificity instead.
|
||||
- **Don't** use `text-transform`. Write the casing in markup.
|
||||
- **Don't** use a `border-left` or `border-right` greater than 1px as an accent stripe.
|
||||
- **Don't** use bounce or elastic easing.
|
||||
- **Don't** use glassmorphism as default decoration.
|
||||
- **Don't** mimic generic dark developer-tool sites, other awesome-\* sites, or SaaS marketing pages.
|
||||
|
||||
## Narrow-Screen Behavior
|
||||
|
||||
The user actively tests `< 960px` and `< 680px`. Narrow screens must stay functional.
|
||||
|
||||
- Do not drop features the user might want (sort affordance, filter chips, sticky header where reasonable). Hiding is a last resort and requires justification.
|
||||
- Always run the `playwright-cli` skill at a narrow viewport after any layout change.
|
||||
|
||||
## Iteration Guide
|
||||
|
||||
Run this audit after generating or modifying a screen. Failure on any item means revise before moving on.
|
||||
|
||||
1. **Width caps.** Inspect every section, card, paragraph, table cell, expanded row, CTA, sponsor description, hero subcopy. Only `.section-shell` (`--shell-max: 84rem`) may cap width. Anything else with a `max-width` is wrong.
|
||||
2. **Accent reservation.** Grep the changed CSS for `--accent`, `--accent-deep`, `--accent-soft`. Each match must back an interactive element (link, button, focus ring, filter tag). Static decoration must use ink tokens (`--ink`, `--ink-soft`, `--ink-muted`) on `--bg-paper-strong` or `--bg-paper`.
|
||||
3. **Shape language.** Containers are square or pill. Anything in the 4px-to-16px radius range is suspect. The lone `0.4rem` on `.sponsorship-body code` is the only allowed exception.
|
||||
4. **Type sizes.** Confirm no rendered text falls below 12px. If a size feels small to a mid-senior reader on a 27-inch display, bump one step up. Never reduce an existing size.
|
||||
5. **Peer consistency.** Compare against the closest peer element (sibling link type, sibling tag variant, sibling label). Hover, focus, color token, and gutter must match unless there is a stated reason to differ.
|
||||
6. **Narrow viewport.** Run the `playwright-cli` skill at `< 960px` and `< 680px`. Sort affordance, filter chips, and sticky header must remain functional.
|
||||
|
||||
## Known Gaps
|
||||
|
||||
- **Color format diverges from the Stitch spec.** The official linter requires hex sRGB (`/^#([0-9a-fA-F]{3,8})$/`). The project mandates OKLCH in `style.css`. The Colors section above resolves this by showing both: OKLCH is canonical, hex is the linter-friendly approximation.
|
||||
- **YAML frontmatter is minimal.** Only `version`, `name`, and `description` are encoded. The project has no JSON / Figma export pipeline that would consume token-level frontmatter, so the prose-led format is preferred for everything else.
|
||||
- **No formal spacing or radius scale.** The codebase uses `clamp()` and ad-hoc rem values rather than a tokenized scale. Adding one would be invention, not documentation.
|
||||
|
||||
## Verification
|
||||
|
||||
After any frontend change, use the `playwright-cli` skill to visually verify in a real browser. Check layout, responsiveness, and interactive behavior. Do not claim a UI change works based on code alone.
|
||||
@@ -1,14 +1,32 @@
|
||||
site_install:
|
||||
pip install -r requirements.txt
|
||||
-include .env
|
||||
export
|
||||
|
||||
site_link:
|
||||
ln -sf $(CURDIR)/README.md $(CURDIR)/docs/index.md
|
||||
install:
|
||||
uv sync --locked
|
||||
|
||||
site_preview: site_link
|
||||
mkdocs serve
|
||||
fetch_github_stars:
|
||||
uv run python website/fetch_github_stars.py
|
||||
|
||||
site_build: site_link
|
||||
mkdocs build
|
||||
fetch_pypi_downloads:
|
||||
uv run python website/fetch_pypi_downloads_via_clickpy.py
|
||||
|
||||
site_deploy: site_link
|
||||
mkdocs gh-deploy --clean
|
||||
test:
|
||||
uv run pytest website/tests/ -v
|
||||
|
||||
lint:
|
||||
uv run ruff check .
|
||||
|
||||
format:
|
||||
uv run ruff format .
|
||||
|
||||
typecheck:
|
||||
uv run ty check website
|
||||
|
||||
build:
|
||||
uv run python website/build.py
|
||||
|
||||
preview: build
|
||||
uv run watchfiles \
|
||||
'uv run python website/build.py' \
|
||||
README.md website/templates website/static website/data & \
|
||||
python -m http.server -b 127.0.0.1 -d website/output/ 8000
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
# Sponsor awesome-python
|
||||
|
||||
**The #10 most-starred repository on all of GitHub.**
|
||||
|
||||
awesome-python is where Python developers go to discover tools. It ranks on the first page of Google for "best Python libraries," is referenced by ChatGPT and other LLMs when recommending Python tools, and is the list developers check when evaluating frameworks.
|
||||
|
||||
Your sponsorship puts your product in front of developers at the exact moment they're choosing what to use.
|
||||
|
||||
## Audience
|
||||
|
||||
| Metric | Value |
|
||||
| ------------ | ---------------------------------------------------------------------------------------------------- |
|
||||
| Stars |  |
|
||||
| Forks |  |
|
||||
| Watchers |  |
|
||||
| Contributors |  |
|
||||
|
||||
**Who visits:** Professional Python developers evaluating libraries and tools for production use. Not beginners browsing tutorials. People making adoption decisions.
|
||||
|
||||
**Where they come from:** Google Search, GitHub, Reddit, YouTube, ChatGPT/LLMs, Hacker News.
|
||||
|
||||
## Sponsorship Tiers
|
||||
|
||||
### Headline Sponsor - $500/month
|
||||
|
||||
- Large logo and one-line description (max 120 characters) pinned at the very top of the README, above all project entries
|
||||
- Logo link in the sponsor section of [awesome-python.com](https://awesome-python.com/)
|
||||
|
||||
### Featured Sponsor - $200/month
|
||||
|
||||
- Text entry (`[Name](URL) - Description.`, max 120 characters) pinned at the top of the README, directly below Headline sponsors
|
||||
- Text link in the sponsor section of [awesome-python.com](https://awesome-python.com/)
|
||||
|
||||
### Indie Dev Sponsor - $99/month or $249/quarter
|
||||
|
||||
- Lower-cost tier for indie developers, solo founders, and one-person companies only
|
||||
- Text entry (`[Name](URL) - Description.`, max 120 characters) in the README sponsor section, directly below Featured sponsors
|
||||
- Text link in the sponsor section of [awesome-python.com](https://awesome-python.com/)
|
||||
|
||||
Indie Dev Sponsor is available only to founder-led (one-person company) products. Funded startups, larger companies, and products with enterprise sales teams should use Featured or Headline Sponsor.
|
||||
|
||||
## Get Started
|
||||
|
||||
Email [sponsorship@awesome-python.com](mailto:sponsorship@awesome-python.com?subject=Awesome%20Python%20Sponsorship) with:
|
||||
|
||||
- **Tier:** Headline Sponsor ($500/mo), Featured Sponsor ($200/mo), or Indie Dev Sponsor ($99/mo or $249/quarter)
|
||||
- **Content:** Product name, URL, logo, and description (Headline tier) or `[Name](URL) - Description.` entry (Featured or Indie Dev tier)
|
||||
- **Duration:** 1, 3, 6 months, or longer
|
||||
- **Payment method:** card via [Open Collective](https://opencollective.com/awesome-python), US bank transfer (ACH/wire), or PayPal
|
||||
|
||||
One upfront payment per term. Setup takes less than 24 hours.
|
||||
|
||||
## Previously Sponsored By
|
||||
|
||||
- [Warp](https://www.warp.dev/) - The terminal for modern developers.
|
||||
- [pyr](https://pyrun.dev) - Zero-config Python project manager.
|
||||
|
||||
## Editorial Independence
|
||||
|
||||
Sponsorship is logo or link placement in the README header. It does not influence which projects are listed. Listings are curated on merit through the normal [contribution process](CONTRIBUTING.md).
|
||||
|
||||
We reserve the right to request changes to sponsor text, logos, or links that are misleading, off-topic, or incompatible with the README formatting.
|
||||
@@ -1 +0,0 @@
|
||||
awesome-python.com
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
status: accepted
|
||||
---
|
||||
|
||||
# awesome-python is a shortlist of obvious choices, not a catalog
|
||||
|
||||
By mid-2026 the list held 576 entries across 75 sections, with entry inflow up 24x year over year (96 additions in the trailing 12 months vs 4 the year before), concentrated in sections like AI and Agents (41 entries). The old acceptance model — three lanes (Industry Standard, Rising Star, Hidden Gem), only the first capped — admitted any project good enough in isolation, so categories grew without bound and stopped answering the reader's actual question: "what do I use for X?" We decided to reposition the list as a curated shortlist, per the Zen of Python: there should be one — and preferably only one — obvious way to do it.
|
||||
|
||||
## The decision
|
||||
|
||||
The list's scope test is "serves Python developers", replacing the old "primarily written in Python (>50%)" requirement: implementation language and packaging are irrelevant when Python developers use the thing in their Python work (uv and ty are Rust; agent skill packs are markdown), while a pure-Python project nobody uses in Python work does not belong. Each Use Case (a subcategory, or a flat section) lists at most its Obvious Choices — up to 3, plus up to 2 marked Challengers, hard maximum 5 (numbers provisional, to be reviewed after the prune). Admission is by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars, and stated as final; judgment overrides the signal's known failure modes (CI/dependency-inflated counts, model releases consumed as weights rather than pip installs, large-but-specific audiences misread as "niche"). Once a Use Case is at cap, the only way in is Displacement: the PR names the entry it replaces and argues the newcomer does that job better. Use Cases are defined by the list's existing structure; an entry PR can never create the subcategory it needs. Standard-library entries hold a slot only where the stdlib module is itself the obvious choice. The existing stock gets the same test retroactively: a staged, worst-first prune (per-section sweep commits), with removed entries deleted outright — git history is the archive. Resources sections (Newsletters, Podcasts, Websites) are out of scope for now.
|
||||
|
||||
## Considered options
|
||||
|
||||
- Keep the three-lane model with caps on every lane: rejected because the lanes answer the wrong question ("is this good enough to enter?") once admission is comparative; rising-star momentum becomes evidence for a Challenger slot or a Displacement rather than an admission ticket of its own, and Hidden Gem is definitionally incompatible with "obvious".
|
||||
- Rules-only, no retroactive prune: rejected because every rejection would face "but X is listed" precedent arguments, and readers would see no change.
|
||||
- Archive removed entries in a separate file: rejected because it recreates the catalog one click away and dilutes the identity the change exists to restore.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The list shrinks substantially: the maintainer-reviewed preview of the three largest sections kept 45 of 80 entries. Most future PRs will be rejected for fullness, not badness.
|
||||
- awesome-python.com loses long-tail search traffic for the hundreds of niche tool names it will no longer carry. Accepted deliberately: reader trust over search surface.
|
||||
- Fast-moving domains (e.g. AI and Agents) list current leaders by usage and absorb churn through Displacement; an oversized Use Case is either trimmed or Split into finer Use Cases by the maintainer.
|
||||
- Linked awesome-* lists (e.g. awesome-python-testing) remain as the escape valve for readers who want exhaustive catalogs.
|
||||
|
||||
See CONTEXT.md for the vocabulary (Use Case, Obvious Choice, Challenger, Displacement, Split).
|
||||
@@ -0,0 +1,67 @@
|
||||
# Audit Log
|
||||
|
||||
awesome-python is audited section by section. Every entry gets re-verified against live data, and every removal lands in a commit whose body carries the reason. Git history is the archive. This file is the at-a-glance register of maintainer decisions that a single commit can't show.
|
||||
|
||||
## Overrides
|
||||
|
||||
[CONTRIBUTING.md](../CONTRIBUTING.md) allows the maintainer to exceed any limit for a specific entry or use case. Each override is recorded here.
|
||||
|
||||
### Naming Exceptions
|
||||
|
||||
Display names follow the canonical PyPI package name. These entries keep a different name by maintainer decision:
|
||||
|
||||
- autobahn-python -- `autobahn`.
|
||||
- django-rest-framework -- `djangorestframework`.
|
||||
- django-rules -- `rules`.
|
||||
- fasthtml -- `python-fasthtml`.
|
||||
- graphify -- `graphifyy`.
|
||||
- jinja -- `Jinja2`.
|
||||
- mem0 -- `mem0ai`.
|
||||
- pangu.py -- `pangu`.
|
||||
- playwright-python -- `playwright`.
|
||||
- pytorch -- `torch`.
|
||||
- strawberry -- `strawberry-graphql`.
|
||||
- strawberry-django -- `strawberry-graphql-django`.
|
||||
- thealgorithms -- not on PyPI; its GitHub repo name is `Python`.
|
||||
|
||||
### Stability Exceptions
|
||||
|
||||
Entries admitted despite the Stable quality requirement, by maintainer decision:
|
||||
|
||||
- zensical -- admitted 2026-08-23 at version 0.0.57, PyPI classifier `Development Status :: 3 - Alpha`, with no 1.0 target announced. Admitted as a challenger displacing mkdocs because the Documentation section was carrying a dying upstream: the Material for MkDocs team's announcement of 2025-11-05 put Material into maintenance mode for twelve months and called MkDocs itself unmaintained since 2024-08 and a supply chain risk. Re-check when 1.0 lands.
|
||||
- openviking -- admitted 2026-08-25 (PR #3302) at version 0.4.16, PyPI classifier `Development Status :: 3 - Alpha`, with a README that said "OpenViking is still in its early stages". Admitted as a Data Layer challenger because the adoption is real: 459,446 downloads/month on pypistats (mirror-excluded) on 2026-08-24, and releases every few days. Re-check when 1.0 lands.
|
||||
- claude-agent-sdk -- kept 2026-10-01 in the AI and Agents audit at version 0.2.163, PyPI classifier `Development Status :: 3 - Alpha`. Kept as a Vendor Agent SDKs obvious choice because the adoption is real: 30,033,689 downloads/month on ClickPy on 2026-10-01, up from 1.2M in March 2026 on pypistats. Re-check when 1.0 lands.
|
||||
- uv-audit -- kept 2026-08-16 as a Supply Chain Security entry, recorded 2026-10-02. uv itself prints "`uv audit` is experimental and may change without warning". Kept because it is part of uv, the obvious package manager. Re-check when uv drops the experimental notice.
|
||||
|
||||
### Challenger Exceptions
|
||||
|
||||
Entries admitted as challengers without adoption-trajectory evidence, by maintainer decision:
|
||||
|
||||
- seleniumbase -- admitted 2026-08-16 (PR #3284) as a Browser Automation challenger. It wraps selenium, the successor trajectory in the use case belongs to playwright, and its distinct value is its stealth/UC mode. Admitted because that stealth value counts toward Browser Automation.
|
||||
- semantica -- kept 2026-10-01 in the AI and Agents audit as a Data Layer challenger at 16,582 downloads/month on ClickPy, with no adoption-trajectory evidence found. Kept by maintainer decision.
|
||||
- jev-ultrafast -- kept 2026-09-24 as a Web Scraping Frameworks challenger, recorded 2026-10-02. Admitted while the repo was under a month old and not on PyPI, so it has no download signal and no adoption-trajectory evidence; its README calls it an MVP and it needs a hosted TypeSafe API key. Kept by maintainer decision.
|
||||
|
||||
### Cap Exceptions
|
||||
|
||||
Use cases holding more than 5 entries, by maintainer decision:
|
||||
|
||||
- Asynchronous Programming > Async I/O -- 6 entries since the 2026-08-16 split (19875cf), recorded 2026-10-02.
|
||||
- Code Analysis > Linters and Formatters -- 6 entries since Security Linters merged in on 2026-08-16 (3290ce5), recorded 2026-10-02.
|
||||
- Package Management > Package Managers -- 6 entries since the subcategory was minted on 2026-08-16 (1b51907), recorded 2026-10-02.
|
||||
|
||||
### Mature-stable Keeps
|
||||
|
||||
These entries sit past the 12-month activity requirement without an override. Each one is kept by editorial judgment: mature, stable, and no successor exists.
|
||||
|
||||
- annoy -- kept 2026-10-02 in the AI & ML audit, last push 2025-10-29. Unlike the others it has a successor, Spotify's own Voyager (2023), but kept by maintainer decision on 911,913 downloads/month on pypistats.
|
||||
- blinker -- recorded 2026-10-02 ahead of its activity-line crossing on 2026-11-19 (last push 2025-11-19, last release 1.9.0 on 2024-11-08). Flask requires it.
|
||||
- diskcache -- kept 2026-10-02, last push 2024-08-10, last release 5.6.3 (2023-08-31). Kept despite an unpatched pickle deserialization advisory, CVE-2025-69872 (GHSA-w8v5-vhqr-4h9v), on 25,526,045 downloads/month on ClickPy.
|
||||
- django-rules -- recorded 2026-10-02 ahead of its activity-line crossing on 2026-10-11 (last push 2025-10-11, last release 3.5 on 2024-09-02).
|
||||
- ftfy
|
||||
- httpie -- kept 2026-08-16, recorded 2026-10-02. Last commit 2024-12-17, last release 3.2.4 on 2024-11-01.
|
||||
- itsdangerous
|
||||
- jieba
|
||||
- jinja
|
||||
- python-pptx -- kept 2026-10-02, last commit 2024-08-06, last release 1.0.2 on 2024-08-07. No successor for generating .pptx files.
|
||||
- schedule -- kept 2026-08-16, recorded 2026-10-02. Last commit 2024-05-25.
|
||||
- sortedcontainers
|
||||
@@ -1,9 +0,0 @@
|
||||
@media (min-width: 960px) {
|
||||
html {
|
||||
scroll-behavior: smooth;
|
||||
}
|
||||
|
||||
.md-content__inner > ul:nth-child(5) {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
-26
@@ -1,26 +0,0 @@
|
||||
site_name: Awesome Python
|
||||
site_url: https://awesome-python.com
|
||||
site_description: A curated list of awesome Python frameworks, libraries and software
|
||||
site_author: Vinta Chen
|
||||
repo_name: vinta/awesome-python
|
||||
repo_url: https://github.com/vinta/awesome-python
|
||||
theme:
|
||||
name: material
|
||||
palette:
|
||||
primary: red
|
||||
accent: pink
|
||||
extra:
|
||||
social:
|
||||
- type: github
|
||||
link: https://github.com/vinta
|
||||
- type: twitter
|
||||
link: https://twitter.com/vinta
|
||||
- type: linkedin
|
||||
link: https://www.linkedin.com/in/vinta
|
||||
google_analytics:
|
||||
- UA-510626-7
|
||||
- auto
|
||||
extra_css:
|
||||
- css/extra.css
|
||||
nav:
|
||||
- "Life is short, you need Python.": "index.md"
|
||||
@@ -0,0 +1,53 @@
|
||||
[project]
|
||||
name = "awesome-python"
|
||||
version = "0.1.0"
|
||||
description = "An opinionated guide to the best Python frameworks, libraries, and tools."
|
||||
authors = [{ name = "Vinta Chen", email = "vinta.chen@gmail.com" }]
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
requires-python = ">=3.14"
|
||||
dependencies = []
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://awesome-python.com/"
|
||||
Repository = "https://github.com/vinta/awesome-python"
|
||||
|
||||
[dependency-groups]
|
||||
build = ["httpx>=0.28.1", "jinja2>=3.1.6", "markdown-it-py>=4.0.0"]
|
||||
lint = ["ruff>=0.15.6", "ty>=0.0.33"]
|
||||
test = ["pytest>=9.0.3"]
|
||||
preview = ["watchfiles>=1.2.0"]
|
||||
dev = [
|
||||
{ include-group = "build" },
|
||||
{ include-group = "lint" },
|
||||
{ include-group = "test" },
|
||||
{ include-group = "preview" },
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
exclude-newer = "3 days"
|
||||
no-build = true
|
||||
|
||||
[tool.uv.pip]
|
||||
only-binary = [":all:"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 200
|
||||
|
||||
[tool.ty.environment]
|
||||
python-version = "3.14"
|
||||
root = ["website"]
|
||||
|
||||
[tool.ty.terminal]
|
||||
error-on-warning = true
|
||||
|
||||
[tool.ty.rules]
|
||||
division-by-zero = "error"
|
||||
possibly-missing-attribute = "error"
|
||||
possibly-missing-import = "error"
|
||||
possibly-unresolved-reference = "error"
|
||||
unused-ignore-comment = "error"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["website/tests"]
|
||||
pythonpath = ["website"]
|
||||
@@ -1,2 +0,0 @@
|
||||
mkdocs==1.0.4
|
||||
mkdocs-material==4.0.2
|
||||
@@ -1,83 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
The approach taken is explained below. I decided to do it simply.
|
||||
Initially I was considering parsing the data into some sort of
|
||||
structure and then generating an appropriate README. I am still
|
||||
considering doing it - but for now this should work. The only issue
|
||||
I see is that it only sorts the entries at the lowest level, and that
|
||||
the order of the top-level contents do not match the order of the actual
|
||||
entries.
|
||||
|
||||
This could be extended by having nested blocks, sorting them recursively
|
||||
and flattening the end structure into a list of lines. Revision 2 maybe ^.^.
|
||||
"""
|
||||
|
||||
def sort_blocks():
|
||||
# First, we load the current README into memory
|
||||
with open('README.md', 'r') as read_me_file:
|
||||
read_me = read_me_file.read()
|
||||
|
||||
# Separating the 'table of contents' from the contents (blocks)
|
||||
table_of_contents = ''.join(read_me.split('- - -')[0])
|
||||
blocks = ''.join(read_me.split('- - -')[1]).split('\n# ')
|
||||
for i in range(len(blocks)):
|
||||
if i == 0:
|
||||
blocks[i] = blocks[i] + '\n'
|
||||
else:
|
||||
blocks[i] = '# ' + blocks[i] + '\n'
|
||||
|
||||
# Sorting the libraries
|
||||
inner_blocks = sorted(blocks[0].split('##'))
|
||||
for i in range(1, len(inner_blocks)):
|
||||
if inner_blocks[i][0] != '#':
|
||||
inner_blocks[i] = '##' + inner_blocks[i]
|
||||
inner_blocks = ''.join(inner_blocks)
|
||||
|
||||
# Replacing the non-sorted libraries by the sorted ones and gathering all at the final_README file
|
||||
blocks[0] = inner_blocks
|
||||
final_README = table_of_contents + '- - -' + ''.join(blocks)
|
||||
|
||||
with open('README.md', 'w+') as sorted_file:
|
||||
sorted_file.write(final_README)
|
||||
|
||||
def main():
|
||||
# First, we load the current README into memory as an array of lines
|
||||
with open('README.md', 'r') as read_me_file:
|
||||
read_me = read_me_file.readlines()
|
||||
|
||||
# Then we cluster the lines together as blocks
|
||||
# Each block represents a collection of lines that should be sorted
|
||||
# This was done by assuming only links ([...](...)) are meant to be sorted
|
||||
# Clustering is done by indentation
|
||||
blocks = []
|
||||
last_indent = None
|
||||
for line in read_me:
|
||||
s_line = line.lstrip()
|
||||
indent = len(line) - len(s_line)
|
||||
|
||||
if any([s_line.startswith(s) for s in ['* [', '- [']]):
|
||||
if indent == last_indent:
|
||||
blocks[-1].append(line)
|
||||
else:
|
||||
blocks.append([line])
|
||||
last_indent = indent
|
||||
else:
|
||||
blocks.append([line])
|
||||
last_indent = None
|
||||
|
||||
with open('README.md', 'w+') as sorted_file:
|
||||
# Then all of the blocks are sorted individually
|
||||
blocks = [
|
||||
''.join(sorted(block, key=str.lower)) for block in blocks
|
||||
]
|
||||
# And the result is written back to README.md
|
||||
sorted_file.write(''.join(blocks))
|
||||
|
||||
# Then we call the sorting method
|
||||
sort_blocks()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,349 @@
|
||||
version = 1
|
||||
revision = 3
|
||||
requires-python = ">=3.14"
|
||||
|
||||
[options]
|
||||
exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
|
||||
exclude-newer-span = "P3D"
|
||||
|
||||
[[package]]
|
||||
name = "anyio"
|
||||
version = "4.14.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "idna" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/61/cc/a381afa6efea9f496eff839d4a6a1aed3bfafc7b3ab4b0d1b243a12573dd/anyio-4.14.2.tar.gz", hash = "sha256:cfa139f3ed1a23ee8f88a145ddb5ac7605b8bbfd8592baacd7ce3d8bb4313c7f", size = 260176, upload-time = "2026-07-12T20:29:07.082Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/da/35/f2287558c17e29fafc8ef3daf819bb9834061cfa43bff8014f7df7f63bdc/anyio-4.14.2-py3-none-any.whl", hash = "sha256:9f505dda5ac9f0c8309b5e8bd445a8c2bf7246f3ce950121e45ea15bc41d1494", size = 125813, upload-time = "2026-07-12T20:29:05.763Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "awesome-python"
|
||||
version = "0.1.0"
|
||||
source = { virtual = "." }
|
||||
|
||||
[package.dev-dependencies]
|
||||
build = [
|
||||
{ name = "httpx" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "markdown-it-py" },
|
||||
]
|
||||
dev = [
|
||||
{ name = "httpx" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "markdown-it-py" },
|
||||
{ name = "pytest" },
|
||||
{ name = "ruff" },
|
||||
{ name = "ty" },
|
||||
{ name = "watchfiles" },
|
||||
]
|
||||
lint = [
|
||||
{ name = "ruff" },
|
||||
{ name = "ty" },
|
||||
]
|
||||
preview = [
|
||||
{ name = "watchfiles" },
|
||||
]
|
||||
test = [
|
||||
{ name = "pytest" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
build = [
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "jinja2", specifier = ">=3.1.6" },
|
||||
{ name = "markdown-it-py", specifier = ">=4.0.0" },
|
||||
]
|
||||
dev = [
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "jinja2", specifier = ">=3.1.6" },
|
||||
{ name = "markdown-it-py", specifier = ">=4.0.0" },
|
||||
{ name = "pytest", specifier = ">=9.0.3" },
|
||||
{ name = "ruff", specifier = ">=0.15.6" },
|
||||
{ name = "ty", specifier = ">=0.0.33" },
|
||||
{ name = "watchfiles", specifier = ">=1.2.0" },
|
||||
]
|
||||
lint = [
|
||||
{ name = "ruff", specifier = ">=0.15.6" },
|
||||
{ name = "ty", specifier = ">=0.0.33" },
|
||||
]
|
||||
preview = [{ name = "watchfiles", specifier = ">=1.2.0" }]
|
||||
test = [{ name = "pytest", specifier = ">=9.0.3" }]
|
||||
|
||||
[[package]]
|
||||
name = "certifi"
|
||||
version = "2026.2.25"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/af/2d/7bf41579a8986e348fa033a31cdd0e4121114f6bce2457e8876010b092dd/certifi-2026.2.25.tar.gz", hash = "sha256:e887ab5cee78ea814d3472169153c2d12cd43b14bd03329a39a9c6e2e80bfba7", size = 155029, upload-time = "2026-02-25T02:54:17.342Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/9a/3c/c17fb3ca2d9c3acff52e30b309f538586f9f5b9c9cf454f3845fc9af4881/certifi-2026.2.25-py3-none-any.whl", hash = "sha256:027692e4402ad994f1c42e52a4997a9763c646b73e4096e4d5d6db8af1d6f0fa", size = 153684, upload-time = "2026-02-25T02:54:15.766Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "h11"
|
||||
version = "0.16.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpcore"
|
||||
version = "1.0.9"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "certifi" },
|
||||
{ name = "h11" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpx"
|
||||
version = "0.28.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "certifi" },
|
||||
{ name = "httpcore" },
|
||||
{ name = "idna" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.15"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jinja2"
|
||||
version = "3.1.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markupsafe" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "markdown-it-py"
|
||||
version = "4.0.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "mdurl" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz", hash = "sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", size = 73070, upload-time = "2025-08-11T12:57:52.854Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl", hash = "sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147", size = 87321, upload-time = "2025-08-11T12:57:51.923Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "markupsafe"
|
||||
version = "3.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mdurl"
|
||||
version = "0.1.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.6.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.20.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pytest"
|
||||
version = "9.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "iniconfig" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pluggy" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruff"
|
||||
version = "0.15.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/51/df/f8629c19c5318601d3121e230f74cbee7a3732339c52b21daa2b82ef9c7d/ruff-0.15.6.tar.gz", hash = "sha256:8394c7bb153a4e3811a4ecdacd4a8e6a4fa8097028119160dffecdcdf9b56ae4", size = 4597916, upload-time = "2026-03-12T23:05:47.51Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/9e/2f/4e03a7e5ce99b517e98d3b4951f411de2b0fa8348d39cf446671adcce9a2/ruff-0.15.6-py3-none-linux_armv6l.whl", hash = "sha256:7c98c3b16407b2cf3d0f2b80c80187384bc92c6774d85fefa913ecd941256fff", size = 10508953, upload-time = "2026-03-12T23:05:17.246Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/60/55bcdc3e9f80bcf39edf0cd272da6fa511a3d94d5a0dd9e0adf76ceebdb4/ruff-0.15.6-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:ee7dcfaad8b282a284df4aa6ddc2741b3f4a18b0555d626805555a820ea181c3", size = 10942257, upload-time = "2026-03-12T23:05:23.076Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/f9/005c29bd1726c0f492bfa215e95154cf480574140cb5f867c797c18c790b/ruff-0.15.6-py3-none-macosx_11_0_arm64.whl", hash = "sha256:3bd9967851a25f038fc8b9ae88a7fbd1b609f30349231dffaa37b6804923c4bb", size = 10322683, upload-time = "2026-03-12T23:05:33.738Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/74/2f861f5fd7cbb2146bddb5501450300ce41562da36d21868c69b7a828169/ruff-0.15.6-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:13f4594b04e42cd24a41da653886b04d2ff87adbf57497ed4f728b0e8a4866f8", size = 10660986, upload-time = "2026-03-12T23:05:53.245Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/a1/309f2364a424eccb763cdafc49df843c282609f47fe53aa83f38272389e0/ruff-0.15.6-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e2ed8aea2f3fe57886d3f00ea5b8aae5bf68d5e195f487f037a955ff9fbaac9e", size = 10332177, upload-time = "2026-03-12T23:05:56.145Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/41/7ebf1d32658b4bab20f8ac80972fb19cd4e2c6b78552be263a680edc55ac/ruff-0.15.6-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:70789d3e7830b848b548aae96766431c0dc01a6c78c13381f423bf7076c66d15", size = 11170783, upload-time = "2026-03-12T23:06:01.742Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/be/6d488f6adca047df82cd62c304638bcb00821c36bd4881cfca221561fdfc/ruff-0.15.6-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:542aaf1de3154cea088ced5a819ce872611256ffe2498e750bbae5247a8114e9", size = 12044201, upload-time = "2026-03-12T23:05:28.697Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/71/68/e6f125df4af7e6d0b498f8d373274794bc5156b324e8ab4bf5c1b4fc0ec7/ruff-0.15.6-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1c22e6f02c16cfac3888aa636e9eba857254d15bbacc9906c9689fdecb1953ab", size = 11421561, upload-time = "2026-03-12T23:05:31.236Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/9f/f85ef5fd01a52e0b472b26dc1b4bd228b8f6f0435975442ffa4741278703/ruff-0.15.6-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:98893c4c0aadc8e448cfa315bd0cc343a5323d740fe5f28ef8a3f9e21b381f7e", size = 11310928, upload-time = "2026-03-12T23:05:45.288Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8c/26/b75f8c421f5654304b89471ed384ae8c7f42b4dff58fa6ce1626d7f2b59a/ruff-0.15.6-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:70d263770d234912374493e8cc1e7385c5d49376e41dfa51c5c3453169dc581c", size = 11235186, upload-time = "2026-03-12T23:05:50.677Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fc/d4/d5a6d065962ff7a68a86c9b4f5500f7d101a0792078de636526c0edd40da/ruff-0.15.6-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:55a1ad63c5a6e54b1f21b7514dfadc0c7fb40093fa22e95143cf3f64ebdcd512", size = 10635231, upload-time = "2026-03-12T23:05:37.044Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d6/56/7c3acf3d50910375349016cf33de24be021532042afbed87942858992491/ruff-0.15.6-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:8dc473ba093c5ec238bb1e7429ee676dca24643c471e11fbaa8a857925b061c0", size = 10340357, upload-time = "2026-03-12T23:06:04.748Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/06/54/6faa39e9c1033ff6a3b6e76b5df536931cd30caf64988e112bbf91ef5ce5/ruff-0.15.6-py3-none-musllinux_1_2_i686.whl", hash = "sha256:85b042377c2a5561131767974617006f99f7e13c63c111b998f29fc1e58a4cfb", size = 10860583, upload-time = "2026-03-12T23:05:58.978Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/1e/509a201b843b4dfb0b32acdedf68d951d3377988cae43949ba4c4133a96a/ruff-0.15.6-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:cef49e30bc5a86a6a92098a7fbf6e467a234d90b63305d6f3ec01225a9d092e0", size = 11410976, upload-time = "2026-03-12T23:05:39.955Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/25/3fc9114abf979a41673ce877c08016f8e660ad6cf508c3957f537d2e9fa9/ruff-0.15.6-py3-none-win32.whl", hash = "sha256:bbf67d39832404812a2d23020dda68fee7f18ce15654e96fb1d3ad21a5fe436c", size = 10616872, upload-time = "2026-03-12T23:05:42.451Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/7a/09ece68445ceac348df06e08bf75db72d0e8427765b96c9c0ffabc1be1d9/ruff-0.15.6-py3-none-win_amd64.whl", hash = "sha256:aee25bc84c2f1007ecb5037dff75cef00414fdf17c23f07dc13e577883dca406", size = 11787271, upload-time = "2026-03-12T23:05:20.168Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7f/d0/578c47dd68152ddddddf31cd7fc67dc30b7cdf639a86275fda821b0d9d98/ruff-0.15.6-py3-none-win_arm64.whl", hash = "sha256:c34de3dd0b0ba203be50ae70f5910b17188556630e2178fd7d79fc030eb0d837", size = 11060497, upload-time = "2026-03-12T23:05:25.968Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ty"
|
||||
version = "0.0.33"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/84/44/9478c50c266826c1bf30d1692e589755bffa8f1c0a3eb7af8a346c255991/ty-0.0.33.tar.gz", hash = "sha256:46d63bda07403322cb6c28ccfdd5536be916e13df725c29f7ccd0a21f06bd9e8", size = 5559373, upload-time = "2026-04-28T10:45:13.18Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/24/e287388c63a19191be26b32ff4dbd06029834068150ebe2532939bc4c851/ty-0.0.33-py3-none-linux_armv6l.whl", hash = "sha256:94d0a9d2234261a8911396d59e506b5923fe0971dbda43b9dcea287936887fcc", size = 11021308, upload-time = "2026-04-28T10:45:43.34Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/ca/ba1eed819895bd239fba8ee35dfcd5fcb266c203b0914a17a59579096bb5/ty-0.0.33-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:e4a2b5ba078f90de342f56b5f7979bb77c9b9b1d8625a041352ffc6ee93c4073", size = 10777272, upload-time = "2026-04-28T10:45:32.905Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/25/a8/c3131d37b44b3fea1d6654a1c929a0cd0873822f77a90482b8ec28f6fbbd/ty-0.0.33-py3-none-macosx_11_0_arm64.whl", hash = "sha256:84ff5707825e9af9668d2bcf66975f93e520a63b524ab494e3a8265735be2563", size = 10201078, upload-time = "2026-04-28T10:45:23.374Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/db/d8e37ff0045810cc65e1ff36aa0da0a2253c05659787ac987df8a16c7897/ty-0.0.33-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e375285736f57886868e7af0b11c7b0ec5b6543fa15e7ad2a714fed9f077d4e0", size = 10732347, upload-time = "2026-04-28T10:45:21.444Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/1a/20e83a412506a918e4684fc67b567cf7cc13b105470b3428cb23c3d5aa13/ty-0.0.33-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5680f6350c3b4e46b8bff6d7bb132366ea239463d6cad4892725d06046e65464", size = 10808238, upload-time = "2026-04-28T10:45:38.565Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/4b/d0a39f4464dc6cb4cc2c159473ce216bd1846bfb684c0323a3cb36dce5c6/ty-0.0.33-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c5535538bad8d0f7e62bcdff02197cdb30e41451d80b35d27e17d128f2e1dc5d", size = 11288348, upload-time = "2026-04-28T10:45:08.419Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/35/7e/f1745e0f9583363d7a83d9a4990fc244f76ecc30840ddad83dc16a33c52d/ty-0.0.33-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:da196c42bbbc069e1e21e3e52107c061aa9660352dae57a41930690b56e2c02d", size = 11789907, upload-time = "2026-04-28T10:45:19.064Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a5/71/25f39f46a12d662859d45bc648555d0661044eb43db6b5648c9947487da9/ty-0.0.33-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9281672921ef6d4460e03146b5e6c18cb1a3e3a3b8a1a88f6f33226d05a469b7", size = 11500774, upload-time = "2026-04-28T10:45:48.012Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/ec/136959ecbb7c71cb90537f5aea441c73f4ab24612868a6ecdc9d7444d32d/ty-0.0.33-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:82c1b8f303f82da64e878108e764be3ecbcd7c9903ac0a7f7031614ed00b97ab", size = 11360314, upload-time = "2026-04-28T10:45:05.402Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cf/95/32809575c222f00beed498cb728e9290a0f5009f930025381bb7253b2206/ty-0.0.33-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:efe3af412c9ff67bce5fa37d0a2b0d8555c24072b145a5bac6c79637f1c83abe", size = 10707785, upload-time = "2026-04-28T10:45:10.836Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/13/89/c8e9531f7aa4a093359e15fa32c8e1277fbbe90d16894d7c6032d29f4b34/ty-0.0.33-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:aeec29c91ea768601747da546c3efc20b72c2fb1bd52bcc786a5c6eeff51d27b", size = 10834987, upload-time = "2026-04-28T10:45:40.738Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/31/16/9835fbcf5338af1a1917bd28fdb8a7193c210b83f243aa286fa9f79cb3ad/ty-0.0.33-py3-none-musllinux_1_2_i686.whl", hash = "sha256:a535977c52bbb5f7e96b8b70a6ad375ad077f4a9ff2492508ea3816a2b403819", size = 10968968, upload-time = "2026-04-28T10:45:30.26Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/36/69/64c76aabc1bc70c7f24b686cd93c3407f8ea430905e395f59bf9603ef571/ty-0.0.33-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:1d732facf39fcb221ba279d469c5040d37883e964f123b1563888efd34818180", size = 11458077, upload-time = "2026-04-28T10:45:45.971Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/91/84/fae27b0c4718776a298690d31ca4cc1995f2e3e1c63a7b59e84c41498e9a/ty-0.0.33-py3-none-win32.whl", hash = "sha256:d90960b574428dc252f85e8598ec5fcb7f619794196b2fc95a90da075ed4681c", size = 10345364, upload-time = "2026-04-28T10:45:16.836Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3c/a0/a2938b23ae3e1a09a2d7c189e2ac5f7113676bae4e0e23948b568e18e5f8/ty-0.0.33-py3-none-win_amd64.whl", hash = "sha256:c1c3aec62c44de610c6e95f0a4e97ac3dbc07934bfdbf1fd90d758c9ff72f48e", size = 11342470, upload-time = "2026-04-28T10:45:26.455Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ab/62/7fb948aace38d2f6329261bb33c035a8484549c74f1db28649c7a4c6fed9/ty-0.0.33-py3-none-win_arm64.whl", hash = "sha256:0d44f99ba1b441e55e2aa301b2ac0a21112784931b46a5f66f4ea9efe5620d97", size = 10742673, upload-time = "2026-04-28T10:45:35.555Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "watchfiles"
|
||||
version = "1.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/cd/41/5e1a4bb12aac5f1493fa1bdc11154eca3b258ca4eba65d39c473fe19d8e9/watchfiles-1.2.0.tar.gz", hash = "sha256:c995fba777f1ea992f090f9236e9284cf7a5d1a0130dd5a3d82c598cacd76838", size = 108252, upload-time = "2026-05-18T04:32:04.251Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/54/a9c7ea9a82a4ac65e7004c0a03920b5cdd2f9c3b678757d9cd425aa51d53/watchfiles-1.2.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b8c8358484d5fa12ef34f05b7f4168eaf1932f408725ff6d023c33ec17bd79d4", size = 400205, upload-time = "2026-05-18T04:32:05.153Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/aa/5d/c9ab3534374a4a67450696905d6ef16a04405448b8dc52bd752ae50423d4/watchfiles-1.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:9f04b092229ad2c50126dd3c922c8822e51e605993764a33058d4a791ab42281", size = 392508, upload-time = "2026-05-18T04:30:54.849Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/26/ca/1ad30103535cf0cecd7b993e8d50edc5351b1820e38f2d22e3df58962feb/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7a7ce236284f002a156f70add88efe5c70879cccbb658be0822c54b1306fc09d", size = 452448, upload-time = "2026-05-18T04:30:53.727Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/a1/ceee2cdf2afbd715fa07758d39c9859513eae411b23196f7fd039e5feedd/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b9909cc2b48468b575eefa944919e1fe8a36c5849d5c7c168f80a8c1db69398e", size = 459605, upload-time = "2026-05-18T04:30:23.312Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e8/f6/421e30fd1cb3907a84ed92ab3f1983e37ba2dca015e9a894a048418417a2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0a37faaed405c67e28e6be45a1fa4f206ef5a2860f27c237db9fa30704c38242", size = 490757, upload-time = "2026-05-18T04:30:47.358Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/b0/55ed1b97ed08be7bba6f9a541cac15f2a858e1d74d2b07b6da70a82aab00/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9649193aa27bd9ff2e80ff29bfaa93085496c7a3a377592823cc58b77ee88add", size = 568672, upload-time = "2026-05-18T04:30:38.915Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/cf/d8ae8a80dd7bafab395ea7681c10237311bbf34d37704a8c744e7cf31fc7/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4e4ff8e37f99cf1da89e255e07c9c4b37c214038c4283707bdec308cb1b0ea1f", size = 464197, upload-time = "2026-05-18T04:30:09.914Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7c/8a/3076c496ca8dafe0e8cd03fcebdfc47be4b1174b4e5b24ff6e396e6b3af2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:054dc20fd2e3132b4c3883b4a00d72fd6e1f56fdaf89fccd12e8057d74cd74d7", size = 453181, upload-time = "2026-05-18T04:30:14.829Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e5/10/9745e17c98e7b8a86454df0a3c7b5686bd650383f1e9f26e4ebcbd6cc0c0/watchfiles-1.2.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:e140ed30ebde76796b686e67c182cff10ea2fbab186fafd1560f74bb5a473a6e", size = 465109, upload-time = "2026-05-18T04:30:28.123Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/95/8ef4a95481d3e0cb52d62a06fa6e972e81424be2d9698b91a2fecca9904c/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:bb7e52ecf68ba46d22df23467b87cffeb2146908aa523ebfe803019618cfda06", size = 630653, upload-time = "2026-05-18T04:31:49.304Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/e4/3b3bf36b0f829b50c6ebcb8d031583863c59f923d6a6af3d485e470d0fac/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:23282a321c8baf9b3a3c4afff673f9fe65eb7fdc2338d765ccad9d3d1916a5ba", size = 657838, upload-time = "2026-05-18T04:31:06.497Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/21/b1/6cbbb50c1f3002ab568777d44aa21206dfb8807a840990c4037523b51812/watchfiles-1.2.0-cp314-cp314-win32.whl", hash = "sha256:c0db965c5f79aa49fe672d297cf1febc5ad149b658594944f49a54a2b96270a7", size = 275108, upload-time = "2026-05-18T04:30:06.891Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/45/190ce6db8dcb4536682cf75d3889ff1a27182a58cb519d343cb6d9ea63d8/watchfiles-1.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:71283b39fd17e5408eb123bd37aeecfd9d54c81fc184421943208aadb879d103", size = 288441, upload-time = "2026-05-18T04:32:12.901Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/74/0d/3eae1c2313ab08378431d907c3f8095ecca00f3eda33111cf4f0f2591799/watchfiles-1.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:c5c19526f4e54a00f2666a6c0e9e40d582c09e865055ea7378bf0009aab857b3", size = 280684, upload-time = "2026-05-18T04:31:26.902Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/75/fb64e6c25d6b5ca636d03df34ffb1c6e9873303e76d27967e045f8df088f/watchfiles-1.2.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:d73a585accffa5ae39c17264c36ec3166d2fad7000c780f5ef83b2722afb9dd2", size = 398857, upload-time = "2026-05-18T04:32:17.108Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/73/4e/9f7adf01754cbf81843722ccfec169d8f26c69778281a302855cecd2ee08/watchfiles-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ae99b14c5f21e026e0e9d96f40e07d8570ebee6cafd9d8fc318354606daa7a28", size = 392413, upload-time = "2026-05-18T04:31:07.911Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/c8/bec626bcc2d69f44b9acb24ce7d60ed7b16b73628eea747fcbd169d8edda/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4429f3b105524a10b72c3a819b091c495d2811d419c1e1e8df773a5a5974f831", size = 452409, upload-time = "2026-05-18T04:31:20.142Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/b7/b6362068e81e7c556d155a34c35d40ac3ef42d747b06d7f6e5bf58e359c2/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:43d818978d06062d9b22c4fab2ebe44cf5213d42dc8e62bda8c2760cfa2eeb33", size = 458827, upload-time = "2026-05-18T04:32:06.219Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/67/f8/9a813fa42afb1e0b4625e75f0479826644d3ee8dc287e093799bc01f390c/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b9f732dc58b2dbe69e464ccf8fff7a03b0dd0be439da4c0720d3558527d3d6b4", size = 490104, upload-time = "2026-05-18T04:31:56.034Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2f/bf/27dfb6094ca4c9aad21298b5525b6c53cb36121ee454331d05161e58d130/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8f200104103feb097de4cab8fe4f5dd18a2026934c7dea98c55a2f5fd6d5a33b", size = 571360, upload-time = "2026-05-18T04:31:57.133Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/39/44a096d67270ea93df91d33877dbe91fbda3aa4f8ec2edf799d93eda8736/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:63ac26eefbf4af1741247d6fb68b11c49a25b2f7413fbd318a83a12aaa9cf666", size = 464644, upload-time = "2026-05-18T04:30:57.33Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0e/80/c7472203bad6268e3ef1ad260739704847898938ad7ea8b63a5131f46b50/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c4997d4e4a55f0d02b6cde327322daf3a0400e5df6c6b15948994bf72497925", size = 454771, upload-time = "2026-05-18T04:30:48.736Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/51/cf/3b10b268b4b7f0fc26e9debb5eef1998b515887840f444cd3ec80c688755/watchfiles-1.2.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:4c887eba18b7945ac73067a8b4a66f21cd46c2539b2bc68588f7be6c7eb6d26b", size = 463494, upload-time = "2026-05-18T04:31:33.826Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/3e/a4302545cd589262a0dc7d140e86f7688eba3f9c72776c27f7e23b8864c4/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:3416ff151bb6b5a8d8d11664974fbef4d9305b9b2957839ab5a270468fd8df30", size = 629383, upload-time = "2026-05-18T04:31:15.596Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/db/99/d5649df0a9a410d45b7c882304d0b790903ac9b6e8f2cfd12114e0c6b9f2/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:0e831a271c035d89789cffc386b6aa1375f39f1cd25eb7ca0997e4970d152fc5", size = 656093, upload-time = "2026-05-18T04:31:58.707Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/b9/362702539275019a54dd2e94511b31a9b89c5f9e6a21966de7eb692549fc/watchfiles-1.2.0-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:37a6721cdf3f65dbb13aa9503510ccb4451603ac837e44d265d7992a597e1374", size = 400109, upload-time = "2026-05-18T04:31:16.879Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/75/71d5ba62db781e5587bded1d944c675374bc4aa37ff33d5018d98e8b6538/watchfiles-1.2.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2b37d10b5a63bd4d87e18472d80fa525bd670586fae62e5dd580452764879b65", size = 392167, upload-time = "2026-05-18T04:31:28.058Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3c/01/c66dd95d0423fe30d31820e2d1d5bda773764131bbb6ac0cb1cf303ac328/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0a105bc2283f67e8fbec74253ec2d94925de92ed72c0393f1206bf326b7b7b69", size = 452372, upload-time = "2026-05-18T04:31:00.836Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/91/15/2fe99557e72f85627c6a8eed50d889e8d101623e060a22ad75b875cb932d/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5327989a465505f05cfe06f04fa9d0c2fd5432bb243e10e6f012b1bdca3c8579", size = 459596, upload-time = "2026-05-18T04:31:34.96Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ed/23/d4acfa0023367428ed48351b3b9b267893037b6cadae55620c61c24bcfd4/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ecb47f183a8025b2aa18b546725c3657e542112ae9c0613a2af79b4fa8d04ad7", size = 490869, upload-time = "2026-05-18T04:31:59.923Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a4/5f/3164cbdce06c9fb95c4f7b9e2f9760b5e2797af43a9ecc317ef42a23a278/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8520a4ab0e37f770afc34459c4f8f7019e153f9124dc101c15538365875d1ab2", size = 571641, upload-time = "2026-05-18T04:32:00.948Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/e6/85d3731c55e65cd7690f3f803d24c139588aaf863e4bf2148fe7a7fa1a19/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:71cd71740ed2c15211ebb237ced4e39a1cdf6f80566e5fe95428da1626f4fde6", size = 464444, upload-time = "2026-05-18T04:30:34.298Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7d/562641012b8b09872742c3b8adf9629ec479fd78f8d68ae4a0c13da8add6/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f88af53d6ddaf72179ef613ddc905e6f4785f712b49b80b3bef9f3525e6194b4", size = 453593, upload-time = "2026-05-18T04:31:23.464Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/56/fe/cb8ef3d6f929d14158fdaaad9925985b7310abc9384dcd4d82dd0016fb59/watchfiles-1.2.0-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:cee9d5efd929efdac5f7e58f72b3376f676b64050a91c5b99a7094c5b2317488", size = 465096, upload-time = "2026-05-18T04:31:30.384Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/25/91/80908e835e100527a9267147b08c0eee1fa6ab0ffec15edc04d1d44885f7/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_aarch64.whl", hash = "sha256:b718bf356bbc15e559bd8ef41782b573b8ae0e3f177ab244b440568d7ea02cfb", size = 630638, upload-time = "2026-05-18T04:30:49.89Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/4b/95ab2f256bb4af3cb2eb23b9317bda984ee6e0f11733a5c004a6c95b06e3/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_x86_64.whl", hash = "sha256:922c0e019fe68b3ae392965a766b02a71ba1168c932cebc3733cd52c5fe5b377", size = 657684, upload-time = "2026-05-18T04:31:32.027Z" },
|
||||
]
|
||||
@@ -0,0 +1,945 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build a single-page HTML site from README.md for the awesome-python website."""
|
||||
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import xml.etree.ElementTree as ET
|
||||
from collections import Counter
|
||||
from collections.abc import Sequence
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import TypedDict
|
||||
|
||||
from fetch_pypi_downloads_via_clickpy import OVERRIDES_FILE, normalize
|
||||
from jinja2 import Environment, FileSystemLoader
|
||||
from markdown_it import MarkdownIt
|
||||
from markdown_it.tree import SyntaxTreeNode
|
||||
from readme_parser import AlsoSee, ParsedGroup, ParsedSection, parse_readme, parse_sponsors, render_inline_text, slugify
|
||||
|
||||
GITHUB_REPO_URL_RE = re.compile(r"^https?://github\.com/([^/]+/[^/]+?)(?:\.git)?/?$")
|
||||
MARKDOWN_LINK_RE = re.compile(r"\[([^\]]+)\]\(([^)\s]+)\)")
|
||||
BULLET_LINE_RE = re.compile(r"^\s*-\s")
|
||||
ANCHOR_LINK_ATTRS_RE = re.compile(r'href="(#[^"]*)" target="_blank" rel="noopener"')
|
||||
SITE_URL = "https://awesome-python.com/"
|
||||
SITEMAP_URL = f"{SITE_URL}sitemap.xml"
|
||||
SITEMAP_NS = "http://www.sitemaps.org/schemas/sitemap/0.9"
|
||||
PLURAL_NOUNS = {"Clients", "Drivers", "Engines", "Files", "Frameworks", "Generators", "Implementations", "Panels", "Queues", "Repositories", "Schedulers", "Servers", "Stdlib", "Tools"}
|
||||
|
||||
BUILTIN_FILTER = "Stdlib"
|
||||
BUILTIN_SLUG = "built-in"
|
||||
BUILTIN_PATH = f"/categories/{BUILTIN_SLUG}/"
|
||||
BUILTIN_PUBLIC_URL = f"{SITE_URL}categories/{BUILTIN_SLUG}/"
|
||||
|
||||
SPONSORSHIP_PATH = "/sponsorship/"
|
||||
SPONSORSHIP_PUBLIC_URL = f"{SITE_URL}sponsorship/"
|
||||
SPONSORSHIP_DESCRIPTION = "Sponsorship for awesome-python: tiers, audience, and how to get your product in front of professional Python developers evaluating tools for production use."
|
||||
|
||||
BUNDLED_PREFIX_RE = re.compile(r"^\(part of ")
|
||||
|
||||
SOURCE_TYPE_DOMAINS = {
|
||||
"docs.python.org": BUILTIN_FILTER,
|
||||
"gitlab.com": "GitLab",
|
||||
"bitbucket.org": "Bitbucket",
|
||||
}
|
||||
|
||||
|
||||
class TemplateSubcategory(TypedDict):
|
||||
name: str
|
||||
value: str
|
||||
slug: str
|
||||
url: str
|
||||
|
||||
|
||||
class TemplateEntry(TypedDict):
|
||||
name: str
|
||||
url: str
|
||||
description: str
|
||||
categories: list[str]
|
||||
groups: list[str]
|
||||
subcategories: list[TemplateSubcategory]
|
||||
stars: int | None
|
||||
downloads: int | None
|
||||
owner: str | None
|
||||
last_commit_at: str | None
|
||||
source_type: str | None
|
||||
bundled: bool
|
||||
badge: str | None
|
||||
also_see: list[AlsoSee]
|
||||
|
||||
|
||||
class EntryGroup(TypedDict):
|
||||
name: str # empty for a page with a single unnamed group
|
||||
slug: str
|
||||
url: str # links the group heading to its own page, empty when it has none
|
||||
lead: str # the section intro's first paragraph as HTML, shown under the heading on group pages
|
||||
entries: list[TemplateEntry]
|
||||
|
||||
|
||||
class SyntheticCategory(TypedDict):
|
||||
name: str
|
||||
slug: str
|
||||
description: str
|
||||
description_html: str
|
||||
|
||||
|
||||
TemplateCategory = ParsedSection | SyntheticCategory
|
||||
|
||||
|
||||
def detect_source_type(url: str) -> str | None:
|
||||
"""Detect source type from URL domain. Returns None for GitHub URLs."""
|
||||
if GITHUB_REPO_URL_RE.match(url):
|
||||
return None
|
||||
for domain, source_type in SOURCE_TYPE_DOMAINS.items():
|
||||
if domain in url:
|
||||
return source_type
|
||||
if "github.com" not in url:
|
||||
return "External"
|
||||
return None
|
||||
|
||||
|
||||
def extract_github_repo(url: str) -> str | None:
|
||||
"""Extract owner/repo from a GitHub repo URL. Returns None for non-GitHub URLs."""
|
||||
m = GITHUB_REPO_URL_RE.match(url)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def load_stars(path: Path) -> dict[str, dict]:
|
||||
"""Load star data from JSON. Returns empty dict if file doesn't exist or is corrupt."""
|
||||
if path.exists():
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
return {}
|
||||
return {}
|
||||
|
||||
|
||||
def load_pypi_badges() -> dict[str, str]:
|
||||
"""Load the badge each entry shows in place of a download count.
|
||||
|
||||
Keyed by normalized README name. Only entries whose missing count needs
|
||||
explaining beyond "Not on PyPI" carry one, e.g. the SDK monorepos that do
|
||||
ship on PyPI, just as many per-service packages.
|
||||
"""
|
||||
raw = json.loads(OVERRIDES_FILE.read_text())
|
||||
return {name: entry["badge"] for name, entry in raw.items() if entry.get("badge")}
|
||||
|
||||
|
||||
def load_downloads(path: Path) -> dict[str, int]:
|
||||
"""Load last-30-day download counts from the TSV cache, keyed by normalized README name.
|
||||
|
||||
Columns: name, package, downloads, fetched_at. Skips the header and
|
||||
NOT_FOUND rows. Returns empty dict if the file doesn't exist.
|
||||
"""
|
||||
if not path.exists():
|
||||
return {}
|
||||
downloads: dict[str, int] = {}
|
||||
for line in path.read_text(encoding="utf-8").splitlines()[1:]:
|
||||
parts = line.split("\t")
|
||||
if len(parts) >= 3 and parts[2].isdigit():
|
||||
downloads[parts[0]] = int(parts[2])
|
||||
return downloads
|
||||
|
||||
|
||||
def sort_entries(entries: Sequence[TemplateEntry]) -> list[TemplateEntry]:
|
||||
"""Sort entries by downloads descending, then name ascending.
|
||||
|
||||
Four tiers: entries with download counts first, then starred entries by
|
||||
stars descending, then stdlib, then the rest.
|
||||
"""
|
||||
|
||||
def sort_key(entry: TemplateEntry) -> tuple[int, int, int, str]:
|
||||
downloads = entry["downloads"]
|
||||
stars = entry["stars"]
|
||||
name = entry["name"].lower()
|
||||
if downloads is not None:
|
||||
return (0, -downloads, 0, name)
|
||||
if stars is not None:
|
||||
builtin = 1 if entry.get("source_type") == BUILTIN_FILTER else 0
|
||||
return (1, -stars, builtin, name)
|
||||
if entry.get("source_type") == BUILTIN_FILTER:
|
||||
return (2, 0, 0, name)
|
||||
return (3, 0, 0, name)
|
||||
|
||||
return sorted(entries, key=sort_key)
|
||||
|
||||
|
||||
def build_robots_txt() -> str:
|
||||
return f"User-agent: *\nContent-Signal: search=yes, ai-input=yes, ai-train=yes\nAllow: /\n\nSitemap: {SITEMAP_URL}\n"
|
||||
|
||||
|
||||
WEBSITE_ID = f"{SITE_URL}#website"
|
||||
ISPARTOF_WEBSITE = {"@type": "WebSite", "@id": WEBSITE_ID}
|
||||
|
||||
|
||||
def _website_node() -> dict:
|
||||
return {
|
||||
"@type": "WebSite",
|
||||
"@id": WEBSITE_ID,
|
||||
"name": "Awesome Python",
|
||||
"url": SITE_URL,
|
||||
"inLanguage": "en",
|
||||
"sameAs": "https://github.com/vinta/awesome-python",
|
||||
}
|
||||
|
||||
|
||||
def _item_list_payload(entries: Sequence[TemplateEntry]) -> dict:
|
||||
return {
|
||||
"@type": "ItemList",
|
||||
"numberOfItems": len(entries),
|
||||
"itemListElement": [
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": i,
|
||||
"name": entry["name"],
|
||||
"url": entry["url"],
|
||||
}
|
||||
for i, entry in enumerate(entries, start=1)
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def build_homepage_json_ld(entries: Sequence[TemplateEntry], total_categories: int) -> dict:
|
||||
description = (
|
||||
"An opinionated guide to the best Python frameworks, libraries, and tools. "
|
||||
f"Explore {len(entries)} curated projects across {total_categories} categories, "
|
||||
"from AI and agents to data science and web development."
|
||||
)
|
||||
return {
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
_website_node(),
|
||||
{
|
||||
"@type": "CollectionPage",
|
||||
"@id": SITE_URL,
|
||||
"name": "Awesome Python",
|
||||
"url": SITE_URL,
|
||||
"description": description,
|
||||
"isPartOf": ISPARTOF_WEBSITE,
|
||||
"inLanguage": "en",
|
||||
"mainEntity": _item_list_payload(entries),
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def category_meta_title(name: str, parent_name: str | None = None) -> str:
|
||||
if parent_name:
|
||||
# Lead with the section's title so the page matches searches like "python jwt library".
|
||||
title = f"{category_meta_title(parent_name).removesuffix(' - Awesome Python')}: {name}"
|
||||
# Google shows the site name above each result, so a long title drops it rather than get cut off.
|
||||
return f"{title} - Awesome Python" if len(title) <= 43 else title
|
||||
# Names ending in one of these nouns already say what the entries are.
|
||||
noun = "" if name.rsplit(" ", 1)[-1] in PLURAL_NOUNS else " Libraries"
|
||||
prefix = "" if name.startswith("Python ") else "Python "
|
||||
title = f"{prefix}{name}{noun} - Awesome Python"
|
||||
if len(title) <= 60:
|
||||
return title
|
||||
return f"{name} - Awesome Python"
|
||||
|
||||
|
||||
def category_meta_description(name: str, entry_count: int, description: str) -> str:
|
||||
count_sentence = f"Explore {entry_count} curated Python project{'s' if entry_count != 1 else ''} in {name}."
|
||||
if description:
|
||||
lead = description if description.endswith((".", "!", "?")) else f"{description}."
|
||||
return f"{lead} {count_sentence}"
|
||||
return f"{count_sentence} Part of the Awesome Python catalog."
|
||||
|
||||
|
||||
def subcategory_meta_description(name: str, parent_name: str, entry_names: Sequence[str]) -> str:
|
||||
names = entry_names[0] if len(entry_names) == 1 else f"{', '.join(entry_names[:-1])}{',' if len(entry_names) > 2 else ''} and {entry_names[-1]}"
|
||||
return f"The {name} picks in Awesome Python's {parent_name} list: {names}."
|
||||
|
||||
|
||||
def load_category_intro(path: Path) -> tuple[str, str, str, str]:
|
||||
"""Render a category intro file to HTML, split at the end of its "How to choose:" list.
|
||||
|
||||
Returns the part shown above the table, the guide shown below it, and the
|
||||
first paragraph as plain text for the meta description and as HTML for
|
||||
group pages. A file without the list keeps everything above the table.
|
||||
Returns empty strings if the category has no intro file.
|
||||
"""
|
||||
if not path.exists():
|
||||
return "", "", "", ""
|
||||
md = MarkdownIt("commonmark")
|
||||
tokens = md.parse(path.read_text(encoding="utf-8"))
|
||||
for token in tokens:
|
||||
for child in token.children or []:
|
||||
if child.type == "link_open":
|
||||
child.attrSet("target", "_blank")
|
||||
child.attrSet("rel", "noopener")
|
||||
lead = next(node for node in SyntaxTreeNode(tokens).children if node.type == "paragraph")
|
||||
split_at = len(tokens)
|
||||
for i, token in enumerate(tokens):
|
||||
if token.type == "inline" and token.level == 1 and token.content == "How to choose:" and i + 2 < len(tokens) and tokens[i + 2].type == "bullet_list_open":
|
||||
split_at = next(j for j in range(i + 3, len(tokens)) if tokens[j].type == "bullet_list_close" and tokens[j].level == 0) + 1
|
||||
break
|
||||
render = md.renderer.render
|
||||
return (
|
||||
render(tokens[:split_at], md.options, {}),
|
||||
render(tokens[split_at:], md.options, {}),
|
||||
render_inline_text(lead.children[0].children),
|
||||
render(lead.to_tokens(), md.options, {}),
|
||||
)
|
||||
|
||||
|
||||
def group_section_entries(section: ParsedSection, entries_by_key: dict[tuple[str, str], TemplateEntry]) -> list[EntryGroup]:
|
||||
"""Group a section's entries by use case (subcategory), both in README order."""
|
||||
groups: dict[str, EntryGroup] = {}
|
||||
for parsed in section["entries"]:
|
||||
name = parsed["subcategory"]
|
||||
slug = slugify(name) if name else ""
|
||||
group = groups.setdefault(name, EntryGroup(name=name, slug=slug, url=subcategory_path(section["slug"], slug) if name else "", lead="", entries=[]))
|
||||
group["entries"].append(entries_by_key[(parsed["url"], parsed["name"])])
|
||||
return list(groups.values())
|
||||
|
||||
|
||||
def group_entries_by_section(sections: Sequence[ParsedSection], entries_by_key: dict[tuple[str, str], TemplateEntry], intros_dir: Path) -> list[EntryGroup]:
|
||||
"""Group a thematic group's entries by section, both in README order, listing each entry once."""
|
||||
placed: set[tuple[str, str]] = set()
|
||||
groups: list[EntryGroup] = []
|
||||
for section in sections:
|
||||
entries: list[TemplateEntry] = []
|
||||
for parsed in section["entries"]:
|
||||
key = (parsed["url"], parsed["name"])
|
||||
if key not in placed:
|
||||
placed.add(key)
|
||||
entries.append(entries_by_key[key])
|
||||
groups.append(EntryGroup(name=section["name"], slug=section["slug"], url=category_path(section), lead=load_category_intro(intros_dir / f"{section['slug']}.md")[3], entries=entries))
|
||||
return groups
|
||||
|
||||
|
||||
def build_breadcrumb_json_ld(items: Sequence[tuple[str, str]]) -> dict:
|
||||
return {
|
||||
"@type": "BreadcrumbList",
|
||||
"itemListElement": [
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": i,
|
||||
"name": name,
|
||||
"item": url,
|
||||
}
|
||||
for i, (name, url) in enumerate(items, start=1)
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def build_category_json_ld(
|
||||
name: str,
|
||||
url: str,
|
||||
description: str,
|
||||
entries: Sequence[TemplateEntry],
|
||||
breadcrumbs: Sequence[tuple[str, str]],
|
||||
) -> dict:
|
||||
return {
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
_website_node(),
|
||||
{
|
||||
"@type": "CollectionPage",
|
||||
"@id": url,
|
||||
"name": name,
|
||||
"url": url,
|
||||
"description": description,
|
||||
"isPartOf": ISPARTOF_WEBSITE,
|
||||
"inLanguage": "en",
|
||||
"mainEntity": _item_list_payload(entries),
|
||||
},
|
||||
build_breadcrumb_json_ld(breadcrumbs),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def build_sponsorship_json_ld() -> dict:
|
||||
return {
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
_website_node(),
|
||||
{
|
||||
"@type": "WebPage",
|
||||
"@id": SPONSORSHIP_PUBLIC_URL,
|
||||
"name": "Sponsor Awesome Python",
|
||||
"url": SPONSORSHIP_PUBLIC_URL,
|
||||
"description": SPONSORSHIP_DESCRIPTION,
|
||||
"isPartOf": ISPARTOF_WEBSITE,
|
||||
"inLanguage": "en",
|
||||
},
|
||||
build_breadcrumb_json_ld(
|
||||
[
|
||||
("Awesome Python", SITE_URL),
|
||||
("Sponsorship", SPONSORSHIP_PUBLIC_URL),
|
||||
]
|
||||
),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def category_path(category: ParsedSection) -> str:
|
||||
return f"/categories/{category['slug']}/"
|
||||
|
||||
|
||||
def category_public_url(category: ParsedSection) -> str:
|
||||
return f"{SITE_URL}categories/{category['slug']}/"
|
||||
|
||||
|
||||
def group_path(group_slug: str) -> str:
|
||||
return f"/categories/{group_slug}/"
|
||||
|
||||
|
||||
def group_public_url(group_slug: str) -> str:
|
||||
return f"{SITE_URL}categories/{group_slug}/"
|
||||
|
||||
|
||||
def subcategory_path(category_slug: str, subcategory_slug: str) -> str:
|
||||
return f"/categories/{category_slug}/{subcategory_slug}/"
|
||||
|
||||
|
||||
def subcategory_public_url(category_slug: str, subcategory_slug: str) -> str:
|
||||
return f"{SITE_URL}categories/{category_slug}/{subcategory_slug}/"
|
||||
|
||||
|
||||
def synthetic_category(name: str, slug: str) -> SyntheticCategory:
|
||||
return {"name": name, "slug": slug, "description": "", "description_html": ""}
|
||||
|
||||
|
||||
def git_last_change_date(repo_root: Path, *log_args: str) -> str:
|
||||
"""Return the date of the last commit matching `git log` args, or "" without git history."""
|
||||
result = subprocess.run(["git", "log", "-1", "--format=%cs", "--no-patch", *log_args], cwd=repo_root, capture_output=True, text=True)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def section_line_range(readme_text: str, name: str) -> str:
|
||||
"""Return a section's README lines as a `git log -L` range, from its heading to the line before the next heading."""
|
||||
lines = readme_text.split("\n")
|
||||
start = next(i for i, line in enumerate(lines) if re.fullmatch(rf"#+ {re.escape(name)}", line))
|
||||
end = next((i for i in range(start + 1, len(lines)) if lines[i].startswith("#")), len(lines))
|
||||
return f"{start + 1},{end}:README.md"
|
||||
|
||||
|
||||
def write_sitemap_xml(path: Path, urls: Sequence[tuple[str, str]]) -> None:
|
||||
ET.register_namespace("", SITEMAP_NS)
|
||||
urlset = ET.Element(f"{{{SITEMAP_NS}}}urlset")
|
||||
for url, lastmod in urls:
|
||||
url_el = ET.SubElement(urlset, f"{{{SITEMAP_NS}}}url")
|
||||
loc_el = ET.SubElement(url_el, f"{{{SITEMAP_NS}}}loc")
|
||||
loc_el.text = url
|
||||
lastmod_el = ET.SubElement(url_el, f"{{{SITEMAP_NS}}}lastmod")
|
||||
lastmod_el.text = lastmod
|
||||
|
||||
tree = ET.ElementTree(urlset)
|
||||
ET.indent(tree, space=" ")
|
||||
tree.write(path, encoding="utf-8", xml_declaration=True)
|
||||
with path.open("ab") as f:
|
||||
f.write(b"\n")
|
||||
|
||||
|
||||
def top_level_heading_text(line: str) -> str | None:
|
||||
stripped = line.strip()
|
||||
match = re.match(r"^(#{1,2})\s+(.+)$", stripped)
|
||||
if match is None:
|
||||
return None
|
||||
return match.group(2).strip().strip("#").strip().strip("*").strip()
|
||||
|
||||
|
||||
def extract_categories_body(markdown: str) -> str:
|
||||
"""Return content from `Categories` through `Projects`, excluding later sections."""
|
||||
lines = markdown.splitlines(keepends=True)
|
||||
start_idx = None
|
||||
end_idx = len(lines)
|
||||
for i, line in enumerate(lines):
|
||||
heading = top_level_heading_text(line)
|
||||
if heading is None:
|
||||
continue
|
||||
if start_idx is None and heading.lower() == "categories":
|
||||
start_idx = i + 1
|
||||
while start_idx < len(lines) and lines[start_idx].strip() == "":
|
||||
start_idx += 1
|
||||
elif start_idx is not None and heading.lower() in ("resources", "contributing"):
|
||||
end_idx = i
|
||||
break
|
||||
if start_idx is None:
|
||||
return ""
|
||||
return "".join(lines[start_idx:end_idx]).rstrip() + "\n"
|
||||
|
||||
|
||||
def github_markdown_anchor(text: str) -> str:
|
||||
anchor = text.strip().lower()
|
||||
anchor = re.sub(r"[^\w\s-]", "", anchor)
|
||||
anchor = re.sub(r"\s", "-", anchor)
|
||||
return f"#{anchor}"
|
||||
|
||||
|
||||
def link_llms_category_index_to_canonical_pages(markdown: str, categories: Sequence[ParsedSection]) -> str:
|
||||
"""Point the README-derived category index at canonical category pages."""
|
||||
category_urls = {}
|
||||
for category in categories:
|
||||
public_url = category_public_url(category)
|
||||
category_urls[f"#{category['slug']}"] = public_url
|
||||
category_urls[github_markdown_anchor(category["name"])] = public_url
|
||||
lines = markdown.splitlines(keepends=True)
|
||||
out: list[str] = []
|
||||
|
||||
def replace_link(match: re.Match[str]) -> str:
|
||||
target = match.group(2)
|
||||
url = category_urls.get(target)
|
||||
if url is None:
|
||||
return match.group(0)
|
||||
return match.group(0).replace(f"({target})", f"({url})", 1)
|
||||
|
||||
for line in lines:
|
||||
out.append(MARKDOWN_LINK_RE.sub(replace_link, line))
|
||||
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def link_description_anchors_to_category_pages(categories: Sequence[ParsedSection]) -> None:
|
||||
"""Point README anchor links in section descriptions at category pages, which lack those anchors."""
|
||||
category_paths = {}
|
||||
for category in categories:
|
||||
category_paths[f"#{category['slug']}"] = category_path(category)
|
||||
category_paths[github_markdown_anchor(category["name"])] = category_path(category)
|
||||
|
||||
def replace_attrs(match: re.Match[str]) -> str:
|
||||
path = category_paths.get(match.group(1))
|
||||
return f'href="{path}"' if path else match.group(0)
|
||||
|
||||
for category in categories:
|
||||
category["description_html"] = ANCHOR_LINK_ATTRS_RE.sub(replace_attrs, category["description_html"])
|
||||
|
||||
|
||||
def build_llms_txt(
|
||||
template_text: str,
|
||||
*,
|
||||
readme_text: str,
|
||||
subtitle: str,
|
||||
build_date: str,
|
||||
stars_data: dict[str, dict],
|
||||
downloads_data: dict[str, int],
|
||||
categories: Sequence[ParsedSection],
|
||||
total_entries: int,
|
||||
) -> str:
|
||||
"""Render the llms.txt entry point with the curated category shortlist."""
|
||||
categories_md = annotate_entries_with_stats(
|
||||
link_llms_category_index_to_canonical_pages(
|
||||
extract_categories_body(readme_text).rstrip(),
|
||||
categories,
|
||||
),
|
||||
stars_data,
|
||||
downloads_data,
|
||||
)
|
||||
text_env = Environment(autoescape=False, trim_blocks=True, lstrip_blocks=True)
|
||||
rendered = text_env.from_string(template_text).render(
|
||||
subtitle=subtitle,
|
||||
build_date=build_date,
|
||||
site_url=SITE_URL,
|
||||
github_repo_url="https://github.com/vinta/awesome-python",
|
||||
contributing_url="https://github.com/vinta/awesome-python/blob/master/CONTRIBUTING.md",
|
||||
sponsorship_url=SPONSORSHIP_PUBLIC_URL,
|
||||
sitemap_url=SITEMAP_URL,
|
||||
categories_md=categories_md,
|
||||
total_entries=total_entries,
|
||||
total_categories=len(categories),
|
||||
)
|
||||
return rendered.rstrip() + "\n"
|
||||
|
||||
|
||||
def annotate_entries_with_stats(
|
||||
markdown: str,
|
||||
stars_data: dict[str, dict],
|
||||
downloads_data: dict[str, int],
|
||||
) -> str:
|
||||
"""Append download and star counts to bullet entry lines.
|
||||
|
||||
Downloads are looked up by the first link's display name (the same key the
|
||||
TSV cache uses); stars by the first GitHub link with known star data.
|
||||
"""
|
||||
lines = markdown.splitlines(keepends=True)
|
||||
out: list[str] = []
|
||||
for line in lines:
|
||||
if not BULLET_LINE_RE.match(line):
|
||||
out.append(line)
|
||||
continue
|
||||
links = MARKDOWN_LINK_RE.findall(line)
|
||||
parts: list[str] = []
|
||||
if links:
|
||||
name, url = links[0]
|
||||
# Category-index bullets link into the site itself, and stdlib entries
|
||||
# would hit same-named PyPI backports (e.g. logging) — no counts for either.
|
||||
if not url.startswith((SITE_URL, "#")) and detect_source_type(url) != BUILTIN_FILTER:
|
||||
downloads = downloads_data.get(normalize(name))
|
||||
if downloads is not None:
|
||||
parts.append(f"PyPI downloads/month: {downloads}")
|
||||
for _, url in links:
|
||||
repo_key = extract_github_repo(url)
|
||||
if not repo_key:
|
||||
continue
|
||||
entry = stars_data.get(repo_key)
|
||||
if entry and "stars" in entry:
|
||||
parts.append(f"GitHub stars: {entry['stars']}")
|
||||
break
|
||||
if not parts:
|
||||
out.append(line)
|
||||
continue
|
||||
stripped = line.rstrip("\n")
|
||||
ending = line[len(stripped) :]
|
||||
out.append(f"{stripped} ({', '.join(parts)}){ending}")
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def remove_sponsors_section(markdown: str) -> str:
|
||||
lines = markdown.splitlines(keepends=True)
|
||||
start_idx = None
|
||||
for i, line in enumerate(lines):
|
||||
heading = top_level_heading_text(line)
|
||||
if heading and heading.lower() == "sponsors":
|
||||
start_idx = i
|
||||
break
|
||||
|
||||
if start_idx is None:
|
||||
return markdown
|
||||
|
||||
end_idx = len(lines)
|
||||
for i, line in enumerate(lines[start_idx + 1 :], start=start_idx + 1):
|
||||
if top_level_heading_text(line):
|
||||
end_idx = i
|
||||
break
|
||||
|
||||
return "".join(lines[:start_idx] + lines[end_idx:])
|
||||
|
||||
|
||||
def extract_entries(
|
||||
categories: list[ParsedSection],
|
||||
groups: list[ParsedGroup],
|
||||
) -> list[TemplateEntry]:
|
||||
"""Flatten categories into individual library entries for table display.
|
||||
|
||||
Entries appearing in multiple categories are merged into a single entry
|
||||
with lists of categories and groups.
|
||||
"""
|
||||
cat_to_group = {cat["name"]: group["name"] for group in groups for cat in group["categories"]}
|
||||
|
||||
seen: dict[tuple[str, str], TemplateEntry] = {} # (url, name) -> entry
|
||||
entries: list[TemplateEntry] = []
|
||||
for cat in categories:
|
||||
group_name = cat_to_group.get(cat["name"], "Other")
|
||||
for entry in cat["entries"]:
|
||||
key = (entry["url"], entry["name"])
|
||||
existing = seen.get(key)
|
||||
if existing is None:
|
||||
existing = TemplateEntry(
|
||||
name=entry["name"],
|
||||
url=entry["url"],
|
||||
description=entry["description"],
|
||||
categories=[],
|
||||
groups=[],
|
||||
subcategories=[],
|
||||
stars=None,
|
||||
downloads=None,
|
||||
owner=None,
|
||||
last_commit_at=None,
|
||||
source_type=detect_source_type(entry["url"]),
|
||||
bundled=bool(BUNDLED_PREFIX_RE.match(entry["description"])),
|
||||
badge=None,
|
||||
also_see=entry["also_see"],
|
||||
)
|
||||
seen[key] = existing
|
||||
entries.append(existing)
|
||||
if cat["name"] not in existing["categories"]:
|
||||
existing["categories"].append(cat["name"])
|
||||
if group_name not in existing["groups"]:
|
||||
existing["groups"].append(group_name)
|
||||
subcat = entry["subcategory"]
|
||||
if subcat:
|
||||
scoped = f"{cat['name']} > {subcat}"
|
||||
if not any(s["value"] == scoped for s in existing["subcategories"]):
|
||||
sub_slug = slugify(subcat)
|
||||
existing["subcategories"].append(
|
||||
TemplateSubcategory(
|
||||
name=subcat,
|
||||
value=scoped,
|
||||
slug=sub_slug,
|
||||
url=f"/categories/{cat['slug']}/{sub_slug}/",
|
||||
)
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def build(repo_root: Path) -> None:
|
||||
"""Main build: parse README, render single-page HTML via Jinja2 templates."""
|
||||
website = repo_root / "website"
|
||||
readme_text = (repo_root / "README.md").read_text(encoding="utf-8")
|
||||
|
||||
subtitle = ""
|
||||
for line in readme_text.split("\n"):
|
||||
stripped = line.strip()
|
||||
if stripped and not stripped.startswith("#"):
|
||||
subtitle = stripped
|
||||
break
|
||||
|
||||
parsed_groups = parse_readme(readme_text)
|
||||
sponsors = parse_sponsors(readme_text)
|
||||
|
||||
categories = [cat for g in parsed_groups for cat in g["categories"]]
|
||||
cat_slugs = [cat["slug"] for cat in categories]
|
||||
group_slugs = [g["slug"] for g in parsed_groups]
|
||||
all_top_level_slugs = cat_slugs + group_slugs + [BUILTIN_SLUG]
|
||||
duplicates = {s for s, n in Counter(all_top_level_slugs).items() if n > 1}
|
||||
if duplicates:
|
||||
raise ValueError(f"slug collision in /categories/ namespace: {sorted(duplicates)}. Rename a category or group so their slugs differ.")
|
||||
link_description_anchors_to_category_pages(categories)
|
||||
total_entries = sum(c["entry_count"] for c in categories)
|
||||
entries = extract_entries(categories, parsed_groups)
|
||||
build_date = datetime.now(UTC)
|
||||
|
||||
stars_data = load_stars(website / "data" / "github_stars.json")
|
||||
downloads_data = load_downloads(website / "data" / "pypi_downloads.tsv")
|
||||
pypi_badges = load_pypi_badges()
|
||||
|
||||
repo_self = stars_data.get("vinta/awesome-python", {})
|
||||
repo_stars = None
|
||||
if "stars" in repo_self:
|
||||
stars_val = repo_self["stars"]
|
||||
repo_stars = f"{stars_val // 1000}k" if stars_val >= 1000 else str(stars_val)
|
||||
|
||||
for entry in entries:
|
||||
repo_key = extract_github_repo(entry["url"])
|
||||
if not repo_key and entry.get("source_type") == BUILTIN_FILTER:
|
||||
repo_key = "python/cpython"
|
||||
if repo_key and repo_key in stars_data:
|
||||
sd = stars_data[repo_key]
|
||||
entry["stars"] = sd["stars"]
|
||||
entry["owner"] = sd["owner"]
|
||||
entry["last_commit_at"] = sd.get("last_commit_at", "")
|
||||
# Stdlib entries would hit same-named PyPI backports (e.g. asyncio), not the stdlib.
|
||||
if entry.get("source_type") != BUILTIN_FILTER:
|
||||
entry["downloads"] = downloads_data.get(normalize(entry["name"]))
|
||||
entry["badge"] = pypi_badges.get(normalize(entry["name"]))
|
||||
|
||||
entries = sort_entries(entries)
|
||||
category_urls = {cat["name"]: category_path(cat) for cat in categories}
|
||||
|
||||
filter_urls: dict[str, str] = dict(category_urls)
|
||||
for group in parsed_groups:
|
||||
filter_urls[group["name"]] = group_path(group["slug"])
|
||||
builtin_entries = [e for e in entries if e.get("source_type") == BUILTIN_FILTER]
|
||||
|
||||
env = Environment(
|
||||
loader=FileSystemLoader(website / "templates"),
|
||||
autoescape=True,
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
)
|
||||
site_dir = website / "output"
|
||||
if site_dir.exists():
|
||||
shutil.rmtree(site_dir)
|
||||
site_dir.mkdir(parents=True)
|
||||
|
||||
homepage_json_ld = json.dumps(
|
||||
build_homepage_json_ld(entries, len(categories)),
|
||||
ensure_ascii=False,
|
||||
).replace("</", "<\\/")
|
||||
|
||||
tpl_index = env.get_template("index.html")
|
||||
(site_dir / "index.html").write_text(
|
||||
tpl_index.render(
|
||||
categories=categories,
|
||||
subtitle=subtitle,
|
||||
entries=entries,
|
||||
total_entries=total_entries,
|
||||
total_categories=len(categories),
|
||||
repo_stars=repo_stars,
|
||||
build_date=build_date.strftime("%B %d, %Y"),
|
||||
sponsors=sponsors,
|
||||
category_urls=category_urls,
|
||||
filter_urls=filter_urls,
|
||||
homepage_json_ld=homepage_json_ld,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
tpl_category = env.get_template("category.html")
|
||||
categories_dir = site_dir / "categories"
|
||||
|
||||
def render_category(
|
||||
category: TemplateCategory,
|
||||
*,
|
||||
category_url: str,
|
||||
entries: Sequence[TemplateEntry],
|
||||
current_path: str,
|
||||
page_dir: Path,
|
||||
parent_category: ParsedSection | None = None,
|
||||
group_categories: Sequence[ParsedSection] | None = None,
|
||||
entry_groups: Sequence[EntryGroup] = (),
|
||||
) -> None:
|
||||
page_dir.mkdir(parents=True, exist_ok=True)
|
||||
parent_name = parent_category["name"] if parent_category else None
|
||||
category_title = category_meta_title(category["name"], parent_name)
|
||||
intro_html, guide_html, intro_lead, _ = load_category_intro(website / "data" / "category_intros" / f"{current_path.removeprefix('/categories/').strip('/')}.md")
|
||||
if intro_lead:
|
||||
category_description = intro_lead
|
||||
elif parent_name:
|
||||
category_description = subcategory_meta_description(category["name"], parent_name, [e["name"] for e in entries])
|
||||
else:
|
||||
category_description = category_meta_description(category["name"], len(entries), category["description"])
|
||||
breadcrumbs = [("Awesome Python", SITE_URL)]
|
||||
if parent_category:
|
||||
breadcrumbs.append((parent_category["name"], category_public_url(parent_category)))
|
||||
breadcrumbs.append((category["name"], category_url))
|
||||
page_entries = [entry for group in entry_groups for entry in group["entries"]] if entry_groups else entries
|
||||
category_json_ld = json.dumps(
|
||||
build_category_json_ld(category_title.removesuffix(" - Awesome Python"), category_url, category_description, page_entries, breadcrumbs),
|
||||
ensure_ascii=False,
|
||||
).replace("</", "<\\/")
|
||||
(page_dir / "index.html").write_text(
|
||||
tpl_category.render(
|
||||
category=category,
|
||||
category_title=category_title,
|
||||
category_url=category_url,
|
||||
category_description=category_description,
|
||||
intro_html=intro_html,
|
||||
guide_html=guide_html,
|
||||
entries=entries,
|
||||
entry_groups=entry_groups,
|
||||
total_categories=len(categories),
|
||||
category_urls=category_urls,
|
||||
current_path=current_path,
|
||||
filter_urls=filter_urls,
|
||||
parent_category=parent_category,
|
||||
group_categories=group_categories,
|
||||
category_json_ld=category_json_ld,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
entries_by_key = {(e["url"], e["name"]): e for e in entries}
|
||||
section_groups = {category["name"]: group_section_entries(category, entries_by_key) for category in categories}
|
||||
for category in categories:
|
||||
render_category(
|
||||
category,
|
||||
category_url=category_public_url(category),
|
||||
entries=[e for e in entries if category["name"] in e["categories"]],
|
||||
current_path=category_path(category),
|
||||
page_dir=categories_dir / category["slug"],
|
||||
entry_groups=section_groups[category["name"]],
|
||||
)
|
||||
|
||||
for group in parsed_groups:
|
||||
render_category(
|
||||
synthetic_category(group["name"], group["slug"]),
|
||||
category_url=group_public_url(group["slug"]),
|
||||
entries=[e for e in entries if group["name"] in e["groups"]],
|
||||
current_path=group_path(group["slug"]),
|
||||
page_dir=categories_dir / group["slug"],
|
||||
group_categories=group["categories"],
|
||||
entry_groups=group_entries_by_section(group["categories"], entries_by_key, website / "data" / "category_intros"),
|
||||
)
|
||||
|
||||
if builtin_entries:
|
||||
render_category(
|
||||
synthetic_category(BUILTIN_FILTER, BUILTIN_SLUG),
|
||||
category_url=BUILTIN_PUBLIC_URL,
|
||||
entries=builtin_entries,
|
||||
current_path=BUILTIN_PATH,
|
||||
page_dir=categories_dir / BUILTIN_SLUG,
|
||||
)
|
||||
|
||||
sponsorship_dir = site_dir / "sponsorship"
|
||||
sponsorship_dir.mkdir(parents=True, exist_ok=True)
|
||||
tpl_sponsorship = env.get_template("sponsorship.html")
|
||||
hero_stats: list[str] = []
|
||||
if repo_stars:
|
||||
hero_stats.append(f"{repo_stars}+ stars on GitHub")
|
||||
hero_stats.append(f"Updated {build_date.strftime('%B %d, %Y')}")
|
||||
(sponsorship_dir / "index.html").write_text(
|
||||
tpl_sponsorship.render(
|
||||
hero_stats=hero_stats,
|
||||
sponsorship_description=SPONSORSHIP_DESCRIPTION,
|
||||
sponsorship_json_ld=json.dumps(build_sponsorship_json_ld(), ensure_ascii=False).replace("</", "<\\/"),
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
subcat_to_entries: dict[str, list[TemplateEntry]] = {}
|
||||
subcat_meta: dict[str, tuple[str, str, str]] = {} # value -> (cat_slug, sub_slug, sub_name)
|
||||
cat_slug_by_url_prefix = {f"/categories/{c['slug']}/": c["slug"] for c in categories}
|
||||
cat_by_slug = {c["slug"]: c for c in categories}
|
||||
for entry in entries:
|
||||
for sub in entry.get("subcategories", []):
|
||||
value = sub["value"]
|
||||
subcat_to_entries.setdefault(value, []).append(entry)
|
||||
if value not in subcat_meta:
|
||||
for prefix, cat_slug in cat_slug_by_url_prefix.items():
|
||||
if sub["url"].startswith(prefix):
|
||||
subcat_meta[value] = (cat_slug, sub["slug"], sub["name"])
|
||||
break
|
||||
|
||||
for value, (cat_slug, sub_slug, sub_name) in subcat_meta.items():
|
||||
render_category(
|
||||
synthetic_category(sub_name, sub_slug),
|
||||
category_url=subcategory_public_url(cat_slug, sub_slug),
|
||||
entries=subcat_to_entries[value],
|
||||
current_path=subcategory_path(cat_slug, sub_slug),
|
||||
page_dir=categories_dir / cat_slug / sub_slug,
|
||||
parent_category=cat_by_slug[cat_slug],
|
||||
entry_groups=[EntryGroup(name="", slug="", url="", lead="", entries=group["entries"]) for group in section_groups[cat_by_slug[cat_slug]["name"]] if group["name"] == sub_name],
|
||||
)
|
||||
|
||||
redirects_file = website / "data" / "redirects.json"
|
||||
redirects = json.loads(redirects_file.read_text(encoding="utf-8")) if redirects_file.exists() else {}
|
||||
for old_path, new_path in redirects.items():
|
||||
tpl_redirect = env.get_template("redirect.html")
|
||||
stub = site_dir / old_path.strip("/") / "index.html"
|
||||
if stub.exists():
|
||||
raise ValueError(f"redirect source {old_path} is a live page; remove it from redirects.json")
|
||||
if not (site_dir / new_path.strip("/") / "index.html").exists():
|
||||
raise ValueError(f"redirect target {new_path} for {old_path} does not exist")
|
||||
stub.parent.mkdir(parents=True, exist_ok=True)
|
||||
stub.write_text(tpl_redirect.render(target_url=SITE_URL + new_path.lstrip("/")), encoding="utf-8")
|
||||
|
||||
static_src = website / "static"
|
||||
static_dst = site_dir / "static"
|
||||
if static_src.exists():
|
||||
shutil.copytree(static_src, static_dst, dirs_exist_ok=True)
|
||||
|
||||
llms_template = (website / "templates" / "llms.txt").read_text(encoding="utf-8")
|
||||
llms_txt = build_llms_txt(
|
||||
llms_template,
|
||||
readme_text=readme_text,
|
||||
subtitle=subtitle,
|
||||
build_date=build_date.strftime("%Y/%m/%d"),
|
||||
stars_data=stars_data,
|
||||
downloads_data=downloads_data,
|
||||
categories=categories,
|
||||
total_entries=total_entries,
|
||||
)
|
||||
(site_dir / "robots.txt").write_text(build_robots_txt(), encoding="utf-8")
|
||||
# Daily star and download refreshes are not significant changes, so lastmod follows commits to the page's own content.
|
||||
build_day = build_date.date().isoformat()
|
||||
readme_date = git_last_change_date(repo_root, "--", "README.md") or build_day
|
||||
section_dates = {
|
||||
c["slug"]: max(
|
||||
git_last_change_date(repo_root, "-L", section_line_range(readme_text, c["name"])),
|
||||
git_last_change_date(repo_root, "--", f"website/data/category_intros/{c['slug']}.md"),
|
||||
)
|
||||
or build_day
|
||||
for c in categories
|
||||
}
|
||||
sitemap_urls = [(SITE_URL, readme_date)]
|
||||
sitemap_urls.extend((category_public_url(c), section_dates[c["slug"]]) for c in categories)
|
||||
sitemap_urls.extend((group_public_url(g["slug"]), max(section_dates[c["slug"]] for c in g["categories"])) for g in parsed_groups)
|
||||
if builtin_entries:
|
||||
sitemap_urls.append((BUILTIN_PUBLIC_URL, readme_date))
|
||||
for cat_slug, sub_slug, _ in sorted(subcat_meta.values()):
|
||||
sitemap_urls.append((subcategory_public_url(cat_slug, sub_slug), section_dates[cat_slug]))
|
||||
sitemap_urls.append((SPONSORSHIP_PUBLIC_URL, git_last_change_date(repo_root, "--", "SPONSORSHIP.md") or build_day))
|
||||
write_sitemap_xml(site_dir / "sitemap.xml", sitemap_urls)
|
||||
(site_dir / "llms.txt").write_text(llms_txt, encoding="utf-8")
|
||||
|
||||
print(f"Built site with {len(parsed_groups)} groups, {len(categories)} categories")
|
||||
print(f"Total entries: {total_entries}")
|
||||
print(f"Output: {site_dir}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
build(Path(__file__).parent.parent)
|
||||
@@ -0,0 +1,18 @@
|
||||
Django saves you from building a Python admin panel: its admin comes built in. Flask-Admin and SQLAdmin add one to Flask and FastAPI apps.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A Flask app: Flask-Admin
|
||||
- SQLAlchemy models in FastAPI or Starlette: SQLAdmin
|
||||
- A Django admin grown into dashboards and internal tools: Unfold
|
||||
- A new look for the Django admin, on your stock ModelAdmin classes: Grappelli
|
||||
|
||||
Flask-Admin [gives each model a `ModelView`](https://flask-admin.readthedocs.io/en/latest/introduction/#adding-model-views): one `admin.add_view(ModelView(User, db.session))` line gets you list, create, and edit pages for it. To fit a model, [subclass `ModelView`](https://flask-admin.readthedocs.io/en/latest/introduction/#customizing-built-in-views) and set class attributes like `can_delete` or `column_list`. Expect fewer defaults than in Django's admin: [it's up to you to tell Flask-Admin](https://flask-admin.readthedocs.io/en/latest/advanced/#design-philosophy) what to display and how. Besides SQLAlchemy, it has [backends](https://flask-admin.readthedocs.io/en/latest/advanced/#using-different-database-backends) for Peewee, pymongo, and MongoEngine.
|
||||
|
||||
SQLAdmin brings Flask-Admin's approach to SQLAlchemy models on FastAPI or Starlette: Flask-Admin [inspired most of its features and configuration](https://smithyhq.github.io/sqladmin/#related-projects-and-inspirations). Create an `Admin` from your app and engine, then [write one `ModelView` subclass per model](https://smithyhq.github.io/sqladmin/#quickstart), declared with `model=User`, and register it with `admin.add_view`.
|
||||
|
||||
Unfold is a theme for Django's own admin, [made for building dashboards, internal tools, and business applications](https://github.com/unfoldadmin/django-unfold). Put `"unfold"` [before `django.contrib.admin`](https://unfoldadmin.com/docs/installation/quickstart/) in `INSTALLED_APPS`, then make your admin classes inherit from `unfold.admin.ModelAdmin`: the stock `ModelAdmin` leaves forms unstyled.
|
||||
|
||||
Grappelli is [a jazzy skin for the Django admin](https://django-grappelli.readthedocs.io/en/latest/) that also adds collapsibles, sortable inlines, and related and autocomplete lookups. Put `'grappelli'` [before `django.contrib.admin`](https://django-grappelli.readthedocs.io/en/latest/quickstart.html) in `INSTALLED_APPS`, and include `grappelli.urls`, which the lookups need. You switch features on with [attributes on your `admin.ModelAdmin` classes](https://django-grappelli.readthedocs.io/en/latest/customization.html#autocomplete-lookups), like `autocomplete_lookup_fields`.
|
||||
|
||||
By default, Django's admin [lets in only users with `is_staff` set](https://docs.djangoproject.com/en/stable/ref/contrib/admin/#overview). Flask-Admin and SQLAdmin leave access control to you: in Flask-Admin, [override `is_accessible`](https://flask-admin.readthedocs.io/en/latest/introduction/#rolling-your-own) on your views, and SQLAdmin [enforces no authentication](https://smithyhq.github.io/sqladmin/authentication/) until you pass it an `AuthenticationBackend`. Django's docs limit its admin to [an internal management tool](https://docs.djangoproject.com/en/stable/ref/contrib/admin/), and once you need a process-centric interface, they say to write your own views.
|
||||
@@ -0,0 +1,55 @@
|
||||
Build a typed agent in Pydantic AI. Long-running, stateful agents suit LangGraph. Apps tied to one model vendor can use that vendor's Python AI agent library.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Typed agents: Pydantic AI, or LangGraph for long-running, stateful ones
|
||||
- Agents on Claude, OpenAI, or Gemini models: Claude Agent SDK, OpenAI Agents SDK, or Google ADK
|
||||
- Skills for your coding agent: Django AI Skills, Sentry Skills, or Trail of Bits Skills
|
||||
- MCP servers and clients: MCP Python SDK, or FastMCP to compose and proxy servers
|
||||
- A personal assistant in your chat apps: Hermes Agent, or AstrBot for QQ, Feishu, and DingTalk
|
||||
- Prompts tuned against a metric: DSPy
|
||||
- Agent data: Instructor for structured output, LlamaIndex for RAG, Mem0 for memory
|
||||
- Open models from the Hugging Face Hub: Transformers
|
||||
- Serving open models: vLLM or SGLang, or MLX LM on Apple silicon
|
||||
- One OpenAI-style API for many model providers: LiteLLM
|
||||
- Image and video generation: Diffusers
|
||||
- Fine-tuning: TRL with PEFT adapters, Unsloth for less GPU memory, Axolotl from YAML
|
||||
- Speech: faster-whisper, FunASR for Chinese, gTTS for text to speech
|
||||
|
||||
Pydantic AI comes from the Pydantic team and is [typed end to end](https://pydantic.dev/docs/ai/overview/). Outputs, tools, and dependencies all carry types, so your type checker knows what an agent returns. Switching models takes one string. [Give an agent an output type and tools](https://github.com/pydantic/pydantic-ai), and every run comes back validated.
|
||||
|
||||
LangGraph is a [low-level orchestration framework](https://docs.langchain.com/oss/python/langgraph/overview) for long-running, stateful agents, and it can mix hand-coded steps with LLM-driven ones in one graph. For your first agent, its docs suggest LangChain's agents instead: [a minimal harness](https://docs.langchain.com/oss/python/langchain/overview) built on LangGraph, which you extend through middleware.
|
||||
|
||||
The Claude Agent SDK gives you [the tools and agent loop behind Claude Code](https://code.claude.com/docs/en/agent-sdk/overview), so it fits agents that read files, run commands, and edit code. Call `query()` for a one-off task, and switch to [`ClaudeSDKClient`](https://code.claude.com/docs/en/agent-sdk/python#choosing-between-query-and-claudesdkclient) when the next step depends on Claude's reply, as in a chat.
|
||||
|
||||
The OpenAI Agents SDK keeps to [a small set of primitives](https://openai.github.io/openai-agents-python/): use it when you want the runtime to handle turns, tool calls, handoffs, and guardrails for you. Despite the name, it [isn't limited to OpenAI models](https://github.com/openai/openai-agents-python).
|
||||
|
||||
Google ADK is [optimized for Gemini but model-agnostic](https://github.com/google/adk-python). Start a project with [`adk create`](https://adk.dev/get-started/python/).
|
||||
|
||||
All three skill collections install from a Claude Code plugin marketplace. Django AI Skills and Sentry Skills follow the open [Agent Skills](https://agentskills.io/) format, so other coding agents can load them too. Sentry Skills is [written for Sentry's own engineers](https://github.com/getsentry/skills), so some of its skills follow Sentry's internal standards.
|
||||
|
||||
CrewAI splits an app into Flows, which manage state and control execution, and Crews, teams of agents that work on one task together. For a production app, [start with a Flow](https://docs.crewai.com/en/introduction), and hand a Crew only the steps that need autonomy.
|
||||
|
||||
The MCP Python SDK reads each tool's schema [from its type hints and docstring](https://py.sdk.modelcontextprotocol.io/get-started/first-steps/), so you write no JSON Schema or request handlers. FastMCP wrote the high-level API that the SDK [took in](https://gofastmcp.com/getting-started/welcome). The standalone project is a fuller framework: it can [mount several servers into one](https://gofastmcp.com/servers/composition) and [proxy another MCP server](https://gofastmcp.com/servers/providers/proxy) through your own.
|
||||
|
||||
Hermes Agent talks to you on Telegram, Discord, Slack, WhatsApp, and Signal [from one gateway process](https://github.com/NousResearch/hermes-agent), and writes itself new skills as it works. Since it runs commands for whoever it lets in, allow only your own account on its gateway with an [allowlist](https://hermes-agent.nousresearch.com/docs/user-guide/security#user-authorization-gateway), and leave approval prompts on for dangerous commands. AstrBot covers chat apps such as QQ, Feishu, and DingTalk. It's AGPL, while Hermes Agent is MIT.
|
||||
|
||||
DSPy has you write [structured signatures, not prompts](https://dspy.ai/current/), and its optimizers tune the prompts for you. Before they can, [define a metric and score a baseline](https://dspy.ai/current/getting-started/metrics/) to beat.
|
||||
|
||||
Instructor [turns an LLM's reply into a validated Pydantic model](https://github.com/567-labs/instructor), and when validation fails, it retries with the error message. Its docs draw the line at [extraction versus agents](https://python.useinstructor.com/): Instructor for the first, Pydantic AI for the second.
|
||||
|
||||
LlamaIndex builds [agents over your own data](https://developers.llamaindex.ai/python/framework/). Its high-level API goes from documents to answers in a few lines, and its lower-level API lets you replace any part.
|
||||
|
||||
Mem0 gives an agent [memory of each user](https://github.com/mem0ai/mem0): search their memories before the model answers, then add the new exchange. Run it [as a library in your app](https://docs.mem0.ai/open-source/overview), or as a self-hosted server for a team.
|
||||
|
||||
Transformers is the [model definition that most training frameworks and inference engines share](https://huggingface.co/docs/transformers/index), Axolotl, Unsloth, vLLM, and SGLang among them. Load a model from the Hugging Face Hub with [`from_pretrained()`](https://huggingface.co/docs/transformers/quicktour), then run it through `Pipeline` for inference or `Trainer` for training. For large production deployments, its docs [send you to vLLM or SGLang](https://huggingface.co/docs/transformers/serve-cli/serving).
|
||||
|
||||
vLLM runs [batch inference offline or an OpenAI-compatible server](https://docs.vllm.ai/en/latest/getting_started/quickstart/), so apps written for the OpenAI API work unchanged. SGLang is [optimized for agentic workloads, RL rollouts, and large-scale serving](https://github.com/sgl-project/sglang). On a Mac, MLX LM [runs and fine-tunes models on Apple silicon](https://github.com/ml-explore/mlx-lm).
|
||||
|
||||
LiteLLM [returns every provider's responses in the OpenAI format](https://docs.litellm.ai/docs/) and maps their errors to OpenAI's exception types, so one code path covers them all. Use its Python SDK inside your app, or run its proxy as a self-hosted gateway for your team.
|
||||
|
||||
Diffusers centers on the [`DiffusionPipeline`](https://huggingface.co/docs/diffusers/index): a few lines generate an image, video, or audio clip.
|
||||
|
||||
Start fine-tuning with LoRA or QLoRA rather than training every weight, as both [PEFT](https://huggingface.co/docs/peft/methods/overview) and [Unsloth](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide) advise. TRL trains with methods from SFT to DPO and GRPO, and every TRL trainer takes a [`peft_config`](https://huggingface.co/docs/trl/peft_integration) for PEFT adapters. Axolotl keeps the whole pipeline, from dataset preprocessing to inference, in [one YAML file](https://github.com/axolotl-ai-cloud/axolotl).
|
||||
|
||||
faster-whisper reimplements Whisper on CTranslate2 for [the same accuracy in less time and memory](https://github.com/SYSTRAN/faster-whisper). For Chinese audio, FunASR's `AutoModel` [chains speech recognition with voice activity detection, punctuation, and speaker models](https://github.com/modelscope/FunASR) in one call. gTTS speaks through [Google Translate's undocumented speech feature](https://github.com/pndurette/gTTS), so upstream changes can break it without notice.
|
||||
@@ -0,0 +1,21 @@
|
||||
If a Python algorithms library has to run in production, Sorted Containers gives you sorted lists, dicts, and sets. To learn, read The Algorithms - Python.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Lists, dicts, and sets that stay sorted as you add and remove: Sorted Containers
|
||||
- Small, typed implementations you can install and read: algorithms
|
||||
- Algorithm code to study, browsed by topic: The Algorithms - Python
|
||||
- How classic design patterns look in Python, and which ones to skip: python-patterns
|
||||
- Workflows as states and transitions, up to full statecharts: python-statemachine
|
||||
|
||||
Python's standard library [is great until you need a sorted collections type](https://grantjenks.com/docs/sortedcontainers/), and Sorted Containers fills that gap in pure Python, as fast as C extensions. SortedList keeps its values in order as you `add()` them, SortedDict is [a dict that keeps a sorted list of its keys](https://grantjenks.com/docs/sortedcontainers/introduction.html#sorted-dict), and SortedSet does the same for a set.
|
||||
|
||||
algorithms collects [minimal, clean, and well-documented implementations](https://github.com/keon/algorithms) of data structures and algorithms. Each file stands alone, with docstrings, type hints, and complexity notes, and its README says they're designed to be read and learned from. Install it with `pip install algorithms`, then [import an implementation by topic](https://github.com/keon/algorithms#quick-start), like a sort or a graph search.
|
||||
|
||||
The Algorithms - Python implements algorithms [for education](https://github.com/TheAlgorithms/Python), and its README says they may be less efficient than the standard library's own: read them to learn, not to ship. Start from [its directory](https://github.com/TheAlgorithms/Python/blob/master/DIRECTORY.md), which sorts them by topic. Its contributing guide asks each algorithm to [carry type hints and doctests](https://github.com/TheAlgorithms/Python/blob/master/CONTRIBUTING.md#what-is-an-algorithm) that test both valid and erroneous input, so the doctests show you how it behaves.
|
||||
|
||||
python-patterns is [a collection of design patterns and idioms](https://github.com/faif/python-patterns) in Python, one file per pattern, grouped as creational, structural, behavioral, and more. Its README asks you to pay more attention to why you choose a pattern than to how you implement it. It also lists the patterns [not recommended in Python](https://github.com/faif/python-patterns#-anti-patterns): skip a Singleton class, since modules are already singletons, and prefer composition and delegation over deep inheritance.
|
||||
|
||||
python-statemachine builds [flat state machines or full statecharts](https://python-statemachine.readthedocs.io/en/latest/) with compound states, parallel regions, and history, in sync and async code. Declare states as class attributes and transitions with `state.to(target)`, and the [events are the names you assign them to](https://python-statemachine.readthedocs.io/en/latest/tutorial.html#your-first-state-machine).
|
||||
|
||||
Before you hand-write python-patterns' State pattern for a real workflow, read python-statemachine's guide [for coming from the State pattern](https://python-statemachine.readthedocs.io/en/latest/how-to/coming_from_state_pattern.html). The whole workflow sits in one class body, and unreachable states fail when the class is defined, not at runtime. For more places to learn and practice algorithms, see [awesome-algorithms](https://github.com/tayllan/awesome-algorithms).
|
||||
@@ -0,0 +1,30 @@
|
||||
I/O-bound or CPU-bound? I/O-bound code wants a Python async library, and asyncio comes built in. CPU-bound work goes to a concurrent.futures process pool.
|
||||
|
||||
How to choose:
|
||||
|
||||
- I/O-bound code written with async/await: asyncio
|
||||
- CPU-bound work, or blocking calls, in a pool of processes or threads: concurrent.futures
|
||||
- Trio-style task groups and cancel scopes on asyncio, or a library that runs on both asyncio and Trio: AnyIO
|
||||
- A faster, drop-in event loop for an asyncio app on Linux or macOS: uvloop
|
||||
- Structured concurrency from the ground up, with its own libraries: Trio
|
||||
- Existing synchronous code, made concurrent without async/await: gevent
|
||||
- Network servers and clients with protocols built in, like SSH, mail, and DNS: Twisted
|
||||
- Processes you manage yourself, talking through queues and pipes: multiprocessing
|
||||
|
||||
asyncio is the standard library's way to [write concurrent code with async/await](https://docs.python.org/3/library/asyncio.html), and many async web servers, database drivers, and task queues build on it. Start your program with `asyncio.run()`, [called once as the main entry point](https://docs.python.org/3/library/asyncio-runner.html#asyncio.run). App code should [rarely need the event loop object](https://docs.python.org/3/library/asyncio-eventloop.html) itself. Run related tasks in an `asyncio.TaskGroup`: when one task fails, it [cancels the rest, which `gather()` doesn't](https://docs.python.org/3/library/asyncio-task.html#asyncio.gather). CPU-heavy code holds up every task on the loop, so [run it in another process](https://docs.python.org/3/library/asyncio-dev.html#asyncio-handle-blocking): hand it to a `ProcessPoolExecutor` with `loop.run_in_executor()`.
|
||||
|
||||
concurrent.futures runs callables on threads or processes behind [the same interface](https://docs.python.org/3/library/concurrent.futures.html). `ThreadPoolExecutor` is [for overlapping I/O](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor). For CPU-bound work on a multi-core machine, the threading docs [advise processes](https://docs.python.org/3/library/threading.html) instead, and `ProcessPoolExecutor` runs them for you. It [takes only picklable functions and arguments](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor), though. Use either executor [in a `with` block](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.shutdown), which shuts it down and waits for its work to finish.
|
||||
|
||||
AnyIO brings [Trio-like structured concurrency to asyncio](https://anyio.readthedocs.io/en/stable/). Code written against its API runs unmodified on asyncio or Trio, so a library built on it doesn't choose for its users. Its docs see [strong merits in its APIs for applications too](https://anyio.readthedocs.io/en/stable/why.html), starting with cancel scopes for [more predictable cancellation](https://anyio.readthedocs.io/en/stable/why.html#design-problems-with-cancellation). Start with `anyio.run(main)`, which [runs on asyncio unless you pass `backend="trio"`](https://anyio.readthedocs.io/en/stable/basics.html#running-async-programs). Spawn tasks in a [task group](https://anyio.readthedocs.io/en/stable/tasks.html): when one child task raises, the rest are cancelled.
|
||||
|
||||
uvloop is [a drop-in replacement for asyncio's event loop](https://github.com/MagicStack/uvloop), built on libuv. Its README prefers `uvloop.run(main())`, which configures `asyncio.run()` to use uvloop, so the rest of your asyncio code stays the same. uvloop runs on Linux and macOS.
|
||||
|
||||
Trio has [an obsessive focus on usability and correctness](https://trio.readthedocs.io/en/stable/). Child tasks run in a nursery, opened with `async with trio.open_nursery()`, and [Trio never discards their exceptions](https://trio.readthedocs.io/en/stable/tutorial.html#okay-let-s-see-something-cool-already). Functions [take no timeout arguments](https://trio.readthedocs.io/en/stable/reference-core.html#blocking-and-non-blocking-methods): you wrap the code in a cancel scope like `trio.move_on_after()`. Trio runs its own event loop, so asyncio functions [don't work inside `trio.run()`](https://trio.readthedocs.io/en/stable/tutorial.html#task-switching-illustrated). Check [the Trio library list](https://trio.readthedocs.io/en/stable/awesome-trio-libraries.html) for what you need first.
|
||||
|
||||
gevent uses greenlets to give you [a synchronous API on top of an event loop](https://www.gevent.org/intro.html). Its monkey patching swaps the standard library's blocking sockets for cooperative ones, so code that knows nothing about gevent runs concurrently. Most programs should [patch everything with `monkey.patch_all()`](https://www.gevent.org/intro.html#beyond-sockets), and the main module should do it [before any other imports](https://www.gevent.org/api/gevent.monkey.html).
|
||||
|
||||
Twisted is [an event-based framework for internet applications](https://github.com/twisted/twisted) that ships clients and servers for HTTP, SSH, IMAP, POP3, SMTP, DNS, IRC, and XMPP. Write new Twisted code as `async def` coroutines, which its docs [prefer over `inlineCallbacks`](https://docs.twisted.org/en/stable/core/howto/defer-intro.html#inline-callbacks-using-yield), and start one with [`Deferred.fromCoroutine()`](https://docs.twisted.org/en/stable/core/howto/defer-intro.html#coroutines-with-async-await).
|
||||
|
||||
multiprocessing runs work in subprocesses, so it can [use every processor on a machine](https://docs.python.org/3/library/multiprocessing.html#introduction). Its own docs point to `ProcessPoolExecutor` as the higher-level interface for pooled tasks. Reach for multiprocessing when you need what it adds, like killing a running process or passing data through queues and pipes. Its guidelines say to [avoid shared state](https://docs.python.org/3/library/multiprocessing.html#all-start-methods) and keep the data moving between processes small.
|
||||
|
||||
On an event loop, a blocking call holds up every other task, so push it to a worker thread: asyncio has [`asyncio.to_thread()`](https://docs.python.org/3/library/asyncio-task.html#asyncio.to_thread), AnyIO [`to_thread.run_sync()`](https://anyio.readthedocs.io/en/stable/threads.html), and Trio [`trio.to_thread.run_sync()`](https://trio.readthedocs.io/en/stable/reference-core.html#threads-if-you-must). When multiprocessing or `ProcessPoolExecutor` runs your code in child processes, define its functions in a module. Guard your entry point with `if __name__ == '__main__':` too, so each new process can [import your main module safely](https://docs.python.org/3/library/multiprocessing.html#multiprocessing-safe-main-import).
|
||||
@@ -0,0 +1,28 @@
|
||||
To track tempo and beats with a Python audio processing library, load music into librosa. MoviePy cuts and joins video in code. Mutagen reads and writes tags.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Tempo, beats, and other music analysis: librosa
|
||||
- Scripted video edits, titles, and compositing: MoviePy
|
||||
- Reading and writing tags: Mutagen
|
||||
- Reading and writing sound files as NumPy arrays: soundfile
|
||||
- Slicing, fading, and joining audio clips: pydub
|
||||
- Frame- and packet-level access to FFmpeg: PyAV
|
||||
- Reading tags only, without GPL code: tinytag
|
||||
- Organizing a music collection with MusicBrainz data: beets
|
||||
|
||||
librosa is a Python library for [audio and music signal processing](https://librosa.org/doc/latest/index.html), with the building blocks for music information retrieval: spectrograms, chroma, onsets, tempo, and beats. Load each file with [`librosa.load()`](https://librosa.org/doc/latest/auto_tutorials/01-intro/01-load.html), which resamples it to a standard rate and mixes it down to mono, the defaults the rest of librosa is designed around.
|
||||
|
||||
soundfile reads and writes [any format libsndfile supports](https://python-soundfile.readthedocs.io/en/latest/#read-write-functions), like WAV, FLAC, and OGG: `sf.read()` returns the samples as a NumPy array along with the sample rate, and `sf.write()` saves them. librosa reads files through soundfile, and its docs [recommend using soundfile directly](https://librosa.org/doc/latest/ioformats.html) to write audio, or when you need more control than loading a whole recording into memory.
|
||||
|
||||
MoviePy is for [automating video edits](https://zulko.github.io/moviepy/getting_started/quick_presentation.html#do-i-need-moviepy): cutting scenes, adding titles and subtitles, or composing many videos into one. A [typical script](https://zulko.github.io/moviepy/getting_started/quick_presentation.html#example-code) loads videos as clips, modifies them, puts them together in a `CompositeVideoClip`, and writes the result with `write_videofile()`.
|
||||
|
||||
PyAV binds FFmpeg's libraries for [direct and precise access](https://pyav.basswood.io/docs/stable/) to your media's containers, streams, packets, codecs, and frames. Open a file with `av.open()` and loop over `container.decode(video=0)` to get frames, which [convert to NumPy arrays](https://pyav.basswood.io/docs/stable/cookbook/numpy.html) with `to_ndarray()` and back with `av.VideoFrame.from_ndarray()`.
|
||||
|
||||
Mutagen reads and writes tags in many audio formats through [one API](https://mutagen.readthedocs.io/en/latest/#why-mutagen) that's roughly the same across them. `mutagen.File()` [guesses the file's type](https://mutagen.readthedocs.io/en/latest/user/gettingstarted.html), you set tags like dict keys, and `save()` writes them. Mutagen is GPL.
|
||||
|
||||
tinytag, which is MIT and pure Python, only reads: `TinyTag.get(path)` returns the [title, artist, album, duration, and more](https://github.com/tinytag/tinytag) as attributes.
|
||||
|
||||
beets runs from the command line: it [catalogs your music collection](https://beets.io/) and improves its metadata from MusicBrainz as it goes. Point [`beet import`](https://beets.readthedocs.io/en/stable/guides/tagger.html) at a folder of albums, and it tags the files as it adds them to your library. The docs [recommend this autotagged import](https://beets.readthedocs.io/en/stable/guides/main.html#importing-your-music): it asks you questions along the way, but gets every song's tags right from the start.
|
||||
|
||||
MoviePy and PyAV both run on FFmpeg. When the `ffmpeg` command [does the job](https://pyav.basswood.io/docs/stable/) on its own, call it directly: MoviePy's docs say that's [faster and more memory-efficient](https://zulko.github.io/moviepy/getting_started/quick_presentation.html#do-i-need-moviepy).
|
||||
@@ -0,0 +1,27 @@
|
||||
Letting users log in with Google takes one Python authentication library: django-allauth on Django, which does password sign-up too, or Authlib elsewhere.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Sign-up, password login, and Google or GitHub login on a Django site: django-allauth
|
||||
- Google or GitHub login, OAuth API calls, or an OAuth server outside Django: Authlib
|
||||
- Adding OAuth to a web framework or HTTP library you maintain: oauthlib
|
||||
- Your own OAuth 2.0 server on Django, or tokens for a Django REST framework API: django-oauth-toolkit
|
||||
- Signing and verifying JSON Web Tokens: PyJWT
|
||||
- Permissions you grant per object to users and groups: django-guardian
|
||||
- Permissions that follow from the object's data, with no extra tables: django-rules
|
||||
|
||||
django-allauth exists to handle [local and social accounts in one app](https://docs.allauth.org/en/latest/introduction/index.html): a user can sign up with a password or log in with Google. Follow the [quickstart](https://docs.allauth.org/en/latest/installation/quickstart.html): add its authentication backend, apps, and middleware, then include `allauth.urls` under `accounts/`. Its login, logout, and password views can replace the ones in `django.contrib.auth.urls`. Give each OAuth provider its client ID and secret in a [`SocialApp` or the `SOCIALACCOUNT_PROVIDERS` setting](https://docs.allauth.org/en/latest/installation/quickstart.html#post-installation). For a single-page or mobile app, add [`allauth.headless`](https://docs.allauth.org/en/latest/headless/introduction.html).
|
||||
|
||||
Authlib covers both sides of OAuth 2.0 and OpenID Connect. As a client, it comes in [two kinds](https://docs.authlib.org/en/latest/oauth2/client/index.html): HTTP clients for scripts and service-to-service calls, and web clients that log users in through Flask, Django, Starlette, or FastAPI. For login, [create an `OAuth` registry and `register()` each provider](https://docs.authlib.org/en/latest/oauth2/client/web/index.html). For an OpenID Connect provider like Google, pass its discovery URL as [`server_metadata_url`](https://docs.authlib.org/en/latest/oauth2/client/web/index.html#parsing-id-token), and Authlib reads the other endpoints from it. To [run your own OAuth 2.0 or OpenID Connect provider](https://docs.authlib.org/en/latest/oauth2/authorization-server/index.html), it has server integrations for Flask and Django.
|
||||
|
||||
PyJWT [encodes and decodes JSON Web Tokens](https://pyjwt.readthedocs.io/en/latest/). Treat every token as untrusted input: [hard-code the `algorithms` you pass to `jwt.decode()`](https://pyjwt.readthedocs.io/en/latest/algorithms.html#specifying-an-algorithm), never read them from the token's own header, and don't mix HS and RS algorithms. For tokens from an OpenID Connect provider, point [`PyJWKClient` at its JWKS endpoint](https://pyjwt.readthedocs.io/en/latest/usage.html#retrieve-rsa-signing-keys-from-a-jwks-endpoint) instead of hard-coding its public keys.
|
||||
|
||||
oauthlib implements OAuth's logic [without assuming a web framework or HTTP request object](https://github.com/oauthlib/oauthlib), so a framework or library maintainer can write a thin layer on top and get OAuth support. Its FAQ says [most people use it indirectly](https://oauthlib.readthedocs.io/en/latest/faq.html#how-do-i-use-oauthlib-with-google-twitter-and-other-providers): django-oauth-toolkit and django-allauth both build on it. To build a provider on it, most of the work is [a `RequestValidator`](https://oauthlib.readthedocs.io/en/latest/oauth2/server.html#implement-a-validator) that maps its checks to your storage.
|
||||
|
||||
django-oauth-toolkit is an [OAuth 2.0 authorization server for Django](https://django-oauth-toolkit.readthedocs.io/en/latest/): it issues and manages tokens from your existing project, and can also protect a Django or Django REST framework API. Django REST framework's docs [recommend it for OAuth 2.0](https://www.django-rest-framework.org/api-guide/authentication/#django-oauth-toolkit). [Install it](https://django-oauth-toolkit.readthedocs.io/en/latest/install.html) by adding `oauth2_provider` to `INSTALLED_APPS`, including its URLs under `o/`, and migrating. For an API, [set `OAuth2Authentication`](https://django-oauth-toolkit.readthedocs.io/en/latest/rest-framework/getting_started.html) as Django REST framework's authentication class and guard views with `TokenHasScope`.
|
||||
|
||||
Django has [a foundation for object permissions but no implementation](https://docs.djangoproject.com/en/stable/topics/auth/customizing/#handling-object-permissions), and django-guardian fills it with an [extra authentication backend](https://django-guardian.readthedocs.io/en/latest/configuration/). Grant a permission on one object with `assign_perm()`, then check it with Django's own [`user.has_perm(perm, obj)`](https://django-guardian.readthedocs.io/en/latest/userguide/checks/). Django REST framework's `DjangoObjectPermissions` [names it as an example backend](https://www.django-rest-framework.org/api-guide/permissions/#djangoobjectpermissions).
|
||||
|
||||
django-rules gives Django object permissions [without a database](https://github.com/dfunckt/django-rules). A permission is a rule built from predicates, plain functions like `is_book_author(user, book)` that you combine with `|`, `&`, and `~`. Add its [backend to `AUTHENTICATION_BACKENDS`](https://github.com/dfunckt/django-rules#configuring-django), register each permission with `rules.add_perm()`, and check it with `user.has_perm()`. Keep predicates and rules in their own modules, and let [`AutodiscoverRulesConfig` import each app's `rules.py`](https://github.com/dfunckt/django-rules#best-practices) at startup.
|
||||
|
||||
On either side of OAuth, use PKCE. django-oauth-toolkit's security guide maps [the OAuth security recommendations of RFC 9700](https://django-oauth-toolkit.readthedocs.io/en/latest/security.html) to its settings: PKCE is one, and the implicit and password grants must not be used. Authlib's client [turns PKCE on with `code_challenge_method`](https://docs.authlib.org/en/latest/oauth2/client/web/index.html#oauth-2-0-code-challenge). Also, don't pick JWTs for sessions just because they sound stateless. django-allauth's docs point out that [if a token must stop working at logout, you need state to revoke it](https://docs.allauth.org/en/latest/headless/token-strategies/jwt-tokens.html). JWTs help most when each service checks tokens without asking a central server.
|
||||
@@ -0,0 +1,16 @@
|
||||
Tasks like running your tests go in pyproject.toml for Poe the Poet to run. C and C++ projects that replace Make with a Python build tool pick SCons.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Project tasks in pyproject.toml, run in your uv or Poetry environment: Poe the Poet
|
||||
- Compiling C or C++ in place of Make: SCons
|
||||
- Tasks written as Python functions that run shell commands: Invoke
|
||||
- Steps that skip when their input files haven't changed: doit
|
||||
|
||||
Poe the Poet reads tasks from a [`[tool.poe.tasks]` table](https://poethepoet.natn.io/tasks/index.html) in pyproject.toml. A string like `test = "pytest"` is a whole task, run as a command without a shell. In a uv or Poetry project, poe [runs tasks in the project's virtualenv](https://poethepoet.natn.io/#quick-start), so you don't type `uv run` or `poetry run` in front.
|
||||
|
||||
Invoke tasks are Python functions in a [`tasks.py` file](https://docs.pyinvoke.org/en/latest/getting-started.html#defining-and-running-task-functions), decorated with `@task` and taking a context as their first argument. Inside, `c.run()` runs a shell command. A function's arguments [become CLI flags](https://docs.pyinvoke.org/en/latest/getting-started.html#task-parameters), and a task can name [pre-tasks](https://docs.pyinvoke.org/en/latest/getting-started.html#declaring-pre-tasks) to always run before it.
|
||||
|
||||
doit reads tasks from a [`dodo.py` file](https://pydoit.org/tutorial-1.html), where each `task_` function returns a dict of actions, file dependencies, and targets instead of running anything. It saves a [signature of each `file_dep`](https://pydoit.org/tasks.html#file-dep) and skips a task whose inputs haven't changed. A task that [declares no input](https://pydoit.org/dependencies.html#doit-up-to-date-definition) never counts as up to date, so list the files each task reads in `file_dep`.
|
||||
|
||||
SCons reads your build from an [`SConstruct` file, which is a Python script](https://scons.org/doc/production/HTML/scons-user.html#sect-sconstruct-python): `Program('hello.c')` there is enough for `scons` to compile a program. Calling a builder like `Program()` [declares what you want built](https://scons.org/doc/production/HTML/scons-user.html#sect-order-independent) instead of building it on that line, and SCons works out when to run each step.
|
||||
@@ -0,0 +1,16 @@
|
||||
Skip hand-written dunder methods, and when you compare Python dataclass alternatives, pick attrs for its validators and converters. Box gives dicts dot access.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Classes with generated dunder methods, validators, and converters: attrs
|
||||
- Dicts read with dot notation, nested ones included: python-box
|
||||
- Looking up a key by its value, with both directions in sync: bidict
|
||||
- A faster drop-in for the `uuid` module: uuid-utils
|
||||
|
||||
attrs [writes the dunder methods](https://www.attrs.org/en/latest/) that implement object protocols, so you don't have to. For new code, its docs [recommend the modern API](https://www.attrs.org/en/latest/names.html#tl-dr): `@define` on the class and `field()` for an attribute, or `@frozen` for immutable instances, with slots on by default. Type annotations stay optional. The standard library's dataclasses [gave up features for simplicity](https://www.attrs.org/en/latest/why.html#data-classes), like validators and converters, so a class that needs them goes to attrs. It [isn't a full serialization library](https://www.attrs.org/en/latest/overview.html#what-attrs-is-not), though: to serialize and validate your attrs classes, its docs point you to its sibling project, cattrs.
|
||||
|
||||
python-box installs Box, a [`dict` subclass](https://github.com/cdgriffith/Box) whose keys you can also read as attributes, even ones like `"imdb stars"`. Nested dicts and lists become Box and BoxList objects, so dot access works all the way down: `movie_box.Robin_Hood_Men_in_Tights.imdb_stars`. It [reads and writes JSON, YAML, TOML, and msgpack](https://github.com/cdgriffith/Box/wiki/Converters) with methods like `from_json()` and `to_json()`, and `to_dict()` gives you plain dicts back. Keep Box for data whose keys change: attrs' docs say a dict with a fixed and known set of keys [is an object, not a hash](https://www.attrs.org/en/latest/why.html#dicts), and belongs in a class.
|
||||
|
||||
bidict gives you [bidirectional mappings](https://bidict.readthedocs.io/intro.html) that work like dicts: look up a value by its key as usual, or a key by its value through `.inverse`, which stays in sync as you update the mapping. A single dict holding both directions mixes keys with values. Modeling the mapping correctly takes two one-way mappings kept in sync, [which is what bidict does](https://bidict.readthedocs.io/intro.html#why-can-t-i-just-use-a-dict) under the hood.
|
||||
|
||||
uuid-utils is a [fast, drop-in replacement for Python's `uuid` module](https://aminalaee.github.io/uuid-utils/latest/), powered by Rust. Import it under the module's name, `import uuid_utils as uuid`, and call `uuid.uuid4()` or `uuid.uuid7()` as usual. Django and some other frameworks require the standard library's own `uuid.UUID` instances. For those, [import `uuid_utils.compat`](https://aminalaee.github.io/uuid-utils/latest/#compatibility-with-python-uuid) instead, which returns them and still outperforms the standard library.
|
||||
@@ -0,0 +1,23 @@
|
||||
In memory, on disk, or on a cache server: the Python caching library you want is cachetools, DiskCache, or dogpile.cache. For HTTP responses, it's Hishel.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Function results in one process's memory, with a size limit or a time-to-live: cachetools
|
||||
- A cache on local disk that processes on one machine share, with no server to run: DiskCache
|
||||
- A Django cache backend on local disk: DiskCache
|
||||
- One cache on memcached or Redis for many processes or servers: dogpile.cache
|
||||
- HTTP responses in HTTPX or Requests: Hishel
|
||||
- `Cache-Control` headers and response caching in FastAPI or another ASGI app: Hishel
|
||||
- Django querysets, invalidated when a model changes: Cacheops
|
||||
|
||||
cachetools offers [variants of the standard library's `@lru_cache`](https://cachetools.readthedocs.io/en/stable/) with more cache algorithms, including a `TTLCache` whose items [expire after a time-to-live](https://cachetools.readthedocs.io/en/stable/#cachetools.TTLCache). Wrap a function in [`@cached`](https://cachetools.readthedocs.io/en/stable/#cachetools.cached) and pass it the cache to use. The cache classes [aren't thread-safe](https://cachetools.readthedocs.io/en/stable/#cache-implementations), so when threads share a cache, give `@cached` a `threading.Lock`. The cache lives in your process's memory, so each worker process of a web app keeps its own copy.
|
||||
|
||||
DiskCache is a [disk and file backed cache library](https://grantjenks.com/docs/diskcache/) in pure Python, built on SQLite, with no other process to run. Create a `Cache` with a directory path: [two `Cache` objects on the same directory](https://grantjenks.com/docs/diskcache/tutorial.html#cache) can live in separate processes, so the workers on one machine share one cache. Wrap a function in [`@cache.memoize()`](https://grantjenks.com/docs/diskcache/tutorial.html#fanoutcache), which takes arguments like `lru_cache`'s. In Django, set [`diskcache.DjangoCache`](https://grantjenks.com/docs/diskcache/tutorial.html#djangocache) as the cache `BACKEND`. Keep the directory on a local disk, since SQLite [isn't recommended on NFS mounts](https://grantjenks.com/docs/diskcache/tutorial.html#caveats).
|
||||
|
||||
dogpile.cache is [a caching API over backends of any variety](https://dogpilecache.sqlalchemy.org/en/latest/), memcached and Redis among them. You ask it for a value and hand it a function that [creates the value only when needed](https://dogpilecache.sqlalchemy.org/en/latest/usage.html#overview). When the value expires, one worker regenerates it instead of every worker at once. Create a region with `make_region()` at import time, decorate functions with `@region.cache_on_arguments()`, and [call `configure()` later](https://dogpilecache.sqlalchemy.org/en/latest/usage.html#rudimentary-usage) with the backend and expiration time from your config file. When several processes share one Redis or memcached server, turn on the backend's [`distributed_lock`](https://dogpilecache.sqlalchemy.org/en/latest/api.html#dogpile.cache.backends.redis.RedisBackend) so that lock covers them all.
|
||||
|
||||
Hishel caches HTTP responses by [RFC 9111](https://hishel.com/overview.html), the HTTP caching rules browsers follow, in both sync and async code. With HTTPX, [swap your client for Hishel's cache-enabled one](https://hishel.com/httpx.html#quick-start), or [give a client you already have its cache transport](https://hishel.com/httpx.html#cache-transports). With Requests, [mount its cache adapter](https://hishel.com/requests.html#quick-start) on a `Session`. In FastAPI, [a dependency sets `Cache-Control` headers](https://hishel.com/fastapi.html#quick-start) for browsers and CDNs, and its ASGI middleware also caches responses on your server.
|
||||
|
||||
Cacheops caches Django querysets in Redis and [invalidates them](https://github.com/Suor/django-cacheops#user-content-invalidation) on `save()`, `delete()`, and many-to-many changes. [Add it to `INSTALLED_APPS`](https://github.com/Suor/django-cacheops#user-content-setup) and point it at its own Redis database, as its docs highly recommend. Turn caching on per app with `'app_name.*'`, since `'*.*'` can also cache tables you don't mean to, like migrations. Call `.cache()` on a queryset to cache it by hand, and wrap a function in [`@cached_as(Article)`](https://github.com/Suor/django-cacheops#user-content-usage) to drop its result whenever an `Article` changes.
|
||||
|
||||
Plan how stale entries leave the cache, too. cachetools' decorators add a [`cache_clear()`](https://cachetools.readthedocs.io/en/stable/#memoizing-decorators) function, DiskCache [evicts every key with a given tag](https://grantjenks.com/docs/diskcache/tutorial.html#cache), and a function decorated by dogpile.cache takes [`invalidate()`](https://dogpilecache.sqlalchemy.org/en/latest/api.html#dogpile.cache.region.CacheRegion.cache_on_arguments) with the same arguments you'd call it with.
|
||||
@@ -0,0 +1,38 @@
|
||||
Python's own argparse covers basic command-line apps. Beyond that, pick Click or Typer as your Python CLI library, and style the output with Rich.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Basic command-line app with no dependencies: argparse
|
||||
- Nested commands, or subcommands loaded lazily: Click
|
||||
- Options declared once as type-hinted function parameters: Typer
|
||||
- Interactive prompts and REPLs: prompt_toolkit
|
||||
- CLI from existing code without writing a parser: Python Fire
|
||||
- Progress bar for a loop: tqdm, or alive-progress for animated bars
|
||||
- Colored text, tables, and logs: Rich, or colorama for ANSI colors on Windows
|
||||
- Full-screen app in the terminal or a browser: Textual
|
||||
- Terminal widgets on an event loop you already run: Urwid
|
||||
- Full-screen forms and ASCII animations: asciimatics
|
||||
|
||||
argparse is in the standard library. Python's docs call it [the recommended choice](https://docs.python.org/3/library/optparse.html#choosing-an-argument-parser) when you have no more specific needs, since it gives you the most out of the box for the least code. It [writes the help and usage messages](https://docs.python.org/3/library/argparse.html) and reports invalid arguments for you. For commands written as decorated functions, the same docs point to Click, and for a CLI that works with static type checking, to Typer.
|
||||
|
||||
Click builds a CLI from [commands declared with decorators](https://click.palletsprojects.com/en/stable/quickstart/). You can nest them to any depth, and Click can [load subcommands lazily](https://click.palletsprojects.com/en/stable/) at runtime. It also [reads option values from environment variables](https://click.palletsprojects.com/en/stable/why/) and comes with helpers for ANSI colors, terminal size, and launching editors.
|
||||
|
||||
Typer builds a CLI from your function signatures: you [declare each argument and option once](https://typer.tiangolo.com/), as a type-hinted function parameter. Write those hints with `Annotated`, which its tutorial [recommends wherever possible](https://typer.tiangolo.com/tutorial/arguments/optional/). Its docs say you get [the best results by pairing it with Rich](https://typer.tiangolo.com/tutorial/printing/): Typer structures the commands and options, and Rich displays the output.
|
||||
|
||||
prompt_toolkit is for interactive input. It can [replace GNU readline](https://python-prompt-toolkit.readthedocs.io/en/stable/) or build full-screen apps. For a REPL, use a [`PromptSession`](https://python-prompt-toolkit.readthedocs.io/en/stable/pages/asking_for_input.html), which keeps the history for the whole session.
|
||||
|
||||
Python Fire turns any Python object into a CLI: [call `fire.Fire()`](https://github.com/google/python-fire/blob/master/docs/guide.md) at the end of your program. Its README pitches it for [developing and debugging your code](https://github.com/google/python-fire), exploring existing code, and turning other people's code into a CLI. It takes each argument's type from the value you pass, not from the function signature.
|
||||
|
||||
tqdm adds a progress bar to any loop: [wrap the iterable in `tqdm()`](https://github.com/tqdm/tqdm). Import it from `tqdm.auto`, which picks the console bar or the Jupyter widget for you. alive-progress wraps the loop in an [`alive_bar` context manager](https://github.com/rsalmei/alive-progress) instead, with a spinner that speeds up and slows down with your throughput.
|
||||
|
||||
Rich writes colored text, tables, Markdown, and syntax-highlighted code to the terminal. Start with its [drop-in `print`](https://rich.readthedocs.io/en/stable/introduction.html), then create [one `Console` at module level](https://rich.readthedocs.io/en/stable/console.html) for the rest of your app. For colored logs, send the logging module's output through [`RichHandler`](https://rich.readthedocs.io/en/stable/logging.html).
|
||||
|
||||
colorama makes ANSI escape codes work on Windows and does nothing on other platforms. If that's all you need, call [`just_fix_windows_console()`](https://github.com/tartley/colorama).
|
||||
|
||||
Textual apps run in the terminal, and the `textual serve` command from textual-dev [serves them in a browser](https://textual.textualize.io/guide/devtools/). Build one by [subclassing `App`](https://textual.textualize.io/guide/app/), and style its widgets with [CSS](https://textual.textualize.io/guide/CSS/).
|
||||
|
||||
Urwid is a [console widget construction set](https://urwid.org/manual/overview.html) rather than a finished UI library. It runs on [your choice of event loop](https://github.com/urwid/urwid): asyncio, or another one you already use. It's [licensed under the LGPL](https://github.com/urwid/urwid/blob/master/COPYING), while Textual and asciimatics use permissive licenses.
|
||||
|
||||
asciimatics does [full-screen text UIs, from interactive forms to ASCII animations](https://github.com/peterbrittain/asciimatics): create a Screen, build a Scene from Effect objects, and let the Screen play it.
|
||||
|
||||
Ship your CLI as an installable package with a `[project.scripts]` entry point, not a file users run with `python`. [Click recommends it](https://click.palletsprojects.com/en/stable/entry-points/), and shell completion in Click and [Typer](https://typer.tiangolo.com/tutorial/package/) works through that entry point. Test your commands in-process: Click and Typer both provide a [`CliRunner`](https://click.palletsprojects.com/en/stable/testing/), and Textual's [`run_test()`](https://textual.textualize.io/guide/testing/) drives your app as if you were using the keyboard and mouse.
|
||||
@@ -0,0 +1,26 @@
|
||||
Swap cURL for HTTPie, and your database's own client for a Python CLI tool with autocompletion: pgcli, mycli, litecli, or IRedis.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Calling HTTP APIs from the terminal: HTTPie
|
||||
- A database shell with autocompletion: pgcli for PostgreSQL, mycli for MySQL, litecli for SQLite, IRedis for Redis
|
||||
- Downloading video or audio from YouTube and other sites: yt-dlp
|
||||
- A new project from a template: Cookiecutter, or Copier to pull in later template changes
|
||||
- A shell you script in Python: xonsh
|
||||
- A tmux session with its windows and panes, from one YAML file: tmuxp
|
||||
|
||||
HTTPie is a command-line HTTP client for [testing, debugging, and generally interacting with APIs](https://httpie.io/docs/cli) and HTTP servers. It formats and colors the output. A request reads like `http PUT pie.dev/put X-API-Token:123 name=John`: `Header:Value` items set headers, and `field=value` items fill the body, which HTTPie [sends as JSON by default](https://httpie.io/docs/cli/json).
|
||||
|
||||
pgcli, mycli, litecli, and IRedis are terminal clients with autocompletion and syntax highlighting, one per database. Give [pgcli](https://github.com/dbcli/pgcli) a database name or a `postgresql://` URI, and [litecli](https://github.com/dbcli/litecli) the path to a SQLite file. mycli also [works with MariaDB](https://github.com/dbcli/mycli), Percona, TiDB, and Apache Doris. IRedis behaves like Redis's own client in most cases, but it's [safer on production servers](https://github.com/laixintao/iredis): it stops you from accidentally running dangerous commands like `KEYS *`.
|
||||
|
||||
yt-dlp downloads audio and video from [thousands of sites](https://github.com/yt-dlp/yt-dlp). Install ffmpeg too: yt-dlp [needs it to merge separate video and audio files](https://github.com/yt-dlp/yt-dlp#dependencies).
|
||||
|
||||
Cookiecutter creates projects from templates, and its templates [work for any language](https://github.com/cookiecutter/cookiecutter), not only Python. [Run `cookiecutter gh:audreyfeldroy/cookiecutter-pypackage`](https://cookiecutter.readthedocs.io/en/stable/usage.html), answer its prompts, and you get a new project from that GitHub template.
|
||||
|
||||
Copier also generates projects from templates, but it calls itself [a code lifecycle management tool](https://copier.readthedocs.io/en/stable/comparisons/). When the template changes, `copier update` brings the changes into projects you already generated. [It works best](https://copier.readthedocs.io/en/stable/updating/) when both are in Git: the template tagged, your project clean, with the `.copier-answers.yml` file that records your answers. Templates can run code: Cookiecutter's [pre- and post-generate scripts](https://github.com/cookiecutter/cookiecutter), Copier's tasks. [Generate projects only from templates you trust](https://copier.readthedocs.io/en/stable/generating/).
|
||||
|
||||
xonsh is a shell whose language is [a superset of Python](https://github.com/xonsh/xonsh), with shell commands built in. It isn't POSIX-compatible, so don't make it your login shell with `chsh`. Its docs [recommend a xonsh profile in your terminal emulator](https://xon.sh/install.html#before-installing) instead.
|
||||
|
||||
tmuxp launches a whole tmux session [from one YAML or JSON file](https://tmuxp.git-pull.com/quickstart/): its windows, its panes, and the commands in them. Save the file as `.tmuxp.yaml` in a project, and [`tmuxp load path/to/project/`](https://github.com/tmux-python/tmuxp) builds the session.
|
||||
|
||||
Several of these tools' docs have you [install them as CLI tools](https://github.com/cookiecutter/cookiecutter), and add them to a project only to use them from Python. Looking for a library to build your own Python CLI tool? That's [CLI Development](/categories/cli-development/).
|
||||
@@ -0,0 +1,12 @@
|
||||
Both Python CMS picks run on Django: Wagtail for page types your developers define, like articles, and django CMS for editors building pages on the live site.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Content types your developers define, like articles and events: Wagtail
|
||||
- Editors composing pages from reusable components on the live site: django CMS
|
||||
|
||||
Wagtail is [not an instant website in a box](https://docs.wagtail.org/en/stable/getting_started/the_zen_of_wagtail.html#wagtail-is-not-an-instant-website-in-a-box): expect to write code. Start a project with [`wagtail start`](https://docs.wagtail.org/en/stable/getting_started/quick_install.html). Each page type is [a Django model that inherits from `Page`](https://docs.wagtail.org/en/stable/topics/pages.html), so give each kind of content its own type with its own fields. An event page with a date and a location [can show up in a calendar](https://docs.wagtail.org/en/stable/getting_started/the_zen_of_wagtail.html#a-cms-should-get-information-out-of-an-editor-s-head-and-into-a-database-as-efficiently-and-directly-as-possible), and a styled heading on a generic page can't. Use [StreamField](https://docs.wagtail.org/en/stable/topics/streamfield.html) for pages without a fixed structure, like blog posts, and [snippets](https://docs.wagtail.org/en/stable/topics/snippets/index.html) for content that doesn't need its own page, like headers and footers. For a headless site, use its [built-in API](https://docs.wagtail.org/en/stable/advanced_topics/api/index.html).
|
||||
|
||||
In django CMS, [editors compose pages from plugins](https://docs.django-cms.org/en/stable/explanation/philosophy.html#three-disciplines-three-surfaces) in a toolbar on the live site, and developers write standard Django. Each template [declares its placeholders](https://docs.django-cms.org/en/stable/tutorials/02-templates-placeholders.html) with `{% placeholder %}`, and `CMS_TEMPLATES` lists the templates editors can pick from. For a region that is the same on every page, like a footer, use [`{% static_alias %}`](https://docs.django-cms.org/en/stable/tutorials/02-templates-placeholders.html#a-reusable-region-with-static-alias), so the content is stored once. For content from your own app, ask [where it lives](https://docs.django-cms.org/en/stable/explanation/composition.html). If it fits in a placeholder on a page, write a plugin. If it has its own list view, detail view, and URL, mount it as an app with an apphook. The core [publishes as you edit](https://docs.django-cms.org/en/stable/explanation/publishing.html), which is rarely enough for a production site with editors, so add a versioning package.
|
||||
|
||||
Both are built on Django. Wagtail [deploys like a Django site](https://docs.wagtail.org/en/stable/deployment/index.html). You can add either one to an existing Django project: Wagtail [integrates into one](https://docs.wagtail.org/en/stable/getting_started/integrating_into_django.html), and django CMS [doesn't make you rebuild around it](https://docs.django-cms.org/en/stable/explanation/philosophy.html#implications-for-projects). Neither fits a very small or static site. For a two-page brochure, django CMS calls itself [overkill](https://docs.django-cms.org/en/stable/explanation/philosophy.html#when-django-cms-may-not-be-the-right-fit), and a static site generator is lighter.
|
||||
@@ -0,0 +1,49 @@
|
||||
Lint and format with Ruff. It doesn't check types, so add a type checker, and let pre-commit run each Python code analysis tool on every commit.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Linting, formatting, and import sorting in one tool: Ruff
|
||||
- Running every check before each commit: pre-commit
|
||||
- Rules on which of your modules may import which: Import Linter
|
||||
- Unused code: Vulture
|
||||
- Functions too complex for people to read: complexipy
|
||||
- Several analyzers under one command, with ready-made profiles: Prospector
|
||||
- Black's own formatting, with imports sorted to match: Black and isort
|
||||
- Checks Ruff can't run: Pylint for deeper inference, Flake8 for your own plugins
|
||||
- Security issues: Bandit
|
||||
- Renaming and moving code from your editor or a script: Rope
|
||||
- Type checking Django or Pydantic code: mypy with its plugins, Pyrefly without
|
||||
- Adding types to a partly typed codebase: ty
|
||||
- Type checking in VS Code: Pyright, which Pylance runs on
|
||||
|
||||
Ruff [can replace Flake8, Black, and isort](https://docs.astral.sh/ruff/), plus dozens of Flake8 plugins. Its docs suggest you [start with a small rule set](https://docs.astral.sh/ruff/linter/#rule-selection), add one group at a time, and list them explicitly in `select`. Ruff is [a linter, not a type checker](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-ty-mypy-pyright-or-pyre), so run a type checker alongside it.
|
||||
|
||||
pre-commit installs and runs the hooks you list in `.pre-commit-config.yaml`, [written in any language](https://pre-commit.com/). Run `pre-commit install` [first thing after you clone](https://pre-commit.com/#usage) a repo that uses it, so the hooks run on every `git commit`, and add `pre-commit run --all-files` [as a CI step](https://pre-commit.com/#usage-in-continuous-integration).
|
||||
|
||||
Import Linter checks [contracts on the imports between your modules](https://import-linter.readthedocs.io/en/stable/): `lint-imports` fails when code breaks one. A [layers contract](https://import-linter.readthedocs.io/en/stable/contract_types/layers/) lets higher layers import lower ones but not the other way around, and it counts chains of imports through other modules too.
|
||||
|
||||
Vulture finds unused code, and [running it on your tests too](https://github.com/jendrikseipp/vulture) shows you untested code.
|
||||
|
||||
complexipy measures [cognitive complexity](https://complexipy.com/): how hard code is for people to understand. Unlike cyclomatic complexity, it counts nesting. Its docs recommend [running it alongside Ruff](https://complexipy.com/comparison-with-ruff/#use-both-recommended), whose complexity rule measures something else.
|
||||
|
||||
Prospector runs Pylint and other analyzers together, and aims to be [useful out of the box](https://prospector.landscape.io/en/stable/), with default profiles so you don't have to sort out which warnings matter. When your project uses Django, Flask, or Celery, it [adapts its checks](https://prospector.landscape.io/en/stable/#adapting-to-dependencies) to that framework.
|
||||
|
||||
With Black, you [give up control over hand-formatting](https://black.readthedocs.io/en/stable/), and in return, code looks the same in every project and diffs stay small. Its style options are [deliberately limited](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html). If you sort imports with isort, set [the black profile](https://isort.readthedocs.io/en/latest/configuration/black_compatibility.html) so the two agree.
|
||||
|
||||
Pylint [infers the actual values](https://pylint.readthedocs.io/en/latest/#what-differentiates-pylint) in your code instead of trusting its type hints. That makes it slower than other linters, but it finds more in code that isn't fully typed, and it's why Ruff [isn't a drop-in replacement](https://docs.astral.sh/ruff/faq/#how-does-ruffs-linter-compare-to-pylint) for Pylint. Because of that speed, run it [in CI or a pre-push hook](https://pylint.readthedocs.io/en/latest/user_guide/installation/pre-commit-integration.html) rather than on every commit.
|
||||
|
||||
Flake8 [wraps pyflakes, pycodestyle, and McCabe](https://github.com/PyCQA/flake8) in one command, and [its plugins](https://flake8.pycqa.org/en/latest/user/using-plugins.html) are much of its appeal. Keep it when you depend on a plugin of your own, since Ruff [doesn't support custom lint rules](https://docs.astral.sh/ruff/faq/#how-does-ruffs-linter-compare-to-flake8).
|
||||
|
||||
Bandit [finds common security issues](https://bandit.readthedocs.io/en/latest/) in Python code. Scan a whole tree with [`bandit -r`](https://bandit.readthedocs.io/en/latest/start.html).
|
||||
|
||||
Rope is a refactoring library that [many IDEs and editors use](https://rope.readthedocs.io/en/latest/overview.html) to rename, extract, and move code, and scripts can run the same refactorings through [its library API](https://rope.readthedocs.io/en/latest/library.html).
|
||||
|
||||
mypy is [designed for gradual typing](https://mypy.readthedocs.io/en/stable/): by default, it [skips functions without annotations](https://mypy.readthedocs.io/en/stable/getting_started.html#dynamic-vs-static-typing), so you can add type hints slowly. Check in one config file, [run mypy in CI](https://mypy.readthedocs.io/en/stable/existing_code.html#run-mypy-consistently-and-prevent-regressions) so new type errors don't land, and aim for `--strict`. Its [plugins](https://mypy.readthedocs.io/en/stable/extending_mypy.html#extending-mypy-using-plugins) teach it libraries the standard type system can't describe.
|
||||
|
||||
Pyrefly is a type checker and language server with [built-in support for Pydantic and Django](https://github.com/facebook/pyrefly), so those work without plugins. It reads an existing mypy or Pyright config, so you can [try it alongside your current checker](https://pyrefly.org/en/docs/compare/#try-it-alongside-your-current-checker) before you switch.
|
||||
|
||||
ty is a type checker and language server [designed for adoption](https://docs.astral.sh/ty/) on partly typed code. Unlike mypy, it [checks the bodies of unannotated functions](https://docs.astral.sh/ty/coming-from-mypy-or-pyright/), and it leaves [missing-annotation checks](https://docs.astral.sh/ty/reference/typing-faq/#why-doesnt-ty-warn-about-missing-type-annotations) to Ruff's `ANN` rules.
|
||||
|
||||
Pyright [checks all code by default](https://github.com/microsoft/pyright/blob/main/docs/mypy-comparison.md#type-checking-unannotated-code), annotated or not, which its language server features rely on. In VS Code, its docs [recommend Pylance](https://github.com/microsoft/pyright/blob/main/docs/installation.md), which includes Pyright. Run it [in CI to keep the project type clean](https://github.com/microsoft/pyright/blob/main/docs/getting-started.md), then turn on `# pyright: strict` file by file.
|
||||
|
||||
Pylint, Prospector, Import Linter, and ty all need to see your project's dependencies, so run them [from the project's own environment](https://pylint.readthedocs.io/en/latest/user_guide/installation/pre-commit-integration.html), not a separate one. For more Python code analysis tools, see [awesome-static-analysis](https://github.com/analysis-tools-dev/static-analysis).
|
||||
@@ -0,0 +1,27 @@
|
||||
OpenCV processes images and video. Training your own detector needs more than a Python computer vision library scoped to inference, so reach for Ultralytics.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Reading, editing, and analyzing images and video: OpenCV
|
||||
- Detecting, segmenting, or tracking objects with a trained model: Ultralytics
|
||||
- Image operations inside a PyTorch model or training loop: Kornia
|
||||
- Browsing a dataset, finding label mistakes, and seeing where a model fails: FiftyOne
|
||||
- Printed text in clean scans: pytesseract
|
||||
- Text in photos or degraded scans: EasyOCR
|
||||
- Tables and page layouts, turned into Markdown or JSON: PaddleOCR
|
||||
|
||||
OpenCV's Python bindings [convert every OpenCV array to and from a NumPy array](https://github.com/opencv/opencv/blob/5.x/doc/py_tutorials/py_setup/py_intro/py_intro.markdown), so the images you load work with any other library that uses NumPy.
|
||||
|
||||
Ultralytics runs YOLO models for detection, segmentation, pose estimation, and tracking from [one Python package and CLI](https://docs.ultralytics.com/). Load a pretrained model and [call `.train()` on your own dataset](https://docs.ultralytics.com/guides/finetuning-guide/): the weights carry over with no extra code. Ultralytics is AGPL, so [using it without open-sourcing your project](https://www.ultralytics.com/license) takes its Enterprise License.
|
||||
|
||||
Kornia is [a differentiable computer vision library like OpenCV, with strong GPU support](https://kornia.readthedocs.io/en/latest/get-started/introduction.html), built on PyTorch tensors. Its FAQ states the split: Kornia [provides operators to train networks, while OpenCV's scope is inference](https://kornia.readthedocs.io/en/latest/community/faqs.html).
|
||||
|
||||
FiftyOne is for building better datasets. [Load your images and labels as a Dataset](https://docs.voxel51.com/user_guide/basics.html) to search and sort samples and spot label mistakes. Then add your model's predictions and [evaluate them sample by sample](https://docs.voxel51.com/user_guide/evaluation/index.html) in the FiftyOne App to see where it goes wrong.
|
||||
|
||||
pytesseract is [a wrapper for Google's Tesseract engine](https://github.com/madmaze/pytesseract), which you install on its own. `image_to_string()` returns the text, and `image_to_data()` adds boxes and confidences.
|
||||
|
||||
EasyOCR reads [many languages and scripts](https://github.com/JaidedAI/EasyOCR) with deep learning models on PyTorch. [Create one `Reader` and reuse it](https://www.jaided.ai/easyocr/tutorial/): it loads the models into memory once, then reads as many images as you give it.
|
||||
|
||||
PaddleOCR does two jobs: [general OCR](https://www.paddleocr.ai/latest/en/version3.x/pipeline_usage/OCR.html) reads text, and document parsing [turns PDFs and document images into Markdown and JSON](https://www.paddleocr.ai/latest/en/index.html) that keep the original structure.
|
||||
|
||||
Load and crop images with OpenCV, then pass the NumPy arrays on: [EasyOCR](https://www.jaided.ai/easyocr/tutorial/), [PaddleOCR](https://www.paddleocr.ai/latest/en/version3.x/pipeline_usage/OCR.html), and [Ultralytics](https://docs.ultralytics.com/modes/predict/) all take them as input.
|
||||
@@ -0,0 +1,21 @@
|
||||
Settings that change between deployments belong in the environment, whatever Python configuration library you use. pydantic-settings reads them as typed fields.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Typed, validated settings from environment variables and `.env` files: pydantic-settings
|
||||
- An INI file your users edit, with nothing to install: configparser
|
||||
- A `.env` file loaded into the environment during development: python-dotenv
|
||||
- Research code that composes its config and overrides it from the command line: Hydra
|
||||
- Layered settings per environment, or settings for Django and Flask: Dynaconf
|
||||
|
||||
pydantic-settings loads [a settings class from environment variables or secrets files](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/). Subclass `BaseSettings` and declare each setting as a type-hinted field. [Any field you don't pass to the initializer gets its value from the environment](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#usage), or its default when the variable isn't set. To read a `.env` file too, [set `env_file` in `model_config`](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#dotenv-env-support), and its values get validated like the rest.
|
||||
|
||||
configparser comes with Python. It reads [a basic configuration language structured like Windows INI files](https://docs.python.org/3/library/configparser.html), so end users can customize your program by editing a file. Read values with mapping access, `config['section']['option']`, which the docs [prefer for new projects](https://docs.python.org/3/library/configparser.html#legacy-api-examples) over the legacy get and set methods. Values always come back as strings, so convert them with [`getint()`, `getfloat()`, and `getboolean()`](https://docs.python.org/3/library/configparser.html#supported-datatypes).
|
||||
|
||||
python-dotenv is for apps that [take their configuration from environment variables](https://saurabh-kumar.com/python-dotenv/#getting-started). In development, setting each one yourself isn't practical, so call `load_dotenv()` before the rest of your code. It reads a `.env` file when there is one and adds its values to `os.environ`, so your code reads them with `os.getenv()` as if they came from the real environment.
|
||||
|
||||
Hydra is a framework for [research and other complex applications](https://hydra.cc/docs/1.3/intro/). It builds a hierarchical configuration by composition, and you override it from config files and the command line. Decorate your entry point with [`@hydra.main()`](https://hydra.cc/docs/1.3/intro/#basic-example) and point it at a YAML config. Then change a value with an argument like `db.user=root`. For options you switch between, like two databases, [create a config group](https://hydra.cc/docs/1.3/intro/#composition-example) with one file per option, and pick one in the config's `defaults` list. With [`--multirun`](https://hydra.cc/docs/1.3/intro/#multirun), one command runs your function once for each configuration you list.
|
||||
|
||||
Dynaconf reads settings from [files in several formats, environment variables, and Vault or Redis](https://www.dynaconf.com/#features). Start with [`dynaconf init -f toml`](https://www.dynaconf.com/#using-python-only), which creates `config.py`, `settings.toml`, and `.secrets.toml`. Then import `settings` from `config` in your code. TOML is its default and most recommended format. To give development and production their own sections in one file, [set `environments=True`](https://www.dynaconf.com/#layered-environments-on-files). In a Django app, `django.conf.settings` [becomes a Dynaconf settings object](https://www.dynaconf.com/#using-django), and in Flask, `app.config` does.
|
||||
|
||||
Keep settings that change between deployments out of your code. The [twelve-factor app stores config in environment variables](https://12factor.net/config), and python-dotenv and Dynaconf both cite it. python-dotenv, pydantic-settings, and Dynaconf let the real environment win over files. `load_dotenv()` [doesn't override variables already set](https://saurabh-kumar.com/python-dotenv/#getting-started), pydantic-settings [ranks environment variables above `.env` and secrets files](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#field-value-priority), and Dynaconf [prioritizes environment variables over files](https://www.dynaconf.com/envvars/). Keep secrets out of Git, too: add `.env` to your `.gitignore`, and `dynaconf init` adds `.secrets.*` there for you.
|
||||
@@ -0,0 +1,19 @@
|
||||
Don't touch a Python cryptography library's raw ciphers: encrypt with cryptography's Fernet or PyNaCl. Paramiko runs SSH, and ItsDangerous signs tokens.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Encrypting data, X.509 certificates, and other standard formats: cryptography
|
||||
- Encryption and signatures between your own apps, with algorithms picked for you: PyNaCl
|
||||
- Hashing passwords for storage: PyNaCl
|
||||
- SSH clients and servers, and SFTP: Paramiko
|
||||
- Signed links, tokens, and cookies that users may read but not change: ItsDangerous
|
||||
|
||||
cryptography has 2 layers: recipes that need few decisions, and low-level primitives it calls the "hazmat" layer. Its docs [recommend the recipes layer whenever possible](https://cryptography.io/en/stable/#layout), and hazmat only when necessary. For data you encrypt with a key, the recipe is [Fernet](https://cryptography.io/en/stable/fernet/): a message encrypted with it can't be read or changed without the key. To encrypt with a password, run it through a key derivation function first. The docs [recommend Argon2id](https://cryptography.io/en/stable/fernet/#using-passwords-with-fernet), and you keep the salt to derive the same key again. When Fernet doesn't fit, the docs point you to [authenticated encryption](https://cryptography.io/en/stable/hazmat/primitives/symmetric-encryption/) before any raw cipher, since encryption alone keeps data secret but doesn't stop tampering. The recipes also cover [X.509 certificates](https://cryptography.io/en/stable/x509/tutorial/), from signing requests to self-signed certs.
|
||||
|
||||
PyNaCl is a binding to libsodium, a fork of NaCl. cryptography's FAQ explains the split: cryptography is [general purpose and interoperable with existing systems](https://cryptography.io/en/stable/faq/#how-does-cryptography-compare-to-nacl-networking-and-cryptography-library), while NaCl gives you a set of hand-selected algorithms. If you prefer NaCl's design, the FAQ recommends PyNaCl. With a shared key, encrypt through `SecretBox` or `Aead` and let PyNaCl [generate a random nonce](https://pynacl.readthedocs.io/en/latest/secret/#nacl.secret.Aead.encrypt) for each message, which its docs strongly recommend. Between 2 parties, `Box` [authenticates both sides](https://pynacl.readthedocs.io/en/latest/public/#nacl-public-box), and `SealedBox` sends a message only the recipient can decrypt, without proving who sent it. To store passwords, `nacl.pwhash.str()` [hashes with argon2id by default](https://pynacl.readthedocs.io/en/latest/password_hashing/#password-storage-and-verification), and `nacl.pwhash.verify()` checks a password against the hash.
|
||||
|
||||
Paramiko implements SSHv2 in pure Python, as both client and server. For common client jobs like running remote commands or transferring files, its homepage [recommends a higher-level library built on it](https://www.paramiko.org/). Use Paramiko directly for low-level primitives or to run an SSH server in Python. Its client API [starts with `SSHClient`](https://docs.paramiko.org/en/latest/), and checking the server's host key is your job: call `load_system_host_keys()` before `connect()`. A host key it can't find is [rejected by default](https://docs.paramiko.org/en/latest/api/client.html#paramiko.client.SSHClient.set_missing_host_key_policy) with an `SSHException`. `open_sftp()` opens an SFTP session on the same connection.
|
||||
|
||||
ItsDangerous signs data, so you can send it somewhere untrusted and get it back: the receiver [can see the data but can't modify it](https://itsdangerous.palletsprojects.com/en/stable/) without your key. Flask's default sessions are [cookies signed with it](https://flask.palletsprojects.com/en/stable/api/#flask.sessions.SecureCookieSessionInterface). Signing hides nothing, so when the data must stay secret, encrypt it with Fernet instead. Its docs say you'll [typically want a serializer, not a signer](https://itsdangerous.palletsprojects.com/en/stable/concepts/#serializer-vs-signer). `URLSafeSerializer` turns data into a URL-safe string, and `URLSafeTimedSerializer` rejects tokens older than the `max_age` you pass to `loads()`. Give each use its own [salt](https://itsdangerous.palletsprojects.com/en/stable/concepts/#the-salt), so an activation link's signature won't pass as an upgrade link's.
|
||||
|
||||
Whichever pick you use, keep secret keys out of your code. ItsDangerous's docs say the key [shouldn't be saved in source code or committed to version control](https://itsdangerous.palletsprojects.com/en/stable/concepts/#the-secret-key), and suggest reading it from an environment variable. When you generate a key or token yourself, use `os.urandom()`, [never the `random` module](https://cryptography.io/en/stable/random-numbers/), which isn't cryptographically secure. Plan for rotation, too: cryptography's [`MultiFernet`](https://cryptography.io/en/stable/fernet/#cryptography.fernet.MultiFernet) and ItsDangerous both [take a list of keys](https://itsdangerous.palletsprojects.com/en/stable/concepts/#key-rotation), so old tokens keep working after you add a new key.
|
||||
@@ -0,0 +1,17 @@
|
||||
Once your data nears the size of your RAM, switch Python data analysis libraries from pandas to Polars. Once it lives in a database, Ibis runs your code there.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Data that fits in memory: pandas
|
||||
- Handing DataFrames to libraries that expect pandas: pandas
|
||||
- Large data on one machine, even bigger than your RAM: Polars
|
||||
- Data already in a database or warehouse: Ibis
|
||||
- Code you prototype locally, then run on a warehouse or cluster: Ibis
|
||||
|
||||
pandas aims to be [the fundamental high-level building block](https://pandas.pydata.org/docs/getting_started/overview.html) for practical data analysis in Python. Its DataFrame fits time series and tables with mixed column types, like an SQL table or a spreadsheet. It's built on NumPy to work with the rest of the scientific Python stack. In production code, the docs [recommend `.loc` and `.iloc` over `[]`](https://pandas.pydata.org/docs/user_guide/indexing.html) to select data. Set values in [one `.loc` call](https://pandas.pydata.org/docs/user_guide/copy_on_write.html#chained-assignment), not chained indexing like `df["foo"][mask] = 100`. pandas keeps everything in memory, so a dataset that takes a sizable share of your RAM [gets unwieldy](https://pandas.pydata.org/docs/user_guide/scale.html), and the docs point you to other libraries.
|
||||
|
||||
Polars is [built for multithreaded computing on a single machine](https://docs.pola.rs/user-guide/misc/comparison/#pandas), and its stricter API leads to fewer schema bugs. It has no index: rows are known by their position, so [no index state can change what a query means](https://docs.pola.rs/user-guide/migration/pandas/#polars-does-not-have-a-multi-indexindex). Make the lazy API [your default](https://docs.pola.rs/user-guide/migration/pandas/#be-lazy): start from a function like `scan_csv()` or call `.lazy()`, then `.collect()` the result. That lets the optimizer [filter rows and pick columns while reading the data](https://docs.pola.rs/user-guide/concepts/lazy-api/). Stay eager for exploratory work, when [you don't know yet what your query will look like](https://docs.pola.rs/user-guide/concepts/lazy-api/#when-to-use-which). Write expressions inside `select`, `with_columns`, `filter`, and `group_by`, since Polars code that [looks like pandas code](https://docs.pola.rs/user-guide/migration/pandas/#key-syntax-differences) likely runs slower than it should. For data that doesn't fit in memory, run the query on the [streaming engine](https://docs.pola.rs/user-guide/concepts/streaming/), which works through it in batches.
|
||||
|
||||
Ibis [compiles one Python dataframe API](https://ibis-project.org/why#how-does-ibis-work) into each backend's native language, mostly SQL, so the database or engine does the work. pandas' ecosystem page lists it for [bridging local Python and remote databases](https://pandas.pydata.org/community/ecosystem.html#ibis). Start on the default DuckDB backend, as the tutorial [recommends](https://ibis-project.org/tutorials/basics#install-ibis), then [change the connection string](https://ibis-project.org/why#scaling-up-and-out) to run the same code on PySpark, BigQuery, or Trino. Expressions are lazy: nothing runs until you call a method like `to_pandas()`, and only then does Ibis [send the compiled query](https://ibis-project.org/tutorials/coming-from/pandas) to the backend.
|
||||
|
||||
You can mix all three. Polars converts a DataFrame [with `to_pandas()`](https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.to_pandas.html), Ibis returns results [as pandas or Polars DataFrames](https://ibis-project.org/reference/expression-tables#ibis.expr.types.relations.Table.to_pandas), and pandas [can use PyArrow](https://pandas.pydata.org/docs/user_guide/pyarrow.html) to trade data with Arrow-based libraries like Polars. Do the heavy work in Polars or Ibis, and hand a pandas DataFrame to the libraries that expect one.
|
||||
@@ -0,0 +1,24 @@
|
||||
dlt loads data from APIs, databases, and files into your warehouse. You pip install the Python ETL library into your own code, with no platform to run.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Data from APIs, databases, or files, loaded into a warehouse or DuckDB: dlt
|
||||
- pandas DataFrames in and out of S3, Athena, Redshift, and other AWS services: awswrangler
|
||||
- Stock prices, financials, and options from Yahoo Finance, for personal research: yfinance
|
||||
- Chinese stocks, futures, and funds, for academic research: AKShare
|
||||
- US company filings and financial statements from SEC EDGAR: EdgarTools
|
||||
- Many data providers behind one Python API, also served over REST and MCP: OpenBB
|
||||
|
||||
dlt [loads data from messy sources into well-structured datasets](https://dlthub.com/docs/intro): it infers the schema and data types, normalizes the data, and handles nested structures. Start a project with [`dlt init rest_api duckdb`](https://dlthub.com/docs/tutorial/rest-api), which sets up a pipeline script with a REST API source and a DuckDB destination. Build and test on DuckDB, then [switch out the destination](https://dlthub.com/docs/reference/explainers/how-dlt-works) when you deploy. Beyond the built-in sources, any Python iterable can feed it, since [a resource is just a generator](https://github.com/dlt-hub/dlt). dlt [runs anywhere Python runs](https://dlthub.com/docs/walkthroughs/deploy-a-pipeline), so schedule it with the orchestrator you already have.
|
||||
|
||||
AWS SDK for pandas [extends pandas to AWS](https://aws-sdk-pandas.readthedocs.io/en/stable/about.html): import it as `awswrangler`, and it connects DataFrames to S3, Athena, Glue, Redshift, and DynamoDB. Its [quick start](https://github.com/aws/aws-sdk-pandas) stores a DataFrame on S3 as a Parquet dataset with `wr.s3.to_parquet()`, passing a database and table name. Then `wr.athena.read_sql_query()` queries that table with SQL.
|
||||
|
||||
yfinance offers [a Pythonic way to fetch financial and market data](https://github.com/ranaroussi/yfinance) from Yahoo Finance. Its [quick start](https://ranaroussi.github.io/yfinance/) uses `Ticker` for one symbol's prices, financial statements, and options, and `download` for prices across several symbols at once.
|
||||
|
||||
AKShare [aims to simplify fetching financial data](https://github.com/akfamily/akshare), with one function call per dataset, like `ak.stock_zh_a_hist()` for the daily history of a Chinese stock. Its README credits mostly Chinese exchanges and finance sites as its sources. Its [docs are in Chinese](https://akshare.akfamily.xyz/introduction.html), and each data interface comes with an example you can copy and paste.
|
||||
|
||||
EdgarTools [turns any SEC filing into a typed Python object](https://github.com/dgunning/edgartools), so a 10-K's revenue is one line instead of an afternoon of XBRL parsing. EDGAR requires an email with every request, so [set your identity](https://edgartools.readthedocs.io/en/latest/configuration/) with `set_identity()` before anything else. Everything starts with a `Company` or a `Filing`. `Company("AAPL").get_financials().income_statement()` returns a standardized income statement, and `.obj()` turns a filing into an object built for its form, with its data as pandas DataFrames.
|
||||
|
||||
OpenBB is a ["connect once, consume everywhere" layer](https://docs.openbb.co/odp) over financial data sources. [Each provider package](https://docs.openbb.co/odp/python) connects one source and adds its own namespace to the `obb` client. The same commands run from Python, as a REST API, and as MCP tools. A call returns an OBBject, and [`to_dataframe()`](https://docs.openbb.co/odp/python/quickstart/user) turns it into a pandas DataFrame.
|
||||
|
||||
Before you build on financial data, check whose terms you're under. yfinance [isn't affiliated with Yahoo](https://ranaroussi.github.io/yfinance/) and is meant for research and education, and Yahoo's API is for personal use only. AKShare's data is [for academic research](https://github.com/akfamily/akshare#statement) only. OpenBB [hosts no data itself](https://docs.openbb.co/odp/python/extensions/providers): each provider sets its own coverage and terms of use.
|
||||
@@ -0,0 +1,18 @@
|
||||
Pydantic checks API input against your type hints. Python data validation extends to dataframes in Pandera, where each column gets a type hint.
|
||||
|
||||
How to choose:
|
||||
|
||||
- API payloads and other records, typed with hints: Pydantic
|
||||
- Dataframes in pandas, Polars, PySpark, and more: Pandera
|
||||
- Data checked against a JSON Schema document: jsonschema
|
||||
- Data quality checks on databases and files, with alerts and reports: Great Expectations
|
||||
|
||||
Pydantic [takes its schema from Python type hints](https://pydantic.dev/docs/validation/latest/get-started/why/#type-hints), so your type checker and IDE read the same code it validates. Subclass `BaseModel` and annotate its fields, then pass untrusted data to the model: if no `ValidationError` is raised, [every field matches its declared type](https://pydantic.dev/docs/validation/latest/concepts/models/). Your models can also [generate JSON Schema](https://pydantic.dev/docs/validation/latest/get-started/why/#json-schema) for self-documenting APIs and other tools.
|
||||
|
||||
Pandera [validates pandas, Polars, PySpark, and other dataframes against one schema](https://pandera.readthedocs.io/en/latest/). Write that schema as a `DataFrameModel`, [much the way you'd define a Pydantic model](https://pandera.readthedocs.io/en/latest/dataframe_models.html), with a typed field per column. Then decorate your pipeline functions with `check_types()`, which [validates both inputs and outputs](https://pandera.readthedocs.io/en/latest/decorators.html#check-inputs-and-outputs) from their type annotations.
|
||||
|
||||
jsonschema [implements the JSON Schema specification](https://python-jsonschema.readthedocs.io/en/latest/), so the schema is a JSON document rather than Python code. That suits a schema that has to work outside Python too, since JSON Schema [establishes a common language for data exchange](https://json-schema.org/overview/what-is-jsonschema) across systems. Declare `$schema` in each schema to [identify which version it's written for](https://python-jsonschema.readthedocs.io/en/latest/referencing/).
|
||||
|
||||
Great Expectations is [a framework for describing data with expressive tests](https://docs.greatexpectations.io/docs/core/introduction/gx_overview/#gx-core-components-and-workflows) and validating data against them, from databases to files in cloud storage. Every script [starts by creating a Data Context](https://docs.greatexpectations.io/docs/core/set_up_a_gx_environment/create_a_data_context/). Group your production Expectations into [Expectation Suites](https://docs.greatexpectations.io/docs/core/define_expectations/organize_expectation_suites/), and run those through [a Checkpoint](https://docs.greatexpectations.io/docs/core/introduction/gx_overview/#run-validations), which can send email or Slack alerts and write the results to Data Docs, a human-readable report.
|
||||
|
||||
Records and tables need different validators, but the picks work together. A Pandera `DataFrameModel` [works as a field on a Pydantic model](https://pandera.readthedocs.io/en/latest/pydantic_integration.html), with Pydantic checking the built-in types next to it. Pandera's maintainer suggests [Pandera for in-memory dataframes and Great Expectations for data on disk](https://github.com/unionai-oss/pandera/discussions/598). Pandera needs zero configuration, while Great Expectations takes some setup up front.
|
||||
@@ -0,0 +1,39 @@
|
||||
Charts for print come from Matplotlib, and interactive web ones from Plotly. Put either Python data visualization library in a Streamlit script for a dashboard.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Publication figures you control down to the last detail: Matplotlib
|
||||
- Interactive charts with little code: Plotly
|
||||
- A dashboard or data app from a plain Python script: Streamlit
|
||||
- Statistical charts straight from a pandas DataFrame: seaborn
|
||||
- Charts you declare by mapping columns to x, y, and color: Vega-Altair
|
||||
- Interactive browser plots with full control, or Python code behind them: Bokeh
|
||||
- Diagrams of nodes and edges, drawn by Graphviz: graphviz
|
||||
- Maps of large areas, in the projection you choose: Cartopy
|
||||
- A knowledge graph of your codebase for your AI coding assistant: graphify
|
||||
- Dashboards built from callbacks and custom layouts: Dash
|
||||
- A demo UI for an ML model or any Python function: Gradio
|
||||
|
||||
Matplotlib [makes easy things easy and hard things possible](https://matplotlib.org/), and it's built for publication-quality plots. Start with `fig, ax = plt.subplots()` and draw on the `Axes` it returns. The docs [suggest this object-oriented style](https://matplotlib.org/stable/users/explain/quick_start.html#coding-styles) for complicated plots and for code you'll reuse. The pyplot style, like `plt.plot()`, is handy for quick interactive work.
|
||||
|
||||
Plotly draws interactive charts that show up in Jupyter or [run inside Dash apps](https://plotly.com/python/getting-started/). Start with Plotly Express, the [recommended starting point](https://plotly.com/python/plotly-express/) for most common figures. Every Plotly Express function returns a graph objects `Figure`, so you can still fine-tune it with `update_layout()` and `add_trace()`.
|
||||
|
||||
Streamlit turns a Python script into a web app: add a few `st.` calls, then run it with `streamlit run`. Whenever something on the screen must update, Streamlit [reruns your entire script from top to bottom](https://docs.streamlit.io/get-started/fundamentals/main-concepts#data-flow). So wrap slow work in a cached function: [`st.cache_data`](https://docs.streamlit.io/develop/concepts/architecture/caching) for data like a loaded DataFrame or a query result, and `st.cache_resource` for ML models and database connections.
|
||||
|
||||
seaborn builds on Matplotlib and [integrates closely with pandas](https://seaborn.pydata.org/tutorial/introduction.html): you name DataFrame columns, and it draws the statistical chart. Its docs [recommend figure-level functions](https://seaborn.pydata.org/tutorial/function_overview.html#relative-merits-of-figure-level-functions) like `displot()` for most plots.
|
||||
|
||||
Vega-Altair is declarative: you [declare links between data columns and encoding channels](https://altair-viz.github.io/getting_started/overview.html) like the x-axis, y-axis, and color, and it handles the rest.
|
||||
|
||||
Bokeh [creates JavaScript-powered visualizations without writing any JavaScript](https://docs.bokeh.org/en/latest/index.html). Build plots with `figure()` from `bokeh.plotting`, its primary interface. Add the Bokeh server when your plots need to [respond to browser events with Python code](https://docs.bokeh.org/en/latest/docs/user_guide/server/server_introduction.html), like widgets that rerun a query.
|
||||
|
||||
graphviz writes graphs in the DOT language, and [renders them with Graphviz](https://graphviz.readthedocs.io/en/latest/), which you install separately. Build a `Graph` or `Digraph`, add nodes and edges, then call [`render()`](https://graphviz.readthedocs.io/en/latest/manual.html) to save the DOT source and the rendered file.
|
||||
|
||||
Cartopy draws maps on Matplotlib, and its docs say it's [especially useful for large areas](https://cartopy.readthedocs.io/latest/), where flat-map assumptions break down at the poles and the dateline. [Give the axes a map projection](https://cartopy.readthedocs.io/latest/matplotlib/intro.html), like `plt.axes(projection=ccrs.PlateCarree())`, then add coastlines and your data. Your data's coordinate system is separate from that projection, so [always pass `transform=`](https://cartopy.readthedocs.io/latest/tutorials/understanding_transform.html) when you plot.
|
||||
|
||||
graphify maps your project into a knowledge graph that your AI coding assistant can [query instead of grepping](https://github.com/Graphify-Labs/graphify) through files. Code is parsed locally, with no LLM, and you can click through the graph in a browser.
|
||||
|
||||
Dash [ties UI elements like dropdowns, sliders, and graphs](https://github.com/plotly/dash) to your Python code, and draws charts with Plotly through `dcc.Graph`. You add interactivity with [callbacks](https://dash.plotly.com/basic-callbacks): functions Dash calls whenever an input changes, to update another component.
|
||||
|
||||
Gradio [builds a demo or web app](https://www.gradio.app/guides/quickstart) for an ML model, an API, or any Python function, with no JavaScript or CSS. Start with `gr.Interface`, which takes your function plus its inputs and outputs. Use `gr.ChatInterface` for a chatbot, and `gr.Blocks` when you need your own layout and data flow.
|
||||
|
||||
Pick the chart library and the way you share it separately. Plotly, Vega-Altair, and Bokeh all save interactive charts as standalone HTML files: [`write_html()`](https://plotly.com/python/interactive-html-export/), [`chart.save("chart.html")`](https://altair-viz.github.io/user_guide/saving_charts.html), and [`output_file()`](https://docs.bokeh.org/en/latest/docs/user_guide/intro.html). Streamlit has [chart elements](https://docs.streamlit.io/develop/api-reference/charts) for Matplotlib, Vega-Altair, Plotly, and Graphviz, and Gradio's [`gr.Plot`](https://www.gradio.app/docs/gradio/plot) takes Matplotlib, Plotly, Vega-Altair, and Bokeh figures.
|
||||
@@ -0,0 +1,36 @@
|
||||
Each database calls for a Python database driver built for it. Reach PostgreSQL through Psycopg, and MySQL through mysqlclient. sqlite3 comes with Python.
|
||||
|
||||
How to choose:
|
||||
|
||||
- PostgreSQL, from sync or async code: Psycopg
|
||||
- MySQL or MariaDB, Django projects included: mysqlclient
|
||||
- SQLite with nothing to install: sqlite3
|
||||
- MySQL in pure Python: PyMySQL
|
||||
- An asyncio-only PostgreSQL service: asyncpg
|
||||
- Loading JSON or CSV into SQLite, from Python or the shell: sqlite-utils
|
||||
- ClickHouse, even behind proxies and load balancers: clickhouse-connect
|
||||
- ClickHouse over its native TCP protocol: clickhouse-driver
|
||||
- Any database with an ODBC driver: pyodbc
|
||||
- Oracle Database: python-oracledb
|
||||
- SQL Server or Azure SQL, with no driver manager to set up: mssql-python
|
||||
- Redis, MongoDB, or Apache Cassandra: redis-py, PyMongo, or cassandra-driver
|
||||
|
||||
Psycopg is the PostgreSQL driver [Django recommends](https://docs.djangoproject.com/en/stable/ref/databases/#postgresql-notes). One design serves sync and async code: [`AsyncConnection` and `AsyncCursor`](https://www.psycopg.org/psycopg3/docs/advanced/async.html#asynchronous-operations) mirror the sync classes, so you mostly add `await`. Open each connection in a [`with` block](https://www.psycopg.org/psycopg3/docs/basic/usage.html#connection-context). It commits when the block ends, rolls back on an exception, and closes the connection either way.
|
||||
|
||||
asyncpg is built for asyncio only, and [skips DB-API on purpose](https://magicstack.github.io/asyncpg/current/faq.html) to fit PostgreSQL better. In a server, [create a pool](https://magicstack.github.io/asyncpg/current/usage.html#connection-pools) with `asyncpg.create_pool()` at startup, then take a connection per request with `async with pool.acquire()`.
|
||||
|
||||
Django calls mysqlclient [the recommended choice](https://docs.djangoproject.com/en/stable/ref/databases/#mysql-db-api-drivers) for MySQL, and ships its own adapter for it. It's a native driver built against MySQL's client library. Where you can't install that library and its headers, use PyMySQL, which SQLAlchemy calls [a pure Python port](https://docs.sqlalchemy.org/en/latest/dialects/mysql.html#mysql-python-compatibility) of MySQLdb that targets full compatibility. The licenses differ too: mysqlclient is [GPL](https://github.com/PyMySQL/mysqlclient/blob/main/LICENSE), and PyMySQL is MIT.
|
||||
|
||||
sqlite3 comes with Python, and SQLite needs no server. Use the connection as a [context manager](https://docs.python.org/3/library/sqlite3.html#sqlite3-connection-context-manager): it commits when the block succeeds and rolls back when it raises.
|
||||
|
||||
sqlite-utils sits on top of sqlite3 to create and fill databases, not to be a full ORM. Pass [`insert_all()`](https://sqlite-utils.datasette.io/en/latest/python-api.html#bulk-inserts) a list of dicts, and it creates the table and columns for you. Its command-line tool pipes JSON or CSV straight into a new database file.
|
||||
|
||||
clickhouse-connect is ClickHouse's official client and [talks HTTP](https://clickhouse.com/docs/integrations/language-clients/python), so it works through load balancers, proxies, and enterprise network controls. clickhouse-driver uses the [native TCP protocol](https://clickhouse-driver.readthedocs.io/en/latest/) instead, which its docs say is more configurable and sends compact binary data.
|
||||
|
||||
pyodbc reaches any database through that database's ODBC driver, plus one [driver manager](https://github.com/mkleehammer/pyodbc/wiki/Drivers-and-Driver-Managers) for all of them. Windows has one built in; on macOS and Unix, you install one first. For SQL Server and Azure SQL, Microsoft's own mssql-python [connects without an external driver manager](https://learn.microsoft.com/en-us/sql/connect/python/mssql-python/python-sql-driver-mssql-python).
|
||||
|
||||
python-oracledb is Oracle's official driver. In its default [Thin mode](https://python-oracledb.readthedocs.io/en/latest/user_guide/initialization.html), it connects straight to Oracle Database without Oracle Client libraries.
|
||||
|
||||
redis-py is [the Python client for Redis](https://redis.io/docs/latest/develop/clients/redis-py/), and MongoDB's docs call PyMongo [the recommended way](https://www.mongodb.com/docs/languages/python/pymongo-driver/current/) to work with MongoDB from Python. cassandra-driver talks to Apache Cassandra over Cassandra's binary protocol, with sync and async APIs.
|
||||
|
||||
With any SQL database, pass values as query parameters instead of formatting them into the SQL string. The sqlite3 docs say this [avoids SQL injection](https://docs.python.org/3/library/sqlite3.html#sqlite3-placeholders). For more per database, see [awesome-mysql](https://github.com/shlomi-noach/awesome-mysql), [awesome-postgres](https://github.com/dhamaniasad/awesome-postgres), and [awesome-sqlite](https://github.com/planetopendata/awesome-sqlite).
|
||||
@@ -0,0 +1,22 @@
|
||||
Inside your process, a Python database library runs without a server. DuckDB handles analytical SQL there; to search vectors, start with Chroma.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Analytical SQL over Parquet, CSV, or DataFrames: DuckDB
|
||||
- Vector search in a prototype or notebook: Chroma
|
||||
- ClickHouse's SQL dialect, or pandas code that runs on ClickHouse: chDB
|
||||
- Vectors stored with their raw images and text, on disk: LanceDB
|
||||
- Vector search embedded in an app or an edge device: Zvec
|
||||
- A small JSON document store for one script: TinyDB
|
||||
|
||||
Start DuckDB with `duckdb.sql()`, which runs on an [in-memory database kept inside the module](https://duckdb.org/docs/current/clients/python/overview#basic-api-usage). It queries Parquet, CSV, and JSON files by name, and [pandas DataFrames, Polars DataFrames, and Arrow tables](https://duckdb.org/docs/current/clients/python/overview#dataframes) by variable name, with no import step. When the data should stay, [pass a file name to `duckdb.connect()`](https://duckdb.org/docs/current/clients/python/overview#persistent-storage).
|
||||
|
||||
chDB runs ClickHouse inside your process, so you get its SQL dialect without a ClickHouse server. If you come from pandas, [start with the DataStore API](https://clickhouse.com/docs/chdb#for-pandas-users): change the import, and your pandas code works unchanged. For SQL, use the [connection-based API](https://clickhouse.com/docs/chdb/install/python#connection-based-api), and query DataFrames with the [`Python()` table engine](https://clickhouse.com/docs/chdb/install/python#python-table-engine-recommended).
|
||||
|
||||
Chroma's [in-memory client](https://docs.trychroma.com/docs/run-chroma/clients) is the quick way to try it in a notebook, and `PersistentClient` keeps the data on disk. Add text documents, and Chroma [embeds and indexes them for you](https://docs.trychroma.com/docs/overview/getting-started), so you can query with text too.
|
||||
|
||||
LanceDB starts as an [embedded library pointed at a local directory](https://docs.lancedb.com/quickstart). Its indexes live on disk, and a table [stores the raw data, metadata, and embeddings together](https://docs.lancedb.com/faq/faq-oss), which suits images and other multimodal data. Its [embedding function registry](https://docs.lancedb.com/embedding) generates vectors as you insert rows, and Python code can then query with text.
|
||||
|
||||
Zvec runs entirely in-process, with no server or daemon, and its docs pitch it [from prototypes to embedded apps and edge devices](https://zvec.org/en/docs/db/). Its [quickstart](https://zvec.org/en/docs/db/quickstart/) defines a schema of scalar and vector fields first, then combines vector search with filters on those scalar fields.
|
||||
|
||||
TinyDB is pure Python with no dependencies, and stores Python dicts in a JSON file: [open it with `TinyDB('db.json')` and search with `Query()`](https://tinydb.readthedocs.io/en/latest/getting-started.html). Its own docs say to [pick something else](https://tinydb.readthedocs.io/en/latest/intro.html#why-not-use-tinydb) when you need access from several processes or threads, indexes, or ACID guarantees.
|
||||
@@ -0,0 +1,21 @@
|
||||
Handle time zones with the built-in zoneinfo. Parsing date strings and adding months is where a Python date library earns its install: python-dateutil.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Time zones by IANA name, like Europe/Paris: zoneinfo
|
||||
- Parsing date strings, adding months, and recurring dates: python-dateutil
|
||||
- Dates as people write them, like "3 days ago", in many languages: dateparser
|
||||
- An easier datetime API whose objects are still datetimes: pendulum
|
||||
- Exact and local times as separate types, with DST-safe math: whenever
|
||||
|
||||
zoneinfo brings the IANA time zone database to Python, and the datetime docs say [its usage is recommended](https://docs.python.org/3/library/datetime.html#tzinfo-objects). Attach a ZoneInfo to a datetime [through the constructor, `replace()`, or `astimezone()`](https://docs.python.org/3/library/zoneinfo.html#using-zoneinfo). Some systems, Windows among them, have no IANA database, so if your code runs across platforms, [declare a dependency on tzdata](https://docs.python.org/3/library/zoneinfo.html#data-sources).
|
||||
|
||||
python-dateutil adds [extensions to the standard datetime module](https://dateutil.readthedocs.io/en/stable/); install it as `python-dateutil` and import it as `dateutil`. Its parser [reads most known formats](https://dateutil.readthedocs.io/en/stable/parser.html) and returns a datetime even for an ambiguous date. For input like 01/05/09, set `dayfirst` or `yearfirst` to match your data. For calendar math, `relativedelta` takes [plural arguments that add and singular ones that replace](https://dateutil.readthedocs.io/en/stable/relativedelta.html): `months=+1` moves a month ahead, and `day=1` jumps to the first. `rrule` builds [recurring dates from iCalendar rules](https://dateutil.readthedocs.io/en/stable/rrule.html).
|
||||
|
||||
dateparser reads relative dates like "two weeks ago" and absolute ones in more than 200 language locales. Its docs say it [stands out](https://dateparser.readthedocs.io/en/latest/#common-use-cases) for scraped pages, logs, and other data from mixed sources, and for letting users type dates in their own words. Call `dateparser.parse()`, and [pass `languages` when you know them](https://dateparser.readthedocs.io/en/latest/#how-to-use), so it skips language detection. When you parse many dates from one source, [use `DateDataParser`](https://dateparser.readthedocs.io/en/latest/usage.html), which remembers the languages it has found.
|
||||
|
||||
pendulum's classes are [drop-in replacements for the native ones](https://pendulum.eustace.io/docs/#introduction), since they inherit from datetime. Every instance is time zone aware and in UTC by default. Its docs call aware datetimes [the preferred and recommended way](https://pendulum.eustace.io/docs/#instantiation) to use it. For tests, install `pendulum[test]` and [travel in time](https://pendulum.eustace.io/docs/#testing).
|
||||
|
||||
whenever puts exact time and local time in [separate types](https://whenever.readthedocs.io/en/latest/guide/choosing-a-type.html): an instant when only the moment matters, a zoned datetime when the local time matters too. Mixing up naive and aware [becomes a type error](https://whenever.readthedocs.io/en/latest/), and DST is handled in all arithmetic. A standard datetime [does no time zone adjustment](https://docs.python.org/3/library/datetime.html#datetime-objects) when you add a timedelta to it. In production, [turn whenever's DST warnings into errors](https://whenever.readthedocs.io/en/latest/faq.html#why-warnings-instead-of-errors) with Python's standard warnings filter.
|
||||
|
||||
Decide whether you extend datetime or replace it. zoneinfo, python-dateutil, and dateparser all use standard datetime objects, so they work together. pendulum's objects are datetimes too, but code that checks the exact type, like sqlite3 and some database drivers, [needs an adapter registered](https://pendulum.eustace.io/docs/#limitations). whenever [doesn't subclass datetime at all](https://whenever.readthedocs.io/en/latest/faq.html#why-no-drop-in-replacement-for-datetime), so [convert to and from standard datetimes](https://whenever.readthedocs.io/en/latest/guide/stdlib-convert.html) where other code needs one.
|
||||
@@ -0,0 +1,36 @@
|
||||
Stepping through code in ipdb gets you pdb with IPython's tab completion. Python debugging tools cover slow code too: py-spy profiles a running Python program.
|
||||
|
||||
How to choose:
|
||||
|
||||
- pdb with tab completion and syntax highlighting: ipdb
|
||||
- Finding where a running program spends its time, without changing it: py-spy
|
||||
- A full-screen debugger in your terminal: PuDB
|
||||
- A timeline of every function call: VizTracer
|
||||
- Memory use and leaks, native extensions included: Memray
|
||||
- A readable call tree of where your script spends wall-clock time: pyinstrument
|
||||
- CPU, GPU, and memory use, line by line: Scalene
|
||||
- SQL queries and request data on Django pages: Django Debug Toolbar
|
||||
- Print debugging that labels each value: IceCream
|
||||
- The same toolbar on Flask pages: Flask-DebugToolbar
|
||||
|
||||
ipdb gives you IPython's debugger, with [tab completion, syntax highlighting, and better tracebacks](https://github.com/gotcha/ipdb), behind the same interface as pdb. Drop into it with `import ipdb; ipdb.set_trace()`, or call `ipdb.pm()` for post-mortem debugging after an exception.
|
||||
|
||||
py-spy is a sampling profiler that shows what a program spends its time on [without restarting it or modifying its code](https://github.com/benfred/py-spy). It runs outside the profiled process, which makes it safe to use on production code. `py-spy record` writes a [flame graph](https://github.com/benfred/py-spy#record) of a process ID or a command you hand it, and `py-spy dump` prints [the call stack where a program is hung](https://github.com/benfred/py-spy#dump).
|
||||
|
||||
PuDB keeps the source, the stack, breakpoints, and variables [all visible at once](https://documen.tician.de/pudb/) in one full-screen terminal view. Start it from your code with `from pudb import set_trace; set_trace()`, or [run a whole script](https://documen.tician.de/pudb/starting.html#starting-the-debugger) with `python -m pudb my-script.py`.
|
||||
|
||||
VizTracer records every function's entry and exit time across a whole run, which [helps catch sporadic performance issues](https://viztracer.readthedocs.io/en/latest/). Run your script [with `viztracer my_script.py`](https://viztracer.readthedocs.io/en/latest/basic_usage.html#command-line) instead of `python`, then open the `result.json` it writes with `vizviewer`.
|
||||
|
||||
Memray tracks allocations [in Python code, native extension modules, and the interpreter itself](https://bloomberg.github.io/memray/overview.html), to find memory leaks and the code behind high memory use. Profiling [takes two steps](https://bloomberg.github.io/memray/getting_started.html): `memray run` saves the allocations to a file, then `memray flamegraph` builds a report from it.
|
||||
|
||||
pyinstrument measures wall-clock time, so time spent [downloading data, reading files, and talking to databases](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#wall-clock-time-not-cpu-time) counts too. Type [`pyinstrument script.py`](https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-python-script) instead of `python script.py`, or wrap a block in `with pyinstrument.profile():`.
|
||||
|
||||
Scalene profiles CPU, GPU, and memory [line by line and per function](https://github.com/plasma-umass/scalene#fast-and-accurate). It [separates time in Python from time in native code](https://github.com/plasma-umass/scalene#cpu-profiling), so you can focus on code you can change. Run [`scalene run your_prog.py`](https://github.com/plasma-umass/scalene#using-scalene), then `scalene view` opens the profile in your browser.
|
||||
|
||||
Django Debug Toolbar adds [panels](https://django-debug-toolbar.readthedocs.io/en/latest/panels.html) to your pages for the current request's SQL queries, templates, cache calls, and more. [Its setup](https://django-debug-toolbar.readthedocs.io/en/latest/installation.html) adds the app, its URLs, and its middleware, and shows the toolbar only to IP addresses in `INTERNAL_IPS`.
|
||||
|
||||
IceCream's `ic()` [prints each expression along with its value](https://github.com/gruns/icecream). With no arguments, it prints [the file, line, and function it runs in](https://github.com/gruns/icecream#inspect-execution). Since `ic()` [returns its arguments](https://github.com/gruns/icecream#return-value), you can wrap an expression in existing code without rewriting the line.
|
||||
|
||||
Flask-DebugToolbar is [a port of Django Debug Toolbar](https://github.com/pallets-eco/flask-debugtoolbar) to Flask. Wrap your app in `DebugToolbarExtension(app)`, and the toolbar [shows up in HTML responses while debug mode is on](https://flask-debugtoolbar.readthedocs.io/en/latest/).
|
||||
|
||||
Both debuggers also open from the built-in `breakpoint()`: set the [`PYTHONBREAKPOINT` environment variable](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONBREAKPOINT) to `ipdb.set_trace` or `pudb.set_trace`. Among profilers, sampling ones like py-spy, pyinstrument, and Scalene [cost much less overhead than tracing ones](https://pyinstrument.readthedocs.io/en/latest/how-it-works.html#statistical-profiling-not-tracing), while a tracer like VizTracer records every call and [gives you more information](https://github.com/gaogaotiantian/viztracer#performance).
|
||||
@@ -0,0 +1,23 @@
|
||||
PyTorch suits new models, since it debugs like plain Python. Gymnasium environments pass arrays to whichever Python deep learning framework trains your agent.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Building and training neural networks: PyTorch
|
||||
- RL environments, built-in or your own: Gymnasium
|
||||
- A standard training loop scaled across GPUs: PyTorch Lightning
|
||||
- NumPy-style code compiled for GPUs and TPUs: JAX
|
||||
- One model API across JAX, TensorFlow, and PyTorch: Keras
|
||||
- An existing TensorFlow codebase: TensorFlow
|
||||
- Ready-made RL algorithms to train an agent: Stable Baselines3
|
||||
|
||||
PyTorch runs each line of code as it's reached, so [a debugger and stack traces read the way you'd expect](https://github.com/pytorch/pytorch#imperative-experiences). Its [quickstart](https://docs.pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html) shows the pieces of a project: a `Dataset` holds your samples and labels, and a `DataLoader` wraps it in an iterable. A model subclasses `nn.Module`, defines its layers in `__init__`, and passes data through them in `forward`. To keep a trained model, [save its `state_dict`](https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html) rather than pickling the whole model, which ties the file to your exact classes and directory layout.
|
||||
|
||||
PyTorch Lightning [organizes PyTorch code](https://github.com/Lightning-AI/pytorch-lightning#why-pytorch-lightning) so you write the model logic and it handles the engineering. A `LightningModule` is still an `nn.Module`, with your training step and `configure_optimizers` added, and the `Trainer` runs the loop around it. Moving to more GPUs, to TPUs, or to 16-bit precision is [a Trainer argument](https://github.com/Lightning-AI/pytorch-lightning#advanced-features), with no change to your model code.
|
||||
|
||||
JAX pairs a NumPy-like API with [composable transformations](https://github.com/jax-ml/jax#transformations): `jax.grad` takes gradients, `jax.jit` compiles, and `jax.vmap` vectorizes. They're designed for [functionally pure functions](https://docs.jax.dev/en/latest/notebooks/Common_Gotchas_in_JAX.html#pure-functions), so pass every input as an argument and return every result. The core holds little that's specific to deep learning. [For neural networks](https://docs.jaxstack.ai/en/latest/), add the JAX AI Stack: Flax for models, Optax for optimizers, and Orbax for checkpoints.
|
||||
|
||||
Gymnasium, a fork of OpenAI's Gym, is [a standard API for reinforcement learning](https://github.com/Farama-Foundation/Gymnasium): one interface between learning algorithms and environments, plus reference environments that follow it. Its [basic loop](https://gymnasium.farama.org/introduction/basic_usage/) comes down to `make()`, `Env.reset()`, `Env.step()`, and `Env.render()`, and wrappers change an environment without touching its code. For your own problem, [subclass `gymnasium.Env`](https://gymnasium.farama.org/introduction/create_custom_env/) with observation and action spaces, and register it so `gymnasium.make()` builds it like a built-in one.
|
||||
|
||||
Stable Baselines3 is a set of [reliable implementations of RL algorithms in PyTorch](https://github.com/DLR-RM/stable-baselines3), and it assumes you already know some RL. It follows [a scikit-learn-like syntax](https://stable-baselines3.readthedocs.io/en/master/guide/quickstart.html): create a model such as `A2C("MlpPolicy", env)`, call `learn()`, then `predict()`. It trains on any environment that [follows the Gymnasium interface](https://stable-baselines3.readthedocs.io/en/master/guide/custom_env.html). Its [RL tips](https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html) pick the algorithm by action space, since DQN handles only discrete actions and SAC only continuous ones.
|
||||
|
||||
For more frameworks, papers, and courses, see [awesome-deep-learning](https://github.com/ChristosChristofidis/awesome-deep-learning).
|
||||
@@ -0,0 +1,50 @@
|
||||
Server setup goes in YAML playbooks that Ansible runs, unless you'd rather write Python with pyinfra. Neither Python DevOps tool needs an agent: SSH is enough.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Server setup over SSH: Ansible in YAML, pyinfra in Python
|
||||
- Cloud APIs: Boto3, Azure SDK for Python, Google Cloud client libraries, and the AWS CLI from your shell
|
||||
- First-boot setup of a new cloud instance: cloud-init
|
||||
- Large fleets, with an agent on every host: Salt
|
||||
- Shell commands on remote servers, run from Python: Fabric
|
||||
- Serverless apps on AWS Lambda: Chalice
|
||||
- CPU, memory, disk, and process stats: psutil
|
||||
- Errors from your app, sent to Sentry: Sentry SDK
|
||||
- Processes kept running and restarted after a crash: Supervisor
|
||||
- Celery workers and tasks in a browser: Flower
|
||||
- Shell programs called as Python functions: sh
|
||||
- Deduplicated, encrypted backups: BorgBackup
|
||||
|
||||
Ansible is [agentless](https://github.com/ansible/ansible): it uses the SSH daemon your servers already run. You install it on one control node and [declare the desired state](https://docs.ansible.com/projects/ansible/latest/getting_started/introduction.html) of each machine in a playbook. When a machine already matches, Ansible changes nothing, however many times the playbook runs.
|
||||
|
||||
pyinfra calls itself ["ansible but Python instead of YAML"](https://github.com/pyinfra-dev/pyinfra). Its getting-started guide puts your hosts in an `inventory.py` and your operations in a `deploy.py`, then runs `pyinfra inventory.py deploy.py`. The target hosts [need nothing but an SSH daemon](https://docs.pyinfra.com/en/latest/getting-started.html). Most operations describe an end state: pyinfra [checks the host first](https://docs.pyinfra.com/en/latest/using-operations.html) and runs commands only if reality doesn't match.
|
||||
|
||||
Boto3 is the AWS SDK for Python. Its [clients](https://docs.aws.amazon.com/boto3/latest/guide/clients.html) map close to 1:1 onto AWS service APIs and support every service operation. Resources are a higher-level, object-oriented layer on top.
|
||||
|
||||
The AWS CLI gives you [direct access to AWS's public APIs](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html) from your terminal, to explore a service or write shell scripts.
|
||||
|
||||
The Azure SDK for Python is a set of per-service packages, so you install only what you use. [Management libraries](https://learn.microsoft.com/en-us/azure/developer/python/sdk/azure-sdk-overview), named `azure-mgmt-*`, create and configure Azure resources, while client libraries work with services already provisioned.
|
||||
|
||||
The Google Cloud client libraries are [the recommended way](https://docs.cloud.google.com/python/docs/reference) to call Google Cloud APIs from code.
|
||||
|
||||
cloud-init handles a new instance's first boot. It [takes an initial configuration you supply](https://docs.cloud-init.io/en/latest/explanation/introduction.html) and applies it when the instance is created, from the hostname and network to user accounts and scripts. [Most clouds and Linux distributions ship it](https://github.com/canonical/cloud-init), so there's nothing to install. Write that configuration as [cloud-config](https://docs.cloud-init.io/en/latest/explanation/format/cloud-config.html), YAML that describes the state you want.
|
||||
|
||||
Salt has [a master managing minions](https://docs.saltproject.io/salt/user-guide/en/latest/topics/overview.html): an agent runs on each host and listens on an event bus. One command can reach thousands of systems in seconds.
|
||||
|
||||
Fabric runs shell commands on remote servers over SSH and [hands you back Python objects](https://www.fabfile.org/). A [`Connection`](https://docs.fabfile.org/en/latest/getting-started.html#run-commands-via-connections-and-run) is the core of its API: `run()` a command on it, and `put()` or `get()` files. For deploys and sysadmin jobs, write `@task` functions in a `fabfile.py` and [run them with `fab -H web1,web2`](https://docs.fabfile.org/en/latest/getting-started.html#addendum-the-fab-command-line-tool).
|
||||
|
||||
Chalice deploys Python apps to AWS Lambda. You write handlers with [a decorator-based API](https://github.com/aws/chalice) for API Gateway, S3, SQS, and other AWS services. Then `chalice deploy` provisions what your app needs, including its IAM policy.
|
||||
|
||||
psutil reads process and system information, like CPU, memory, disks, and network, across platforms. It covers [what tools like `ps`, `top`, and `netstat` show](https://github.com/giampaolo/psutil).
|
||||
|
||||
The Sentry SDK reports your app's errors to Sentry. [Call `sentry_sdk.init()`](https://docs.sentry.io/platforms/python/#initialize-the-sentry-sdk) with your DSN as early as possible, in your app's entry point. Its [integrations](https://docs.sentry.io/platforms/python/integrations/) for Django, Flask, Celery, and many other libraries turn on by themselves.
|
||||
|
||||
Supervisor runs your programs as its subprocesses on UNIX-like systems and [can restart them after a crash](https://supervisord.org/introduction.html). Add a [`[program:x]` section](https://supervisord.org/running.html#adding-a-program) per program to its INI-style config, start `supervisord`, and control processes with `supervisorctl`.
|
||||
|
||||
Flower is Celery's [recommended monitor](https://docs.celeryq.dev/en/stable/userguide/monitoring.html): a web app that shows workers and tasks in real time, and can restart worker pools or shut workers down. Start it with your app's configuration, [`celery -A tasks.app flower`](https://flower.readthedocs.io/en/latest/install.html).
|
||||
|
||||
sh lets you [call any program as if it were a function](https://sh.readthedocs.io/en/latest/): `from sh import git`, then `git("status", "--short")`. It runs the real binary on your `$PATH`, and a non-zero exit code raises an exception like `sh.ErrorReturnCode_2`.
|
||||
|
||||
BorgBackup is a [deduplicating backup program](https://borgbackup.readthedocs.io/en/stable/). It stores only changes, so it suits daily backups, and its authenticated encryption suits backup targets you don't fully trust. Initialize an encrypted repository, create an archive per backup, and prune old ones on a schedule. The repository is useless without its key, so the quick start says to [back up the key](https://borgbackup.readthedocs.io/en/stable/quickstart.html#repository-encryption) and test your backups.
|
||||
|
||||
Keep secrets out of your code and plain files, whichever tool you pick. Let each cloud's SDK find credentials: Boto3 [searches a chain of locations](https://docs.aws.amazon.com/boto3/latest/guide/credentials.html#configuring-credentials), Azure recommends [token-based authentication through Microsoft Entra ID](https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication/overview#recommended-approach-for-app-authentication), and Google's client libraries pick up [Application Default Credentials](https://docs.cloud.google.com/docs/authentication/application-default-credentials). In config management, encrypt secrets with [Ansible Vault](https://docs.ansible.com/projects/ansible/latest/tips_tricks/ansible_tips_tricks.html#keep-vaulted-variables-safely-visible) or keep them in [Salt's pillar](https://docs.saltproject.io/en/latest/topics/best_practices.html#storing-secure-data).
|
||||
@@ -0,0 +1,22 @@
|
||||
From a laptop to a cluster, Python distributed computing comes down to the workload: joblib runs loops, Dask scales pandas, Ray scales ML, and PySpark runs SQL.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Parallel for loops on one machine: joblib
|
||||
- Scaling pandas, NumPy, or scikit-learn code: Dask
|
||||
- Training, tuning, and serving ML models, or GPU jobs: Ray
|
||||
- Your own Python code with stateful workers on a cluster: Ray Core
|
||||
- ETL and SQL on structured data, or a JVM shop: PySpark
|
||||
- MPI programs on HPC clusters and supercomputers: mpi4py
|
||||
|
||||
joblib is a [package for parallel computing and disk-based caching](https://joblib.readthedocs.io/en/stable/) that leaves your code as unmodified as possible. You [write a parallel for loop](https://joblib.readthedocs.io/en/stable/user_guide/parallel.html#common-usage) as a generator expression: `Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in range(10))`. By default, joblib runs the calls in separate worker processes. When one machine isn't enough, [switch its backend](https://joblib.readthedocs.io/en/stable/user_guide/parallel.html#setting-up-joblib-s-backend-with-parallel-config), and the same loop runs on a Dask, Ray, or Spark cluster. scikit-learn's [`n_jobs` runs on joblib](https://scikit-learn.org/stable/computing/parallelism.html) too, so it follows the backend you pick.
|
||||
|
||||
Dask [scales pandas, scikit-learn, and NumPy workflows](https://docs.dask.org/en/stable/why.html) with minimal rewriting. A Dask DataFrame is [a collection of pandas dataframes](https://docs.dask.org/en/stable/) on different computers, and Dask Arrays parallelize NumPy. Dask [runs without any setup](https://docs.dask.org/en/stable/deploying.html#local-machine) on your laptop. Its `LocalCluster` follows the same interface as every other Dask cluster manager, so you swap it out when you're ready to scale up.
|
||||
|
||||
Ray is a [unified framework for scaling AI and Python applications](https://docs.ray.io/en/latest/ray-overview/index.html). Its libraries each distribute one ML task: Ray Data, Train, Tune, Serve, and RLlib. Ray Core runs your own code: [decorate a function with `@ray.remote`](https://docs.ray.io/en/latest/ray-core/walkthrough.html), call it with `.remote()`, and fetch the result with `ray.get()`. For workers that keep state between calls, decorate a class the same way to get an actor. Ray runs on one machine with `ray.init()`, and on several nodes once you [deploy a Ray cluster](https://docs.ray.io/en/latest/cluster/getting-started.html). Ray Data [suits GPU workloads for deep learning inference](https://docs.ray.io/en/latest/data/comparisons.html#how-does-ray-data-compare-to-other-solutions-for-offline-inference) better than Spark does, but unlike Spark, it has no SQL interface.
|
||||
|
||||
PySpark is the Python API for Apache Spark, for large-scale data processing. Dask's own comparison suggests Spark [when you prefer SQL, run mostly JVM infrastructure, or want an all-in-one solution](https://docs.dask.org/en/stable/spark.html#reasons-you-might-choose-spark). PySpark's docs [recommend DataFrames over RDDs](https://spark.apache.org/docs/latest/api/python/index.html), so Spark builds the most efficient query for you. Write it in [SQL or the DataFrame API](https://spark.apache.org/docs/latest/api/python/user_guide/sql.html), whichever you think in, and switch between the two as you go. Installing PySpark with pip is [for local use or as a client](https://spark.apache.org/docs/latest/api/python/getting_started/install.html) that connects to a cluster, not for setting up the cluster itself. The `pyspark` package also needs Java.
|
||||
|
||||
mpi4py provides [Python bindings for MPI](https://mpi4py.readthedocs.io/en/stable/), the Message Passing Interface, so your Python code runs across workstations, clusters, and supercomputers. Lowercase methods like `comm.send` pass any picklable Python object, and uppercase ones like `comm.Send` [pass NumPy arrays the fast way](https://mpi4py.readthedocs.io/en/stable/tutorial.html). Run your script with `mpiexec -n 4 python -m mpi4py script.py`, so an unhandled exception [aborts the whole MPI run instead of deadlocking](https://mpi4py.readthedocs.io/en/stable/mpi4py.run.html#exceptions-and-deadlocks). mpi4py runs on an MPI implementation like MPICH or Open MPI, and in production its docs [recommend a custom-built or system-provided one](https://mpi4py.readthedocs.io/en/stable/install.html).
|
||||
|
||||
Start on one machine: parallelism [brings extra complexity and overhead](https://docs.dask.org/en/stable/best-practices.html#start-small), and often you don't need it. When you do scale out, collect results at the end, since Dask's `compute()` and Ray's `ray.get()` both block until the work finishes. Call [`compute()` once](https://docs.dask.org/en/stable/best-practices.html#avoid-calling-compute-repeatedly) rather than in a loop, and [`ray.get()` as late as possible](https://docs.ray.io/en/latest/ray-core/tips-for-first-time.html#tip-1-delay-ray-get).
|
||||
@@ -0,0 +1,21 @@
|
||||
Users without Python installed can still run your app once PyInstaller takes it from Python script to exe. Pyarmor obfuscates the code you ship.
|
||||
|
||||
How to choose:
|
||||
|
||||
- An executable with the least setup: PyInstaller
|
||||
- Obfuscated scripts, or ones that expire or run on one machine only: Pyarmor
|
||||
- Your code compiled to C rather than bundled as bytecode: Nuitka
|
||||
- One file to copy to your servers and laptops: pex
|
||||
- A Windows installer or macOS disk image along with the executable: cx_Freeze
|
||||
|
||||
PyInstaller [reads your script, finds every module and library it needs](https://pyinstaller.org/en/stable/operating-mode.html), and collects them with the Python interpreter into one folder, or optionally into one executable file. For most programs that takes one command, `pyinstaller myscript.py`, and your users don't need Python installed at all. It [bundles major packages like NumPy and the Qt bindings out of the box](https://github.com/pyinstaller/pyinstaller). The bundle holds compiled `.pyc` files instead of your source code, but those [could in principle be decompiled](https://pyinstaller.org/en/stable/operating-mode.html#hiding-the-source-code) to reveal your logic.
|
||||
|
||||
Pyarmor is a command-line tool for [obfuscating Python scripts, binding them to specific machines, and setting expiration dates](https://pyarmor.readthedocs.io/en/latest/tutorial/getting-started.html#what-s-pyarmor). The obfuscated scripts stay standard `.py` files: `pyarmor gen foo.py` writes them to `dist/` with a runtime package, and you [copy all of `dist/`](https://pyarmor.readthedocs.io/en/latest/tutorial/getting-started.html#distributing-the-obfuscated-script) to the target. For an executable, Pyarmor [packs the obfuscated scripts with PyInstaller](https://pyarmor.readthedocs.io/en/latest/tutorial/obfuscation.html#packing-obfuscated-scripts). Pyarmor is commercial software: its free version [covers scripts that won't make you much money](https://pyarmor.readthedocs.io/en/latest/licenses.html), and anything else needs a paid license.
|
||||
|
||||
Nuitka [translates your Python modules into a C program](https://github.com/Nuitka/Nuitka) that runs on `libpython` the way CPython does, so building takes [a C compiler](https://nuitka.net/user-documentation/user-manual.html#c-compiler). Its homepage pitches that compile step for [protecting your source code](https://nuitka.net/) and for speed. Run it as [`python -m nuitka`](https://nuitka.net/user-documentation/user-manual.html#recommended-way), so you know which interpreter compiles your program. Its default mode still needs Python on the target. For users without Python, build in standalone or onefile mode.
|
||||
|
||||
pex builds `.pex` files, [self-contained executable Python virtual environments](https://docs.pex-tool.org/whatispex.html) in a single zip file, so deploying your app comes down to `scp`. Pass it your requirements as you would to pip, [`pex -r requirements.txt -o my_application.pex`](https://docs.pex-tool.org/buildingpex.html#specifying-requirements), and give it an entry point to run, or the file starts an interpreter. A plain `.pex` runs on the target's own Python; [build it as a scie](https://docs.pex-tool.org/scie.html) to get a native executable with the interpreter inside.
|
||||
|
||||
cx_Freeze [normally produces a folder](https://cx-freeze.readthedocs.io/en/stable/overview.html) with your executable and the shared libraries it needs, and it can also wrap that in a simple Windows installer or a macOS disk image. Keep its options in a [`[tool.cxfreeze]` table in `pyproject.toml`](https://cx-freeze.readthedocs.io/en/stable/setup_script.html), then run `cxfreeze build`.
|
||||
|
||||
Get the folder build working before you switch to a single file, since both [PyInstaller](https://pyinstaller.org/en/stable/operating-mode.html#bundling-to-one-file) and [Nuitka](https://nuitka.net/user-documentation/use-cases.html#standalone-program-distribution) say problems are much easier to debug that way. Then build on each operating system you ship for, as [PyInstaller](https://pyinstaller.org/en/stable/usage.html#supporting-multiple-operating-systems) and [cx_Freeze](https://cx-freeze.readthedocs.io/en/stable/faq.html#freezing-for-other-platforms) both tell you to.
|
||||
@@ -0,0 +1,21 @@
|
||||
Write only docstrings, and pdoc is all the Python documentation generator you need. Write guides too, and build the site with Sphinx or Material for MkDocs.
|
||||
|
||||
How to choose:
|
||||
|
||||
- API docs straight from docstrings, with no configuration: pdoc
|
||||
- Handwritten docs plus an API reference, as HTML, PDF, and more: Sphinx
|
||||
- A searchable Markdown site, with no HTML, CSS, or JavaScript to learn: Material for MkDocs
|
||||
- Architecture diagrams as code, kept in version control: Diagrams
|
||||
- A Material for MkDocs site, new or existing, on its team's own generator: Zensical
|
||||
|
||||
pdoc [aims to do one thing and do it well](https://pdoc.dev/docs/pdoc.html#what-is-pdoc): API documentation that follows your module hierarchy, with no configuration. Docstrings are Markdown, and it understands Google and numpydoc styles too. Run `pdoc ./demo.py` or `pdoc my_module_name` for a [preview in your browser that reloads](https://pdoc.dev/docs/pdoc.html#quickstart) when you edit the code, then `pdoc ./demo.py -o ./docs` to export the HTML. Its output is self-contained HTML, and for substantially more complex documentation needs, [pdoc's docs recommend Sphinx](https://pdoc.dev/docs/pdoc.html#limitations).
|
||||
|
||||
Sphinx [focuses on handwritten documentation](https://www.sphinx-doc.org/en/stable/usage/quickstart.html) and turns one set of source files into HTML, a PDF via LaTeX, man pages, and more. Its default markup is reStructuredText, and it can [read Markdown through MyST-Parser](https://www.sphinx-doc.org/en/stable/usage/markdown.html). Run `sphinx-quickstart` to set up a source directory with a `conf.py`, then list your pages in the root document's toctree. `make html` builds the site, and `make latexpdf` the PDF. To document your code, [autodoc](https://www.sphinx-doc.org/en/stable/usage/extensions/autodoc.html) pulls in its docstrings, which you mix with your handwritten pages.
|
||||
|
||||
Material for MkDocs is [a documentation framework on top of MkDocs](https://squidfunk.github.io/mkdocs-material/getting-started/), and `pip install mkdocs-material` installs MkDocs with it. You write Markdown and get a searchable static site, with [no HTML, CSS, or JavaScript to know](https://squidfunk.github.io/mkdocs-material/). Run `mkdocs new .`, then [set `site_name`, `site_url`, and `theme: name: material`](https://squidfunk.github.io/mkdocs-material/creating-your-site/#minimal-configuration) in `mkdocs.yml`, and `mkdocs serve` previews the site as you write. For reference docs from docstrings, try [mkdocstrings](https://squidfunk.github.io/mkdocs-material/alternatives/#sphinx) before switching to Sphinx: Material's alternatives page says it builds on MkDocs and adds Sphinx-like functionality.
|
||||
|
||||
Zensical is [a static site generator from the creators of Material for MkDocs](https://zensical.org/docs/get-started/), with the same batteries-included approach. After `pip install zensical`, [`zensical new .`](https://zensical.org/docs/create-your-site/) creates a `docs/` folder, a `zensical.toml` config, and a GitHub Actions workflow. `zensical serve` previews as you write, and `zensical build` writes the static site. It also [builds existing MkDocs projects without changes](https://zensical.org/docs/compatibility/mkdocs/) from their `mkdocs.yml`, and its classic theme variant keeps the Material for MkDocs look.
|
||||
|
||||
Diagrams [draws cloud system architecture in Python code](https://diagrams.mingrammer.com/), so you can track changes to a diagram in version control. It renders with Graphviz, so [install Graphviz first](https://diagrams.mingrammer.com/docs/getting-started/installation), then `pip install diagrams`. Describe the system in a `with Diagram("Web Service", show=False):` block and chain nodes with `>>`. Running `python diagram.py` saves it as a PNG in your working directory.
|
||||
|
||||
Install [Sphinx](https://www.sphinx-doc.org/en/stable/usage/installation.html), [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/getting-started/#with-pip), or [Zensical](https://zensical.org/docs/get-started/#install-with-pip) into your project's virtual environment, as each one's docs recommend. The documentation generators here all write static HTML, so publish it from CI. All four document a GitHub Actions workflow that deploys to GitHub Pages: [Sphinx](https://www.sphinx-doc.org/en/stable/tutorial/deploying.html#publishing-your-html-documentation), [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/publishing-your-site/), [Zensical](https://zensical.org/docs/publish-your-site/), and [pdoc](https://pdoc.dev/docs/pdoc.html#deploying-to-github-pages).
|
||||
@@ -0,0 +1,14 @@
|
||||
Sending from Django, you learn no new Python email library API: django-anymail runs send_mail(). Async code sends mail through aiosmtplib.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A Django app sending through Amazon SES, Mailgun, Postmark, or another email service: django-anymail
|
||||
- A Django app that tracks delivery or receives email through that service: django-anymail
|
||||
- Async code talking to an SMTP server: aiosmtplib
|
||||
- A script sending from Gmail, or any SMTP account, with attachments: yagmail
|
||||
|
||||
django-anymail extends Django's `django.core.mail` with features email services add, through [one API that doesn't lock your code to one service](https://anymail.dev/en/latest/). [Change Django's email backend to Anymail's backend for your service](https://anymail.dev/en/latest/installation/), and [`send_mail()`, `EmailMessage`, and even `mail_admins()`](https://anymail.dev/en/latest/sending/django_email/) send through it unchanged. Send from a background task, not the view: Anymail calls that a [best practice for sending email from Django](https://anymail.dev/en/latest/tips/transient_errors/), since your views respond faster. To track delivery or receive email, connect the service's webhooks to [Anymail's Django signals](https://anymail.dev/en/latest/sending/tracking/), and [protect them with https and a shared secret](https://anymail.dev/en/latest/tips/securing_webhooks/).
|
||||
|
||||
aiosmtplib is an [async version of smtplib](https://aiosmtplib.readthedocs.io/en/latest/), with similar APIs. Build each message as the standard library's `EmailMessage`, which makes headers easier to work with. Then pass it to the [`send()` coroutine](https://aiosmtplib.readthedocs.io/en/latest/usage.html), which the docs recommend for most use cases.
|
||||
|
||||
yagmail aims to make sending email as simple as possible, with Gmail as its default server. Its [contents argument is guessed](https://yagmail.readthedocs.io/en/latest/usage.html#magical-contents) item by item: a path to a local file becomes an attachment, valid HTML becomes HTML, and anything else is text. For Gmail, sign in with an [app password or OAuth2](https://github.com/kootenpv/yagmail).
|
||||
@@ -0,0 +1,15 @@
|
||||
Since uv both downloads Python and creates virtual environments, one Python environment manager does the jobs that pyenv and virtualenv split between them.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Python versions and virtual environments from one tool: uv
|
||||
- A Python API for creating environments, or plugins: virtualenv
|
||||
- Switching the `python` command between versions, and nothing more: pyenv
|
||||
|
||||
uv [manages Python versions](https://docs.astral.sh/uv/) as well as packages. When a command needs a Python you don't have, uv [downloads it for you](https://docs.astral.sh/uv/guides/install-python/), so you don't need Python installed to get started. A project's virtual environment lives in a `.venv` folder [inside the project, where editors can find it](https://docs.astral.sh/uv/concepts/projects/layout/#the-project-environment); keep it out of version control. Outside a project, `uv venv --python <version>` [creates an environment](https://docs.astral.sh/uv/pip/environments/) with that version, and downloads it if needed.
|
||||
|
||||
virtualenv [creates isolated Python environments](https://virtualenv.pypa.io/en/latest/), and a subset of it ships with Python as the venv module. Its docs place it [between venv and uv](https://virtualenv.pypa.io/en/latest/explanation.html#virtualenv-vs-venv-vs-uv): faster and more featureful than venv, and still pure Python. Pick it when you need plugins, or a Python API to create environments. It uses the Python it runs under unless you [pass `-p`](https://virtualenv.pypa.io/en/latest/how-to/usage.html#select-a-python-version) with another installed version. As a command-line tool, it [belongs in an isolated environment](https://virtualenv.pypa.io/en/latest/how-to/install.html), not your system Python.
|
||||
|
||||
pyenv does one job: it [lets you switch between multiple versions of Python](https://github.com/pyenv/pyenv), and leaves virtual environments [to you or its pyenv-virtualenv plugin](https://github.com/pyenv/pyenv#in-contrast-with-pythonbrew-and-pythonz-pyenv-does-not). Install it with the [automatic installer](https://github.com/pyenv/pyenv#1-automatic-installer-recommended) or, on macOS, Homebrew, then [set up your shell](https://github.com/pyenv/pyenv#b-set-up-your-shell-environment-for-pyenv) for it. [Choose the version](https://github.com/pyenv/pyenv#switch-between-python-versions) with `pyenv shell` for the session, `pyenv local` for a directory, or `pyenv global` for your user account. [Most versions are built from source](https://github.com/pyenv/pyenv#install-additional-python-versions) as you install them, so install Python's build dependencies first. pyenv [doesn't work on Windows](https://github.com/pyenv/pyenv#windows) outside the Windows Subsystem for Linux, so on Windows, use uv, which [supports Windows](https://docs.astral.sh/uv/).
|
||||
|
||||
Whichever tools you combine, name each project's Python in a `.python-version` file. `pyenv local` [writes that file](https://github.com/pyenv/pyenv#understanding-python-version-selection), and so does `uv python pin`, whose docs [recommend a plain version number](https://docs.astral.sh/uv/concepts/python-versions/#python-version-files) there so other tools can read it. virtualenv [uses the version pyenv selected](https://virtualenv.pypa.io/en/latest/how-to/usage.html#using-version-managers-pyenv-mise-asdf) with no extra configuration. Don't install packages into the Python that came with your operating system, which [often manages its packages itself](https://docs.astral.sh/uv/pip/environments/); a virtual environment keeps yours apart.
|
||||
@@ -0,0 +1,19 @@
|
||||
Out of the box, Odoo's CRM, eCommerce, and accounting apps run alone or combine into a full Python ERP. Anything they lack, you add as a module.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Trying Odoo, or customizing it without code: Odoo Online
|
||||
- Writing your own modules: a source install
|
||||
- Hosting your own modules in the cloud: Odoo.sh
|
||||
- Free and open source: Odoo Community
|
||||
- More features, with support and upgrades: Odoo Enterprise
|
||||
|
||||
Everything in Odoo starts and ends with modules, and [the main user-facing ones are flagged as Apps](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/01_architecture.html#odoo-modules). Odoo Enterprise is [extra modules installed on top of Community](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/01_architecture.html#odoo-editions). [Community is free and open source under the LGPL](https://www.odoo.com/documentation/latest/administration.html#editions). Enterprise is shared source, and its license [ties running it to an Enterprise subscription](https://www.odoo.com/documentation/latest/legal/licenses.html).
|
||||
|
||||
[Odoo Online](https://www.odoo.com/documentation/latest/administration/odoo_online.html) runs in your browser with nothing to install, and handles customizations that need no code. Your own modules need Odoo.sh or your own server. Odoo.sh is Odoo's official cloud platform, and it [builds your modules from a GitHub repository](https://www.odoo.com/documentation/latest/administration/odoo_sh/create_module.html).
|
||||
|
||||
To develop modules, Odoo's developer docs prefer [a source install](https://www.odoo.com/documentation/latest/developer/tutorials/setup_guide.html), which runs Odoo straight from its code. Odoo's logic is [written in Python, and it stores data only in PostgreSQL](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/01_architecture.html#multitier-application). Keep your modules in a directory of their own, and [start the server with `odoo-bin`](https://www.odoo.com/documentation/latest/administration/on_premise/source.html#running-odoo), adding that directory to `--addons-path`. Then work through [Server framework 101](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101.html), which builds one module chapter by chapter.
|
||||
|
||||
A module can [add new business logic or change what's already there](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/01_architecture.html#odoo-modules). To change a standard model, extend it from your own module: [model inheritance](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/12_inheritance.html#model-inheritance) adds fields and overrides methods on a model another module defines. Screens work the same way: [view inheritance](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/12_inheritance.html#view-inheritance) applies your extension views on top of the originals instead of overwriting them.
|
||||
|
||||
Before you write a module, [check whether Odoo already covers the case](https://www.odoo.com/documentation/latest/developer/tutorials/server_framework_101/02_newapp.html). A database with custom modules [can't be upgraded until they're ready for the new version](https://www.odoo.com/documentation/latest/administration/upgrade.html), so [cut what duplicates the standard modules](https://www.odoo.com/documentation/latest/developer/howtos/upgrade_custom_db.html#step-1-stop-the-developments).
|
||||
@@ -0,0 +1,54 @@
|
||||
PDFs you merge, split, or encrypt need only pypdf. Which Python file format library you need beyond that depends on whether you read a file or create it.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Merging, splitting, cropping, or encrypting PDFs: pypdf
|
||||
- ELF binaries and DWARF debug info: pyelftools
|
||||
- A table you import or export as CSV, JSON, YAML, or Excel: Tablib
|
||||
- Documents turned into Markdown for an LLM: MarkItDown, or Docling for complex layouts and scans
|
||||
- Excel: openpyxl to edit a workbook, XlsxWriter to create one with charts and formatting
|
||||
- Word documents: python-docx
|
||||
- PowerPoint decks: python-pptx
|
||||
- Text and images out of PDFs, or pages rendered as images: PyMuPDF, or pdfminer.six for each character's position and font
|
||||
- PDFs drawn from Python code: ReportLab
|
||||
- PDFs from HTML and CSS: WeasyPrint
|
||||
- Markdown to HTML: markdown-it-py for CommonMark, Python-Markdown for its extensions, Mistune for speed
|
||||
- Config files: tomllib for TOML, PyYAML for YAML
|
||||
|
||||
pypdf is a [pure-Python PDF library](https://pypdf.readthedocs.io/en/stable/) for splitting, merging, cropping, and transforming pages, and it adds passwords too. To merge files, [`append()` each one to a `PdfWriter`](https://pypdf.readthedocs.io/en/stable/user/merging-pdfs.html) and write it out. With [no C dependency](https://pypdf.readthedocs.io/en/stable/meta/comparisons.html), it's easy to install, but it doesn't generate new PDFs. It reads text with `page.extract_text()`, while a scanned page needs [OCR software](https://pypdf.readthedocs.io/en/stable/user/extract-text.html).
|
||||
|
||||
pyelftools [parses ELF files and DWARF debug info](https://github.com/eliben/pyelftools) in pure Python. Pass an open file to [`ELFFile`](https://github.com/eliben/pyelftools/blob/main/doc/user-guide.md), its main entry point, and stay on the high-level API: its guide says to use the low-level structs at your own peril.
|
||||
|
||||
Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format.
|
||||
|
||||
MarkItDown converts PDF, Word, Excel, PowerPoint, HTML, and more into Markdown [for LLMs and text analysis](https://github.com/microsoft/markitdown). Its README says it may not be the best option for high-fidelity conversions that people read. Don't pass untrusted input straight to it, and when you only read local files, [call `convert_local()` instead of `convert()`](https://github.com/microsoft/markitdown#security-considerations), which also fetches URLs.
|
||||
|
||||
Docling is for documents where layout matters: it [understands page layout, reading order, and tables, and runs OCR](https://docling-project.github.io/docling/) on scanned PDFs. It converts each file into a `DoclingDocument`, which you export to Markdown.
|
||||
|
||||
openpyxl reads and writes Excel workbooks, so it's the one to use when the file already exists. Install defusedxml when you open spreadsheets from other people, since openpyxl [doesn't guard against XML attacks](https://openpyxl.readthedocs.io/en/stable/#security) by default.
|
||||
|
||||
XlsxWriter [can't read or modify existing files](https://xlsxwriter.readthedocs.io/introduction.html). In return, it supports more Excel features than the alternatives, and its files are close to what Excel itself writes.
|
||||
|
||||
python-docx works on `.docx` files, not the older `.doc`. [Start from a document that holds your styles](https://python-docx.readthedocs.io/en/latest/user/documents.html), headers, and footers, and open it with `Document('template.docx')`: python-docx can only use a style that [the starting document defines](https://python-docx.readthedocs.io/en/latest/user/styles-understanding.html).
|
||||
|
||||
python-pptx [creates and updates PowerPoint files](https://python-pptx.readthedocs.io/en/latest/user/intro.html). A deck's look comes from its theme, slide master, and layouts, so [open a template deck](https://python-pptx.readthedocs.io/en/latest/user/presentations.html) with `Presentation('template.pptx')`. Add slides from its layouts and fill their [placeholders](https://python-pptx.readthedocs.io/en/latest/user/placeholders-understanding.html), which carry the template's formatting.
|
||||
|
||||
PyMuPDF is a [high-performance library](https://pymupdf.readthedocs.io/en/latest/) for extracting, converting, and editing PDFs and other documents, built on MuPDF, and it also renders pages to images. It's [available under the AGPL or a commercial license](https://pymupdf.readthedocs.io/en/latest/about.html#license-and-copyright). For reading text and editing pages without that condition, use pypdf or pdfminer.six.
|
||||
|
||||
ReportLab [creates PDFs directly from Python](https://docs.reportlab.com/reportlab/userguide/ch1_intro/), like reports generated on a web server, with charts and tables. Lay out documents with Platypus: [create a document from a `DocTemplate` class and pass a list of flowables](https://docs.reportlab.com/reportlab/userguide/ch5_platypus/), such as paragraphs and tables, to its `build()` method.
|
||||
|
||||
pdfminer.six [extracts text from a PDF's source](https://github.com/pdfminer/pdfminer.six), with the exact location, font, and color of each piece, but it can't write PDFs. The simplest start is [`extract_text()`](https://pdfminersix.readthedocs.io/en/latest/tutorial/highlevel.html).
|
||||
|
||||
WeasyPrint turns HTML and CSS into PDFs, like [reports, invoices, and tickets](https://doc.courtbouillon.org/weasyprint/stable/), with a layout engine built for pagination instead of a browser engine. It runs [no JavaScript](https://doc.courtbouillon.org/weasyprint/stable/going_further.html). Call `HTML(...).write_pdf()`. If users supply the HTML or CSS, its docs say you'll need [extra configuration](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#security) against high memory use, endless renderings, and local file leaks.
|
||||
|
||||
markdown-it-py [follows the CommonMark spec](https://markdown-it-py.readthedocs.io/en/latest/), and plugins add syntax or replace existing rules.
|
||||
|
||||
Python-Markdown implements John Gruber's original Markdown. It's [not a CommonMark implementation](https://python-markdown.github.io/), and its docs say to look elsewhere if you want one. Pick it for its extension API and bundled [extensions](https://python-markdown.github.io/extensions/), like tables, fenced code blocks, and a table of contents.
|
||||
|
||||
Mistune is a [fast parser with renderers and plugins](https://mistune.lepture.com/en/latest/), compatible with "sane CommonMark rules".
|
||||
|
||||
tomllib comes with Python and parses TOML, but doesn't write it. With untrusted input, [limit its size](https://docs.python.org/3/library/tomllib.html), since a malicious TOML string can use a lot of CPU and memory.
|
||||
|
||||
PyYAML parses and emits YAML. Load it with `yaml.safe_load()`, since [`yaml.load` can call any Python function](https://pyyaml.org/wiki/PyYAMLDocumentation) when the data comes from an untrusted source.
|
||||
|
||||
Markdown your users write can carry HTML, so give each parser its safe setting. With markdown-it-py, use the [`js-default` preset](https://markdown-it-py.readthedocs.io/en/latest/security.html), which its docs strongly recommend for user content in web apps. Python-Markdown [doesn't sanitize its output](https://python-markdown.github.io/sanitization/), so run a sanitizer over the HTML. With Mistune, build your parser with [`mistune.create_markdown()`](https://mistune.lepture.com/en/latest/guide.html), which escapes HTML, instead of `mistune.html()`, which doesn't.
|
||||
@@ -0,0 +1,19 @@
|
||||
Without installing anything, two Python file manipulation libraries cover paths and file types by name: pathlib and mimetypes. python-magic reads the bytes.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Building and joining paths, and listing directories: pathlib
|
||||
- A MIME type from a file name or URL: mimetypes
|
||||
- A file's type from its content, like the Unix `file` command: python-magic
|
||||
- Rerunning or reloading code when files change: watchfiles
|
||||
- Your own handler for each created, modified, or moved file: watchdog
|
||||
|
||||
If you aren't sure which pathlib class you need, the docs say [`Path` is most likely it](https://docs.python.org/3/library/pathlib.html): it makes a concrete path for the platform your code runs on. [Join paths with the `/` operator](https://docs.python.org/3/library/pathlib.html#basic-use), and list a directory with `iterdir()` or `glob()`.
|
||||
|
||||
mimetypes [maps a file name's extension to a MIME type](https://docs.python.org/3/library/mimetypes.html), and a MIME type back to extensions. A type guess returns 2 values: a type for the Content-Type header, and an encoding like gzip for the Content-Encoding header. The type is [`None` for a missing or unknown suffix](https://docs.python.org/3/library/mimetypes.html#mimetypes.guess_type).
|
||||
|
||||
python-magic wraps libmagic, which [identifies file types by checking their headers](https://github.com/ahupp/python-magic), the way the Unix `file` command does. It's a thin wrapper, so install libmagic too, with `apt-get install libmagic1` or `brew install libmagic`. Call `magic.from_file(path, mime=True)` for a MIME type, or `magic.from_buffer()` on bytes you already have.
|
||||
|
||||
watchfiles is built for [file watching and code reload](https://watchfiles.helpmanual.io/), and its Rust core groups changes into batches instead of firing once per file. `watch()` is a generator that [yields sets of changes](https://watchfiles.helpmanual.io/api/watch/#watchfiles.watch), and `awatch()` is the async version. To restart code when files change, use [`run_process()`](https://watchfiles.helpmanual.io/api/run_process/#watchfiles.run_process) with a function or a command. From a shell, the [`watchfiles` CLI](https://watchfiles.helpmanual.io/cli/) does the same: `watchfiles --filter python 'pytest --lf' src tests`.
|
||||
|
||||
watchdog gives you [an API and a shell tool](https://python-watchdog.readthedocs.io/en/latest/) for monitoring directories. Its [quickstart](https://python-watchdog.readthedocs.io/en/latest/quickstart.html) has you subclass `FileSystemEventHandler`, override methods like `on_created()` and `on_modified()`, schedule the handler on an `Observer`, and start that thread. An observer skips subdirectories unless you pass `recursive=True` to `schedule()`.
|
||||
@@ -0,0 +1,21 @@
|
||||
Beyond functools, install more-itertools, as the itertools docs suggest. toolz is a fuller Python functional programming library, and returns adds typed errors.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Partial application, decorators, and caching: functools
|
||||
- More iterator tools, the itertools recipes included: more-itertools
|
||||
- Composing functions into pipelines, currying, and dict helpers: toolz, or cytoolz for speed
|
||||
- Everyday helpers for collections, decorators, retries, and debugging: funcy
|
||||
- Errors and missing values as typed containers checked by mypy: returns
|
||||
|
||||
functools is the standard library's module [for higher-order functions](https://docs.python.org/3/library/functools.html), functions that act on or return other functions. Python's Functional Programming HOWTO calls `partial()` [the most useful tool in the module](https://docs.python.org/3/howto/functional.html#the-functools-module): it fills in some of a function's arguments and gives you a new function. When you write a decorator, wrap its inner function with [`wraps`](https://docs.python.org/3/library/functools.html#functools.wraps), so the decorated function keeps its name and docstring. The same HOWTO finds many uses of `reduce()` [clearer as a `for` loop](https://docs.python.org/3/howto/functional.html#small-functions-and-the-lambda-expression).
|
||||
|
||||
The itertools docs point you to more-itertools for [their recipes and many more](https://docs.python.org/3/library/itertools.html#itertools-recipes). It collects [building blocks beyond itertools](https://more-itertools.readthedocs.io/en/stable/), for grouping, windowing, lookahead, and more. The itertools recipes sit in its top-level package, so `from more_itertools import flatten` works.
|
||||
|
||||
toolz [extends itertools and functools](https://toolz.readthedocs.io/en/latest/) with functions that are composable, pure, and lazy, and its API [follows Clojure's standard library](https://toolz.readthedocs.io/en/latest/heritage.html). Each function takes and returns only iterables, dictionaries, and functions, so they [compose to solve your own problems](https://toolz.readthedocs.io/en/latest/composition.html). [`pipe`](https://toolz.readthedocs.io/en/latest/api.html#toolz.functoolz.pipe) runs a value through a sequence of functions, like pipes in Unix. Stick with `partial` at first, and once it shows up several times in your code, [switch to the `toolz.curried` namespace](https://toolz.readthedocs.io/en/latest/curry.html#curry). toolz is a general-purpose library, and for data analytics its docs say [a library built for it](https://toolz.readthedocs.io/en/latest/streaming-analytics.html#disclaimer) may serve you better. [cytoolz](https://github.com/pytoolz/cytoolz) implements the same API in Cython, as a drop-in replacement when you need more speed.
|
||||
|
||||
funcy is a collection of functional tools [focused on practicality](https://github.com/Suor/funcy), inspired by Clojure and underscore. Next to sequence tools, it has [collection functions that keep the type](https://funcy.readthedocs.io/en/stable/overview.html) of a dict or set. It also has control flow helpers, like `@retry` and `silent`, and debugging helpers, like `tap` and `log_calls`. Many of its functions take a regex, a mapping, or a set [where you'd pass a function](https://funcy.readthedocs.io/en/stable/extended_fns.html#extended-function-semantics).
|
||||
|
||||
returns puts results in typed containers: [`Maybe` for None and `Result` for exceptions](https://returns.readthedocs.io/en/latest/pages/quickstart.html#why), plus `IO` for impure code and `Future` for async code. Its docs [really recommend mypy](https://returns.readthedocs.io/en/latest/pages/quickstart.html#typechecking-and-other-integrations), and typing [only works correctly with its mypy plugin](https://returns.readthedocs.io/en/latest/pages/result.html). So it fits projects that check types with mypy. Turn functions that raise into ones that return a `Result` with [`@safe`](https://returns.readthedocs.io/en/latest/pages/result.html#safe), and chain the steps with [`flow`](https://returns.readthedocs.io/en/latest/pages/pipeline.html#flow), which its docs call the recommended way to write code with returns.
|
||||
|
||||
Mix functional style with the rest of your code: Python's HOWTO says functional-style programs usually [give a functional-appearing interface](https://docs.python.org/3/howto/functional.html) and use non-functional features inside. For a plain map or filter, toolz's own docs call comprehensions [more Pythonic](https://toolz.readthedocs.io/en/latest/streaming-analytics.html). Learn a core set of functions; toolz says [about a dozen covers most tasks](https://toolz.readthedocs.io/en/latest/control.html), and the right word only helps when your readers know it too.
|
||||
@@ -0,0 +1,20 @@
|
||||
2D games need a game loop: pygame-ce has you write it; Arcade runs it. Ren'Py is built for visual novels, as Panda3D is for 3D in Python game development.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A game loop you write and control yourself: pygame-ce
|
||||
- A game loop, physics, and screen switching built in: Arcade
|
||||
- 3D games, visualizations, and simulations: Panda3D
|
||||
- Visual novels and other writing-heavy games: Ren'Py
|
||||
- Windowing, OpenGL graphics, and media with no dependencies: pyglet
|
||||
- Code already written for pygame: pygame-ce
|
||||
|
||||
pygame-ce installs as `pygame-ce` and still imports as `pygame`. It [gives you full control of program execution](https://pyga.me/docs/): you write the loop that handles events, draws the frame, and flips the display. pygame-ce is LGPL, which [still allows closed-source and commercial games](https://github.com/pygame-community/pygame-ce); only changes to pygame-ce itself must be released under a compatible license.
|
||||
|
||||
Arcade is built on pyglet, and it's [ideal for beginning programmers](https://github.com/pythonarcade/arcade) or anyone who wants 2D games without learning a complex framework. Give each screen its own `arcade.View` subclass that overrides `on_draw()` and `on_update()`, then [hand control to `arcade.run()`](https://api.arcade.academy/en/stable/tutorials/views/index.html). Draw sprites through a `SpriteList`, [the only way to draw a sprite](https://api.arcade.academy/en/stable/programming_guide/performance_tips.html), which batches them for you. For movement and collisions, start with the [built-in physics engines](https://api.arcade.academy/en/stable/api_docs/api/physics_engines.html): a simple one for top-down games, and a platformer one with gravity and moving platforms.
|
||||
|
||||
Panda3D is a C++ engine scripted in Python, and [its manual says you must be a skilled programmer](https://docs.panda3d.org/latest/python/introduction/index) to use it. Subclass `ShowBase`, which opens the 3D window, and [call `run()` once, as the last line](https://docs.panda3d.org/latest/python/introduction/tutorial/starting-panda3d). A model shows up only once it's [attached to the scene graph](https://docs.panda3d.org/latest/python/programming/scene-graph/index), a tree rooted at `render`.
|
||||
|
||||
To use Ren'Py, you [download it and work in its launcher](https://www.renpy.org/doc/html/quickstart.html), which creates, edits, and runs your projects. Write the story in its script language, and reach for [its screen language and Python](https://www.renpy.org/why.html) when you need custom interfaces or game logic.
|
||||
|
||||
pyglet is a windowing and multimedia library [with no external dependencies](https://pyglet.readthedocs.io/en/latest/), for games and other visually rich apps. Attach handlers like `on_draw()` with `@window.event` and call `pyglet.app.run()`, since [writing your own event loop is generally not necessary](https://pyglet.readthedocs.io/en/latest/programming_guide/quickstart.html). Draw many sprites in [a `Batch`, which the docs strongly recommend](https://pyglet.readthedocs.io/en/latest/programming_guide/image.html).
|
||||
@@ -0,0 +1,20 @@
|
||||
Addresses become coordinates with geopy, whose Python geolocation API covers many geocoding services. GeoPandas analyzes map data, and GeoDjango serves it.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Addresses to coordinates, and coordinates back to addresses: geopy
|
||||
- The distance between two points: geopy
|
||||
- Geographic data in tables and files, like shapefiles and GeoJSON: GeoPandas
|
||||
- Web apps that store and query geographic data: GeoDjango
|
||||
- A visitor's country or city from their IP address, in a Django project: GeoDjango
|
||||
- Building, encoding, and validating GeoJSON objects by hand: geojson
|
||||
|
||||
geopy is [a client for geocoding services, not a service itself](https://geopy.readthedocs.io/en/stable/#geopy-is-not-a-service), and each service has its own terms of use, quotas, and pricing. Every geocoder has a `geocode()` method that turns an address into a location, and most also have `reverse()` for the other way around. For OpenStreetMap's Nominatim, [set a `user_agent` that names your app](https://geopy.readthedocs.io/en/stable/#geopy.geocoders.Nominatim) and follow [its usage policy](https://operations.osmfoundation.org/policies/nominatim/). The policy bans heavy use, auto-complete, and systematic queries. It also asks you to cache results and show attribution, and it discourages bulk geocoding. To geocode a DataFrame, [wrap the call in `RateLimiter`](https://geopy.readthedocs.io/en/stable/#usage-with-pandas), which adds delays between requests and retries failed ones. Check first that your service allows bulk requests at all. For distances, `geopy.distance.distance` [computes the geodesic distance](https://geopy.readthedocs.io/en/stable/#module-geopy.distance) between two points.
|
||||
|
||||
GeoPandas adds geometry columns to pandas, so you can [do in Python what would otherwise take a spatial database such as PostGIS](https://geopandas.org/en/stable/). Load a shapefile, GeoJSON, or GeoPackage with `read_file()`, which [detects the file type and returns a GeoDataFrame](https://geopandas.org/en/stable/getting_started/introduction.html), and write it back with `to_file()`. For a table of latitudes and longitudes, [build the points with `points_from_xy()`](https://geopandas.org/en/stable/gallery/create_geopandas_from_pandas.html) and set `crs="EPSG:4326"`. To geocode a column of addresses, GeoPandas [calls geopy for you](https://geopandas.org/en/stable/docs/user_guide/geocoding.html), so the geocoding service's terms still apply.
|
||||
|
||||
GeoDjango is [a contrib module that ships with Django](https://docs.djangoproject.com/en/stable/ref/contrib/gis/tutorial/). It adds model fields for geometries, spatial queries to the ORM, and geometry editing to the admin. Its tutorial assumes you already know Django. Run it on PostGIS, which its docs recommend as [the most mature and feature-rich open source spatial database](https://docs.djangoproject.com/en/stable/ref/contrib/gis/install/#spatial-database). To load a shapefile, [`ogrinspect` writes the model and a `LayerMapping` imports the rows](https://docs.djangoproject.com/en/stable/ref/contrib/gis/tutorial/#importing-spatial-data). For IP geolocation, [GeoIP2](https://docs.djangoproject.com/en/stable/ref/contrib/gis/geoip2/) looks up a country or city in a MaxMind or DB-IP database file you download.
|
||||
|
||||
geojson has [a class for every object in the GeoJSON spec](https://github.com/jazzband/geojson), and `geojson.dumps()` and `geojson.loads()` [wrap the standard json functions](https://github.com/jazzband/geojson#geojson-encodingdecoding) to encode and decode them. Check an object with [its `is_valid` property and `errors()` method](https://github.com/jazzband/geojson#validation). To encode your own classes the same way, [give them a `__geo_interface__`](https://github.com/jazzband/geojson#custom-classes), which GeoPandas' `from_features()` [also accepts](https://geopandas.org/en/stable/docs/reference/api/geopandas.GeoDataFrame.from_features.html).
|
||||
|
||||
Latitude and longitude are angles, so measure distance and area in a projected coordinate system, in meters or feet. In GeoPandas, you [always need one](https://geopandas.org/en/stable/getting_started/introduction.html#Projections) for those operations, so reproject with `to_crs()` first. GeoDjango calls choosing a geometry field's SRID [an important decision](https://docs.djangoproject.com/en/stable/ref/contrib/gis/model-api/#selecting-an-srid), since projected systems ease distance calculations. geopy's `distance` works on an ellipsoidal model of the earth, so it takes latitudes and longitudes as they are.
|
||||
@@ -0,0 +1,40 @@
|
||||
Desktop apps with room to grow start on Qt, through PySide6. A small tool needs no third-party Python GUI library, since tkinter is in the standard library.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A desktop app that will grow, even a closed-source one: PySide6
|
||||
- A small tool, built with the standard library: tkinter
|
||||
- GNOME apps, or anything else on GTK: PyGObject
|
||||
- Fast, GPU-drawn tools and live plots: Dear PyGui
|
||||
- Touch UIs that also run on Android and iOS: Kivy
|
||||
- Native widgets on Windows, macOS, and Linux: wxPython
|
||||
- Native widgets on desktop and mobile: Toga
|
||||
- A Qt app under the GPL, or with a commercial license: PyQt6
|
||||
- A modern look and dark mode for tkinter apps: CustomTkinter
|
||||
- Your own HTML, CSS, and JavaScript in a desktop window: pywebview
|
||||
- A pure Python UI in the browser or a desktop window: NiceGUI
|
||||
- One Python codebase for desktop, mobile, and web: Flet
|
||||
|
||||
PySide6 is [the official Python binding for Qt](https://doc.qt.io/qtforpython-6/), under the LGPL, which [lets your app's code stay closed](https://www.qt.io/licensing/open-source-lgpl-obligations) as long as you meet its terms. Build the UI with Qt Widgets for an app that [looks native](https://doc.qt.io/qtforpython-6/faq/whatisqt.html#widgets), or with QML, which was [first made for mobile apps](https://doc.qt.io/qtforpython-6/faq/whatisqt.html#qml) and runs on desktop too. To lay out forms visually, draw them in Qt Widgets Designer, and pyside6-uic [turns the .ui files into Python](https://doc.qt.io/qtforpython-6/tutorials/basictutorial/uifiles.html).
|
||||
|
||||
PyQt6 binds the same Qt, and PySide6 [aims to be API compatible with PyQt](https://doc.qt.io/qtforpython-6/considerations.html#api-changes), with some exceptions. The license is what splits them: PyQt6 is [GPL or commercial, never LGPL](https://www.riverbankcomputing.com/static/Docs/PyQt6/introduction.html#license), so with the GPL version your own code must use a compatible license.
|
||||
|
||||
tkinter is part of the standard library. Use the themed `tkinter.ttk` widgets, which [adapt to each platform's native theme](https://docs.python.org/3/library/tkinter.ttk.html). Most tutorials online still teach the old API, so [the docs point you to TkDocs](https://docs.python.org/3/library/tkinter.html), which teaches the modern one.
|
||||
|
||||
CustomTkinter widgets [work like normal Tkinter widgets](https://github.com/TomSchimansky/CustomTkinter) and mix with them, and they follow the system's light or dark mode.
|
||||
|
||||
PyGObject is [the way to go](https://pygobject.gnome.org/) for a GNOME app or any GTK app. Start with the GNOME Developer Documentation's [beginner lessons](https://developer.gnome.org/documentation/tutorials/beginners.html). In code, subclass `Gtk.Application` and build your window in `do_activate()`, as [the getting-started example](https://pygobject.gnome.org/getting_started.html) does.
|
||||
|
||||
Dear PyGui is built on Dear ImGui and [draws in immediate mode on your GPU](https://github.com/hoffstadt/DearPyGui), for [scientific, engineering, and data science apps](https://dearpygui.readthedocs.io/en/latest/) that need fast, interactive interfaces. Every main script [runs the same steps](https://dearpygui.readthedocs.io/en/latest/tutorials/first-steps.html#first-run): create the context and the viewport, set up, show the viewport, start, then destroy the context.
|
||||
|
||||
Kivy was built [from scratch for multi-touch](https://kivy.org/doc/stable/philosophy.html), and the same code runs on Windows, macOS, Linux, Android, and iOS. Subclass `App` and [return your root widget from `build()`](https://kivy.org/doc/stable/guide/basic.html#create-an-application). As the UI grows, move it into the KV language, which [separates your app's logic from its interface](https://kivy.org/doc/stable/guide/lang.html#concept-behind-the-language).
|
||||
|
||||
wxPython wraps the wxWidgets C++ library and, in most cases, [uses each platform's native widgets](https://wxpython.org/pages/overview/#what-is-wxpython). Its [Hello World](https://wxpython.org/pages/overview/#hello-world) subclasses `wx.Frame`, puts the widgets on a `wx.Panel`, and lays them out with a sizer.
|
||||
|
||||
Toga [uses native system widgets, not themes](https://toga.beeware.org/en/latest/about/philosophy/#native-widgets-not-themes), on macOS, Windows, Linux, Android, and iOS. It's part of BeeWare, whose [tutorial](https://tutorial.beeware.org/) builds an app with Toga.
|
||||
|
||||
pywebview [shows your HTML, CSS, and JavaScript in a native window](https://pywebview.flowrl.com/). It doesn't bundle a GUI toolkit or web renderer, so a frozen app [stays small](https://pywebview.flowrl.com/guide/). Point the window at a web server you already run, or call Python from JavaScript through its [JS API bridge](https://pywebview.flowrl.com/guide/architecture.html#js-api-with-internal-http-server) and serve static files from the built-in HTTP server.
|
||||
|
||||
NiceGUI keeps [all UI logic in Python](https://github.com/zauberzeug/nicegui#architecture) on a FastAPI backend and handles the web details for you. The UI shows up in your browser, or in a native desktop window.
|
||||
|
||||
Flet makes [web, desktop, and mobile apps](https://flet.dev/docs/) without HTML, CSS, or JavaScript, and [builds them with Flutter](https://flet.dev/docs/publish/). `flet run` [opens your app](https://github.com/flet-dev/flet) in a desktop window, and `flet run --web` in the browser. As the app gets more interactive, its docs [favor the declarative style](https://flet.dev/docs/cookbook/declarative-vs-imperative/), where the UI is derived from your app's state.
|
||||
@@ -0,0 +1,19 @@
|
||||
Serial ports open with pySerial. Past those, pick a Python hardware library by the device you talk to, like Bleak for Bluetooth LE sensors.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Serial ports on Windows, macOS, Linux, or BSD: pySerial
|
||||
- Bluetooth Low Energy devices, like sensors: Bleak
|
||||
- Controlling or monitoring the keyboard and mouse: pynput
|
||||
- Global hotkeys: pynput
|
||||
- Automated tests on real or virtual hardware, shared across a team or CI: Jumpstarter
|
||||
|
||||
pySerial is imported as `serial`, and `serial.Serial` opens a port as a context manager. [Set a timeout when you open it](https://pyserial.readthedocs.io/en/latest/shortintro.html#readline), or `readline()` can block forever when no newline arrives. The docs also suggest [`serial_for_url()`](https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.serial_for_url) over creating `Serial` directly, so the same code takes a local port, a remote one, or a `loop://` loopback for your tests.
|
||||
|
||||
pynput sends input through a `Controller` and watches it through a `Listener`, for the keyboard and the mouse alike. A listener is a thread that runs your callbacks. On some platforms, notably Windows, a slow callback risks freezing input for every process, so [keep callbacks short](https://pynput.readthedocs.io/en/latest/keyboard.html#the-keyboard-listener-thread): hand events to a queue, and let another thread work through it.
|
||||
|
||||
Bleak is async, so [call `asyncio.run()` once](https://bleak.readthedocs.io/en/latest/troubleshooting.html#calling-asyncio-run-more-than-once) and do all your Bluetooth work in one async main function. Scan with `BleakScanner` and connect with `BleakClient`, each [in an `async with` block](https://bleak.readthedocs.io/en/latest/api/client.html#connecting-and-disconnecting), so the device disconnects when the block exits.
|
||||
|
||||
Jumpstarter puts your device behind an exporter, which manages its interfaces, and your tests reach it through a client. [Local mode](https://jumpstarter.dev/main/introduction/#local-mode) needs no Kubernetes or other infrastructure: run `jmp shell --exporter <name>`, then pytest inside it. When teams share hardware, especially in CI, distributed mode adds a Kubernetes-based controller that leases devices out. Write tests as subclasses of [`JumpstarterTest`](https://jumpstarter.dev/main/getting-started/guides/examples/testing.html), whose `client` fixture handles the connection.
|
||||
|
||||
Jumpstarter builds on the other picks: its serial driver runs on pySerial and takes the port [in pySerial's format](https://jumpstarter.dev/main/reference/package-apis/drivers/pyserial.html), and its Bluetooth LE driver runs on Bleak.
|
||||
@@ -0,0 +1,21 @@
|
||||
Scraped HTML, however broken, parses in Beautiful Soup on lxml's parser. JustHTML packs a sanitizer into a Python HTML library and runs it by default.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Pulling data out of scraped pages: Beautiful Soup, on lxml's parser
|
||||
- Speed, XPath, or XML documents: lxml
|
||||
- Sanitizing HTML your users submit: JustHTML
|
||||
- XML you'd rather handle like JSON: xmltodict
|
||||
- Escaping text you put into HTML: MarkupSafe
|
||||
|
||||
Beautiful Soup is for [pulling data out of HTML and XML files](https://www.crummy.com/software/BeautifulSoup/bs4/doc/). It sits on a parser you pick and gives you one way to navigate, search, and change the tree. Its docs [recommend lxml as that parser](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-a-parser) for speed. Name the parser in the constructor, as in `BeautifulSoup(markup, "lxml")`. [Different parsers build different trees](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#differences-between-parsers) from the same broken page, and your script should parse it the same way on every machine. The docs also say when to skip Beautiful Soup: [work directly atop lxml](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#improving-performance) when computer time costs more than programmer time, and [parse with lxml](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors-through-the-css-property) when CSS selectors are all you need.
|
||||
|
||||
lxml is [a Pythonic binding for libxml2 and libxslt](https://lxml.de/): the speed and XML features of those C libraries, with an API mostly compatible with ElementTree's. For HTML, use [lxml.html](https://lxml.de/lxmlhtml.html), which adds HTML-specific methods to lxml's elements, and every element takes `.xpath()`. On broken pages, lxml's docs say you often need only [Beautiful Soup's encoding detection](https://lxml.de/lxmlhtml.html#really-broken-pages). Leave the rest to lxml's own parser, which is several times faster. XPath has the same injection problem as SQL. When a value comes from outside, [pass it as an XPath variable](https://lxml.de/FAQ.html#how-do-i-use-lxml-safely-as-a-web-service-endpoint) instead of formatting it into the expression.
|
||||
|
||||
JustHTML [parses HTML like a browser](https://github.com/EmilStenstrom/justhtml), including broken markup. `JustHTML(html)` [sanitizes by default](https://emilstenstrom.github.io/justhtml/sanitization.html) against a strict allowlist. The output is safe in a page body, but [not automatically safe inside a `<script>` tag or an attribute](https://emilstenstrom.github.io/justhtml/sanitization.html#important-context-is-king); JustHTML has separate escaping helpers for those. It's pure Python, with no C extension to install. For terabytes of trusted HTML, its README says to use a C or Rust parser like lxml.
|
||||
|
||||
xmltodict [makes working with XML feel like working with JSON](https://github.com/martinblech/xmltodict): `parse()` turns a document into dicts, and `unparse()` turns dicts back into XML. It covers the common 90% of cases and doesn't keep every XML detail, like attribute order. For exact fidelity, its README says to use lxml.
|
||||
|
||||
MarkupSafe [escapes characters so text is safe to use in HTML and XML](https://markupsafe.palletsprojects.com/en/latest/). `escape()` returns a `Markup` string, and any text you join to it gets escaped too. To build HTML, [use `Markup` as the format string](https://markupsafe.palletsprojects.com/en/latest/formatting/): the values you format into it are escaped first. Passing text to `Markup()` itself [marks it safe without escaping](https://markupsafe.palletsprojects.com/en/latest/escaping/#markupsafe.Markup), so save that for markup you trust.
|
||||
|
||||
Escape user text with MarkupSafe, and sanitize user HTML you want to keep with JustHTML. For XML you didn't write, lxml's FAQ says to [keep network access and external DTDs off](https://lxml.de/FAQ.html#how-do-i-use-lxml-safely-as-a-web-service-endpoint) and parse with `resolve_entities=False`, then reject any document that still holds entity references. With xmltodict, pass `disable_entities=True`.
|
||||
@@ -0,0 +1,20 @@
|
||||
Requests makes synchronous HTTP calls simple. A Python HTTP client for async code or HTTP/2 means httpx2, whose API stays requests-compatible.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Scripts and other synchronous code: Requests
|
||||
- One API for sync and async code, or HTTP/2: httpx2
|
||||
- Code already on HTTPX: httpx2, the same API under a new name
|
||||
- An asyncio app making many requests at once: aiohttp
|
||||
- Direct control over connection pools and retries: urllib3
|
||||
- Building and editing URLs: yarl
|
||||
|
||||
Requests is an [HTTP library built for human beings](https://requests.readthedocs.io/en/latest/), with urllib3 pooling connections underneath. Nearly every call in production code should [pass a timeout](https://requests.readthedocs.io/en/latest/user/quickstart/#timeouts), since requests never time out unless you set one. For several calls to the same host, make them through a [Session used as a context manager](https://requests.readthedocs.io/en/latest/user/advanced/#session-objects): it reuses the TCP connection and closes when the block exits.
|
||||
|
||||
httpx2 has a [broadly requests-compatible API](https://pydantic.dev/docs/httpx2/get-started/) in both sync and async code, over HTTP/1.1 or HTTP/2. It's a fork of HTTPX with the [same public API](https://pydantic.dev/docs/httpx2/get-started/migration/#in-a-hurry) under a new name. Beyond one-off scripts, send requests through a [Client used as a context manager](https://pydantic.dev/docs/httpx2/advanced/clients/#why-use-a-client), which pools connections where the top-level functions open a new one per request. In async code, share [one AsyncClient](https://pydantic.dev/docs/httpx2/guides/async/#opening-and-closing-clients) instead of opening one inside a loop.
|
||||
|
||||
aiohttp is async only, and its docs call that choice a trade of [more verbosity for better performance](https://docs.aiohttp.org/en/latest/http_request_lifecycle.html#using-a-session-as-a-best-practice). Create [one ClientSession per application](https://docs.aiohttp.org/en/latest/client_quickstart.html#make-a-request) and reuse it for every request, since each session holds its own connection pool.
|
||||
|
||||
urllib3 is the layer Requests runs on, with [thread safety, connection pooling, and retries](https://urllib3.readthedocs.io/en/latest/) built in. Make requests through a [PoolManager you create](https://urllib3.readthedocs.io/en/latest/user-guide.html#making-requests): the top-level `urllib3.request()` uses a module-global one, so its side effects can reach other libraries that call it too.
|
||||
|
||||
yarl's [URL objects are immutable](https://yarl.aio-libs.org/en/latest/#introduction): every change returns a new URL, and strings you pass in get encoded for you. aiohttp's requests [take a yarl URL](https://docs.aiohttp.org/en/latest/client_quickstart.html#make-a-request) as well as a plain string.
|
||||
@@ -0,0 +1,30 @@
|
||||
Pillow handles everyday edits, so it's the default Python image processing library. Use scikit-image for scientific analysis, and pyvips when memory is tight.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Resizing, cropping, and converting images: Pillow
|
||||
- Scientific image analysis on NumPy arrays: scikit-image
|
||||
- Large images, or many of them, with little memory: pyvips
|
||||
- ImageMagick's features from Python: Wand
|
||||
- Removing photo backgrounds: rembg
|
||||
- Resizing and cropping images on demand over HTTP: thumbor
|
||||
- QR codes: qrcode
|
||||
- Barcodes: python-barcode
|
||||
|
||||
Pillow is [ideal for batch processing](https://pillow.readthedocs.io/en/stable/handbook/overview.html), like making thumbnails and converting between file formats. Open each file in a `with Image.open(path) as im:` block. Opening is [fast and independent of the file size](https://pillow.readthedocs.io/en/stable/handbook/tutorial.html), since Pillow reads the pixels only when it has to. To fit an image to a size, use `ImageOps.contain()`, `cover()`, `fit()`, or `pad()`. `thumbnail()` works too, but it [changes the image in place](https://pillow.readthedocs.io/en/stable/handbook/tutorial.html#relative-resizing).
|
||||
|
||||
scikit-image [aims to be the reference library for scientific image analysis](https://scikit-image.org/docs/stable/about/values.html) in Python, and puts science ahead of photo editing. Images are plain NumPy arrays, so [standard NumPy operations](https://scikit-image.org/docs/stable/user_guide/numpy_images.html) work on them. To change an image's dtype, use `img_as_float()` or `img_as_ubyte()`, [never `astype`](https://scikit-image.org/docs/stable/user_guide/data_types.html), which doesn't rescale the values to the new dtype's range.
|
||||
|
||||
pyvips builds a pipeline of operations and runs it only when you write the result. It streams the image a section at a time, so it [doesn't keep whole images in memory](https://github.com/libvips/pyvips). To shrink an image, use `pyvips.Image.thumbnail()` instead of resizing: it [loads and resizes in one step](https://www.libvips.org/API/current/developer-checklist.html), which is faster and uses less memory. When you read an image from top to bottom, open it with [`access="sequential"`](https://libvips.github.io/pyvips/intro.html).
|
||||
|
||||
Wand is a [ctypes-based ImageMagick binding](https://docs.wand-py.org/en/latest/), so install ImageMagick's MagickWand library first. Its objects are resources like open files: [use them in a `with` block](https://docs.wand-py.org/en/latest/guide/resource.html) so they get closed. Wand's docs say to [never use Wand directly in an HTTP service](https://docs.wand-py.org/en/latest/guide/security.html) or on any public server. Hand the images to a background worker through a queue, and limit ImageMagick's resources and formats in its `policy.xml`.
|
||||
|
||||
rembg runs as a [CLI, a Python library, an HTTP server, or a Docker container](https://github.com/danielgatis/rembg). In code, create a session once with `new_session()` and pass it to each `remove()` call, since `remove` otherwise [starts a new session every call](https://github.com/danielgatis/rembg/blob/main/USAGE.md). The model weights [carry their own licenses](https://github.com/danielgatis/rembg#models), separate from rembg's MIT license, so check the one you use before you ship it in a commercial product.
|
||||
|
||||
thumbor is an HTTP server: you [set the size and crop in the image URL](https://github.com/thumbor/thumbor), and it [detects faces and important features](https://thumbor.readthedocs.io/en/latest/) to crop around them. Set a `SECURITY_KEY` so [every URL is signed](https://thumbor.readthedocs.io/en/latest/security.html) and nobody can tamper with it, and build those URLs in Python with [libthumbor](https://thumbor.readthedocs.io/en/latest/libraries.html). In production, [turn off `ALLOW_UNSAFE_URL`](https://thumbor.readthedocs.io/en/latest/configuration.html) and run [more than one instance](https://thumbor.readthedocs.io/en/latest/hosting.html) behind a load balancer.
|
||||
|
||||
qrcode makes a QR code in one call: `qrcode.make("Some data")`. For more control, use the `QRCode` class, and pass `version=None` with `make(fit=True)` to pick the size for you. For SVG output, the README [recommends the path factory](https://github.com/lincolnloop/python-qrcode), `SvgPathImage`. If you style the code or embed an image, set error correction to high, since styled codes aren't guaranteed to work with all readers.
|
||||
|
||||
python-barcode writes SVG with [no external dependencies](https://python-barcode.readthedocs.io/en/latest/). For PNG and other images, install the `python-barcode[images]` extra, which adds Pillow. Its docs [recommend SVG](https://python-barcode.readthedocs.io/en/latest/getting-started.html) unless your target can't use it, since vectors scale better. It calculates the checksum for you.
|
||||
|
||||
Treat every uploaded image as untrusted. Pillow [guards against decompression bombs](https://pillow.readthedocs.io/en/stable/reference/Image.html) with a pixel limit, and its `Image.open(formats=...)` argument restricts the formats it tries. With pyvips, check the image dimensions before you process it, and [block the loaders](https://www.libvips.org/API/current/developer-checklist.html) libvips hasn't tested for security. With Wand, check each file's [magic bytes](https://docs.wand-py.org/en/latest/guide/security.html), never its extension or MIME type.
|
||||
@@ -0,0 +1,21 @@
|
||||
Unless you need MicroPython on a microcontroller or Pyodide in the browser, CPython is the Python implementation to run. Cython compiles your slow code to C.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Most projects: CPython
|
||||
- Microcontrollers and other constrained devices: MicroPython
|
||||
- Python in the browser or Node.js: Pyodide
|
||||
- Slow code to compile, or a C library to wrap: Cython
|
||||
- Long-running programs in pure Python: PyPy
|
||||
|
||||
CPython is the [original and most-maintained implementation](https://docs.python.org/3/reference/introduction.html#alternate-implementations) of Python, written in C, and new language features generally show up there first. Give each app its own [virtual environment](https://docs.python.org/3/installing/index.html) from venv, and install packages into it with pip.
|
||||
|
||||
MicroPython is a lean implementation of Python 3, [optimized to run on microcontrollers](https://micropython.org/) and in constrained environments. It includes a small subset of the standard library, plus modules like `machine` for the hardware. Flash your board with firmware from [the download page](https://micropython.org/download/), then manage it from your computer with [mpremote](https://docs.micropython.org/en/latest/reference/mpremote.html). `mpremote mip install` [installs packages](https://docs.micropython.org/en/latest/reference/packages.html#installing-packages-with-mpremote) from micropython-lib by default, not PyPI. When RAM runs short, [freeze the modules that rarely change](https://docs.micropython.org/en/latest/reference/packages.html#freezing-packages) into the firmware.
|
||||
|
||||
Pyodide is [a port of CPython to WebAssembly](https://pyodide.org/en/stable/) that runs in the browser and Node.js. It installs any pure Python wheel from PyPI, and many packages with C extensions, like NumPy and pandas, have been ported for it. Install packages with [`micropip.install()`](https://pyodide.org/en/stable/usage/loading-packages.html#how-to-chose-between-micropip-install-and-pyodide-loadpackage), which its docs recommend for almost everything. In a web page, run Pyodide [in a web worker](https://pyodide.org/en/stable/usage/webworker.html), so long computations don't freeze your UI.
|
||||
|
||||
Cython isn't a separate interpreter: it [translates Python code to C](https://cython.readthedocs.io/en/latest/src/quickstart/overview.html) that runs inside CPython, to speed up slow code or wrap C libraries. Write your code in [pure Python syntax](https://cython.readthedocs.io/en/latest/src/tutorial/pure.html) to keep the file runnable by the plain interpreter, and use `.pyx` files for what that syntax can't express. The annotation report from `cython -a` shows [where types help](https://cython.readthedocs.io/en/latest/src/quickstart/cythonize.html#determining-where-to-add-types). Build your package with [a build backend](https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiling-with-a-build-backend), and ship [prebuilt wheels](https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiling-with-pyximport) to your users.
|
||||
|
||||
PyPy is a replacement for CPython, and speed is the reason to use it. It works best on [long-running programs that spend much of their time in Python code](https://pypy.org/features.html#speed), not on short scripts. Code built on C extension modules is a poor fit, since they [often run much slower on PyPy than on CPython](https://doc.pypy.org/faq.html#do-c-extension-modules-work-with-pypy). Packages installed for CPython aren't available to PyPy, so [install them for PyPy](https://doc.pypy.org/faq.html#module-xyz-does-not-work-with-pypy-importerror) in its own virtual environment with `pypy -m pip`.
|
||||
|
||||
Before you switch implementations or compile anything for speed, [measure first](https://pypy.org/performance.html#profiling-vmprof) to confirm the slowdown is real. Then profile to find the slow parts, and only optimize those.
|
||||
@@ -0,0 +1,18 @@
|
||||
Exploring code is easier in IPython than in the default Python interactive interpreter. Jupyter Notebook mixes code with notes and plots.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A better terminal prompt, with help on any object and magic commands: IPython
|
||||
- Code, notes, and plots in one shareable document: Jupyter Notebook
|
||||
- A reactive notebook saved as a Python file you can run as a script or app: marimo
|
||||
- A terminal REPL with syntax highlighting, multiline editing, and Vi or Emacs keys: ptpython
|
||||
|
||||
IPython takes whatever you'd type at the standard Python prompt and adds [tab completion, object introspection, and system shell access](https://ipython.readthedocs.io/en/stable/overview.html). Its tutorial covers [the everyday commands](https://ipython.readthedocs.io/en/stable/interactive/tutorial.html): `object?` shows an object's details, Tab after `object_name.` lists its attributes, and `!` runs a shell command. Magic commands start with `%`: `%timeit` times a statement, `%run` runs a script and loads its variables into your session, and `%debug` opens the debugger after an exception.
|
||||
|
||||
Start Jupyter Notebook with [`jupyter notebook`](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html#starting-the-notebook-server), which runs a server on your machine and opens the app in your browser. Its docs suggest you [work on a problem in pieces](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html#basic-workflow): put related ideas into cells, and move on once the earlier ones work. Python code runs in IPython, the default kernel. Notebooks are saved as `.ipynb` files, and nbconvert [turns them into HTML, PDF, or slides](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html#notebook-documents).
|
||||
|
||||
marimo is [a reactive notebook](https://docs.marimo.io/): run a cell, and marimo runs the cells that depend on it, so code and outputs stay consistent. Each notebook is [stored as a plain Python file](https://docs.marimo.io/faq/#how-is-marimo-different-from-jupyter), so you can diff it in Git, run it as a script with `python`, or serve it as an app with `marimo run`. That model comes with two rules. [Each global variable is defined in only one cell](https://docs.marimo.io/guides/reactivity/#global-variable-names-must-be-unique), so prefix a scratch variable with `_` to keep it local to its cell. And since marimo doesn't track mutations, [mutate an object only in the cell that creates it](https://docs.marimo.io/guides/best_practices/). Unlike a Jupyter notebook, a marimo file [doesn't store outputs like plots](https://docs.marimo.io/guides/coming_from/jupyter/), and IPython magic commands and `!` shell commands don't work in it.
|
||||
|
||||
ptpython is [a better Python REPL](https://github.com/prompt-toolkit/ptpython) for the terminal, built on prompt_toolkit. It adds syntax highlighting, multiline editing where the up arrow works, autocompletion, Vi or Emacs key bindings, and a configuration menu. Run `ptpython` to start it, or `ptipython` to add IPython's magic functions and shell integration.
|
||||
|
||||
Both terminal REPLs can also open inside your program, with that scope's variables at hand: call [`IPython.embed()`](https://ipython.readthedocs.io/en/stable/interactive/reference.html#embedding-ipython), or ptpython's [`embed(globals(), locals())`](https://github.com/prompt-toolkit/ptpython#embedding-the-repl). For more Jupyter tools, see [awesome-jupyter](https://github.com/markusschanta/awesome-jupyter).
|
||||
@@ -0,0 +1,21 @@
|
||||
A function that runs every hour fits in APScheduler inside your app. A pipeline of tasks outgrows a Python scheduler and needs Airflow, Prefect, or Dagster.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Jobs inside a running app, on cron or one-off triggers, kept across restarts: APScheduler
|
||||
- Batch pipelines with a clear start and end that run on a schedule: Airflow
|
||||
- Your Python functions as pipelines, with tasks created at runtime: Prefect
|
||||
- Pipelines built around the data assets they produce, like tables and models: Dagster
|
||||
- A simple loop of periodic jobs in one script: schedule
|
||||
|
||||
APScheduler [schedules your Python code to run later](https://github.com/agronholm/apscheduler/blob/3.x/README.rst), once or periodically. It runs inside your existing application, not as a service. Give each job [a trigger](https://apscheduler.readthedocs.io/en/latest/userguide.html#choosing-the-right-scheduler-job-store-s-executor-s-and-trigger-s): `date` runs it once, `interval` at fixed intervals, and `cron` at set times of day. Jobs [live in memory by default](https://apscheduler.readthedocs.io/en/latest/userguide.html#basic-concepts). When they must survive restarts and crashes, add a persistent job store.
|
||||
|
||||
schedule is an [in-process scheduler for periodic jobs](https://schedule.readthedocs.io/en/latest/), with no extra processes and no dependencies. Write `schedule.every(10).minutes.do(job)`, then call `schedule.run_pending()` in a loop. Its docs call it [a simple solution for simple scheduling problems](https://schedule.readthedocs.io/en/latest/#when-not-to-use-schedule), and say to look elsewhere when jobs must persist between restarts or run concurrently.
|
||||
|
||||
Airflow is [a platform for orchestrating batch workflows](https://airflow.apache.org/docs/apache-airflow/stable/index.html#why-airflow). Its docs say workflows with a clear start and end that run on a schedule are a great fit. It comes with a wide range of built-in operators for integrating other technologies. It's a set of services: [a minimal install](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/overview.html#required-components) runs a scheduler, a processor that parses your workflow files, and an API server with the UI. It also needs a metadata database, usually PostgreSQL or MySQL. Write tasks with [the TaskFlow API](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html): decorate plain Python functions, and Airflow creates the tasks, wires their dependencies, and passes data between them.
|
||||
|
||||
Prefect [turns your Python functions into data pipelines](https://docs.prefect.io/v3/get-started), with no DSLs or complex config files. Put [`@flow` on your script's entrypoint and `@task` on each function it calls](https://docs.prefect.io/v3/get-started/quickstart). Prefect tracks each task's state, so a failed run can resume from its point of failure. With the open-source server running, call `.serve()` on your flow with a `cron` schedule: it starts a process that runs the flow on that schedule. Its docs call serving [simple to reason about](https://docs.prefect.io/v3/concepts/deployments#static-infrastructure) for flows on a machine you control.
|
||||
|
||||
Dagster is [a data orchestrator built for data engineers](https://docs.dagster.io/), with lineage and observability built in. You [declare data assets like tables, datasets, and ML models as Python functions](https://github.com/dagster-io/dagster). Dagster runs them at the right time to keep them up to date. If you're just starting out, its docs [strongly recommend assets rather than ops](https://docs.dagster.io/guides/build/ops). Run assets on [a cron schedule](https://docs.dagster.io/guides/automate/schedules).
|
||||
|
||||
In an orchestrator, make every task safe to run twice. Airflow [can retry a failed task](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#creating-a-task), so its docs say a task should produce the same outcome on every re-run. Prefect tasks are [retryable units of work](https://docs.prefect.io/v3/concepts/tasks) too.
|
||||
@@ -0,0 +1,17 @@
|
||||
Keep the built-in logging module until you need what a Python logging library adds: structlog's key-value logs in JSON, or Loguru's logger, ready on import.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A library other people import: logging, with only a `NullHandler`
|
||||
- Key-value events, pretty in development and JSON in production: structlog
|
||||
- An app already on logging that needs structured output: structlog, which wraps it
|
||||
- A script you want logging in with no setup: Loguru
|
||||
- Log files that rotate, expire, and compress: Loguru
|
||||
|
||||
Every Python module [can take part in the standard library's logging](https://docs.python.org/3/library/logging.html), so your app's log holds messages from third-party packages next to your own. In each module, create a logger with [`logging.getLogger(__name__)`](https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial), so logger names follow your package layout. Use `basicConfig()` for a quick setup, and move to [`dictConfig()`](https://docs.python.org/3/howto/logging.html#configuring-logging), which the docs recommend for new applications. In a library, [add no handler other than `NullHandler`](https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library) and don't log to the root logger: handlers are for the app developer to pick.
|
||||
|
||||
structlog logs [events in a context of key-value pairs](https://www.structlog.org/en/latest/why.html#structured-logging) instead of prose, so each entry is a dictionary instead of a string. [Bind values to a logger](https://www.structlog.org/en/latest/getting-started.html#building-a-context), and every entry it logs carries them. In a web app, [call `clear_contextvars()` at the start of each request](https://www.structlog.org/en/latest/contextvars.html), then `bind_contextvars()` for values like a request ID. Render [pretty, colored output during development and JSON in production](https://www.structlog.org/en/latest/logging-best-practices.html#pretty-printing-vs-structured-output), since log aggregators parse JSON more easily. structlog can also [wrap the standard library's logging and add structure to it](https://www.structlog.org/en/latest/getting-started.html#structlog-and-standard-librarys-logging), so an app already built on logging keeps it.
|
||||
|
||||
Loguru has [one logger, ready to use](https://loguru.readthedocs.io/en/latest/overview.html#ready-to-use-out-of-the-box-without-boilerplate) after `from loguru import logger`, and it writes to stderr out of the box. Instead of handlers, formatters, and filters, you configure it with [one function, `add()`](https://loguru.readthedocs.io/en/latest/overview.html#no-handler-no-formatter-no-filter-one-function-to-rule-them-all). In an app, [call `remove()` first](https://loguru.readthedocs.io/en/latest/resources/troubleshooting.html#how-do-i-create-and-configure-a-logger) to drop the default handler, then `add()` where your logs should go. Given a file path, `add()` handles [rotation, retention, and compression](https://loguru.readthedocs.io/en/latest/overview.html#easier-file-logging-with-rotation-retention-compression), and [`serialize=True`](https://loguru.readthedocs.io/en/latest/overview.html#structured-logging-as-needed) turns each message into JSON. In a library, [call `disable()` instead of `add()`](https://loguru.readthedocs.io/en/latest/overview.html#suitable-for-scripts-and-libraries), and the app using it can `enable()` your logs again.
|
||||
|
||||
Whichever you pick, configure logging once, where your app starts, and have each module only get its logger. With the standard library, usually [only the root logger needs configuring](https://docs.python.org/3/library/logging.html), since module loggers pass their messages up to it. structlog wants [`structlog.configure()` on app initialization](https://www.structlog.org/en/latest/configuration.html), and with Loguru, [your other modules inherit the configuration](https://loguru.readthedocs.io/en/latest/resources/troubleshooting.html#how-do-i-create-and-configure-a-logger) from your entry point. Third-party packages that use the standard library's logging keep logging there. Send their messages to your output too: structlog [formats them with `ProcessorFormatter`](https://www.structlog.org/en/latest/standard-library.html#processor-formatter), and Loguru [intercepts them with a handler](https://loguru.readthedocs.io/en/latest/overview.html#entirely-compatible-with-standard-logging).
|
||||
@@ -0,0 +1,37 @@
|
||||
Fit models with scikit-learn, which covers a Python machine learning library's basic tasks, boosted trees included. Deep learning takes a framework of its own.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Classification, regression, and clustering on tabular data: scikit-learn
|
||||
- Boosted trees with no extra dependency: scikit-learn
|
||||
- Causal graphs and Bayesian networks: pgmpy
|
||||
- Feature engineering that keeps your DataFrame's column names: Feature-engine
|
||||
- Boosted trees trained across a cluster: XGBoost
|
||||
- Boosted trees on large data, fast and light on memory: LightGBM
|
||||
- Tables full of category columns, with little tuning: CatBoost
|
||||
- Series with strong seasonality and holiday effects: Prophet
|
||||
- Many series on classical models like ARIMA and ETS: StatsForecast
|
||||
- Forecasting next to classification and other time-series tasks: sktime
|
||||
- Forecasts with no model to train: TimesFM
|
||||
|
||||
scikit-learn does [supervised and unsupervised learning](https://scikit-learn.org/stable/getting_started.html), plus the tools around it: preprocessing, model selection, and evaluation. Every estimator learns with `fit`. Put your preprocessing and model in one Pipeline, and run cross-validation and searches on that, since the pipeline [keeps test data out of training](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage). You may not need a separate boosting library: its histogram-based gradient boosting is [inspired by LightGBM](https://scikit-learn.org/stable/modules/ensemble.html) and supports missing values and categorical data natively. Deep learning is another matter: it [doesn't fit scikit-learn's design constraints](https://scikit-learn.org/stable/faq.html#why-is-there-no-support-for-deep-or-reinforcement-learning-will-there-be-such-support-in-the-future), so it's out of scope.
|
||||
|
||||
pgmpy does [causal and probabilistic reasoning with graphical models](https://pgmpy.org/). Its [quickstart](https://pgmpy.org/started/quickstart.html) follows the tasks: learn a graph's structure from data, fit the distributions for a graph you know, then query the model for probabilities or causal effects.
|
||||
|
||||
Feature-engine is [designed to work with dataframes](https://feature-engine.trainindata.com/en/latest/): a DataFrame goes in, and the same DataFrame comes out, with no column order or name changes. Instead of wrapping transformers in a ColumnTransformer, you tell each one which variables to transform. Its transformers [work just like any scikit-learn transformer](https://feature-engine.trainindata.com/en/latest/quickstart/index.html), so they fit in a Pipeline.
|
||||
|
||||
XGBoost is [an optimized distributed gradient boosting library](https://xgboost.readthedocs.io/en/stable/). Its Python package has [a native interface, a scikit-learn interface, and a Dask interface](https://xgboost.readthedocs.io/en/stable/python/python_intro.html).
|
||||
|
||||
LightGBM is designed for [faster training, lower memory use, and large-scale data](https://lightgbm.readthedocs.io/en/latest/). Pass category columns as they are: LightGBM [uses them directly, with no one-hot encoding](https://lightgbm.readthedocs.io/en/latest/Python-Intro.html).
|
||||
|
||||
CatBoost aims for [great results with default parameters](https://catboost.ai/), and it takes non-numeric features without you turning them into numbers. Name the category columns in `cat_features`, and [don't one-hot encode them](https://catboost.ai/docs/en/features/categorical-features), which hurts both training speed and quality.
|
||||
|
||||
Prophet fits an additive model of the trend, yearly, weekly, and daily seasonality, and holiday effects. It [works best with series that have strong seasonal effects](https://facebook.github.io/prophet/) and several seasons of history. It [follows the scikit-learn model API](https://facebook.github.io/prophet/docs/quick_start.html): pass `fit` a DataFrame with `ds` and `y` columns, then call `predict`.
|
||||
|
||||
StatsForecast offers [widely used univariate models](https://github.com/Nixtla/statsforecast), like automatic ARIMA, ETS, and Theta, optimized for speed, plus benchmark models for baselines. Data goes in as a long-format DataFrame with `unique_id`, `ds`, and `y` columns, and for many series the docs [recommend the `forecast` method](https://nixtlaverse.nixtla.io/statsforecast/docs/getting-started/getting_started_short.html).
|
||||
|
||||
sktime gives forecasting, classification, clustering, and anomaly detection [one unified interface](https://github.com/sktime/sktime), with interfaces to scikit-learn and Prophet. Split the series with `temporal_train_test_split`, set a `ForecastingHorizon`, then `fit` and `predict`, as in [Get Started](https://www.sktime.net/docs/get-started/). Through [reduction](https://www.sktime.net/docs/user-guide/introduction/), a scikit-learn regressor can solve a forecasting task.
|
||||
|
||||
TimesFM is [a pretrained foundation model from Google Research](https://github.com/google-research/timesfm) for time-series forecasting. It forecasts [zero-shot](https://research.google/blog/a-decoder-only-foundation-model-for-time-series-forecasting/), with no training on your data: load a checkpoint and call `predict` with a horizon. Check the license of the weights you download: some are [for non-commercial, non-production use only](https://github.com/google-research/timesfm#license-notice-for-pretrained-weights).
|
||||
|
||||
To evaluate a model on time-series data, test it on observations from after its training data. scikit-learn's [TimeSeriesSplit](https://scikit-learn.org/stable/modules/cross_validation.html#time-series-split) does that in cross-validation. For more, see [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning#python).
|
||||
@@ -0,0 +1,18 @@
|
||||
Instead of a Python messaging library per broker, an asyncio service can run on FastStream. Otherwise, use confluent-kafka, pika, or paho-mqtt.
|
||||
|
||||
How to choose:
|
||||
|
||||
- An asyncio service on Kafka, RabbitMQ, MQTT, NATS, or Redis: FastStream
|
||||
- Kafka, including Confluent's Schema Registry: confluent-kafka
|
||||
- RabbitMQ, with the client its team recommends: pika
|
||||
- IoT devices and sensors that speak MQTT: paho-mqtt
|
||||
|
||||
confluent-kafka is Confluent's Kafka client, [a binding on top of librdkafka](https://docs.confluent.io/kafka-clients/python/current/overview.html), the C client. `produce()` only queues a message, so [call `poll()` to get delivery reports](https://docs.confluent.io/kafka-clients/python/current/overview.html#asynchronous-writes). Its README says production apps [should serialize with Schema Registry](https://github.com/confluentinc/confluent-kafka-python#basic-producer-example) instead of producing raw bytes.
|
||||
|
||||
pika is [a pure-Python implementation of AMQP 0-9-1](https://pika.readthedocs.io/en/stable/), and RabbitMQ's tutorials use it as [the Python client the RabbitMQ team recommends](https://www.rabbitmq.com/tutorials/tutorial-one-python). Start with `BlockingConnection`, as those tutorials do: it's the adapter for [a procedural style](https://pika.readthedocs.io/en/stable/modules/adapters/index.html). [Ack each message after processing it](https://www.rabbitmq.com/tutorials/tutorial-two-python#message-acknowledgment), so RabbitMQ redelivers it when a worker dies.
|
||||
|
||||
paho-mqtt is the Eclipse Foundation's MQTT client, and MQTT is [a lightweight publish/subscribe protocol](https://eclipse.dev/paho/files/paho.mqtt.python/html/index.html) for IoT devices and low-bandwidth links. Create a client, connect it, and run [one of its network loops](https://eclipse.dev/paho/files/paho.mqtt.python/html/index.html#network-loop). `loop_forever()` blocks until you disconnect, and `loop_start()` runs the loop in a background thread. Both reconnect for you. [Subscribe in `on_connect()`](https://eclipse.dev/paho/files/paho.mqtt.python/html/index.html#getting-started), so a reconnect renews your subscriptions.
|
||||
|
||||
FastStream is [an asynchronous framework for event-driven services](https://faststream.ag2.ai/latest/): FastAPI's decorators, type-driven validation, and dependency injection, pointed at Kafka, RabbitMQ, NATS, Redis, or MQTT instead of HTTP. Write handlers with `@broker.subscriber()` and `@broker.publisher()`, run the app with `faststream run`, and [test it in memory](https://faststream.ag2.ai/latest/#testing-the-service) with no broker running. It [wraps your broker's own client](https://faststream.ag2.ai/latest/#your-broker-in-full) and leaves out retries, delayed delivery, and task orchestration by design: for background jobs, see [Task Queues](/categories/task-queues/).
|
||||
|
||||
In asyncio code, pick a client that won't block the event loop. FastStream is asynchronous throughout, confluent-kafka has [AsyncIO clients](https://docs.confluent.io/kafka-clients/python/current/overview.html#when-to-use-asyncio-clients) for apps like a FastAPI service, and pika has [an asyncio connection adapter](https://pika.readthedocs.io/en/stable/modules/adapters/asyncio.html).
|
||||
@@ -0,0 +1,16 @@
|
||||
On Windows, pywin32 gives Python the Win32 API and COM, from registry keys to Excel. Calling .NET needs a Python Windows library of its own: pythonnet.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Win32 API calls, the registry, the event log, or COM automation of apps like Excel: pywin32
|
||||
- .NET assemblies from Python, or Python embedded in a .NET app: pythonnet
|
||||
- Several Python versions on one machine, switched per folder: pyenv-win
|
||||
- A portable Python that runs from a folder, a network share, or a USB key: WinPython
|
||||
|
||||
pywin32 provides [access to many of the Windows APIs, including COM](https://github.com/mhammond/pywin32). Install it with `python -m pip install --upgrade pywin32`. Outside a virtual environment, its post-install script sets up COM objects and services. [Never run it inside one](https://github.com/mhammond/pywin32#installing-via-pip). To drive an app over COM, [pass its name to `win32com.client.Dispatch()`](https://mhammond.github.io/pywin32/html/com/win32com/HTML/QuickStartClientCom.html), like `"Excel.Application"`. Then call its methods and set its properties as on any Python object.
|
||||
|
||||
pythonnet [integrates CPython with the .NET runtime](https://pythonnet.github.io/pythonnet/) instead of compiling Python to .NET code, so your existing Python code and C extensions keep working. Install it with `pip install pythonnet`. [Pick the runtime before you import `clr`](https://pythonnet.github.io/pythonnet/python.html#loading-a-runtime), with `load("coreclr")` or the `PYTHONNET_RUNTIME` environment variable, or you get the default one. Then [load each assembly with `clr.AddReference()`](https://pythonnet.github.io/pythonnet/python.html#importing-modules) and import its namespaces like Python packages. It also works the other way, to [embed Python in a .NET app](https://pythonnet.github.io/pythonnet/dotnet.html).
|
||||
|
||||
pyenv-win [brings pyenv to Windows](https://github.com/pyenv-win/pyenv-win#introduction), so you get pyenv's commands for switching between Python versions. Install it with [the PowerShell script from its quick start](https://github.com/pyenv-win/pyenv-win#quick-start), then set your default with `pyenv install <version>` and `pyenv global <version>`. In a project folder, [`pyenv local <version>`](https://github.com/pyenv-win/pyenv-win#usage) picks the Python that `python` runs there, with nothing to activate.
|
||||
|
||||
WinPython is a [portable Python distribution](https://winpython.github.io/). Unzip it anywhere, like a network share or a USB key, and it runs, with nothing installed or written to the registry. Pick a build by [how much comes preinstalled](https://winpython.github.io/#builds): a bare Python you fill yourself, or the scientific stack with Spyder and JupyterLab ready to open. [Add packages with pip or wppm](https://winpython.github.io/#about), then zip the folder and hand it to your students or colleagues.
|
||||
@@ -0,0 +1,10 @@
|
||||
Some helpers aren't in the standard library, so before writing your own Python utility library, check boltons. Blinker dispatches signals between modules.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Helpers the standard library lacks, like atomic file saves and chunked iteration: boltons
|
||||
- Events inside one process, between modules that don't import each other: Blinker
|
||||
|
||||
boltons is a set of [pure-Python utilities missing from the standard library](https://boltons.readthedocs.io/en/latest/), like atomic file saves and chunked iteration. Run `pip install boltons`, then [import what you need from its modules](https://boltons.readthedocs.io/en/latest/#installation-and-integration), e.g. `from boltons.cacheutils import LRU`. It depends on no other packages and its modules are self-contained, so you can also [copy the whole package into your project, or just one module](https://boltons.readthedocs.io/en/latest/architecture.html#integration). Most modules aim to be good enough for basic uses. When you outgrow one, [its docs often point to a third-party alternative](https://boltons.readthedocs.io/en/latest/#third-party-packages).
|
||||
|
||||
Blinker lets [any number of interested parties subscribe to events, or signals](https://github.com/pallets-eco/blinker), inside one Python process. Create a named signal with [`signal('name')`](https://blinker.readthedocs.io/en/latest/#decoupling-with-named-signals): every call with that name returns the same object, so modules and plugins share it without importing each other. Register receivers with `connect()`, and when you call `send()`, [pass the object that emits the signal](https://blinker.readthedocs.io/en/latest/#emitting-signals), usually `self`. A receiver can then [subscribe to one sender only](https://blinker.readthedocs.io/en/latest/#subscribing-to-specific-senders).
|
||||
@@ -0,0 +1,25 @@
|
||||
Whether you're building an app or learning the field, there's a Python NLP library for it: spaCy for apps, NLTK for learning. Stanza covers many languages.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Production NLP pipelines: spaCy
|
||||
- Learning NLP, or lexical resources like WordNet: NLTK
|
||||
- Many languages, or CoreNLP from Python: Stanza
|
||||
- Topic models and word embeddings: gensim
|
||||
- Chinese word segmentation: jieba
|
||||
- Chinese characters to pinyin: pypinyin
|
||||
- Spaces between CJK text and letters or digits: pangu.py
|
||||
|
||||
spaCy is [designed for production use](https://spacy.io/usage/facts-figures#comparison-usage): you build and train NLP pipelines, then package them to deploy. Its trained pipelines are Python packages. In an automated build, [install them with pip from a direct link](https://spacy.io/usage/models#download-pip) instead of spaCy's download command, and put that link in your requirements.txt. In a larger code base, [import the pipeline as a module](https://spacy.io/usage/models#models-loading), so a missing one raises an ImportError right away. Run your texts through `nlp.pipe` [in batches](https://spacy.io/usage/processing-pipelines#processing), and disable the components you don't need. To train your own pipeline, run [`spacy train` with one config file](https://spacy.io/usage/training#quickstart).
|
||||
|
||||
NLTK comes with [corpora and lexical resources such as WordNet](https://www.nltk.org/), plus libraries to tokenize, tag, parse, and classify text. Its creators wrote a book that teaches NLP with it, and you can read it online. The book also says NLTK [isn't highly optimized for runtime performance](https://www.nltk.org/book/ch00.html#natural-language-toolkit-nltk), so use it to learn and experiment. The data ships separately: [get it with NLTK's data downloader](https://www.nltk.org/data.html), and set `NLTK_DATA` when you install it somewhere other than the standard locations.
|
||||
|
||||
Stanza is [designed to work across many languages, using the Universal Dependencies formalism](https://stanfordnlp.github.io/stanza/#about). It's also the official Python interface to Stanford's Java CoreNLP, though its own pipeline doesn't need CoreNLP. You can [list the processors to load](https://stanfordnlp.github.io/stanza/getting_started.html#specifying-processors) with `processors=`. Pass all your documents to the pipeline [at once](https://stanfordnlp.github.io/stanza/getting_started.html#processing-multiple-documents), since a for loop over one sentence at a time is very slow. For a lot of text, [run it on a GPU](https://stanfordnlp.github.io/stanza/getting_started.html#controlling-devices). To keep the pipeline from downloading anything at runtime, [download the models ahead of time](https://stanfordnlp.github.io/stanza/getting_started.html#downloading-models-for-offline-usage).
|
||||
|
||||
jieba [segments Chinese text into words](https://github.com/fxsjy/jieba): accurate mode suits text analysis, and search engine mode cuts long words into short ones for better recall. Add your own words with `jieba.load_userdict()` to get higher accuracy, and for Traditional Chinese, switch to its bigger dictionary with `jieba.set_dictionary()`. The other picks work with it: spaCy can [use jieba as its Chinese segmenter](https://spacy.io/usage/models#chinese), and Stanza [supports it as a tokenizer](https://stanfordnlp.github.io/stanza/pipeline.html).
|
||||
|
||||
pypinyin [matches pinyin by whole words](https://github.com/mozillazg/python-pinyin), so it handles characters with more than one reading. It also writes zhuyin (Bopomofo) and Wade-Giles. When a wrong word split gives a wrong reading, [segment the text with jieba first](https://pypinyin.readthedocs.io/zh-cn/latest/faq.html) and pass in the list of words. For readings that are still wrong, [add your own](https://pypinyin.readthedocs.io/zh-cn/latest/usage.html#custom-dict) with `load_phrases_dict()` or `load_single_dict()`.
|
||||
|
||||
pangu.py [inserts spaces between CJK characters and letters, digits, and symbols](https://github.com/vinta/pangu.py). Call `pangu.space_text()` on a string or `pangu.space_file()` on a file. From the command line, `pangu-py -c` prints the corrected text and exits with 1 when the spacing needed fixing.
|
||||
|
||||
Check the license of the models and data, not only the library. spaCy is [MIT](https://github.com/explosion/spaCy/blob/master/LICENSE), but its [Spanish pipelines](https://spacy.io/models/es) are GPL and its [Italian ones](https://spacy.io/models/it) are for non-commercial use only. NLTK is [Apache](https://github.com/nltk/nltk/blob/develop/LICENSE.txt), and its corpora come [under various licenses](https://github.com/nltk/nltk/wiki/FAQ), each listed in its own README.
|
||||
@@ -0,0 +1,13 @@
|
||||
Netmiko is where to start with Python network automation libraries; when many vendors need one API, move to NAPALM. Scapy works a layer down, on raw packets.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Running show and config commands on devices over SSH: Netmiko
|
||||
- Config changes across vendors, with a diff before commit: NAPALM
|
||||
- Forging, sending, sniffing, or dissecting packets: Scapy
|
||||
|
||||
Netmiko gathers show-command output and makes config changes [across a very broad set of platforms](https://github.com/ktbyers/netmiko#why-netmiko), and spares you most of the low-level regex matching on device prompts. Describe the device in a dict with its `device_type`, host, and credentials, and pass it to `ConnectHandler`, which [picks the right class and opens the SSH connection](https://pynet.twb-tech.com/blog/netmiko-python-library.html). Run show commands with `send_command()`, and config commands with `send_config_set()`, which [enters config mode for you](https://github.com/ktbyers/netmiko#getting-started-1).
|
||||
|
||||
NAPALM puts [a unified API over different network operating systems](https://napalm.readthedocs.io/en/latest/). Get a driver with `get_network_driver()` and open the device in a `with` block, which [the docs say to stick with for most situations](https://napalm.readthedocs.io/en/latest/tutorials/context_manager.html), since it opens and closes the session for you. To change config, load a candidate that replaces or merges into the running one, check it with `compare_config()`, then [commit or discard it](https://napalm.readthedocs.io/en/latest/tutorials/changing_the_config.html). The docs also say to [test your workflow and try to break things on a lab first](https://napalm.readthedocs.io/en/latest/support/index.html#configuration-support-matrix).
|
||||
|
||||
Scapy [sends, sniffs, dissects, and forges network packets](https://scapy.readthedocs.io/en/latest/introduction.html#about-scapy), as an interactive shell or as a library you import. Build a packet by [stacking layers with the `/` operator](https://scapy.readthedocs.io/en/latest/usage.html#stacking-layers), like `IP()/TCP()`, and override only the fields whose defaults you don't want. [Sending packets needs root privileges](https://scapy.readthedocs.io/en/latest/usage.html#starting-scapy), so start the shell with `sudo scapy`.
|
||||
@@ -0,0 +1,32 @@
|
||||
SQLAlchemy maps classes to tables in apps outside Django, which has its own ORM. Write Python instead of raw SQL, and a Python ORM returns rows as objects.
|
||||
|
||||
How to choose:
|
||||
|
||||
- An app outside Django, sync or async: SQLAlchemy
|
||||
- A Django project: Django ORM
|
||||
- A small app, with few concepts to learn: Peewee
|
||||
- One set of model classes for Pydantic and SQLAlchemy, as in FastAPI: SQLModel
|
||||
- Amazon DynamoDB tables: PynamoDB
|
||||
- MongoDB from synchronous code: MongoEngine
|
||||
- MongoDB from async code, with Pydantic models: Beanie
|
||||
- A Django project on MongoDB: Django MongoDB Backend
|
||||
|
||||
SQLAlchemy builds its ORM [on top of Core](https://docs.sqlalchemy.org/en/latest/tutorial/index.html), its SQL toolkit, which you can also use alone. The ORM follows the [data mapper pattern](https://www.sqlalchemy.org/philosophy.html), so your classes and your database schema can change independently of each other. Declare models with typed `Mapped[]` attributes, as in the [ORM Quick Start](https://docs.sqlalchemy.org/en/latest/orm/quickstart.html#declare-models). Keep the session [outside the functions that query data](https://docs.sqlalchemy.org/en/latest/orm/session_basics.html#session-faq-whentocreate). Open it with `with Session(engine) as session:` around one unit of work, like a web request, and keep transactions short. For schema changes over time, its docs point you to [Alembic](https://docs.sqlalchemy.org/en/latest/tutorial/metadata.html#tutorial-emitting-ddl). In asyncio code, the [asyncio extension](https://docs.sqlalchemy.org/en/latest/orm/extensions/asyncio.html) covers both Core and the ORM.
|
||||
|
||||
Django comes with an ORM in which you [describe your database layout in Python code](https://docs.djangoproject.com/en/stable/intro/overview/). It follows [the Active Record pattern](https://docs.djangoproject.com/en/stable/misc/design-philosophies/#include-all-relevant-domain-logic), so a model holds its domain logic along with its fields. Change a model, then run `makemigrations` and `migrate`, and [migrations](https://docs.djangoproject.com/en/stable/topics/migrations/) carry the change into your schema. In async code, every QuerySet method that runs a query has [an `a`-prefixed variant](https://docs.djangoproject.com/en/stable/topics/async/#queries-the-orm), like `afirst()`.
|
||||
|
||||
Peewee is [a simple and small ORM](https://docs.peewee-orm.com/en/latest/) with few concepts, in a single module with no required dependencies. It works with SQLite, MySQL, MariaDB, and PostgreSQL. As in [its README](https://github.com/coleifer/peewee), set the database once in a `BaseModel`'s `Meta.database` and subclass that for every model. A [`with db:` block](https://docs.peewee-orm.com/en/latest/peewee/database.html#context-managers) opens a connection and a transaction, commits when the block exits normally, rolls back on an exception, and closes the connection.
|
||||
|
||||
SQLModel is [a thin layer on top of Pydantic and SQLAlchemy](https://sqlmodel.tiangolo.com/): each model is both a Pydantic model and a SQLAlchemy model. FastAPI's author built it for FastAPI apps, but it [works in any other kind of app](https://sqlmodel.tiangolo.com/features/) too. Mark the classes that are tables with `table=True`, and keep what your API reads and returns in [data models](https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/#multiple-models-with-inheritance), which are only Pydantic models.
|
||||
|
||||
PynamoDB wraps DynamoDB's verbose API in [a simple, elegant one](https://github.com/pynamodb/PynamoDB). Write [a model per table](https://pynamodb.readthedocs.io/en/latest/tutorial.html#defining-a-model), with the table's name in `Meta.table_name` and its hash key marked `hash_key=True`, since every DynamoDB table has one.
|
||||
|
||||
MongoEngine is [an ORM-like layer on top of PyMongo](https://github.com/MongoEngine/mongoengine), and synchronous: it's [based on PyMongo alone](https://mongoengine-odm.readthedocs.io/faq.html), not on async drivers. Your document classes define a schema that MongoDB never sees, since it's [enforced at the application level](https://mongoengine-odm.readthedocs.io/tutorial.html#defining-our-documents). Store data that belongs to one document, like a post's comments, [as embedded documents](https://mongoengine-odm.readthedocs.io/tutorial.html#comments) on it.
|
||||
|
||||
Beanie is [an asynchronous ODM whose data models are based on Pydantic](https://beanie-odm.dev/). [Setting it up](https://beanie-odm.dev/getting-started/) takes three steps: subclass `beanie.Document` instead of `pydantic.BaseModel`, create PyMongo's async client, and call `init_beanie()` with your database and models inside your event loop. That call also [creates the collections and sets up the indexes](https://beanie-odm.dev/tutorial/initialization/) your models define.
|
||||
|
||||
Django MongoDB Backend is [a Django database backend that uses PyMongo](https://github.com/mongodb/django-mongodb-backend). MongoDB's docs say it [translates Django ORM methods into MongoDB queries](https://www.mongodb.com/docs/drivers/odm/), so your project keeps Django's models and ORM. Start from [its project template](https://django-mongodb-backend.readthedocs.io/en/latest/intro/configure/), which sets `ObjectIdAutoField` as the default primary key.
|
||||
|
||||
With SQLAlchemy, the Django ORM, or Peewee, load related rows up front instead of running one query per row in a loop. SQLAlchemy calls [`selectinload()` the best strategy](https://docs.sqlalchemy.org/en/latest/orm/queryguide/relationships.html#what-kind-of-loading-to-use) for collections and `joinedload()` the most general one for many-to-one. Django has [`select_related()` and `prefetch_related()`](https://docs.djangoproject.com/en/stable/topics/db/optimization/#use-queryset-select-related-and-prefetch-related), and Peewee [joins or eager-loads](https://docs.peewee-orm.com/en/latest/peewee/relationships.html#the-n-1-problem) depending on which way the relationship goes.
|
||||
|
||||
For more SQLAlchemy extensions and resources, see [awesome-sqlalchemy](https://github.com/dahlia/awesome-sqlalchemy).
|
||||
@@ -0,0 +1,34 @@
|
||||
uv manages your project with a lockfile, and doubles as a Python package manager. Its uv-build backend packages pure-Python code with reasonable defaults.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A new project, from installing Python to locking dependencies: uv
|
||||
- Packaging pure-Python code in a uv project: uv-build
|
||||
- Installing packages with only what ships with Python: pip
|
||||
- A project already on Poetry, or one that relies on its plugins: Poetry
|
||||
- Tests across a matrix of Python versions: Hatch
|
||||
- Command-line apps, each in its own environment: pipx, or `uv tool` on uv
|
||||
- Scientific stacks with non-Python libraries: conda
|
||||
- C or C++ extension modules: setuptools
|
||||
- Build hooks, or a layout uv-build doesn't expect: Hatchling
|
||||
- Building a Poetry project without Poetry installed: poetry-core
|
||||
|
||||
uv is [a single tool](https://docs.astral.sh/uv/) for Python versions, virtual environments, dependencies, and command-line tools. Start a project with `uv init`, add packages with `uv add`, and run code with `uv run`, which [checks that the lockfile and environment are up to date](https://docs.astral.sh/uv/guides/projects/#running-commands) before every run. The `uv pip` commands are [for projects not ready to move away from pip](https://docs.astral.sh/uv/pip/), not the main interface.
|
||||
|
||||
uv-build is uv's own build backend, and `uv init` [sets it up](https://docs.astral.sh/uv/concepts/projects/init/) with your code in `src/<project_name>/`. When you need build scripts or a more flexible layout, uv's docs [suggest Hatchling instead](https://docs.astral.sh/uv/concepts/build-backend/#choosing-a-build-backend).
|
||||
|
||||
pip [installs packages](https://pip.pypa.io/en/stable/) from PyPI and other indexes. It comes with Python, but managing environments, the interpreter, and the project as a whole is [outside its scope](https://pip.pypa.io/en/stable/topics/workflow/). Run it as `python -m pip`, so you [know which Python](https://pip.pypa.io/en/stable/user_guide/#running-pip) gets the package.
|
||||
|
||||
Poetry [manages your dependencies and builds your package](https://python-poetry.org/docs/), with a lockfile for repeatable installs. For a project you won't publish, [turn off package mode](https://python-poetry.org/docs/basic-usage/#operating-modes) and use Poetry for dependencies only. Its build backend, poetry-core, lets pip [build and install your project](https://github.com/python-poetry/poetry-core) without Poetry.
|
||||
|
||||
Hatch [manages environments, builds, and Python installs](https://hatch.pypa.io/latest/) for your project. `hatch run test:pytest` [runs pytest in a `test` environment](https://hatch.pypa.io/latest/tutorials/environment/basic-usage/) without activating it, and creates that environment on first use. A [matrix](https://hatch.pypa.io/latest/environment/#matrix) repeats an environment across Python versions.
|
||||
|
||||
Hatchling is Hatch's build backend, [developed with Hatch but separate from it](https://packaging.python.org/en/latest/guides/tool-recommendations/). It [takes the files to ship from your `.gitignore`](https://hatch.pypa.io/latest/why/#build-backend) and builds reproducible wheels and sdists by default.
|
||||
|
||||
pipx installs command-line apps [each in its own virtual environment](https://pipx.pypa.io/latest/), so their dependencies never collide, and puts their commands on your PATH. To try an app once, [`pipx run`](https://pipx.pypa.io/latest/tutorial/getting-started.html) runs it in a temporary environment. `uv tool` does the same job on uv, and pipx's docs say [running both is fine](https://pipx.pypa.io/latest/explanation/comparisons.html#picking-one).
|
||||
|
||||
conda manages [packages and environments for any language](https://docs.conda.io/projects/conda/en/latest/), and [its packages](https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/packages.html) can hold system libraries and programs, not only Python modules. Keep your requirements in a file, and [use pip only after conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#pip-in-env) has installed everything it can.
|
||||
|
||||
setuptools [builds C and C++ extension modules](https://setuptools.pypa.io/en/latest/userguide/ext_modules.html), and Hatch's docs [recommend it](https://hatch.pypa.io/latest/why/#build-backend) for them. Keep its configuration in pyproject.toml and `setup.py` [minimal, if you need one at all](https://setuptools.pypa.io/en/latest/userguide/quickstart.html#setuppy-discouraged), and don't run `setup.py` as a script.
|
||||
|
||||
Whichever tool writes your lockfile, commit it to version control, as [uv](https://docs.astral.sh/uv/guides/projects/#uvlock) and [Poetry](https://python-poetry.org/docs/basic-usage/#committing-your-poetrylock-file-to-version-control) both recommend, so every machine installs the same versions.
|
||||
@@ -0,0 +1,16 @@
|
||||
Private packages can live on a Python package repository you host, such as pypiserver. devpi serves them too, next to a cache of PyPI.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Your own packages, served from a plain folder: pypiserver
|
||||
- A PyPI cache that keeps installs working when PyPI is down: devpi
|
||||
- Your own indexes, with releases tested and staged before they go to PyPI: devpi
|
||||
- A copy of PyPI, whole or filtered, for a network that can't reach it: bandersnatch
|
||||
|
||||
pypiserver is a [minimal PyPI-compatible server](https://github.com/pypiserver/pypiserver) that serves packages from regular directories, so pip and twine talk to it as they would to PyPI. [Start it on a folder](https://github.com/pypiserver/pypiserver#quickstart-installation-and-usage) with `pypi-server run -p 8080 ~/packages`, then copy your packages in, or upload them with twine. Its docs strongly advise you to [password-protect uploads](https://github.com/pypiserver/pypiserver#uploading-packages-remotely) with an htpasswd file. A package it doesn't have [redirects pip to PyPI](https://github.com/pypiserver/pypiserver#client-side-configurations).
|
||||
|
||||
devpi pairs a [PyPI-compatible server with a command-line tool](https://devpi.net/docs/devpi/devpi/latest/+doc/index.html) for packaging, testing, and release. For a cache alone, install devpi-server and [point pip at its `root/pypi` index](https://devpi.net/docs/devpi/devpi/latest/+doc/quickstart-pypimirror.html). Once pip has fetched a file through it, that file installs offline too. For your own packages, add devpi-web and devpi-client, and [create an index based on `root/pypi`](https://devpi.net/docs/devpi/devpi/latest/+doc/quickstart-releaseprocess.html), so all of PyPI shows up beside your uploads. `devpi upload` publishes a release there, `devpi test` runs tox against it and records the result, and `devpi push` sends it on to a staging index or to PyPI. A package you upload [hides PyPI's packages of the same name](https://devpi.net/docs/devpi/devpi/latest/+doc/userman/devpi_indices.html#modifying-the-mirror-whitelist) by default, which prevents dependency confusion attacks.
|
||||
|
||||
bandersnatch is the Python Packaging Authority's [PyPI mirror client](https://bandersnatch.readthedocs.io/en/latest/): it syncs PyPI's packages to disk as static files, and [any web server can serve them](https://bandersnatch.readthedocs.io/en/latest/serving.html). [Run `bandersnatch mirror`](https://github.com/pypa/bandersnatch#quickstart) once to create its config file, adjust the config, then run it again to fill the mirror. To mirror only what you use, list projects in an allowlist, which can also [read your requirements files](https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html#requirements-files-filtering).
|
||||
|
||||
Whichever you run, set it as pip's `index-url`, so it replaces PyPI instead of sitting beside it. pip's docs call [`--extra-index-url` for private packages unsafe](https://pip.pypa.io/en/latest/cli/pip_install/#cmdoption-extra-index-url), since anyone can publish a package with the same name on PyPI. Serve it over HTTPS too, as the [Python Packaging User Guide highly recommends](https://packaging.python.org/en/latest/guides/hosting-your-own-index/).
|
||||
@@ -0,0 +1,21 @@
|
||||
With permission to test a web app, intercept its traffic in mitmproxy and test its inputs for SQL injection with sqlmap: two Python penetration testing tools.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Intercepting, inspecting, and scripting HTTP(S) traffic: mitmproxy
|
||||
- Detecting and exploiting SQL injection in a web app: sqlmap
|
||||
- Windows network protocols like SMB, MSRPC, and Kerberos: Impacket
|
||||
- Writing exploits and solving CTF challenges: pwntools
|
||||
- Finding a username's accounts across social networks: Sherlock
|
||||
|
||||
mitmproxy is [an interactive, SSL/TLS-capable intercepting proxy](https://docs.mitmproxy.org/stable/) with 3 front ends to the same core: mitmproxy in the console, mitmweb in the browser, and mitmdump on the command line. It [starts as a regular HTTP proxy on localhost:8080](https://docs.mitmproxy.org/stable/overview/getting-started/), so point your browser or device at it, then visit mitm.it to install the mitmproxy CA certificate, which lets it decrypt HTTPS. To script the traffic, write an [addon](https://docs.mitmproxy.org/stable/addons/overview/): a module with a `request(flow)` function is enough. Load it with `-s`.
|
||||
|
||||
sqlmap [automates detecting and exploiting SQL injection flaws and taking over database servers](https://github.com/sqlmapproject/sqlmap). Pass it a URL with a parameter, and it [finds the vulnerable parameter, the injection techniques that work, and the back-end database](https://github.com/sqlmapproject/sqlmap/wiki/Introduction).
|
||||
|
||||
Impacket is [a collection of Python classes for working with network protocols](https://github.com/fortra/impacket), with low-level access to packets and, for protocols like SMB and MSRPC, the protocol implementation itself. The scripts it ships are [tools written as examples of what the library can do](https://github.com/fortra/impacket/tree/master/examples).
|
||||
|
||||
pwntools is [a CTF framework and exploit development library](https://github.com/Gallopsled/pwntools), split into [2 modules](https://docs.pwntools.com/en/stable/about.html): `pwn` for CTFs and `pwnlib` as a clean Python library. `from pwn import *` gives you everything you need to write an exploit, but it also puts your terminal in raw mode and parses `sys.argv`, so import `pwnlib` when pwntools is part of another project. Either way, [tubes](https://docs.pwntools.com/en/stable/intro.html#making-connections) give you one interface to talk to processes, sockets, and SSH sessions.
|
||||
|
||||
Sherlock [hunts down social media accounts by username](https://github.com/sherlock-project/sherlock). Run `sherlock user123`, or pass several usernames at once, and it [saves the accounts it finds to a text file per username](https://github.com/sherlock-project/sherlock#general-usage), like `user123.txt`.
|
||||
|
||||
Run any of these tools only against systems you own or have explicit permission to test. That's [sqlmap's answer to where to practice](https://github.com/sqlmapproject/sqlmap/wiki/FAQ#where-can-i-practise-using-sqlmap), and every sqlmap run prints a disclaimer that attacking targets without prior mutual consent is illegal.
|
||||
@@ -0,0 +1,18 @@
|
||||
Qiskit for most circuits, PennyLane for circuits you train, Cirq for device-level control, QuTiP for physics: Python quantum computing libraries you can mix.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Most circuit work, including IBM hardware: Qiskit
|
||||
- Circuits you train with gradients, like quantum machine learning: PennyLane
|
||||
- Device-level control over circuits and noise: Cirq
|
||||
- Simulating the dynamics of open quantum systems: QuTiP
|
||||
|
||||
Qiskit is an [SDK for working with quantum computers](https://github.com/Qiskit/qiskit) at the level of circuits, operators, and primitives. Its docs lay out the development workflow as a [Qiskit pattern](https://quantum.cloud.ibm.com/docs/en/guides/intro-to-patterns): map your problem to circuits, optimize them for the target hardware, run them, and post-process the results. Run circuits through the [Sampler and Estimator primitives](https://quantum.cloud.ibm.com/docs/en/guides/primitives). Sampler samples outcomes, and Estimator estimates expectation values. Before a circuit goes to a device, [transpile it](https://quantum.cloud.ibm.com/docs/en/guides/transpile) to the gates and qubit connections that device supports. Test it in [local testing mode](https://quantum.cloud.ibm.com/docs/en/guides/local-testing-mode) first: once your program works there, moving to a QPU takes only a backend name change.
|
||||
|
||||
PennyLane lets you [train a quantum computer the same way as a neural network](https://docs.pennylane.ai/en/stable/): it differentiates quantum circuits and connects them to PyTorch, JAX, and NumPy. Write each circuit as a Python function under the [`qnode` decorator](https://docs.pennylane.ai/en/stable/introduction/circuits.html#the-qnode-decorator), which ties it to the device that runs it. Pick the [interface](https://docs.pennylane.ai/en/stable/introduction/interfaces.html) for your machine learning library, then train with that library's own optimizers.
|
||||
|
||||
Cirq is for [writing, manipulating, and optimizing quantum circuits](https://quantumai.google/cirq) when the details of the hardware matter. A circuit is [a collection of Moments](https://quantumai.google/cirq/build/circuits), each a set of operations that act in the same time slice. Model your target processor as a [Device](https://quantumai.google/cirq/hardware/devices) and validate your circuits against it. Then [compile them with transformers](https://quantumai.google/cirq/transform/transformers) into circuits that device can run. Test small circuits on the [built-in simulators](https://quantumai.google/cirq/simulate/simulation), then move to qsim, which its docs recommend for most users. Before Google hardware, run on the [Quantum Virtual Machine](https://quantumai.google/cirq/simulate/quantum_virtual_machine), which mimics Google's processors with noise data.
|
||||
|
||||
QuTiP [simulates the dynamics of open quantum systems](https://qutip.org/), like the ones in quantum optics, trapped ions, and superconducting circuits. Every state and operator is a [`Qobj`](https://qutip.readthedocs.io/en/stable/guide/guide-basics.html), and QuTiP has predefined ones for a variety of them. Pick the solver [by the kind of system](https://qutip.readthedocs.io/en/stable/guide/dynamics/dynamics-intro.html): a closed system is a state vector, and an open one needs a density matrix. [`mesolve`](https://qutip.readthedocs.io/en/stable/guide/dynamics/dynamics-master.html) covers both, and switches to the master equation when you give it collapse operators. For large systems, the docs recommend the [Monte Carlo solver](https://qutip.readthedocs.io/en/stable/guide/dynamics/dynamics-monte.html). To simulate quantum circuits, use its [qutip-qip](https://qutip.readthedocs.io/en/stable/guide/guide-family.html) package.
|
||||
|
||||
You don't have to stay with one library. PennyLane [imports Qiskit circuits](https://docs.pennylane.ai/en/stable/introduction/circuits.html#importing-circuits-from-other-frameworks), and its plugins run on [Qiskit devices](https://docs.pennylane.ai/projects/qiskit/en/stable/) and [Cirq's simulators](https://docs.pennylane.ai/projects/cirq/en/stable/). QuTiP's qutip-qip [simulates circuits made in Qiskit](https://qutip-qip.readthedocs.io/en/stable/qip-qiskit.html).
|
||||
@@ -0,0 +1,15 @@
|
||||
Match the Python recommender system library to your data: Surprise if users rate items, implicit if they only buy or watch. Annoy finds similar items fast.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Ratings users give, like 1 to 5 stars: Surprise
|
||||
- Purchases, watches, or page views, with no ratings: implicit
|
||||
- Fast lookups of similar items from a trained model: Annoy
|
||||
|
||||
Surprise is built for [explicit rating data](https://surpriselib.com/), and it doesn't support implicit ratings. Its [basic usage](https://surprise.readthedocs.io/en/latest/getting_started.html#basic-usage) cross-validates an algorithm like SVD on a built-in dataset in a few lines of code. For your own ratings, define a `Reader` and [load them from a file or a pandas dataframe](https://surprise.readthedocs.io/en/latest/getting_started.html#use-a-custom-dataset). Surprise predicts ratings, so to recommend items, [predict the ratings a user hasn't given and keep the top N](https://surprise.readthedocs.io/en/latest/FAQ.html#how-to-get-the-top-n-recommendations-for-each-user).
|
||||
|
||||
implicit offers fast Python implementations of popular algorithms for [implicit feedback datasets](https://benfred.github.io/implicit/), such as Alternating Least Squares and Bayesian Personalized Ranking. Every model [implements one interface](https://benfred.github.io/implicit/api/models/index.html) for training and recommending. Train with `fit` on a [sparse CSR matrix of users by items](https://benfred.github.io/implicit/api/models/recommender_base.html#implicit.recommender_base.RecommenderBase.fit), whose values are how confident you are that the user likes each item. Then `recommend` returns items for a user, and `similar_items` finds related items.
|
||||
|
||||
Annoy is a C++ library with Python bindings that [searches for points close to a query point](https://github.com/spotify/annoy). For recommendations, it runs [after matrix factorization](https://github.com/spotify/annoy#background): every user and item becomes a vector, and Annoy finds similar ones. Its indexes are static files, so you build an index once and save it, and every process [loads it with mmap](https://github.com/spotify/annoy#python-code-example) and shares the same data.
|
||||
|
||||
implicit and Annoy work together: implicit can use an Annoy index to [speed up `recommend` and `similar_items`](https://benfred.github.io/implicit/api/ann.html) on any matrix factorization model, at the risk of missing some relevant results.
|
||||
@@ -0,0 +1,42 @@
|
||||
Underneath most Python scientific computing libraries sits NumPy's array. SciPy adds algorithms on top, and Numba compiles loops that won't vectorize.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Arrays and vectorized math: NumPy
|
||||
- Optimization, integration, interpolation, and linear algebra: SciPy
|
||||
- Numeric loops that won't vectorize: Numba
|
||||
- Exact, symbolic math: SymPy
|
||||
- Regression, time series, and hypothesis tests: statsmodels
|
||||
- Biological sequences and file formats: Biopython; molecules: RDKit
|
||||
- Physical units: Pint; astronomy: Astropy; seismology: ObsPy
|
||||
- Bayesian models: PyMC
|
||||
- Discrete-event simulations: SimPy; agent-based models: Mesa
|
||||
- Graphs and networks: NetworkX
|
||||
- Planar geometry: Shapely
|
||||
- Colour science: Colour; math animations: Manim
|
||||
|
||||
NumPy gives you a multidimensional array and fast routines that run on it. Write whole-array expressions instead of Python loops: [vectorization and broadcasting](https://numpy.org/doc/stable/user/whatisnumpy.html#why-is-numpy-fast) push the element-by-element work into compiled C code, and your code reads closer to the math. For random numbers, [create a Generator with `default_rng()`](https://numpy.org/doc/stable/reference/random/index.html#random-quick-start) and call its methods.
|
||||
|
||||
SciPy is [a collection of algorithms built on NumPy](https://docs.scipy.org/doc/scipy/tutorial/index.html), split into subpackages by domain, like `optimize`, `integrate`, `interpolate`, `linalg`, and `stats`. Its docs recommend [using those subpackages as namespaces](https://docs.scipy.org/doc/scipy/reference/index.html#guidelines-for-importing-functions-from-scipy): `from scipy import optimize`, then `optimize.curve_fit()`. For linear algebra, use `scipy.linalg` over `numpy.linalg`. It has everything NumPy's module has and more, and it's [always compiled with BLAS/LAPACK](https://docs.scipy.org/doc/scipy/tutorial/linalg.html#scipy-linalg-vs-numpy-linalg), so it may run faster.
|
||||
|
||||
Numba is a just-in-time compiler that [works best on code that uses NumPy arrays and functions, and loops](https://numba.readthedocs.io/en/stable/user/5minguide.html). NumPy wants vector operations, but Numba is [happy with plain loops](https://numba.readthedocs.io/en/stable/user/performance-tips.html#loops), so it fits the numeric code you can't vectorize. Put `@jit` on the function and [let Numba decide when and how to optimize](https://numba.readthedocs.io/en/stable/user/jit.html#lazy-compilation). Keep pandas out of those functions: Numba doesn't understand it, so that code [runs in the interpreter](https://numba.readthedocs.io/en/stable/user/5minguide.html#will-numba-work-for-my-code) with Numba's overhead on top.
|
||||
|
||||
SymPy does symbolic math: expressions stay [exact, not approximate](https://docs.sympy.org/latest/tutorials/intro-tutorial/intro.html#what-is-symbolic-computation), and it's written entirely in Python. Its best practices keep symbolic and numeric code apart: [model the problem in SymPy, then turn the result into a function with `lambdify()`](https://docs.sympy.org/latest/explanation/best-practices.html#separate-symbolic-and-numeric-code) that runs on NumPy arrays. Define symbols with `symbols()` and [the assumptions you know](https://docs.sympy.org/latest/explanation/best-practices.html#defining-symbols), like `positive=True`, so more expressions simplify. Build expressions from those symbols, [not from strings](https://docs.sympy.org/latest/explanation/best-practices.html#avoid-string-inputs).
|
||||
|
||||
statsmodels [estimates statistical models, runs hypothesis tests, and explores data](https://www.statsmodels.org/stable/index.html), and [most of its results are verified](https://www.statsmodels.org/stable/about.html#testing) against another statistical package. SciPy's stats docs [send regression, linear models, and time series analysis to statsmodels](https://docs.scipy.org/doc/scipy/reference/stats.html), while `scipy.stats` keeps the probability distributions and statistical tests. Write your model as an [R-style formula](https://www.statsmodels.org/stable/example_formulas.html) on a pandas DataFrame, like `smf.ols("y ~ x", data=df).fit()`, and read the fit with `summary()`.
|
||||
|
||||
Biopython [parses bioinformatics file formats](https://biopython.org/docs/latest/Tutorial/chapter_introduction.html) like FASTA and GenBank, queries online services like NCBI, and gives you a standard sequence class. Read sequence files with `Bio.SeqIO.parse()`, or [`Bio.SeqIO.read()` when a file holds one record](https://biopython.org/docs/latest/Tutorial/chapter_seqio.html#parsing-or-reading-sequences). When you query NCBI through `Bio.Entrez`, [pass your email](https://biopython.org/docs/latest/Tutorial/chapter_entrez.html#entrez-guidelines) so NCBI can contact you if there's a problem. RDKit is a [cheminformatics toolkit with its core in C++](https://www.rdkit.org/docs/Overview.html), for 2D and 3D molecular operations and descriptors for machine learning. To compare molecules, [create a fingerprint generator](https://www.rdkit.org/docs/GettingStartedInPython.html#fingerprinting-and-molecular-similarity) for the fingerprint type you want, the docs' most consistent way to get fingerprints.
|
||||
|
||||
Pint handles [physical quantities](https://pint.readthedocs.io/en/stable/getting/overview.html): a value times a unit, in any numeric type. In a package, [create one `UnitRegistry` in a single place](https://pint.readthedocs.io/en/stable/getting/pint-in-your-projects.html#having-a-shared-registry) and import it everywhere, since quantities from different registries don't mix. Astropy is [a common core package for astronomy](https://www.astropy.org/), with affiliated packages around it. [Import the subpackage you need](https://docs.astropy.org/en/stable/importing_astropy.html#importing-astropy-and-sub-packages), like `from astropy import units as u`, and never import with `*`. ObsPy is [a framework for processing seismological data](https://docs.obspy.org/): `read()` loads [SAC, MiniSEED, and other formats into a Stream](https://docs.obspy.org/tutorial/code_snippets/reading_seismograms.html) of Traces, and its [FDSN client](https://docs.obspy.org/packages/obspy.clients.fdsn.html#basic-fdsn-client-usage) fetches data from data centers.
|
||||
|
||||
PyMC [builds Bayesian models with a simple Python API](https://www.pymc.io/welcome.html) and fits them with Markov chain Monte Carlo or variational inference. Declare the variables inside a `with pm.Model():` block, which [adds them to the model for you](https://www.pymc.io/projects/docs/en/stable/learn/core_notebooks/pymc_overview.html#model-specification), and let `pm.sample()` pick the samplers. Validate the model with [posterior predictive checks](https://www.pymc.io/projects/docs/en/stable/learn/core_notebooks/posterior_predictive.html), and run prior predictive checks too, which its docs call a crucial part of the Bayesian workflow.
|
||||
|
||||
SimPy runs discrete-event simulations. Each process is a Python generator that [yields events and waits for them](https://simpy.readthedocs.io/en/latest/simpy_intro/basic_concepts.html), and shared resources model congestion points like servers and checkout counters. Request a resource in a `with` block so it's [released for you](https://simpy.readthedocs.io/en/latest/simpy_intro/shared_resources.html#basic-resource-usage). Its docs call it [overkill for fixed-step simulations](https://simpy.readthedocs.io/en/latest/) whose processes don't interact or share resources. Mesa builds [agent-based models](https://mesa.readthedocs.io/latest/) from core components like spatial grids, shows them in the browser, and hands the results to Python's data tools. Its best practices put [the model class in `model.py` and the agents in `agents.py`](https://mesa.readthedocs.io/latest/best-practices.html#model-layout), with an optional browser visualization in `app.py`. Its [DataCollector](https://mesa.readthedocs.io/latest/getting_started.html) collects model-level and agent-level data.
|
||||
|
||||
NetworkX is for [the creation, manipulation, and study of complex networks](https://networkx.org/documentation/stable/index.html). [Pick the graph class first](https://networkx.org/documentation/stable/reference/introduction.html#graphs): Graph, DiGraph, MultiGraph, or MultiDiGraph. Store numeric edge data under [the `weight` key](https://networkx.org/documentation/stable/reference/introduction.html#nodes-and-edges), which algorithms like Dijkstra's shortest path read by default. Its drawing functions are basic, since its [main goal is graph analysis, not visualization](https://networkx.org/documentation/stable/reference/drawing.html), so export the graph to a dedicated visualization tool.
|
||||
|
||||
Shapely wraps the GEOS library for [manipulation and analysis of planar geometric objects](https://shapely.readthedocs.io/en/stable/). For many geometries, put them in a NumPy array and call its [vectorized functions instead of a Python loop](https://shapely.readthedocs.io/en/stable/#usage). It doesn't read or write data files or transform coordinate systems. Operations [presume the features sit in the same Cartesian plane](https://shapely.readthedocs.io/en/stable/manual.html), so project your data to a plane before you measure it.
|
||||
|
||||
Colour provides [algorithms and datasets for colour science](https://colour.readthedocs.io/en/latest/). Manim [generates animations of technical concepts from Python code](https://docs.manim.community/en/stable/). It comes in more than one edition, and the one listed here is [the community-maintained one its docs recommend](https://docs.manim.community/en/stable/faq/installation.html#which-version-should-i-use), especially for beginners. Each animation lives in [the `construct()` method of a `Scene` subclass](https://docs.manim.community/en/stable/tutorials/quickstart.html), with helper functions outside the class.
|
||||
|
||||
Keep your data in NumPy arrays as it moves between most of these libraries. ObsPy already holds each trace's samples in one, and Colour [recommends them as input](https://colour.readthedocs.io/en/latest/basics.html). For results you can reproduce, seed your random numbers: [pass a seed to `default_rng()`](https://numpy.org/doc/stable/reference/random/index.html#random-quick-start) in NumPy, and [give your Mesa model a `seed` argument](https://mesa.readthedocs.io/latest/best-practices.html#randomization).
|
||||
@@ -0,0 +1,18 @@
|
||||
The engine comes first, then its official Python search library. Meilisearch keeps a search bar simple, while analytics call for Elasticsearch or OpenSearch.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A typo-tolerant search bar for your site or app: meilisearch
|
||||
- Log analytics and aggregations beyond search: elasticsearch
|
||||
- The same under Apache instead of the AGPL, or on Amazon OpenSearch Service: opensearch-py
|
||||
- Search over Django models, with a backend you can swap later: django-haystack
|
||||
|
||||
Meilisearch is [an open-source search engine](https://github.com/meilisearch/meilisearch-python), and meilisearch is its Python client. Its docs call it [a perfect choice for a typo-tolerant search bar](https://www.meilisearch.com/docs/resources/comparisons/alternatives), built for instant search aimed at end users. Meilisearch [queues writes and processes them asynchronously](https://www.meilisearch.com/docs/capabilities/indexing/tasks_and_batches/async_operations), so after `add_documents()`, the Python quick start [waits for indexing to complete](https://www.meilisearch.com/docs/getting_started/sdks/python) with `client.wait_for_task(task.task_uid)`.
|
||||
|
||||
For log analytics and aggregations beyond search, Meilisearch's own docs point you to [Elasticsearch](https://www.meilisearch.com/docs/resources/comparisons/elasticsearch#when-to-choose-elasticsearch) or [OpenSearch](https://www.meilisearch.com/docs/resources/comparisons/opensearch#when-to-choose-opensearch). elasticsearch is Elastic's official Python client. It's [unopinionated and extensible](https://www.elastic.co/docs/reference/elasticsearch/clients/python) and covers the entire Elasticsearch API. Its docs call [the bulk helpers the recommended way to ingest data](https://www.elastic.co/docs/reference/elasticsearch/clients/python/getting-started): unlike calling `client.bulk` yourself, they handle retries and send documents chunk by chunk.
|
||||
|
||||
OpenSearch is [a fork of Elasticsearch, and all of its software is Apache-licensed](https://opensearch.org/faq/). The Elasticsearch server is available [under the AGPL or source-available licenses](https://www.elastic.co/pricing/faq/licensing). OpenSearch's Python client is opensearch-py, [a fork of elasticsearch-py](https://github.com/opensearch-project/opensearch-py) that [wraps the OpenSearch REST API in Python methods](https://docs.opensearch.org/latest/clients/python-low-level/#low-level-python-client). Use it for any OpenSearch cluster, as OpenSearch's docs [recommend OpenSearch clients for OpenSearch clusters](https://docs.opensearch.org/latest/clients/#legacy-clients). On Amazon OpenSearch Service, [sign requests with `AWSV4SignerAuth`](https://docs.opensearch.org/latest/clients/python-low-level/#connecting-to-amazon-opensearch-service) and your IAM credentials.
|
||||
|
||||
django-haystack gives Django [one API over pluggable search backends](https://django-haystack.readthedocs.io/en/latest/), such as Elasticsearch, so you can switch engines without rewriting your search code. Its FAQ [advises against it](https://django-haystack.readthedocs.io/en/latest/faq.html#when-should-i-not-be-using-haystack) for data that isn't in Django models and for ultra-high volume, since the abstraction costs performance and some engine features. [Create a `SearchIndex` for each model](https://django-haystack.readthedocs.io/en/latest/tutorial.html#creating-searchindexes) you index, in a `search_indexes.py` file in its app. Load your data with [`./manage.py rebuild_index`](https://django-haystack.readthedocs.io/en/latest/tutorial.html#reindex), then keep the index current with a cron job running `update_index`.
|
||||
|
||||
Keep your data in your database, and send the search engine a copy of what people search for. Meilisearch [wasn't designed to be your main data container](https://www.meilisearch.com/docs/capabilities/indexing/advanced/indexing_best_practices#do-not-use-meilisearch-as-your-main-database), and django-haystack's docs call the database [authoritative and the search index non-authoritative](https://django-haystack.readthedocs.io/en/latest/signal_processors.html).
|
||||
@@ -0,0 +1,18 @@
|
||||
When your Python serialization library should validate too, msgspec decodes JSON straight into typed objects. Without a schema, orjson is fast and correct.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Decoding into typed, validated objects: msgspec
|
||||
- Faster JSON that still gives you dicts and lists: orjson
|
||||
- Compact binary data other languages can read: msgpack
|
||||
- Schemas that convert ORM or app objects for a web API: marshmallow
|
||||
|
||||
msgspec works as a faster JSON or MessagePack library on its own. Still, its docs recommend it for the [full serialization and validation workflow](https://github.com/msgspec/msgspec): define your schemas with type annotations, encode them, then decode them back with validation. [Structs are the preferred way](https://msgspec.dev/structs) to define those types. Pass the type when you decode, and msgspec [validates while decoding](https://msgspec.dev/usage#typed-decoding) at no added runtime cost: `msgspec.json.decode(data, type=User)`. A consistent interface covers MessagePack, YAML, and TOML too.
|
||||
|
||||
Moving from the standard `json` module to orjson? The [largest difference](https://github.com/ijl/orjson?tab=readme-ov-file#migrating) is that `orjson.dumps` returns `bytes`, not `str`. orjson serializes dataclasses, datetimes, and UUIDs natively. Decoding gives you only dicts, lists, and other builtins, and orjson leaves schemas to [validation libraries a level above](https://github.com/ijl/orjson?tab=readme-ov-file#questions).
|
||||
|
||||
msgpack reads and writes MessagePack, a binary format that [lets you exchange data among languages like JSON, but faster and smaller](https://github.com/msgpack/msgpack-python). Call `packb` and `unpackb` for one-shot use, and read many objects from one stream with an [`Unpacker`](https://github.com/msgpack/msgpack-python?tab=readme-ov-file#streaming-unpacking). When the data comes from an untrusted source, [set `max_buffer_size`](https://msgpack-python.readthedocs.io/en/latest/api.html#msgpack.Unpacker) to limit the buffer.
|
||||
|
||||
marshmallow [makes no assumption about your web framework or database layer](https://marshmallow.readthedocs.io/en/latest/why.html#agnostic), so its schemas work with just about any ORM, or none. [Declare a schema](https://marshmallow.readthedocs.io/en/latest/quickstart.html#declaring-schemas) as a class that maps attribute names to fields. Its `dump` method turns your objects into primitive Python types, and `load` [validates and deserializes](https://marshmallow.readthedocs.io/en/latest/quickstart.html#deserializing-objects-loading) incoming data, raising `ValidationError` on invalid input. To get objects back instead of dicts, [decorate a schema method with `post_load`](https://marshmallow.readthedocs.io/en/latest/quickstart.html#deserializing-to-objects).
|
||||
|
||||
orjson, msgspec, and msgpack all pitch speed, but msgspec's own benchmark page [encourages you to write your own benchmarks](https://msgspec.dev/benchmarks) before deciding.
|
||||
@@ -0,0 +1,14 @@
|
||||
Of the two Python static site generators here, Pelican builds your blog. Nikola also builds sites that aren't only a blog, and takes Jupyter notebooks as posts.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A blog: Pelican
|
||||
- A site with more than a blog, or no blog at all: Nikola
|
||||
- Jupyter notebooks as posts: Nikola
|
||||
- Themes written in Mako: Nikola
|
||||
|
||||
Pelican splits content into [articles and pages](https://docs.getpelican.com/en/latest/content.html#articles-and-pages): articles are dated, like blog posts, and pages hold content that rarely changes, like an About page. Start a project with [`pelican-quickstart`](https://docs.getpelican.com/en/latest/quickstart.html#create-a-project). Then write each post as a Markdown or reStructuredText file in `content/`, with metadata headers at the top. While you write, [`pelican --autoreload --listen`](https://docs.getpelican.com/en/latest/publish.html#site-generation) rebuilds the site on every change and serves it on localhost. Themes are Jinja2 templates. The default theme is plain HTML without styling, so start from a [community theme](https://docs.getpelican.com/en/latest/themes.html) or write your own. Add features with [plugins you install with pip](https://docs.getpelican.com/en/latest/plugins.html#how-to-use-plugins), which Pelican can discover on its own. It [isn't only for blogs](https://docs.getpelican.com/en/latest/faq.html#is-pelican-only-suitable-for-blogs), though a site without one takes some theme and configuration changes.
|
||||
|
||||
Nikola [started as a blog generator but supports most kinds of sites](https://getnikola.com/handbook.html#what-s-nikola-and-what-can-you-do-with-it). It comes with [batteries included](https://getnikola.com/): comments, tags, archives, feeds, multilingual support, image galleries, and code listings. Out of the box, it takes reStructuredText, Markdown, Jupyter notebooks, and HTML. Create a site with [`nikola init --demo`](https://getnikola.com/getting-started.html#init), which runs a setup wizard. Write each post with [`nikola new_post -e`](https://getnikola.com/getting-started.html#newpost), which adds the metadata headers for you. Posts default to reStructuredText; add `-f markdown` for Markdown. `nikola build` fills the `output` directory, and [`nikola serve --browser`](https://getnikola.com/getting-started.html#serve) opens the site in your browser. For a server that rebuilds on every change, run `nikola auto --browser`. Themes are [Mako or Jinja2 templates](https://getnikola.com/theming.html). For a site with no blog, follow its [guide to building a site that isn't a blog](https://getnikola.com/creating-a-site-not-a-blog-with-nikola.html).
|
||||
|
||||
Both write plain HTML files: Pelican's output is [easy to host anywhere](https://docs.getpelican.com/en/latest/), and a Nikola site runs [on any web server](https://getnikola.com/). For GitHub Pages, push your Pelican sources and let [a GitHub Actions workflow](https://docs.getpelican.com/en/latest/tips.html#publishing-to-github-pages-using-a-custom-github-actions-workflow) build the site. Nikola's [`nikola github_deploy`](https://getnikola.com/handbook.html#deploying-to-github) builds the site and pushes the output to a `gh-pages` branch. Moving off WordPress works with either: Pelican [imports a WordPress XML export](https://docs.getpelican.com/en/latest/importer.html), and Nikola has [`nikola import_wordpress`](https://getnikola.com/handbook.html#importing-your-wordpress-site-into-nikola).
|
||||
@@ -0,0 +1,12 @@
|
||||
Audit your dependencies for known vulnerabilities as part of Python supply chain security. A uv project has uv audit built in; other projects run pip-audit.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A project managed by uv: uv audit
|
||||
- Environments, requirements files, and projects outside uv: pip-audit
|
||||
|
||||
`uv audit` comes with uv. Run it in your project, and it [audits the project's dependencies](https://docs.astral.sh/uv/reference/cli/#uv-audit) for known vulnerabilities and for statuses like deprecation and quarantine. By default, it covers every extra and dependency group.
|
||||
|
||||
pip-audit [scans Python environments for packages with known vulnerabilities](https://github.com/pypa/pip-audit), using the Python Packaging Advisory Database. Run `pip-audit` for the current environment, `pip-audit -r requirements.txt` for a requirements file, or `pip-audit .` for a local project. In CI, run it with its [official GitHub Action](https://github.com/pypa/pip-audit#github-actions). Only audit a requirements file you would install, since `pip-audit -r` is [functionally equivalent to `pip install -r`](https://github.com/pypa/pip-audit#security-model).
|
||||
|
||||
Also control what gets installed, since an audit only finds vulnerabilities someone has already reported. For requirements files, pip's docs recommend [hash-checking mode](https://pip.pypa.io/en/stable/topics/secure-installs/) to protect against remote tampering. In a uv project, uv's docs suggest a [dependency cooldown](https://docs.astral.sh/uv/concepts/resolution/#dependency-cooldowns), which holds back new releases until the community has had a chance to vet them.
|
||||
@@ -0,0 +1,22 @@
|
||||
Two questions pick a Python task queue: is your app async, and what's your broker? Taskiq is for asyncio. Celery takes RabbitMQ or Redis, RQ Redis, Huey SQLite.
|
||||
|
||||
How to choose:
|
||||
|
||||
- An asyncio app, like FastAPI: Taskiq
|
||||
- A batteries-included queue on RabbitMQ, Redis, or Amazon SQS, or tasks sent from non-Python code: Celery
|
||||
- The simplest job queue, on Redis or Valkey: RQ
|
||||
- No broker server to run, with the queue in SQLite or the Postgres you already have: Huey
|
||||
- A production backend for Django's task framework: Huey
|
||||
- Retries with backoff and acks after processing by default, on RabbitMQ or Redis: Dramatiq
|
||||
|
||||
Taskiq's docs call it [an asyncio Celery implementation](https://taskiq-python.github.io/): you mark tasks with `@broker.task` and send them with `await task.kiq()`. Since sending is async only, its docs [point fully synchronous projects to Celery or Dramatiq](https://taskiq-python.github.io/guide/). For production, its docs [highly recommend](https://taskiq-python.github.io/guide/getting-started.html) taskiq-aio-pika or taskiq-nats as the broker and taskiq-redis as the result backend. In a FastAPI app, [taskiq-fastapi](https://taskiq-python.github.io/framework_integrations/taskiq-with-fastapi.html) connects the broker to your app.
|
||||
|
||||
Celery is a task queue [with batteries included](https://docs.celeryq.dev/en/stable/getting-started/first-steps-with-celery.html): [celery beat](https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html) runs periodic tasks, and [Django support](https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html) comes out of the box. It runs on RabbitMQ, Redis, or Amazon SQS. Its docs call RabbitMQ [an excellent choice for production](https://docs.celeryq.dev/en/stable/getting-started/first-steps-with-celery.html#choosing-a-broker) and warn that Redis is more likely to lose data on a sudden shutdown or power failure. Its protocol has clients in other languages, so non-Python code can send tasks too. Celery [doesn't support Windows](https://docs.celeryq.dev/en/stable/getting-started/introduction.html), though.
|
||||
|
||||
RQ is a simple job queue on Redis or Valkey, built for [a low barrier to entry](https://python-rq.org/). [Any Python function call](https://python-rq.org/docs/) can go on a queue, and there are [no queues, exchanges, or routing rules](https://python-rq.org/docs/#on-the-design) to set up first. Jobs are pickled, so RQ is Python-only. Put job functions in a module the worker [can import, not in `__main__`](https://python-rq.org/docs/#considerations-for-jobs), and run workers on the same source code as your app. Start workers with `rq worker --with-scheduler` to run jobs you schedule with `enqueue_in` or `enqueue_at`.
|
||||
|
||||
Huey is [a lightweight alternative](https://huey.readthedocs.io/en/latest/) with zero dependencies, and its queue can live in Redis, Postgres, SQLite, files, or memory. Its docs [match the storage to your workload](https://huey.readthedocs.io/en/latest/guide.html#storage-options): Redis for busy workloads, SQLite for moderate ones without a separate server, and Postgres when your app already runs it. Recurring tasks come built in with the `periodic_task()` decorator. For Django, Huey has [its own integration](https://huey.readthedocs.io/en/latest/contrib.html#django), and it provides [a production backend for Django's task framework](https://huey.readthedocs.io/en/latest/contrib.html#django-task-framework).
|
||||
|
||||
Dramatiq aims for [sane defaults for most SaaS workloads](https://dramatiq.io/motivation.html) on RabbitMQ or Redis. Mark a function with `@dramatiq.actor` and enqueue it with `.send()`, [passing only JSON-encodable arguments](https://dramatiq.io/guide.html). When an actor raises, Dramatiq [retries it with exponential backoff](https://dramatiq.io/guide.html#error-handling). It [acknowledges a message only after processing it](https://dramatiq.io/advanced.html#message-persistence), while Celery by default does so [just before running the task](https://docs.celeryq.dev/en/stable/userguide/tasks.html#Task.acks_late). For cron-style jobs, pair it with [a separate scheduler](https://dramatiq.io/cookbook.html#scheduling). Dramatiq is [licensed under the LGPL](https://dramatiq.io/).
|
||||
|
||||
Make tasks safe to run twice: after a worker failure, the same message [can arrive again](https://dramatiq.io/best_practices.html#retriable-actors). Pass a task the ID of a database row rather than the object, and [re-fetch it when the task runs](https://docs.celeryq.dev/en/stable/userguide/tasks.html#state), since old data leads to race conditions. In Django, enqueue a task only [after the transaction commits](https://docs.celeryq.dev/en/stable/userguide/tasks.html#database-transactions), with Celery's `delay_on_commit()` or Huey's `on_commit_task()`. With celery beat, Huey, or Taskiq, run [only one scheduler](https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html) for periodic tasks, or you'll get duplicate tasks. In production, run workers under [a process manager](https://python-rq.org/docs/workers/).
|
||||
@@ -0,0 +1,15 @@
|
||||
Whether templates may hold Python code splits the Python template engines: Jinja keeps code out and sandboxes untrusted templates, Mako embeds plain Python.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Templates without embedded Python: Jinja
|
||||
- Templates your users write: Jinja, in its sandbox
|
||||
- Plain Python inside your templates: Mako
|
||||
|
||||
Jinja [doesn't allow arbitrary Python code in templates](https://jinja.palletsprojects.com/en/stable/faq/): you get blocks, filters, and function calls, and the rest of your logic stays in Python. Outside a framework, [create one `Environment`](https://jinja.palletsprojects.com/en/stable/api/) when your app starts and load templates through a `PackageLoader`. Put the layout your pages share in a base template, and let each page [extend it and override its blocks](https://jinja.palletsprojects.com/en/stable/templates/#template-inheritance). Flask [sets up Jinja for you](https://flask.palletsprojects.com/en/stable/templating/#jinja-setup), Django has a [built-in Jinja2 backend](https://docs.djangoproject.com/en/stable/topics/templates/#django.template.backends.jinja2.Jinja2), and FastAPI [supports it with `Jinja2Templates`](https://fastapi.tiangolo.com/advanced/templates/).
|
||||
|
||||
When your users write templates, like a report layout, render them in Jinja's [sandbox](https://jinja.palletsprojects.com/en/stable/sandbox/#sandbox), which can block attribute access, method calls, and other operations. Its docs say the sandbox alone [is not a solution for perfect security](https://jinja.palletsprojects.com/en/stable/sandbox/#security-considerations): catch errors when a template renders, limit CPU and memory, and pass the template only the data it needs.
|
||||
|
||||
Mako is [an embedded Python language](https://www.makotemplates.org/): inside `<% %>` tags you [write regular Python](https://docs.makotemplates.org/en/latest/syntax.html#python-blocks). A real application [loads its templates from a `TemplateLookup`](https://docs.makotemplates.org/en/latest/usage.html#using-templatelookup) with a `module_directory`, which caches each compiled template on disk as a Python module.
|
||||
|
||||
Jinja leaves HTML escaping [off by default](https://jinja.palletsprojects.com/en/stable/faq/#why-is-html-escaping-not-the-default), since it also renders plain text, emails, and config files. When you create the `Environment` yourself, turn it on with [`select_autoescape()`](https://jinja.palletsprojects.com/en/stable/api/#jinja2.select_autoescape); Flask turns it on for HTML templates, and Django's Jinja2 backend turns it on for all. Mako escapes HTML only through [the `h` filter](https://docs.makotemplates.org/en/latest/filtering.html#expression-filtering): add it to `default_filters` on your `TemplateLookup` to escape every expression.
|
||||
@@ -0,0 +1,45 @@
|
||||
pytest runs your unittest suites unchanged, so switching Python testing frameworks costs no rewrite. Hypothesis and Playwright tests run inside it too.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Unit and integration tests: pytest
|
||||
- Tests that generate their own inputs: Hypothesis
|
||||
- End-to-end browser tests: Playwright
|
||||
- Acceptance tests that non-programmers can read: Robot Framework
|
||||
- Testing across Python versions: tox, or Nox to configure it in Python
|
||||
- WebDriver tests on remote machines: Selenium, or SeleniumBase with waits and reports built in
|
||||
- Load tests written as Python code: Locust
|
||||
- API tests generated from an OpenAPI or GraphQL schema: Schemathesis
|
||||
- Replacing objects in a test: unittest.mock, or time-machine for the clock
|
||||
- Faking HTTP calls: responses for Requests, RESPX for HTTPX, VCR.py to record real traffic
|
||||
- Test objects: factory_boy for ORM models, Polyfactory for typed classes
|
||||
- Code coverage: Coverage.py
|
||||
- Fake names and addresses: Faker, or Mimesis for speed
|
||||
|
||||
Write pytest tests as plain functions with plain `assert` statements: [pytest's assertion introspection](https://docs.pytest.org/en/stable/getting-started.html) reports the intermediate values when one fails, so there are no assert methods to remember. Share setup through [fixtures](https://docs.pytest.org/en/stable/explanation/fixtures.html#improvements-over-xunit-style-setup-teardown-functions), which a test requests by name, instead of xUnit-style setup and teardown. pytest also [runs unittest-based suites](https://docs.pytest.org/en/stable/how-to/unittest.html), so you can move an old suite over a test at a time.
|
||||
|
||||
Hypothesis tests take a [`@given` decorator](https://hypothesis.readthedocs.io/en/latest/quickstart.html#write-your-first-test) with a strategy describing the inputs, and Hypothesis picks which inputs to try, edge cases included. They're still regular functions that pytest or unittest runs.
|
||||
|
||||
Playwright's docs call [its pytest plugin](https://playwright.dev/python/docs/intro#installing-playwright-pytest) the recommended way to write end-to-end tests, and every test gets a fresh browser context. Playwright [waits for elements to be actionable](https://playwright.dev/python/docs/actionability) before each action, and its assertions wait for the expected condition. [Locate elements by role](https://playwright.dev/python/docs/locators#quick-guide) and other user-facing attributes, not CSS or XPath, which break as the DOM changes.
|
||||
|
||||
Robot Framework writes test cases in a [tabular, keyword-driven syntax](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#why-robot-framework), builds higher-level keywords from existing ones, and reports results in HTML. Custom keyword libraries are plain Python. In [behavior-driven style](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#behavior-driven-style), a test case reads as a requirement stakeholders who don't code can follow.
|
||||
|
||||
tox and Nox both create a virtual environment per Python version or dependency set and run your tests in each. tox defines its environments in [a config file](https://tox.wiki/en/latest/tutorial/getting-started.html#creating-your-first-configuration). Nox is [configured in a `noxfile.py`](https://nox.thea.codes/en/stable/tutorial.html#writing-the-configuration-file), where each session is a Python function, so any logic your test matrix needs is ordinary code.
|
||||
|
||||
Selenium drives browsers through WebDriver, a W3C standard, and [Selenium Grid](https://www.selenium.dev/documentation/overview/) runs those tests on other machines and platforms. Its guidelines model each page as a [page object](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) that makes no assertions, and use [explicit waits](https://www.selenium.dev/documentation/webdriver/waits/#explicit-waits) for the exact condition each step needs. [SeleniumBase](https://github.com/seleniumbase/SeleniumBase) builds on Selenium and runs under pytest: its methods wait for an element before acting, and failing tests save screenshots.
|
||||
|
||||
A Locust test is [a Python program](https://docs.locust.io/en/stable/quickstart.html): an `HttpUser` class whose tasks make requests, so complex user flows are just code.
|
||||
|
||||
Schemathesis reads your OpenAPI or GraphQL schema and generates property-based tests from it, with no per-endpoint tests to maintain. Its FAQ [recommends the CLI](https://schemathesis.readthedocs.io/en/stable/faq/#how-should-i-run-schemathesis) for most users, with a pytest integration for existing suites.
|
||||
|
||||
With unittest.mock, [patch a name where it's looked up](https://docs.python.org/3/library/unittest.mock.html#where-to-patch), which isn't always where it's defined. [Autospec](https://docs.python.org/3/library/unittest.mock.html#autospeccing) your mocks, so a test fails when your code calls an API the real object lacks. For the clock, time-machine [mocks time functions everywhere they're referenced](https://time-machine.readthedocs.io/en/latest/comparison.html), where a patch only reaches the import location it targets.
|
||||
|
||||
For HTTP, pick by client. responses mocks Requests calls in a test [wrapped in `@responses.activate`](https://github.com/getsentry/responses#basics), and RESPX mocks HTTPX, with a [`respx_mock` fixture](https://lundberg.github.io/respx/) for pytest. VCR.py [records real HTTP interactions](https://vcrpy.readthedocs.io/en/latest/) to a cassette file on the first run and replays them after, across many client libraries.
|
||||
|
||||
factory_boy replaces static fixtures with factories, where a test declares only the fields it cares about. It has [base classes for Django, SQLAlchemy, and MongoEngine](https://factoryboy.readthedocs.io/en/stable/#orm-integration) models. Polyfactory instead [reads the type hints](https://polyfactory.litestar.dev/latest/getting-started.html#example) on dataclasses, TypedDicts, Pydantic models, and msgspec Structs to generate the data.
|
||||
|
||||
Run your suite under Coverage.py with [`coverage run -m pytest`](https://coverage.readthedocs.io/en/latest/#quick-start), then `coverage report` prints the results. For most purposes, you need no pytest plugin.
|
||||
|
||||
Faker generates names, addresses, and other data [per locale](https://faker.readthedocs.io/en/master/#localization). Mimesis covers the same ground, fully typed, and [its benchmarks](https://mimesis.name/latest/benchmarks.html) show it faster than Faker.
|
||||
|
||||
Random test data makes a failing build hard to reproduce, so seed it: [Faker](https://faker.readthedocs.io/en/master/#seeding-the-generator), [factory_boy](https://factoryboy.readthedocs.io/en/stable/#reproducible-random-values), and [Mimesis](https://mimesis.name/latest/random_and_seed.html) all take a seed. For more tools, see [awesome-python-testing](https://github.com/cleder/awesome-python-testing).
|
||||
@@ -0,0 +1,44 @@
|
||||
Mojibake like "café" is ftfy's to fix, and charset-normalizer detects encodings. For fuzzy matching, add RapidFuzz to your Python text processing libraries.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Text decoded with the wrong encoding: ftfy
|
||||
- Bytes in an unknown encoding: charset-normalizer
|
||||
- Fuzzy matching against a long list of strings: RapidFuzz
|
||||
- Diffs, or close matches with nothing to install: difflib
|
||||
- ASCII-art banners: pyfiglet
|
||||
- Translations, plus dates and numbers in each user's locale: Babel
|
||||
- Syntax highlighting: Pygments
|
||||
- A grammar written in Python code: pyparsing, or parsy for small languages
|
||||
- Splitting and formatting SQL: sqlparse
|
||||
- Phone numbers: phonenumbers
|
||||
- URL slugs: python-slugify, or Unidecode for plain ASCII
|
||||
- Short IDs that users will read: shortuuid
|
||||
|
||||
ftfy [fixes Unicode that's broken in various ways](https://ftfy.readthedocs.io/en/latest/): bad Unicode goes in, good Unicode comes out. It's [a mojibake detector and fixer, not an encoding detector](https://ftfy.readthedocs.io/en/latest/detect.html#is-ftfy-an-encoding-detector), so give it text you've already tried to decode, never bytes. The function you'll call most is [`ftfy.fix_text()`](https://ftfy.readthedocs.io/en/latest/explain.html), which applies every fix it can. ftfy aims never to change text that was decoded correctly.
|
||||
|
||||
charset-normalizer [reads text from an unknown charset encoding](https://github.com/jawah/charset_normalizer). Call `from_bytes(data).best()`, or `from_path(path).best()` for a file, and [`str()` of the result is your decoded text](https://charset-normalizer.readthedocs.io/en/latest/user/getstarted.html). chardet does the job differently: charset-normalizer decodes candidate encodings and scores the text, while chardet [scores raw bytes against per-language models](https://chardet.readthedocs.io/en/latest/faq.html#how-is-chardet-different-from-charset-normalizer).
|
||||
|
||||
RapidFuzz [does fuzzy string matching with various string metrics](https://rapidfuzz.github.io/RapidFuzz/), mostly in C++, with a pure Python fallback for every algorithm. To compare one string against a list, [use the `process` module](https://github.com/rapidfuzz/RapidFuzz#process), like `process.extractOne()`.
|
||||
|
||||
difflib comes with Python and [compares sequences](https://docs.python.org/3/library/difflib.html), mostly lines of text, producing unified, context, and HTML diffs like the ones `diff` and `git diff` print. For a "did you mean" suggestion, [`get_close_matches()`](https://docs.python.org/3/library/difflib.html#difflib.get_close_matches) returns the best "good enough" matches for a word.
|
||||
|
||||
pyfiglet is [a full port of FIGlet into pure Python](https://github.com/pwaller/pyfiglet): it renders text in ASCII-art fonts, as in `pyfiglet.figlet_format("text", font="slant")`.
|
||||
|
||||
Babel [does two jobs](https://babel.pocoo.org/en/latest/intro.html): it builds gettext message catalogs without the GNU gettext tools for common tasks, and it formats dates, numbers, and locale names from CLDR data. Run catalogs through the `pybabel` command: [`extract` messages, `init` a catalog per language, `update` it, and `compile` it](https://babel.pocoo.org/en/latest/cmdline.html).
|
||||
|
||||
Pygments is [a generic syntax highlighter](https://github.com/pygments/pygments) for code hosting, forums, wikis, and other apps that show source code. If you highlight code your users submit, its docs recommend you [stop the Pygments process after a short timeout](https://pygments.org/docs/security/#security-considerations).
|
||||
|
||||
pyparsing [builds a grammar directly in Python code](https://github.com/pyparsing/pyparsing) from a library of classes, instead of lex/yacc or regular expressions. It handles whitespace, quoted strings, and embedded comments for you. Its best practices start with [writing a BNF](https://github.com/pyparsing/pyparsing/wiki/Best-Practices) before any code. [Use results names](https://pyparsing-docs.readthedocs.io/en/latest/HowToUsePyparsing.html#usage-notes) to get tokens by field name rather than by position. parsy [combines small parsers into larger ones](https://github.com/python-parsy/parsy) and gives you building blocks only. Its docs say it [excels at easy-to-read parsers for relatively small languages](https://parsy.readthedocs.io/en/latest/overview.html#other-python-projects).
|
||||
|
||||
sqlparse is [a non-validating SQL parser](https://github.com/andialbrecht/sqlparse): it splits scripts into statements, formats them, and walks their token tree, whatever the dialect. Its [three module-level functions](https://github.com/andialbrecht/sqlparse#usage), `split()`, `format()`, and `parse()`, cover most needs.
|
||||
|
||||
phonenumbers is [a Python port of Google's libphonenumber](https://github.com/daviddrysdale/python-phonenumbers). Parse a number together with [the country it's dialed from](https://github.com/daviddrysdale/python-phonenumbers#example-usage), unless it's in E.164 format. Then check it with `is_possible_number()` or `is_valid_number()`, and print it with `format_number()`.
|
||||
|
||||
python-slugify [makes Unicode-aware slugs](https://github.com/un33k/python-slugify) with your choice of transliteration backend: by default, Unidecode if it's installed, otherwise text-unidecode. Its own code is MIT, while [Unidecode is GPL](https://github.com/un33k/python-slugify#licensing), so check that license before you install Unidecode next to it.
|
||||
|
||||
Unidecode makes [lossy ASCII transliterations of Unicode text](https://github.com/avian2/unidecode), close to what someone on a US keyboard would type. Use it for ASCII machine identifiers. Its README says it's best kept off strings your users see.
|
||||
|
||||
shortuuid [generates concise, unambiguous, URL-safe UUIDs](https://github.com/skorokithakis/shortuuid) for IDs users will see: it takes UUIDs from Python's `uuid` module, writes them in base57, and drops look-alike characters such as l, 1, I, O, and 0. Call `shortuuid.uuid()` for a new ID.
|
||||
|
||||
Decode with the encoding you were told whenever there is one. charset-normalizer's FAQ calls detection [a last resort](https://charset-normalizer.readthedocs.io/en/latest/community/faq.html#should-i-bother-using-detection), and ftfy's docs say to [assume UTF-8](https://ftfy.readthedocs.io/en/latest/avoid.html#assume-utf-8) unless you have a specific reason to believe otherwise.
|
||||
@@ -0,0 +1,31 @@
|
||||
Type hints alone let FastAPI validate requests and document your API. Python API frameworks for Django include Django Ninja, which reads them too.
|
||||
|
||||
How to choose:
|
||||
|
||||
- A new API, not tied to Django or Flask: FastAPI
|
||||
- Typed endpoints in a Django project: Django Ninja
|
||||
- CRUD endpoints over Django models, with a browsable API: Django REST framework
|
||||
- GraphQL types generated from Django models: Strawberry Django
|
||||
- msgspec, attrs, or dataclass schemas on Django: django-modern-rest
|
||||
- Flask resources grouped into namespaces, with Swagger docs: Flask-RESTX
|
||||
- marshmallow schemas on Flask blueprints: flask-smorest
|
||||
- A Flask app validated with Pydantic models or marshmallow: APIFlask
|
||||
- GraphQL on FastAPI or any other framework: Strawberry
|
||||
- An OpenAPI spec written before the code: Connexion
|
||||
- Calls between services written in different languages: gRPC
|
||||
|
||||
FastAPI is [based on standard Python type hints](https://fastapi.tiangolo.com/). Declare path, query, and body parameters as typed function arguments, and it validates them and serves interactive docs at `/docs`. As the app grows, split it into modules with [`APIRouter`, a "mini FastAPI"](https://fastapi.tiangolo.com/tutorial/bigger-applications/).
|
||||
|
||||
Django Ninja was [heavily inspired by FastAPI](https://django-ninja.dev/) and integrates with Django's core and ORM. You declare parameter and body types once, as function arguments, and the interactive docs show up at `/api/docs`. For a bigger project, [create an `api.py` with a `Router` in each Django app](https://django-ninja.dev/guides/routers/).
|
||||
|
||||
Django REST framework [splits an API into](https://www.django-rest-framework.org/) serializers for the representation, viewsets for the behavior, and routers for the URLs. [Viewsets get you running quickly](https://www.django-rest-framework.org/api-guide/viewsets/) and keep URLs consistent across a large API, while plain views are more explicit. For OpenAPI schemas, its docs [recommend drf-spectacular](https://www.django-rest-framework.org/topics/documenting-your-api/).
|
||||
|
||||
APIFlask has you [use `APIFlask` instead of `Flask`](https://apiflask.com/), then declare request and response schemas with [`@app.input` and `@app.output`](https://apiflask.com/usage/). flask-smorest [organizes resources as `MethodView` classes in blueprints](https://flask-smorest.readthedocs.io/en/latest/quickstart.html), with marshmallow schemas on `Blueprint.arguments` and `Blueprint.response`. Flask-RESTX [splits your app into reusable namespaces](https://flask-restx.readthedocs.io/en/latest/scaling.html), much like blueprints, and describes them for Swagger through decorators.
|
||||
|
||||
Strawberry [builds on dataclasses and type hints](https://strawberry.rocks/docs), and FastAPI's docs [recommend it for GraphQL](https://fastapi.tiangolo.com/how-to/graphql/). Before going to production, [disable GraphiQL and introspection](https://strawberry.rocks/docs/operations/deployment), and turn on Strawberry's security extensions that limit query depth, aliases, and tokens.
|
||||
|
||||
Strawberry Django [creates GraphQL types from your Django models](https://strawberry.rocks/docs/django). Add its [`DjangoOptimizerExtension`](https://strawberry.rocks/docs/django/guide/optimizer), which calls `select_related()` and `prefetch_related()` for you to avoid N+1 queries.
|
||||
|
||||
Connexion flips the usual order: you [write the OpenAPI spec first](https://connexion.readthedocs.io/en/latest/), then the code, and Connexion validates requests against the spec. Each `operationId` in the spec [names the Python function that handles it](https://connexion.readthedocs.io/en/latest/routing.html). Start new projects on [`AsyncApp`, or on `FlaskApp` to use the Flask ecosystem](https://connexion.readthedocs.io/en/latest/quickstart.html).
|
||||
|
||||
gRPC lets you [define a service once in a `.proto` file](https://grpc.io/docs/languages/python/basics/) and generate clients and servers in any language it supports. For asyncio code, gRPC has [`grpc.aio`](https://grpc.github.io/grpc/python/grpc_asyncio.html).
|
||||
@@ -0,0 +1,15 @@
|
||||
Static files can skip a separate server: with WhiteNoise, Django static files are served by your app itself. Uploads go to cloud storage via django-storages.
|
||||
|
||||
How to choose:
|
||||
|
||||
- Static files served by your Django or other WSGI app: WhiteNoise
|
||||
- User uploads on Amazon S3, Azure Storage, Google Cloud Storage, or SFTP: django-storages
|
||||
- CSS and JavaScript from your Django templates, combined into one file: Django Compressor
|
||||
|
||||
WhiteNoise lets your app [serve its own static files](https://whitenoise.readthedocs.io/en/latest/), with no separate web server. It works with any WSGI app, with extra auto-configuration for Django. On Django, switch to its [compressed manifest storage backend](https://whitenoise.readthedocs.io/en/latest/django.html#add-compression-and-caching-support), which compresses your files and gives them hashed names, so browsers can cache them forever. Its docs advise against [pushing static files to S3](https://whitenoise.readthedocs.io/en/latest/#shouldn-t-i-be-pushing-my-static-files-to-s3-using-something-like-django-storages) with django-storages: that adds libraries, configuration, keys, and deploy steps.
|
||||
|
||||
django-storages is a [collection of storage backends](https://django-storages.readthedocs.io/en/latest/) for Django, covering Amazon S3 and S3-compatible services, Azure Storage, Google Cloud Storage, Dropbox, FTP, and SFTP. Each backend has its own settings: to save uploads to S3, for example, set `storages.backends.s3.S3Storage` as the [`default` backend in `STORAGES`](https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#configuration-settings) and pass its settings under `OPTIONS`.
|
||||
|
||||
Django Compressor [combines the CSS and JavaScript](https://django-compressor.readthedocs.io/en/latest/) that your templates link or inline into one cached file, and browsers can cache that file forever. Wrap the tags in a [`{% compress %}` block](https://django-compressor.readthedocs.io/en/latest/usage.html#examples). By default it compresses during requests, but when you run several servers or [WhiteNoise serves the files](https://whitenoise.readthedocs.io/en/latest/django.html#django-compressor), turn on [offline compression](https://django-compressor.readthedocs.io/en/latest/scenarios.html#offline-compression) and run its `compress` command on deploy.
|
||||
|
||||
Keep user uploads out of your static files. WhiteNoise [isn't suitable for serving them](https://whitenoise.readthedocs.io/en/latest/django.html#serving-media-files): uploads served from your app's own domain are a security risk, and local disk makes scaling across machines harder. Its docs point to django-storages instead. Django's own docs recommend serving uploads [from a separate domain](https://docs.djangoproject.com/en/stable/topics/security/#user-uploaded-content), like usercontent-example.com for example.com, not a subdomain.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user