mirror of
https://github.com/vinta/awesome-python.git
synced 2026-11-14 18:16:27 -06:00
Merge branch 'audit/file-manipulation-archives'
This commit is contained in:
@@ -1005,6 +1005,12 @@ _Libraries for file manipulation._
|
||||
- [zstandard](https://github.com/indygreg/python-zstandard) - Python bindings to the Zstandard (zstd) compression library.
|
||||
- [brotli](https://github.com/google/brotli) - Python bindings for Google's Brotli compression library.
|
||||
- [lz4](https://github.com/python-lz4/python-lz4) - Python bindings for the LZ4 fast compression library.
|
||||
- Archives
|
||||
- [tarfile](https://docs.python.org/3/library/tarfile.html) - (Python standard library) Read and write tar archives, including gzip, bz2, lzma, and zstd compressed ones.
|
||||
- [zipfile](https://docs.python.org/3/library/zipfile.html) - (Python standard library) Read and write ZIP archives.
|
||||
- [py7zr](https://github.com/miurahr/py7zr) - A pure-Python library for reading and writing 7-Zip archives.
|
||||
- [pyzipper](https://github.com/danifus/pyzipper) - Read and write AES-encrypted ZIP files ([zipfile](https://docs.python.org/3/library/zipfile.html) fork).
|
||||
- [rarfile](https://github.com/markokr/rarfile) - A RAR archive reader with a zipfile-like API.
|
||||
|
||||
**Media**
|
||||
|
||||
|
||||
+2
-2
@@ -111,8 +111,8 @@ Projects the maintainer decided not to list, with the evidence at decision time,
|
||||
- pycompression/python-zlib-ng -- 2026-11-15, 3,387,853 downloads/month (pypistats, package zlib-ng): A faster drop-in for zlib and gzip with a niche audience (16 stars).
|
||||
- pycompression/python-isal -- 2026-11-15, 2,960,582 downloads/month (pypistats, package isal): A faster drop-in for zlib and gzip using Intel ISA-L, with a niche audience (61 stars).
|
||||
- python-hyper/brotlicffi -- 2026-11-15, 1,946,691 downloads/month (pypistats): The PyPy version of the listed brotli. HTTP clients pick it automatically on non-CPython.
|
||||
- uktrade/stream-zip -- 2026-11-15, 980,636 downloads/month (pypistats): Builds ZIP files as a stream. Niche (21 stars), and no archives use case exists.
|
||||
- Changaco/python-libarchive-c -- 2026-11-15, 824,581 downloads/month (pypistats, package libarchive-c): A libarchive binding with low demand (85 stars) that needs the system library. No archives use case exists.
|
||||
- uktrade/stream-zip -- 2026-11-15, 980,636 downloads/month (pypistats): Builds ZIP files as a stream. Niche (21 stars), and Archives is at cap.
|
||||
- Changaco/python-libarchive-c -- 2026-11-15, 824,581 downloads/month (pypistats, package libarchive-c): A libarchive binding with low demand (85 stars) that needs the system library. Archives is at cap.
|
||||
|
||||
## Last Audited
|
||||
|
||||
|
||||
@@ -14,6 +14,8 @@ How to choose:
|
||||
- Reading and writing .gz files: gzip
|
||||
- Denser compression than gzip at a similar speed: brotli
|
||||
- Compressing and decompressing as fast as possible: lz4
|
||||
- Tar and ZIP archives: tarfile and zipfile, or pyzipper for AES-encrypted ZIP files
|
||||
- 7-Zip and RAR archives: py7zr, or rarfile to read .rar files
|
||||
|
||||
If you aren't sure which pathlib class you need, the docs say [`Path` is most likely it](https://docs.python.org/3/library/pathlib.html): it makes a concrete path for the platform your code runs on. [Join paths with the `/` operator](https://docs.python.org/3/library/pathlib.html#basic-use), and list a directory with `iterdir()` or `glob()`.
|
||||
|
||||
@@ -40,3 +42,13 @@ zstandard is [the most feature-rich zstd binding](https://python-zstandard.readt
|
||||
brotli is the Python module of the Brotli project, whose format is [similar in speed to deflate but denser](https://github.com/google/brotli#introduction). Call `brotli.compress()` and `brotli.decompress()` on bytes, or [feed a `brotli.Compressor` piece by piece](https://github.com/google/brotli/blob/master/python/brotli.py) with `process()`, then call `finish()`.
|
||||
|
||||
lz4 binds the LZ4 library, built for [extremely fast compression and an extremely fast decoder](https://github.com/lz4/lz4). Its docs recommend [the `lz4.frame` bindings](https://python-lz4.readthedocs.io/en/stable/quickstart.html#simple-usage), since the frame format works with other LZ4 implementations and language bindings. `lz4.frame.open()` is [the most convenient way to work with compressed files](https://python-lz4.readthedocs.io/en/stable/quickstart.html#working-with-compressed-files), and it works like `gzip.open()`.
|
||||
|
||||
tarfile reads and writes tar archives through [`tarfile.open()`](https://docs.python.org/3/library/tarfile.html#tarfile.open), and a mode like `'r:gz'` or `'w:zst'` picks the compression. To extract, use `extractall()` with the [`'data'` filter](https://docs.python.org/3/library/tarfile.html#extraction-filters), which is meant for cross-platform data archives. Still, never extract an archive from an untrusted source without inspecting it first.
|
||||
|
||||
zipfile works through [`ZipFile` as a context manager](https://docs.python.org/3/library/zipfile.html#zipfile-objects): `ZipFile('spam.zip', 'w')` to write, then `write()` each file. Before you call `extractall()` on an archive you didn't make, [inspect it first](https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile.extractall), since members with absolute paths or `..` can land outside the target folder.
|
||||
|
||||
py7zr opens 7-Zip archives with [`SevenZipFile` in a `with` block](https://github.com/miurahr/py7zr#sevenzipfile-class-usage): call `extractall()` in `'r'` mode, or `writeall()` in `'w'` mode to pack a whole folder.
|
||||
|
||||
pyzipper is [a replacement for `zipfile` that reads and writes AES-encrypted ZIP files](https://github.com/danifus/pyzipper#usage), with the same API. Open the archive with `pyzipper.AESZipFile`, pass `encryption=pyzipper.WZ_AES` when writing, and call `setpassword()` with a bytes password.
|
||||
|
||||
rarfile reads RAR archives through [an interface in the style of `zipfile`](https://github.com/markokr/rarfile). It parses archives in pure Python, but extracts compressed files with an external tool, so install unrar, the one it recommends.
|
||||
|
||||
@@ -40,6 +40,7 @@
|
||||
"sqlite3": { "package": null, "reason": "stdlib module" },
|
||||
"strawberry": { "package": "strawberry-graphql", "reason": "PyPI strawberry is an unrelated bookmarking service" },
|
||||
"strawberry-django": { "package": "strawberry-graphql-django", "reason": null },
|
||||
"tarfile": { "package": null, "reason": "stdlib module" },
|
||||
"thealgorithms": { "package": null, "reason": "educational monorepo, not a package" },
|
||||
"tkinter": { "package": null, "reason": "stdlib module" },
|
||||
"tomllib": { "package": null, "reason": "stdlib module" },
|
||||
@@ -48,5 +49,6 @@
|
||||
"uv-audit": { "package": null, "reason": "part of uv (the `uv audit` subcommand); PyPI uv-audit is an unrelated third-party tool by rocshers" },
|
||||
"winpython": { "package": null, "reason": "distribution with its own installer; PyPI name is a 2012 relic" },
|
||||
"wshobson-agents": { "package": null, "reason": "agent skills repo, not a package" },
|
||||
"zipfile": { "package": null, "reason": "stdlib module" },
|
||||
"zoneinfo": { "package": null, "reason": "stdlib module" }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user