awesome-python/website/data/category_intros/file-manipulation.md

Without installing anything, two Python file manipulation libraries cover paths and file types by name: pathlib and mimetypes. python-magic reads the bytes.

How to choose:

• Building and joining paths, and listing directories: pathlib
• A MIME type from a file name or URL: mimetypes
• A file's type from its content, like the Unix file command: python-magic
• Rerunning or reloading code when files change: watchfiles
• Your own handler for each created, modified, or moved file: watchdog
• Telling text file types apart, like source code from config files: Magika

If you aren't sure which pathlib class you need, the docs say Path is most likely it: it makes a concrete path for the platform your code runs on. Join paths with the / operator, and list a directory with iterdir() or glob().

mimetypes maps a file name's extension to a MIME type, and a MIME type back to extensions. A type guess returns 2 values: a type for the Content-Type header, and an encoding like gzip for the Content-Encoding header. The type is None for a missing or unknown suffix.

python-magic wraps libmagic, which identifies file types by checking their headers, the way the Unix file command does. It's a thin wrapper, so install libmagic too, with apt-get install libmagic1 or brew install libmagic. Call magic.from_file(path, mime=True) for a MIME type, or magic.from_buffer() on bytes you already have.

watchfiles is built for file watching and code reload, and its Rust core groups changes into batches instead of firing once per file. watch() is a generator that yields sets of changes, and awatch() is the async version. To restart code when files change, use run_process() with a function or a command. From a shell, the watchfiles CLI does the same: watchfiles --filter python 'pytest --lf' src tests.

watchdog gives you an API and a shell tool for monitoring directories. Its quickstart has you subclass FileSystemEventHandler, override methods like on_created() and on_modified(), schedule the handler on an Observer, and start that thread. An observer skips subdirectories unless you pass recursive=True to schedule().

Magika runs a compact deep-learning model on a few chunks of a file, without loading large files entirely into memory. On its own test set, it outperforms existing approaches, especially on textual content types. Create a Magika() instance, then call identify_path() on a file or identify_bytes() on bytes you already have. Check result.ok before you read result.output.label or result.output.mime_type.