From 7dd39794ac1db84b16a61ad1dd31ed7e6df1f91d Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Fri, 2 Oct 2026 16:37:46 +0800 Subject: [PATCH] docs: rewrite File Format Processing category intro The intro predated the audit and never mentioned pymupdf, which the README now lists. Co-Authored-By: Claude --- .../category_intros/file-format-processing.md | 55 ++++++++++--------- 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/website/data/category_intros/file-format-processing.md b/website/data/category_intros/file-format-processing.md index 57e0328c..e5c52292 100644 --- a/website/data/category_intros/file-format-processing.md +++ b/website/data/category_intros/file-format-processing.md @@ -1,51 +1,54 @@ -pypdf reads PDFs, python-docx Word files, and openpyxl Excel sheets: one Python file format library per format. MarkItDown turns any of them into Markdown. +PDFs you merge, split, or encrypt need only pypdf. Which Python file format library you need beyond that depends on whether you read a file or create it. How to choose: -- Splitting, merging, and reading PDFs: pypdf -- Word documents: python-docx -- Excel: openpyxl to read or edit, XlsxWriter for new reports -- Documents to Markdown for an LLM: MarkItDown +- Merging, splitting, cropping, or encrypting PDFs: pypdf - ELF binaries and DWARF debug info: pyelftools -- One table exported to CSV, JSON, Excel, and more: Tablib -- Scanned PDFs, tables, and complex layouts: Docling +- A table you import or export as CSV, JSON, YAML, or Excel: Tablib +- Documents turned into Markdown for an LLM: MarkItDown, or Docling for complex layouts and scans +- Excel: openpyxl to edit a workbook, XlsxWriter to create one with charts and formatting +- Word documents: python-docx - PowerPoint decks: python-pptx -- New PDFs drawn from Python code: ReportLab -- PDF text with its position and font: pdfminer.six +- Text and images out of PDFs, or pages rendered as images: PyMuPDF, or pdfminer.six for each character's position and font +- PDFs drawn from Python code: ReportLab - PDFs from HTML and CSS: WeasyPrint - Markdown to HTML: markdown-it-py for CommonMark, Python-Markdown for its extensions, Mistune for speed - Config files: tomllib for TOML, PyYAML for YAML -pypdf works on PDFs that already exist: it [splits, merges, crops, and transforms pages](https://pypdf.readthedocs.io/en/stable/), adds passwords, and pulls out text and metadata. It's [pure Python with no C dependency](https://pypdf.readthedocs.io/en/stable/meta/comparisons.html), and it doesn't create PDFs. pypdf [isn't OCR software](https://pypdf.readthedocs.io/en/stable/user/extract-text.html), so run scanned pages through OCR instead. +pypdf is a [pure-Python PDF library](https://pypdf.readthedocs.io/en/stable/) for splitting, merging, cropping, and transforming pages, and it adds passwords too. To merge files, [`append()` each one to a `PdfWriter`](https://pypdf.readthedocs.io/en/stable/user/merging-pdfs.html) and write it out. With [no C dependency](https://pypdf.readthedocs.io/en/stable/meta/comparisons.html), it's easy to install, but it doesn't generate new PDFs. It reads text with `page.extract_text()`, while a scanned page needs [OCR software](https://pypdf.readthedocs.io/en/stable/user/extract-text.html). -python-docx [only edits existing documents](https://python-docx.readthedocs.io/en/latest/user/documents.html): `Document()` opens a built-in template with no content. Start from your own .docx instead, so its styles, headers, and footers carry over. +pyelftools [parses ELF files and DWARF debug info](https://github.com/eliben/pyelftools) in pure Python. Pass an open file to [`ELFFile`](https://github.com/eliben/pyelftools/blob/main/doc/user-guide.md), its main entry point, and stay on the high-level API: its guide says to use the low-level structs at your own peril. -openpyxl reads and writes Excel files. For big workbooks, open them with `read_only=True` or create them with `write_only=True`, which [keep memory near constant](https://openpyxl.readthedocs.io/en/stable/optimized.html). A cell with a formula loads the formula; pass [`data_only=True`](https://openpyxl.readthedocs.io/en/stable/tutorial.html) to get the value Excel last calculated. +Tablib is a [format-agnostic tabular dataset library](https://tablib.readthedocs.io/en/stable/). `Dataset().load(fh)` [detects the format of the file](https://tablib.readthedocs.io/en/stable/tutorial.html) you load, and `export()` writes the same data as CSV, JSON, or another format. -XlsxWriter [only writes new files](https://xlsxwriter.readthedocs.io/introduction.html) and can't read or modify existing ones, but it supports more Excel features than the alternatives. Open the workbook [in a `with` block](https://xlsxwriter.readthedocs.io/workbook.html) so it gets closed and saved. For large files, turn on [`constant_memory`](https://xlsxwriter.readthedocs.io/working_with_memory.html) and write the rows in order. From pandas, pass [`engine='xlsxwriter'`](https://xlsxwriter.readthedocs.io/working_with_pandas.html) to `pd.ExcelWriter`. +MarkItDown converts PDF, Word, Excel, PowerPoint, HTML, and more into Markdown [for LLMs and text analysis](https://github.com/microsoft/markitdown). Its README says it may not be the best option for high-fidelity conversions that people read. Don't pass untrusted input straight to it, and when you only read local files, [call `convert_local()` instead of `convert()`](https://github.com/microsoft/markitdown#security-considerations), which also fetches URLs. -MarkItDown converts files to Markdown [for LLMs and text analysis](https://github.com/microsoft/markitdown), not for high-fidelity conversions that people read. Install `markitdown[all]`, or only the extras for your formats, like `markitdown[pdf, docx, pptx]`. +Docling is for documents where layout matters: it [understands page layout, reading order, and tables, and runs OCR](https://docling-project.github.io/docling/) on scanned PDFs. It converts each file into a `DoclingDocument`, which you export to Markdown. -Docling [understands PDF layout](https://docling-project.github.io/docling/): reading order, tables, and formulas. It also runs OCR on scanned pages. Its models [run locally and send no data out](https://docling-project.github.io/docling/usage/advanced_options/#using-remote-services) unless you turn remote services on. Each file becomes a DoclingDocument, which you export to Markdown or [split into chunks](https://docling-project.github.io/docling/concepts/chunking/) for an embedding model. +openpyxl reads and writes Excel workbooks, so it's the one to use when the file already exists. Install defusedxml when you open spreadsheets from other people, since openpyxl [doesn't guard against XML attacks](https://openpyxl.readthedocs.io/en/stable/#security) by default. -python-pptx builds decks from data, like a database query or analytics output, and [doesn't need PowerPoint installed](https://python-pptx.readthedocs.io/en/latest/). It [only edits existing presentations](https://python-pptx.readthedocs.io/en/latest/user/presentations.html), so start from your own deck: its theme, slide master, and slide layouts set how the slides look. Add each slide from one of those layouts, picked by [its index in your deck](https://python-pptx.readthedocs.io/en/latest/user/slides.html). +XlsxWriter [can't read or modify existing files](https://xlsxwriter.readthedocs.io/introduction.html). In return, it supports more Excel features than the alternatives, and its files are close to what Excel itself writes. -ReportLab draws new PDFs from Python code. Learn it on [`pdfgen`, its lowest-level interface](https://docs.reportlab.com/developerfaqs/), then build multi-page documents with [Platypus](https://docs.reportlab.com/reportlab/userguide/ch5_platypus/). Platypus lets you keep paragraph styles and page layouts in one shared file, so restyling takes a few lines. +python-docx works on `.docx` files, not the older `.doc`. [Start from a document that holds your styles](https://python-docx.readthedocs.io/en/latest/user/documents.html), headers, and footers, and open it with `Document('template.docx')`: python-docx can only use a style that [the starting document defines](https://python-docx.readthedocs.io/en/latest/user/styles-understanding.html). -pdfminer.six [focuses on text](https://github.com/pdfminer/pdfminer.six): it gets each piece of text with its exact location, font, and color. Start with `extract_text()` from its [high-level API](https://pdfminersix.readthedocs.io/en/latest/tutorial/highlevel.html). A PDF [stores only characters and their positions](https://pdfminersix.readthedocs.io/en/latest/topic/converting_pdf_to_text.html), so pdfminer.six guesses words, lines, and paragraphs from the layout. Tune those guesses with `LAParams`. +python-pptx [creates and updates PowerPoint files](https://python-pptx.readthedocs.io/en/latest/user/intro.html). A deck's look comes from its theme, slide master, and layouts, so [open a template deck](https://python-pptx.readthedocs.io/en/latest/user/presentations.html) with `Presentation('template.pptx')`. Add slides from its layouts and fill their [placeholders](https://python-pptx.readthedocs.io/en/latest/user/placeholders-understanding.html), which carry the template's formatting. -WeasyPrint turns HTML and CSS into PDFs, like [reports, invoices, and tickets](https://doc.courtbouillon.org/weasyprint/stable/). It runs [no JavaScript](https://doc.courtbouillon.org/weasyprint/stable/going_further.html). Set page size and margins [with the CSS `@page` rule](https://doc.courtbouillon.org/weasyprint/stable/common_use_cases.html). +PyMuPDF is a [high-performance library](https://pymupdf.readthedocs.io/en/latest/) for extracting, converting, and editing PDFs and other documents, built on MuPDF, and it also renders pages to images. It's [available under the AGPL or a commercial license](https://pymupdf.readthedocs.io/en/latest/about.html#license-and-copyright). For reading text and editing pages without that condition, use pypdf or pdfminer.six. -markdown-it-py [follows the CommonMark spec](https://markdown-it-py.readthedocs.io/en/latest/) and takes plugins for more syntax. For content your users submit, use the [`js-default` preset](https://markdown-it-py.readthedocs.io/en/latest/security.html), since the default settings aren't safe for it. +ReportLab [creates PDFs directly from Python](https://docs.reportlab.com/reportlab/userguide/ch1_intro/), like reports generated on a web server, with charts and tables. Lay out documents with Platypus: [create a document from a `DocTemplate` class and pass a list of flowables](https://docs.reportlab.com/reportlab/userguide/ch5_platypus/), such as paragraphs and tables, to its `build()` method. -Python-Markdown [isn't a CommonMark implementation](https://python-markdown.github.io/): it follows the original Markdown syntax and has an extension API. It [doesn't sanitize its HTML output](https://python-markdown.github.io/sanitization/), so sanitize it yourself when the input is untrusted. +pdfminer.six [extracts text from a PDF's source](https://github.com/pdfminer/pdfminer.six), with the exact location, font, and color of each piece, but it can't write PDFs. The simplest start is [`extract_text()`](https://pdfminersix.readthedocs.io/en/latest/tutorial/highlevel.html). -Mistune is [fast and has no dependencies](https://mistune.lepture.com/en/latest/). For untrusted input, build the parser with [`mistune.create_markdown()`](https://mistune.lepture.com/en/latest/guide.html), which escapes HTML tags, since `mistune.html()` doesn't. +WeasyPrint turns HTML and CSS into PDFs, like [reports, invoices, and tickets](https://doc.courtbouillon.org/weasyprint/stable/), with a layout engine built for pagination instead of a browser engine. It runs [no JavaScript](https://doc.courtbouillon.org/weasyprint/stable/going_further.html). Call `HTML(...).write_pdf()`. If users supply the HTML or CSS, its docs say you'll need [extra configuration](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#security) against high memory use, endless renderings, and local file leaks. -tomllib [only reads TOML](https://docs.python.org/3/library/tomllib.html), from a file opened in binary mode. +markdown-it-py [follows the CommonMark spec](https://markdown-it-py.readthedocs.io/en/latest/), and plugins add syntax or replace existing rules. -Tablib holds one dataset and exports it to many formats; Excel, YAML, and pandas [are optional extras](https://tablib.readthedocs.io/en/stable/formats.html), like `tablib[xlsx]`. +Python-Markdown implements John Gruber's original Markdown. It's [not a CommonMark implementation](https://python-markdown.github.io/), and its docs say to look elsewhere if you want one. Pick it for its extension API and bundled [extensions](https://python-markdown.github.io/extensions/), like tables, fenced code blocks, and a table of contents. -pyelftools is [pure Python with no dependencies](https://github.com/eliben/pyelftools); start from its [`ELFFile` class](https://github.com/eliben/pyelftools/blob/main/doc/user-guide.md) and stay on the high-level API. +Mistune is a [fast parser with renderers and plugins](https://mistune.lepture.com/en/latest/), compatible with "sane CommonMark rules". -Treat every file you didn't create as untrusted. With PyYAML, call [`yaml.safe_load()`](https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml), never `yaml.load()`, which can run any Python function. Install [defusedxml](https://openpyxl.readthedocs.io/en/stable/#security) next to openpyxl to guard against XML attacks like billion laughs. [Catch pypdf's exceptions](https://pypdf.readthedocs.io/en/stable/user/security.html) yourself, so a broken PDF can't crash your service. For MarkItDown, call [`convert_local()` or `convert_stream()`](https://github.com/microsoft/markitdown#security-considerations) instead of `convert()`, which also fetches remote URIs. Cap Docling's input with [`max_num_pages` and `max_file_size`](https://docling-project.github.io/docling/usage/advanced_options/#impose-limits-on-the-document-size). Run WeasyPrint on untrusted HTML [as a user with limited access](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#security), with a URL fetcher that blocks local files. With tomllib, [limit the size](https://docs.python.org/3/library/tomllib.html) of the data you parse. +tomllib comes with Python and parses TOML, but doesn't write it. With untrusted input, [limit its size](https://docs.python.org/3/library/tomllib.html), since a malicious TOML string can use a lot of CPU and memory. + +PyYAML parses and emits YAML. Load it with `yaml.safe_load()`, since [`yaml.load` can call any Python function](https://pyyaml.org/wiki/PyYAMLDocumentation) when the data comes from an untrusted source. + +Markdown your users write can carry HTML, so give each parser its safe setting. With markdown-it-py, use the [`js-default` preset](https://markdown-it-py.readthedocs.io/en/latest/security.html), which its docs strongly recommend for user content in web apps. Python-Markdown [doesn't sanitize its output](https://python-markdown.github.io/sanitization/), so run a sanitizer over the HTML. With Mistune, build your parser with [`mistune.create_markdown()`](https://mistune.lepture.com/en/latest/guide.html), which escapes HTML, instead of `mistune.html()`, which doesn't.