This page documents CPython's Sphinx-based documentation build system, which generates comprehensive documentation from reStructuredText sources and produces multiple output formats including HTML, PDF, EPUB, and runtime help data. For information about the build infrastructure that compiles CPython itself, see 7.1 Configuration and Build Process For CI/CD documentation testing, see 7.2 CI/CD Infrastructure
The documentation system transforms reStructuredText (.rst) source files located in the Doc/ directory into multiple output formats:
pydoc and help() system, specifically topics.py and module_docs.pyThe system is built on Sphinx (pinned to version < 9.0.0) and extended with custom directives and builders specific to Python's documentation needs.
Sources: Doc/conf.py1-110 Doc/Makefile1-51 Doc/requirements.txt6-10
The build process is orchestrated by Doc/Makefile and configured via Doc/conf.py. It integrates source text, version metadata from the C headers, and news entries from the blurb tool.
Documentation Build Pipeline Diagram The build pipeline processes reStructuredText sources through Sphinx with custom extensions to produce multiple output formats and runtime help data.
Sources: Doc/conf.py14-42 Doc/Makefile52-76 Doc/tools/extensions/pydoc_topics.py106-189
The main configuration file Doc/conf.py controls all aspects of the documentation build. It is executed as Python code during the Sphinx initialization phase.
CPython uses a mix of built-in Sphinx extensions and bespoke internal tools.
| Extension Name | File Path | Role |
|---|---|---|
pyspecific | Doc/tools/extensions/pyspecific.py | Python-specific markup (opcodes, pdb commands) |
pydoc_topics | Doc/tools/extensions/pydoc_topics.py | Generates data for the pydoc module |
c_annotations | Doc/tools/extensions/c_annotations.py | Adds refcount and stable ABI info to C API docs |
availability | Doc/tools/extensions/availability.py | Tracks platform availability (e.g., POSIX, Windows) |
audit_events | Doc/tools/extensions/audit_events.py | Documents sys.audit events |
Sources: Doc/conf.py24-42 Doc/tools/extensions/pyspecific.py1-129
The c_annotations.py extension enriches the C API documentation by reading external data files and injecting nodes into the Sphinx doctree.
Doc/data/refcounts.dat (via read_refcount_data) to annotate function return values (e.g., "Return value: New reference").Doc/data/threadsafety.dat (via read_threadsafety_data) to mark functions as atomic, shared, or compatible.Misc/stable_abi.toml to identify functions that are part of the Limited API.Sources: Doc/tools/extensions/c_annotations.py81-155 Doc/tools/extensions/c_annotations.py158-204 Doc/data/threadsafety.dat1-15
One of the most critical outputs of the documentation system is the data used by the interactive help() function in the Python REPL.
The PydocTopicsBuilder (defined in pydoc_topics.py) is a custom Sphinx builder that extracts specific sections of the documentation based on labels defined in _PYDOC_TOPIC_LABELS.
Pydoc Data Generation Diagram
Associates the PydocTopicsBuilder logic with the generation of topics.py and module_docs.py.
Sources: Doc/tools/extensions/pydoc_topics.py106-112 Doc/tools/extensions/pydoc_topics.py118-158 Doc/tools/extensions/pydoc_topics.py159-189
The Lib/pydoc.py module consumes the generated data. Key functions include:
getdoc(object): Retrieves the docstring or comments for an object. Lib/pydoc.py137-141_getargspec(object): Uses inspect.signature to format function signatures for the help display. Lib/pydoc.py151-170render_doc(thing, ...): The core engine that produces the text or HTML representation of the help page. Lib/pydoc.py1750-1810 (inferred from general pydoc structure).The generated Lib/pydoc_data/topics.py contains a dictionary topics where keys are keywords (like 'assert') and values are raw strings of the documentation for that keyword.
Sources: Lib/pydoc_data/topics.py4-35 Lib/pydoc.py132-171
The Doc/Makefile defines the interface for developers to build and test documentation.
| Target | Command | Purpose |
|---|---|---|
html | sphinx-build -b html | Builds standard HTML documentation |
pydoc-topics | sphinx-build -b pydoc-topics | Regenerates topics.py and module_docs.py |
doctest | sphinx-build -b doctest | Runs all code snippets in the docs to ensure they work |
linkcheck | sphinx-build -b linkcheck | Verifies all external URLs are reachable |
venv | python3 -m venv venv | Sets up a local environment with Sphinx and dependencies |
Sources: Doc/Makefile79-146 Doc/Makefile169-186
The pyspecific extension provides directives that bridge the gap between documentation and CPython internals:
opcode directive uses parse_opcode_signature to document bytecode instructions. Doc/tools/extensions/pyspecific.py46-62pdbcommand directive documents debugger commands. Doc/tools/extensions/pyspecific.py65-88PyAwaitableMixin adds an "awaitable" prefix to function/method signatures in the documentation. Doc/tools/extensions/pyspecific.py27-44Sources: Doc/tools/extensions/pyspecific.py121-127
Documentation quality is maintained through several automated checks:
nitpick_ignore. Doc/conf.py124-227Lib/test/test_pydoc/test_pydoc.py script performs extensive testing on the help system, verifying that the generated text matches expected patterns. Lib/test/test_pydoc/test_pydoc.py54-144Sources: Doc/conf.py231-260 Lib/test/test_pydoc/test_pydoc.py1-40
Refresh this wiki
This wiki was recently refreshed. Please wait 5 days to refresh again.