This document describes CPython's bytecode instruction format and the opcode definition system. It covers the structure of bytecode instructions, how opcodes are defined in the source, and the code generation process that produces execution handlers for both Tier 1 (interpreted bytecode) and Tier 2 (optimized Micro-Ops or UOps).
For information about the compilation process that produces bytecode, see AST, Compiler, and Bytecode Assembly. For information about bytecode execution, see Bytecode Interpreter and Evaluation Loop. For optimizer details, see Tier 2 Optimizer and JIT Compilation.
CPython bytecode consists of a sequence of instructions, each encoded as a _Py_CODEUNIT. The fundamental unit is 16 bits (2 bytes), containing an 8-bit opcode and an 8-bit argument:
Title: Bytecode Unit Structure
Instruction Structure:
For arguments larger than 8 bits, the EXTENDED_ARG opcode is used. Multiple EXTENDED_ARG instructions can precede an instruction to build up arbitrarily large arguments by shifting and combining the values. Python/specialize.c108-111
Key Properties:
next_instr) advances by the instruction length, which includes the opcode unit plus any inline cache entries. Python/generated_cases.c.h27Sources: Python/bytecodes.c1-100 Include/internal/pycore_opcode_metadata.h1-50 Python/generated_cases.c.h21-30
The file Python/bytecodes.c serves as the single source of truth for all instruction definitions in CPython. It uses a domain-specific language (DSL) that is processed by code generators to produce multiple outputs.
Title: Opcode Definition Data Flow
DSL Constructs in bytecodes.c:
| Construct | Purpose | Example |
|---|---|---|
inst(name, ...) | Define a complete Tier 1 instruction | inst(NOP, (--)) Python/bytecodes.c148 |
op(name, ...) | Define a reusable micro-operation component | op(_CHECK_PERIODIC, (--)) Python/bytecodes.c158 |
macro(name) | Combine ops into a single bytecode | macro(NOT_TAKEN) = NOP; Python/bytecodes.c156 |
family(name, ...) | Define a specialization family | family(RESUME, 1) = {...}; Python/bytecodes.c151 |
pseudo(name) | Define pseudo-instruction (compiler only) | pseudo(LOAD_CLOSURE) Include/internal/pycore_opcode_metadata.h21 |
Annotations:
pure: Instruction has no side effects. Python/bytecodes.c148guard: Used for type guards and specialization checks. Python/bytecodes.c62specializing: Indicates an instruction that can trigger specialization. Python/bytecodes.c64tier1: Logic restricted to the Tier 1 interpreter. Python/bytecodes.c175Sources: Python/bytecodes.c1-68 Python/bytecodes.c148-200 Include/internal/pycore_opcode_metadata.h20-32
The code generation process transforms bytecodes.c into several critical header files used across the runtime.
Generated File Purposes:
| File | Purpose | Used By |
|---|---|---|
generated_cases.c.h | Tier 1 bytecode handler implementations for the evaluation loop. | Python/ceval.c Python/generated_cases.c.h1-20 |
executor_cases.c.h | Tier 2 UOp execution handlers with register caching. | Python/executor.c Python/executor_cases.c.h1-50 |
optimizer_cases.c.h | Logic for symbolic execution during optimization. | Python/optimizer.c Python/optimizer_cases.c.h1-40 |
pycore_opcode_metadata.h | Metadata such as stack effects (_PyOpcode_num_popped). | Compiler & Optimizer Include/internal/pycore_opcode_metadata.h35-40 |
pycore_uop_metadata.h | UOp-specific flags and replication ranges. | Tier 2 Runtime Include/internal/pycore_uop_metadata.h1-20 |
pycore_uop_ids.h | Constant definitions for Micro-Op IDs. | Optimizer Include/internal/pycore_uop_ids.h1-50 |
Sources: Python/generated_cases.c.h1-10 Python/executor_cases.c.h1-10 Python/optimizer_cases.c.h1-10 Include/internal/pycore_opcode_metadata.h1-10
CPython uses an adaptive interpreter that replaces generic opcodes with specialized versions after a "warmup" period.
BINARY_OP Python/generated_cases.c.h21BINARY_OP_ADD_FLOAT, BINARY_OP_ADD_INT Python/generated_cases.c.h95-170Specialized instructions often include Inline Cache Entries. For example, BINARY_OP reserves 5 cache entries to store specialization data. Python/generated_cases.c.h105-180
UOps are the primitive operations used in Tier 2 execution traces. They are more granular than Tier 1 bytecodes.
Title: Tier 1 to Tier 2 Mapping (Example: BINARY_OP)
_GUARD_TOS_INT, _GUARD_TYPE_VERSION. Include/internal/pycore_uop_ids.h199-206_LOAD_FAST, _STORE_ATTR_INSTANCE_VALUE. Python/optimizer_bytecodes.c120-137_BINARY_OP_ADD_INT, _CALL_BUILTIN_FAST. Include/internal/pycore_uop_ids.h18-48Sources: Python/generated_cases.c.h95-180 Include/internal/pycore_uop_ids.h1-210 Python/optimizer_bytecodes.c111-150
The runtime must know the "stack effect" of every opcode (how many items it pops and pushes) to maintain the evaluation stack.
Stack Effect Metadata:
The function _PyOpcode_num_popped(opcode, oparg) returns the number of items an instruction removes from the stack. Include/internal/pycore_opcode_metadata.h35-37
| Opcode | Stack Popped |
|---|---|
BINARY_OP | 2 Include/internal/pycore_opcode_metadata.h42 |
BINARY_SLICE | 3 Include/internal/pycore_opcode_metadata.h76 |
CALL | 2 + oparg Include/internal/pycore_opcode_metadata.h96 |
BUILD_LIST | oparg Include/internal/pycore_opcode_metadata.h80 |
UOp Flags:
UOps carry flags defining their properties, such as HAS_PURE_FLAG (no side effects) or HAS_ERROR_FLAG (can raise exceptions). Include/internal/pycore_uop_metadata.h38-42
Sources: Include/internal/pycore_opcode_metadata.h35-150 Include/internal/pycore_uop_metadata.h15-100
Unlike Tier 1, which interacts directly with the stack_pointer, Tier 2 execution handlers in executor_cases.c.h utilize a register caching mechanism. This minimizes memory traffic by keeping the top elements of the stack in CPU registers (represented as C local variables like _tos_cache0). Python/executor_cases.c.h22-53
CHECK_CURRENT_CACHED_VALUES(n): Validates the number of items currently in the register cache. Python/executor_cases.c.h12SET_CURRENT_CACHED_VALUES(n): Updates the cache state after an operation. Python/executor_cases.c.h14Sources: Python/executor_cases.c.h11-70 Include/internal/pycore_uop_metadata.h22-34
Python/bytecodes.c: DSL source for all opcodes.generated_cases.c.h: The generated switch-case body for the Tier 1 interpreter.executor_cases.c.h: The generated switch-case body for Tier 2 executors.pycore_opcode_metadata.h: Static properties of bytecodes (stack depth, deoptimization targets)._PyOpcode_Deopt[opcode]: Array mapping specialized opcodes back to their generic base for deoptimization. Python/generated_cases.c.h118-128Sources: Python/bytecodes.c1-7 Python/generated_cases.c.h1-20 Python/executor_cases.c.h1-20 Include/internal/pycore_opcode_metadata.h1-40
Refresh this wiki
This wiki was recently refreshed. Please wait 4 days to refresh again.