Fix dict unpacking order and improve error messages#7085
Fix dict unpacking order and improve error messages#7085youknowone merged 6 commits intoRustPython:mainfrom
Conversation
📝 WalkthroughWalkthroughRefactors dict construction and star-unpack compilation paths in the codegen, tightens runtime error messages for starred/mapping/unpack operations, and adds a function-descriptor helper used across frame and builtin error reporting. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
Client->>Compiler: compile source with dict/** and *args usage
note right of Compiler: emits bytecode sequences\n(BuildMap, DictUpdate, CallFunctionEx)
Compiler->>VM: load bytecode
Client->>VM: invoke function (runtime)
VM/Frame->>VM/Frame: object_function_str(func) -> func_str
VM/Frame->>Runtime: handle DictUpdate / iterate_mapping_keys using func_str
alt mapping is invalid
Runtime->>Client: raise TypeError("X argument after ** must be a mapping, not Y")
else star-unpack single tuple-like
VM->>Runtime: CallFunctionEx with compiled starred value
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
No actionable comments were generated in the recent review. 🎉 🧹 Recent nitpick comments
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
b0eaa37 to
6aa01a4
Compare
- Rewrite compile_dict to preserve insertion order with ** unpacking - Fix function call error messages to include function name and keyword-only argument counts - Fix missing keyword-only argument error to list all missing args - Fix starred expression error messages to match CPython - Fix "cannot unpack non-iterable" error message - Remove resolved expected failure markers in test_extcall and test_unpack_ex
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@crates/vm/src/frame.rs`:
- Around line 2806-2815: The current error-rewrite around value.try_to_value(vm)
replaces any TypeError with "cannot unpack non-iterable ..." and therefore can
hide real TypeErrors raised by a custom __iter__; change the condition so the
rewrite only happens when the object truly lacks an __iter__ implementation
(mirror the guard used in ListExtend and collect_ex_args). Concretely, inside
the map_err closure around value.try_to_value(vm), keep the existing check of
e.class().is(vm.ctx.exceptions.type_error) but add a second check that the
value's type does not provide __iter__ (e.g., via the same API used elsewhere
such as checking value.class().has_attr("__iter__") or using
vm.get_attribute_opt(value, "__iter__") == None); only then return
vm.new_type_error(...), otherwise return the original error e. Reference
symbols: elements (Vec<_>), value.try_to_value, vm.new_type_error, and the
analogous guards in ListExtend and collect_ex_args.
🧹 Nitpick comments (2)
crates/codegen/src/compile.rs (1)
6649-6707: Deduplicate pending-pair flush logic.The BuildMap/DictUpdate sequence is repeated in multiple branches. Consider extracting the common logic so the differing part (whether a dict already exists) is handled once.
♻️ Suggested refactor
fn compile_dict(&mut self, items: &[ast::DictItem]) -> CompileResult<()> { let has_unpacking = items.iter().any(|item| item.key.is_none()); if !has_unpacking { // Simple case: no ** unpacking, build all pairs directly for item in items { self.compile_expression(item.key.as_ref().unwrap())?; self.compile_expression(&item.value)?; } emit!( self, Instruction::BuildMap { size: u32::try_from(items.len()).expect("too many dict items"), } ); return Ok(()); } + fn flush_pending( + compiler: &mut Compiler, + elements: u32, + have_dict: &mut bool, + ) { + if elements == 0 { + return; + } + emit!(compiler, Instruction::BuildMap { size: elements }); + if *have_dict { + emit!(compiler, Instruction::DictUpdate { index: 1 }); + } else { + *have_dict = true; + } + } + // Complex case with ** unpacking: preserve insertion order. // Collect runs of regular k:v pairs and emit BUILD_MAP + DICT_UPDATE // for each run, and DICT_UPDATE for each ** entry. let mut have_dict = false; let mut elements: u32 = 0; for item in items { if let Some(key) = &item.key { // Regular key: value pair self.compile_expression(key)?; self.compile_expression(&item.value)?; elements += 1; } else { // ** unpacking entry - if elements > 0 { - // Flush pending regular pairs - if !have_dict { - emit!(self, Instruction::BuildMap { size: elements }); - have_dict = true; - } else { - emit!(self, Instruction::BuildMap { size: elements }); - emit!(self, Instruction::DictUpdate { index: 1 }); - } - elements = 0; - } + flush_pending(self, elements, &mut have_dict); + elements = 0; if !have_dict { emit!(self, Instruction::BuildMap { size: 0 }); have_dict = true; } self.compile_expression(&item.value)?; emit!(self, Instruction::DictUpdate { index: 1 }); } } - if elements > 0 || !have_dict { - if !have_dict { - emit!(self, Instruction::BuildMap { size: elements }); - } else { - emit!(self, Instruction::BuildMap { size: elements }); - emit!(self, Instruction::DictUpdate { index: 1 }); - } - } + flush_pending(self, elements, &mut have_dict); + if !have_dict { + emit!(self, Instruction::BuildMap { size: 0 }); + } Ok(()) }As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."
crates/vm/src/builtins/function.rs (1)
395-444: Duplicate formatting logic for missing-argument error messages.Lines 417–444 replicate the conjunction-formatting pattern from lines 349–376 almost verbatim (pop last, choose
"' and '"vs"', and '", join remaining, format). Consider extracting a small helper (e.g.,fn format_missing_args(kind: &str, qualname: &str, names: &mut Vec<…>) -> String) and calling it from both sites.♻️ Sketch
// Somewhere accessible to both call sites: fn format_missing_args_error( qualname: &str, kind: &str, // "positional" or "keyword-only" missing: &mut Vec<&impl AsRef<str>>, ) -> String { let missing_len = missing.len(); let last = if missing.len() > 1 { missing.pop() } else { None }; let (and, right) = if let Some(last) = last { ( if missing.len() == 1 { "' and '" } else { "', and '" }, last.as_ref(), ) } else { ("", "") }; format!( "{}() missing {} required {} argument{}: '{}{}{}'", qualname, missing_len, kind, if missing_len == 1 { "" } else { "s" }, missing.iter().map(|m| m.as_ref()).join("', '"), and, right, ) }As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code".
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [ ] test: cpython/Lib/test/test_extcall.py (TODO: 8) dependencies: dependent tests: (no tests depend on extcall) [x] lib: cpython/Lib/inspect.py dependencies:
dependent tests: (42 tests)
[x] lib: cpython/Lib/pickle.py dependencies:
dependent tests: (72 tests)
[ ] test: cpython/Lib/test/test_positional_only_arg.py (TODO: 5) dependencies: dependent tests: (no tests depend on positional_only_arg) [ ] test: cpython/Lib/test/test_unpack.py dependencies: dependent tests: (no tests depend on unpack) Legend:
|
Summary by CodeRabbit