Bytecode parity: align borrow optimization with CFG traversal

[youknowone]

Summary

Align LOAD_FAST_BORROW optimization with CPython-style CFG traversal instead of post-processing bytecode after borrow optimization.

Details

• Marks assertion-success and unprotected debug>=4 tails before optimize_load_fast_borrow, so those blocks are excluded during traversal rather than rewritten afterward.
• Treats exception handlers that POP_EXCEPT and then return/continue normally as resuming handlers, avoiding over-deoptimization after conditional reraises.
• Narrows async-finally early-return borrow deoptimization so async-for GET_ANEXT/SEND iterator paths are not mistaken for relevant async-finally sends.
• Adds regression coverage for imaplib.Idler.__exit__-style debug tails and the test_compile.py async comprehension capture case.

Validation

• cargo fmt
• cargo test -p rustpython-codegen test_match_async_comprehension_iter_keeps_capture_borrow -- --nocapture
• cargo test -p rustpython-codegen test_imap_idle_status_debug_tail_keeps_borrow -- --nocapture
• cargo test -p rustpython-codegen deopts -- --nocapture
• cargo build --release
• python scripts/compare_bytecode.py --filter 'test/test_compile.py' --detail --max-diffs 300 -j 1 -o /tmp/compare_test_compile.after_rebase.report
• python scripts/compare_bytecode.py --filter 'imaplib.py' --detail --max-diffs 200 -j 1 -o /tmp/compare_imaplib.after_rebase.report

Summary by CodeRabbit

• Chores
  • Updated spell-check word allowlist configuration with additional correctly-spelled terms for enhanced recognition.
  • Improved symbol-table free-variable analysis within the compiler for more accurate scoping behavior in functions.
  • Enhanced development tooling with better temporary file creation and explicit UTF-8 encoding support.

[coderabbitai]

📝 Walkthrough

Walkthrough

Three independent changes: spell-check word additions, compiler symbol-table free-variable analysis refinement for cell scopes, and temporary file handling in bytecode comparison script.

Changes

Spell-Check Configuration

Layer / File(s)	Summary
Word Allowlist Update .cspell.json	Added deoptimized, fnfe, reborrow, reraises, and reraising to the spell-check allowlist.

Symbol-Table Free-Variable Analysis

Layer / File(s)	Summary
Analysis Predicate crates/codegen/src/symboltable.rs	Introduced remove_owned_cells_from_free predicate to enable cell-scope removal in function-like scopes (Function, AsyncFunction, Lambda, Comprehension, Annotation).
Cell Scope Filtering crates/codegen/src/symboltable.rs	During per-symbol analysis, symbols with scope == Cell are removed from the newfree set via shift_remove when the predicate is enabled.
Lambda Scope Finalization crates/codegen/src/symboltable.rs	After entering lambda scope with parameters, explicitly set the current scope's typ to CompilerScope::Lambda.

Bytecode Comparison Script

Layer / File(s)	Summary
Temporary File Creation scripts/compare_bytecode.py	Modified NamedTemporaryFile calls to remove dir=PROJECT_ROOT and add explicit prefix values while retaining UTF-8 encoding and delete=False behavior.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🎯 3 (Moderate) | ⏱️ ~25 minutes

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 Five words now pass the spell-check's gate,
Cell-scoped symbols find their proper fate,
Lambdas scope themselves with crystal care,
Temp files bloom with prefix in the air,
Three patches dance, yet none shall interfere!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Bytecode parity: align borrow optimization with CFG traversal' directly reflects the main objective of the PR, which is to realign the LOAD_FAST_BORROW optimization with CFG traversal instead of bytecode post-processing.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] test: cpython/Lib/test/test_coroutines.py (TODO: 18)

dependencies:

dependent tests: (7 tests)
- [ ] asyncio: test_asyncio test_contextlib_async test_inspect test_logging test_os test_sys_settrace test_unittest

[x] test: cpython/Lib/test/test_compile.py (TODO: 16)
[x] test: cpython/Lib/test/test_compiler_assemble.py
[x] test: cpython/Lib/test/test_compiler_codegen.py
[x] test: cpython/Lib/test/test_peepholer.py (TODO: 6)

dependencies:

dependent tests: (no tests depend on compile)

Legend:

• [+] path exists in CPython
• [x] up-to-date, [ ] outdated

[coderabbitai]

🧹 Nitpick comments (2)

crates/codegen/src/symboltable.rs (2)

594-643: 💤 Low value

Optional: deduplicate the function-like scope predicate.

remove_owned_cells_from_free (594–601) and promote_inlined_cells_to_cell (628–635) test the exact same set of CompilerScope variants. Extracting a small helper such as is_function_like_scope(typ) (or a single local bound once) would prevent the two lists from drifting if future scope variants are added (e.g. a new type-param-like scope that should — or shouldn't — be considered function-like for both passes).

♻️ Sketch

fn is_function_like(typ: CompilerScope) -> bool {
    matches!(
        typ,
        CompilerScope::Function
            | CompilerScope::AsyncFunction
            | CompilerScope::Lambda
            | CompilerScope::Comprehension
            | CompilerScope::Annotation
    )
}

Then reuse at both call sites.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/codegen/src/symboltable.rs` around lines 594 - 643, The two boolean
predicates remove_owned_cells_from_free and promote_inlined_cells_to_cell
duplicate the same matches(...) over symbol_table.typ; refactor by extracting a
small helper (e.g. is_function_like or is_function_like_scope) that accepts a
CompilerScope (or the symbol_table.typ) and returns whether it is one of
Function | AsyncFunction | Lambda | Comprehension | Annotation, then replace
both match expressions with calls to that helper (update the uses around
symbol_table.typ so remove_owned_cells_from_free and
promote_inlined_cells_to_cell are computed via
is_function_like(symbol_table.typ) or by binding the helper result once).

---

2131-2153: 💤 Low value

Lambda scope type set post-hoc — consider threading the scope type through the helper instead.

enter_scope_with_parameters always creates the scope as Function/AsyncFunction based on is_async, so the new line 2146 patches the typ back to Lambda only on the with-parameters path; the without-parameters branch (line 2148) already passes Lambda directly. Functionally this works because parameter registration doesn't observe typ in a way that differs between Function and Lambda, and the new cell-pruning logic at lines 594–601 already treats Lambda as function-like.

That said, the post-hoc mutation is easy to overlook for future readers. A cleaner long-term shape would be to pass the scope kind into enter_scope_with_parameters (e.g. an enum or a dedicated is_lambda: bool) so both branches construct the table with the final typ from the start. Optional in this PR.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/codegen/src/symboltable.rs` around lines 2131 - 2153, The Lambda
branch mutates the scope.kind after calling enter_scope_with_parameters because
that helper currently infers Function/AsyncFunction from is_async; modify
enter_scope_with_parameters to accept an explicit scope kind (e.g., a new enum
parameter like scope_kind or a boolean is_lambda) and use that when constructing
the new CompilerScope table so the created scope is Lambda from the start;
update the Expr::Lambda call site to pass the Lambda kind (and preserve existing
async handling for other callers), and adjust any internal logic in
enter_scope_with_parameters that previously derived typ from is_async to instead
use the provided scope kind.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/codegen/src/symboltable.rs`:
- Around line 594-643: The two boolean predicates remove_owned_cells_from_free
and promote_inlined_cells_to_cell duplicate the same matches(...) over
symbol_table.typ; refactor by extracting a small helper (e.g. is_function_like
or is_function_like_scope) that accepts a CompilerScope (or the
symbol_table.typ) and returns whether it is one of Function | AsyncFunction |
Lambda | Comprehension | Annotation, then replace both match expressions with
calls to that helper (update the uses around symbol_table.typ so
remove_owned_cells_from_free and promote_inlined_cells_to_cell are computed via
is_function_like(symbol_table.typ) or by binding the helper result once).
- Around line 2131-2153: The Lambda branch mutates the scope.kind after calling
enter_scope_with_parameters because that helper currently infers
Function/AsyncFunction from is_async; modify enter_scope_with_parameters to
accept an explicit scope kind (e.g., a new enum parameter like scope_kind or a
boolean is_lambda) and use that when constructing the new CompilerScope table so
the created scope is Lambda from the start; update the Expr::Lambda call site to
pass the Lambda kind (and preserve existing async handling for other callers),
and adjust any internal logic in enter_scope_with_parameters that previously
derived typ from is_async to instead use the provided scope kind.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: df128704-a615-42bb-b02d-227922125963

📥 Commits

Reviewing files that changed from the base of the PR and between 0325fd4 and e04ca46.

⛔ Files ignored due to path filters (2)

• Lib/test/test_coroutines.py is excluded by !Lib/**
• Lib/test/test_peepholer.py is excluded by !Lib/**

📒 Files selected for processing (5)

• .cspell.json
• crates/codegen/src/compile.rs
• crates/codegen/src/ir.rs
• crates/codegen/src/symboltable.rs
• scripts/compare_bytecode.py