Skip to content

marshal: allow runtime bags to preserve raw code bytes - #8552

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowone:marshal-invalid-code
Aug 18, 2026
Merged

marshal: allow runtime bags to preserve raw code bytes#8552
youknowone merged 1 commit into
RustPython:mainfrom
youknowone:marshal-invalid-code

Conversation

@youknowone

@youknowone youknowone commented Aug 18, 2026

Copy link
Copy Markdown
Member

Summary

  • add a MarshalBag::code_units_from_bytes hook for runtime-specific decoding of marshalled co_code
  • add a make_code_with_constants_and_bytes hook so a runtime can retain the exact public byte string
  • preserve the existing strict CodeUnits behavior by default

Motivation

Runtime code objects may accept raw opcode bytes that are not representable by the compiler-core Instruction enum. CPython and PyPy allow such code objects to round-trip through marshal; execution rejects the unknown opcode later. The current shared deserializer rejects these bytes before a runtime bag can preserve them.

These defaulted hooks keep compiler-core behavior unchanged while allowing runtimes with a raw-byte fallback to implement that behavior without duplicating the marshal parser.

Validation

  • cargo fmt --all -- --check
  • cargo test -p rustpython-compiler-core

This is a draft while the downstream runtime integration is validated.

Summary by CodeRabbit

  • New Features
    • Added customizable handling for converting serialized code bytes into executable code units.
    • Added a construction hook that can access both decoded constants and the original code bytes during deserialization.
    • Preserved existing default behavior for standard runtime code loading.

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 425e958d-26bc-4533-b7d7-f954c2a5ecba

📥 Commits

Reviewing files that changed from the base of the PR and between 67cf607 and 3964ce0.

📒 Files selected for processing (1)
  • crates/compiler-core/src/marshal.rs

📝 Walkthrough

Walkthrough

MarshalBag now supports custom conversion of marshal bytecode and custom code construction with access to the original code bytes. Default implementations preserve the existing strict conversion and construction behavior.

Changes

Marshal deserialization hooks

Layer / File(s) Summary
Add and invoke marshal hooks
crates/compiler-core/src/marshal.rs
MarshalBag adds default hooks for bytecode conversion and code construction. Runtime deserialization uses these hooks and forwards the original co_code bytes.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: shaharnaveh

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone
youknowone marked this pull request as ready for review August 18, 2026 13:21
@youknowone
youknowone merged commit 189bc97 into RustPython:main Aug 18, 2026
27 of 28 checks passed
@youknowone
youknowone deleted the marshal-invalid-code branch August 18, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant