[RFC] What are the compatibility goals for RustPython? #1940
Description
Summary
As a long-time Pythonista and short-time Rustacean I of course searched for this project and was happy to find it existed. Looks like a lot of fun, and if I get the itch I'll definitely try to contribute (with the caveat that like most of you, I won't be able to dedicate a ton of time).
I was wondering about the long-term vision for RustPython in terms of CPython interop (if not necessarily parity). I think this question is the core of issues like #1849, #1815, #1690, #1204 and #1192, and having read those discussions it seems like it's an ongoing discussion.
Is RustPython a replacement for CPython, or a new interpreter to enable Python in new places (e.g. the web) but doesn't try to be compatible?
I think this is the big question, and answering it would define a roadmap for the future. The two answers define two different projects that are both exciting for different ways:
| Goal | Pros | Cons |
|---|---|---|
| CPython replacement | Greenfield implementation of core code | Faithfully recreating stdlib is complicated |
| Rust is simpler, less buggy than C | Tons of complex dynamic behavior to support | |
| Compatible with a huge ecosystem | Limited ability to change/optimize any syntax or code behavior | |
| Potential for major impact/user uptake | Potential for huge support surface | |
| Novel interpreter / Wasm compiler | Can selectively implement libraries/syntax | More difficult for new users, less "pythonic" |
| Can optimize the interpreter (like PyPy) | Libraries that aren't pure Python require active support or won't work | |
| Maybe more exciting to work on, for some? | Much less potential for broad impact IMO |
To me, RustPython is a cool idea in large part because Rust has good compatibility with C, and so RustPython could (maybe) work with the huge number of libraries that mix C with Python†. Most of the heavy hitters in the CPython space fall into this category–the whole data stack, for instance. I think e.g. PyPy suffers a lot from not being trivially compatible with such popular projects, even though it has lots of other advantages. Going this route means supporting all the complex meta-programming, introspection, etc that those projects rely on. That might be fun, if the goal is to learn about interpreters!
This approach doesn't mean not compiling to Wasm or anything like that. But if the goal is primarily web-native Python, I think that changes priorities and the direction of the project.
† not to mention Cython, C++, Fortran, etc: if it works with CPython then it could work with RustPython.
On the other hand, if the idea is to build a Python-to-wasm compiler/interpreter (or something like that), then CPython compatibility is a lot less important. This is more like Łukasz Langa 's proposal: you can discard weird code behavior and syntax, optimize how the language works (e.g. #1690, #1831), etc. That path is also super exciting to work on but is probably going to remain a toy project for a long time because you won't support very much outside of the stdlib. As Łukasz points out, a lot of defunct projects started out that way, and even PyPy is still a fairly niche project.
Drawbacks, Rationale, and Alternatives
There are pluses and minuses to both approaches. Both versions need to implement stdlib one way or another, so this decision doesn't immediately affect the direction of the project, but I think it's useful to discuss (and I was curious).
A somewhat "middle-road" answer would be to target compatibility with Python 3.10 or something, in the sense that the CPython roadmap defines some simplifications of the language. e.g. PEP 594 which removes some libraries from the stdlib (and you could be even more aggressive, as mentioned in #1204 and this post), and PEP 611 which defines some internal limits. I could imagine RustPython as a place that provides almost-complete compatibility with CPython but exposes more of the future, both in terms of new features and in terms of removing obsolete things.
Unresolved Questions
I don't think this is a decision that needs to be made, but having a general consensus among the core developers would help clarify what this project is for and give newcomers a sense of what to work on.
This turned into more of a screed than I intended, but I think it's a super interesting project. As someone more on the data science side of Python, I'm always yearning for a faster, cleaner core runtime that I could actually help contribute to. So I'm pretty firmly on the side of Option A, but that's just my perspective. Curious to hear your thoughts.