➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pablogsal
Recipients Andrew.C, Anthony Sottile, Jim Fasarakis-Hilliard, amaury.forgeotdarc, berker.peksag, djmitche, effbot, kirkshorts, meador.inge, pablogsal, serhiy.storchaka, superluser
Date 2021-01-27.20:58:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <[email protected]>
In-reply-to
Content 
I have built a draft of how the changes required to make what you describe, in case you want to finish them:

https://github.com/pablogsal/cpython/tree/tokenizer_mod
History
Date User Action Args
2021-01-27 20:58:53pablogsalsetrecipients: + pablogsal, effbot, amaury.forgeotdarc, djmitche, kirkshorts, meador.inge, berker.peksag, serhiy.storchaka, superluser, Andrew.C, Anthony Sottile, Jim Fasarakis-Hilliard
2021-01-27 20:58:53pablogsalsetmessageid: <[email protected]>
2021-01-27 20:58:53pablogsallinkissue3353 messages
2021-01-27 20:58:53pablogsalcreate