Skip to content

bpo-25643: Refactor the C tokenizer#25050

Merged
pablogsal merged 6 commits intopython:masterfrom
pablogsal:better_token
Mar 28, 2021
Merged

bpo-25643: Refactor the C tokenizer#25050
pablogsal merged 6 commits intopython:masterfrom
pablogsal:better_token

Conversation

@pablogsal
Copy link
Member

@pablogsal pablogsal commented Mar 28, 2021

https://bugs.python.org/issue25643

Based on the original patch by Serhiy Storchaka

Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pablogsal, @serhiy-storchaka: I added some PEP 7 comments, some comments about deprecated memory functions, and some decref / free control questions. Hope you don't mind.

pablogsal and others added 3 commits March 28, 2021 22:28
@pablogsal
Copy link
Member Author

pablogsal commented Mar 28, 2021

@pablogsal, @serhiy-storchaka: I added some PEP 7 comments, some comments about deprecated memory functions, and some decref / free control questions. Hope you don't mind.

Thanks a lot for your comments and for the review. I have applied most of the PEP7 and made some changes to error returning paths so they are a bit more clear.

Regarding PEP7, normally we only adapt it when we are reworking things, but given that this change was quite big I had opted to not make it worse by changing the surrounding style, but I think it doesn't hurt that much.

@pablogsal
Copy link
Member Author

@erlend-aasland Once you are ok with the changes, you can approve the PR if you wish :)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erlend-aasland Once you are ok with the changes, you can approve the PR if you wish :)

LGTM :)

@pablogsal pablogsal merged commit 261a452 into python:master Mar 28, 2021
@pablogsal pablogsal deleted the better_token branch March 28, 2021 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants