Skip to content

Python tokenizer rewriting #69829

@serhiy-storchaka

Description

@serhiy-storchaka
BPO 25643
Nosy @brettcannon, @vstinner, @serhiy-storchaka, @1st1, @matrixise, @DimitrisJim, @pablogsal
PRs
  • bpo-25643: Refactor the C tokenizer #25050
  • bpo-25643: Fix tokenizer error when raw decoding null bytes #25080
  • Dependencies
  • bpo-26581: Double coding cookie
  • Files
  • tokenize_input.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2021-03-28.22:49:06.587>
    created_at = <Date 2015-11-17.01:27:32.904>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = 'Python tokenizer rewriting'
    updated_at = <Date 2021-03-29.21:53:38.609>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2021-03-29.21:53:38.609>
    actor = 'pablogsal'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2021-03-28.22:49:06.587>
    closer = 'pablogsal'
    components = ['Interpreter Core']
    creation = <Date 2015-11-17.01:27:32.904>
    creator = 'serhiy.storchaka'
    dependencies = ['26581']
    files = ['41058']
    hgrepos = []
    issue_num = 25643
    keywords = ['patch']
    message_count = 7.0
    messages = ['254778', '255082', '255355', '262091', '376742', '389654', '389692']
    nosy_count = 8.0
    nosy_names = ['brett.cannon', 'vstinner', 'python-dev', 'serhiy.storchaka', 'yselivanov', 'matrixise', 'Jim Fasarakis-Hilliard', 'pablogsal']
    pr_nums = ['25050', '25080']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue25643'
    versions = ['Python 3.7']

    Metadata

    Metadata

    Labels

    3.7 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions