These .pickle files aren't created by the tests themselves, but they do
show up after running 'make test', or more specifically after running
'./python Lib/test/regrtest.py -v test_lib2to3'.

This is because a grammar generated from a .txt grammar file is cached
as a pickle for faster access by load_grammar in
Lib/lib2to3/pgen2/driver.py. A pickle file then ends up in the same
directory as the .txt file.

IMHO, though it may be preferable to remove just the pickle files in
Lib/lib2to3, it should never be required for tests to have pickle files
already available in the source tree. Instead, the test should create a
temporary pickle file, e.g. from a bytes object. Furthermore, binary
files shouldn't be versioned without good reason. Thus, removing all
pickle files should be safe for the future.