Introduction to OpenRefine
1 like1,439 views
The document introduces OpenRefine, a data cleaning and transformation tool, presented by Heather Myers at the Central PA Open Source Conference in 2015. It provides guidance on getting started with OpenRefine using a dataset from the Pennsylvania Heritage Magazine's subject index, with steps for importing, filtering, and exporting data. Resources such as the OpenRefine website, a user guide, and an online course are also mentioned for further learning.
Introduction to OpenRefine
- 1. INTRODUCTION TO OPENREFINE CENTRAL PA OPEN SOURCE CONFERENCE OCTOBER 17, 2015 Heather Myers / @privatestorm https://www.linkedin.com/in/heathercmyers
- 2. ABOUT ME Web administrator in the government and cultural heritage sectors. Currently working at the Pennsylvania Historical and Museum Commission.
- 3. OPENREFINE "a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data."
- 4. GETTING STARTED Choose dataset Decide what you want to accomplish with data Install OpenRefine Run OpenRefine http://openrefine.org/download.html http://127.0.0.1:3333
- 5. ABOUT THE DATASET Pennsylvania Heritage magazine subject index. Index of 12,000+ magazine terms for issues dated 1975– 2002. http://bit.ly/1Udha8D
- 6. DATA TO DO LIST Create lists of terms for specific issues Extract list of terms
- 7. IMPORT ALL DATA File upload Web download Copy from clipboard Google data import
- 10. APPLY A TEXT FILTER Click down arrow on column Choose text filter from menu
- 11. APPLY A TEXT FILTER Type text Choose additional options
- 14. EXPORT DATA Click export button Choose export type Save file
- 15. DATA TO DO LIST Create lists of terms for specific issues Extract list of terms
- 16. IMPORT SELECTION OF DATA Choose CSV /TSV/ separator- based files Use semicolon as custom separator
- 17. DATA SEPARATED INTO COLUMNS
- 18. SPLIT INTO SEVERAL COLUMNS Click down arrow on column Choose edit column Choose split into several columns
- 19. SPLIT INTO SEVERAL COLUMNS Split by separator or field length Choose after splitting options
- 20. EDIT SINGLE CELL Hover over cell & click edit Update data type Update text Click apply
- 21. TRANSFORM TEXT Click down arrow on column Choose edit cells Choose transform
- 23. TERMS SPLIT INTO COLUMNS
- 24. MOVE COLUMN Click down arrow on column Choose edit column Choose move column left
- 25. TRIM WHITESPACE Click down arrow Choose edit cells, common trans- formations Choose trim leading & trailing whitespace
- 27. HIDE COLUMNS Click down arrow Choose view Choose collapse all other columns
- 28. SORT VALUES Click down arrow Choose sort Choose additional options
- 29. UNDO / REDO Choose undo/redo tab in left column Click previous step to undo
- 30. FILTER UNDO / REDO Type keywords in filter box
- 31. EXTRACT OPERATION HISTORY Choose extract button Choose steps to save in left column Copy JSON in right column
- 32. APPLY OPERATION HISTORY Open or create new project Click apply button in undo/redo Paste JSON Click perform operations
- 33. EXPORT DATA Click export button Choose export type Save file
- 34. LEARN MORE Website: OpenRefine http://openrefine.org/ Book: Using OpenRefine http://bit.ly/1QC0oNS Course: Big Data University http://bit.ly/1QC1sl1
- 35. THE END