• A quick roundup of important pieces of information to understand and use data as part of normal reporting
• You don’t need to be an expert to use data!
• Keep an eye out for common pitfalls
• Some parts of this might be basic to some of you, but hopefully everyone will find at least some of this helpful

• BASIC: You can parse a spreadsheet, request data, and avoid simple pitfalls
• INTERMEDIATE: You’re comfortable with spreadsheets, working with formulas and pivot tables, and making simple charts to visualize data
• ADVANCED: You’re familiar with one or more tools for working with big databases, code, geographic data, and how to request this data

My opinion: Not everyone needs to be a data expert. But in a mid-sized newsroom like MPR, every journalist should have at least Basic levels of data literacy, a majority should be Intermediate, and a few should be Advanced.

• GIS: Geographic Information System, the discipline of using computers to analyze geographic data and make maps
• ArcGIS/QGIS: Two popular GIS programs. ArcGIS is the major commercial GIS program, while QGIS is the free open-source variant
• Shapefile: A common file format for GIS data. Actually not one file but a collection of files with the same filename and different extensions — the map.shp file has the geographic data, the map.dbf file has accompanying data, and so on

• R: A programming language optimized for statistics and data visualization
• Python: A general-purpose programming language with strong support for data
• JavaScript: A programming language focused on scripting web sites
• D3: A JavaScript library focused on interactive visualizations
• SQL: “Structured Query Language,” a programming language used for interacting with databases

• Pivot table: A function in Excel and other spreadsheet programs to summarize a larger spreadsheet

• Summary data/raw data

## # A tibble: 8 × 3
##   carat cut       price
##   <dbl> <ord>     <chr>
## 1  0.23 Ideal     $326 
## 2  0.21 Premium   $326 
## 3  0.23 Good      $327 
## 4  0.29 Premium   $334 
## 5  0.31 Good      $335 
## 6  0.24 Very Good $336 
## 7  0.24 Very Good $336 
## 8  0.26 Very Good $337

## # A tibble: 5 × 3
##   cut       diamonds average_price
##   <ord>     <chr>    <chr>        
## 1 Fair      1,610    $4,359       
## 2 Good      4,906    $3,929       
## 3 Very Good 12,082   $3,982       
## 4 Premium   13,791   $4,584       
## 5 Ideal     21,551   $3,458

• Edit in a copy, not the original
• Record your steps, maybe save as new files periodically
• Never encode information in color or formatting

• “Wide” vs. “Tall”/“Long” data

• “Wide” vs. “Tall”/“Long” data

• “Wide” vs. “Tall”/“Long” data
  • “Wide” data is often more easily read by humans; “Long” data is easier for machines to parse
  • “Long” data can lead to tidier column names
  • Sometimes one or the other will be easier to graph with whatever program you’re using
  • It’s not that hard to switch data between them, so don’t stress too much about this

• Calculator to compare a few numbers.
• Spreadsheet (Excel, Google Sheets, LibreOffice) to compare hundreds or thousands.
• Database or code (SQL, Access, R, Python) for hundreds of thousands of numbers

• Missing data
• Duplicated data
• Spelling differences (“Saint Louis” vs. “St. Louis”)
• Date formats (“12/25/2021” vs. “2021-12-25”)
• Sequences like 999999
• Default dates like Jan. 1, 1970, or Jan. 1, 1900

• Excel row limits (1,048,576 rows, or 65,536)

• Excel converting things to dates

• Keep it simple
  • Avoid pointless embellishments like 3D bars or dozens of colors
• Charts should tell a story
• Recognize when words or tables would be better

• Are you comparing the relationship between two different values for each point of data? Scatterplot
• Are you comparing values from several different categories? Bar chart
• Are you showing a change in one metric over time? A line chart if your data is relatively smooth or continuous, perhaps a bar chart if not
• Is geography important for your data? If so, make a map. If it’s not important, stick with a chart.

• No pie charts!
  • People are bad at judging which wedge of a pie chart is larger, if sizes are at all close
  • Instead, use a bar chart. It’s easier to compare relative values
• No dual-axis charts!
  • Your choice of scales can drastically change the apparent relationship between your two pieces of data
  • Instead, convert the metrics you want to compare into percent change from a fixed starting date

• You can truncate your y-axis — in the right situation
• Bar charts must always have y-axes that start at 0; if one bar is twice as tall as another, our minds assume the value of Bar A is twice as big as Bar B’s value, and truncated y-axes break that
• For other kinds of charts, it’s generally OK to cut out wasted space — no one expects a summer weather chart to go down to 0º, for example
• Be conscious of the story your choice of axis limits tells. Are you highlighting change or continuity? Is this fair?

• In a linear scale, you add a regular increment: 10, 20, 30, 40
• In a logarithmic scale, you multiply a regular increment: 10, 100, 1,000, 10,000

When to use a log scale

• When your growth is exponential (watch for J swoops!)
• When you want to plot really small numbers and really large numbers next to each other
• When the point is NOT to highlight outliers

Important log scales:

• log2 is a sequence of doubling: 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048
• log10 is a sequence increasing by 10: 10, 100, 1,000

Always explain!

• PDFs are not data!
• Ask for data in “machine-readable” format such as CSV, Excel, or a database
• Ask for the name of their database software
• Ask for the “record layout” — the names of the rows and columns in the database, minus the actual data
• Ask to talk to their IT person, who might actually know what’s going on
• If they won’t give raw data, ask for summary data
• Suggest they’d save themselves a lot of effort by just automatically publishing the data online

• U.S. Census
• Census Reporter
• NHGIS
• Our World In Data
• Natural Earth
• Wikipedia (as a starting point!)

Always check the data!

• Spreadsheets:
  • Excel: Industry standard
  • Google Sheets: Free online version
  • LibreOffice: Open-source alternative
  • Numbers: Default on Macs; avoid
• GIS: QGIS is free and open-source
• Extracting data from PDFs: Tabula
• Programming: R is the primary language I use. You’ll need to download R, and you’ll want to download the app RStudio to provide a more user-friendly interface
• A plaintext editor like Sublime Text or BBEdit

• Edward Tufte, The Visual Display of Quantitative Information
• Dona M. Wong, The Wall Street Journal Guide to Information Graphics
• Scott Berinato, Good Charts: The HBR Guide to Making Smarter, More Persuasive Data Visualizations
• Hadley Wickham & Garrett Grolemund, R for Data Science
• “The Quartz guide to bad data”