Overview
Module 7 examines literature reviews: What they are, what they are meant to do, and how we can go about generating them. The Module also explores the importance of surveying existing literature in order to conduct good research more broadly. Indeed, existing literature is fundamental for developing effective questions, arguments, and analysis plans. The weekly R exercise then invites you to reflect on the question of how we can go about visualizing uncertainty and variability in our data analysis.
Objectives
- Internalize the importance of existing literature for virtually every aspect of research.
- Lay out the purpose of a literature review and describe the basic process for writing an effective one.
- Examine two approaches to writing an effective literature review as they crop up in actual research.
- Use statistical software to visualize uncertainty and variability in your own data.
Due
- Fri @ 8pm CST: 15-page draft of the research report. See guidelines below. Note that over the weekend, you should read drafts that will be workshopped in Week 8. I will disseminate a workshop schedule toward the end of the week.
- Thu/Fri @ 8pm CST: Module 7 Q & A
Draft of the research report
Your draft should be roughly 15 pages. It should be double-spaced and use 12-point Serif font. Your draft should lay out (1) your research question and its motivation; (2) a brief literature review; (3) your argument; (4) your data and analysis plan with justification; and (5) two plots, tables, or other visualization that motivate your question and/or support your argument.
Your draft should be complete and demonstrate thought and effort. That is, whereas this is meant to be a draft that you will revise and augment between now and the end of the quarter, it should nevertheless reflect that you have been thinking about and refining your question, argument, analysis, and grasp of the relevant literature during these past 7 weeks. In addition, your draft should not come across as having been hastily written; you should therefore revise your draft for clarity, precision, and concision before submitting it.
The Module
Review of Module 6
Module 7 focuses on the literature review. As with previous Modules, however, it begins with an overview of the Module videos as well as a very brief review of Module 6 on “Analysis Plans”. Watch the video on “Module 6 Review.”
Uses of Literature
The next video examines the role played by existing literature in different components of research. We have of course already examined this idea to a degree. For instance, in our discussion of research questions, we underscored the importance of surveying prior studies for generating and refining our questions. The video builds on this intuition by examining how we should be incorporating prior work at different points in the research process.
Watch the video on “Uses of Literature.”
The Literature Review
The next video is the real meat of the Module in that it lays out the literature review: What it is, what it is meant to do, and how we can go about writing one effectively. We also examine two different approaches to writing a literature review and what these approaches look like in actual scholarship. Toward this end, you should start by reading the articles by Schillbach on “Alcohol and Self Control” and Kocher et al. on “Rabbit in the Hat.”
Both of these articles are fascinating, so you should read them in their entirety. But, given this week’s topic, you should focus on how the authors engage existing literature on their topic in order to build their own arguments. Accordingly, here are some questions that you should try to answer as you read:
- What is the focal relationship in each piece (i.e., What is the main IV and DV)?
- How do the authors organize their engagement of the literature? Do they seem to focus on literature surrounding their IV, or their DV?
- How do the authors group different citations? Why do you think that they grouped them in the way that they did?
- Find one or two citations in each piece. What does the citation do for the author? What point does it help them make?
Now read Schillbach and Kocher et al. Once you have finished reading, watch the video on “The Literature Review.”
Zotero
I should note that Zotero is an excellent resource for organizing citations and generating bibliographies. If you have not used Zotero before, I highly recommend it. A very brief introduction to using Zotero is available at the UC Library website: https://guides.lib.uchicago.edu/c.php?g=297676&p=1986555
R Exercise on Depicting Uncertainty
This week’s R exercise asks you to reflect on uncertainty: How to think about it, and how to report it visually. You will work with your own data to generate some tables and plots that aim to capture uncertainty and variability in your data. You are likely to include some of these in your final research report, so I highly encourage everyone to complete the exercise in R or Stata.
First read the very brief article by Hullman on “Confronting Unknowns.” Once you have read, proceed to the Module 7 R Exercise, which is available for download at Canvas. If you are using Stata, I have copy-pasted a portion of the Exercise instructions below:
—–
The Module 7 RMD invites you to reflect on uncertainty: How to think about it, and how to capture it visually. I have therefore asked you to begin by reading Hullman’s brief article on “Confronting Unknowns.”
Critically, we often confuse uncertainty and variability. Uncertainty means that we do not know a particular quantity. For instance, we we may not know with precision the proportion of voters that will vote Democrat in an upcoming election. By contrast, variability means that a particular variable can take on a range of different values. For instance, each time we conduct a survey, a different proportion of respondents will state that they plan to vote Democrat in an upcoming election.
As with last week’s RMD, you will work with your own data. Start by cleaning your data as necessary. Then proceed below.
In this section, use your own data to create a plot or other graphic that depicts some quantity of interest AND the uncertainty OR variability around it. For example, your quantity of interest could be something along the lines of:
- Average life expectancy across sub-Saharan Africa.
- Average perceived level of inequality across countries.
- Average decrease in enrollment across colleges and universities under Covid-19.
- Proportion of Divvy users who own annual memberships (or month-to-month memberships).
- Average decrease in expected educational achievement for each additional child added to a family.
You can choose from a wide range of ways for visually depicting these quantities. Here are some of these ways. Note that whereas I have included some potentially useful resources for generating these graphs in R, Stata users may need to use an internet search in order to find comparable resources:
- Barplot with error bars (see https://www.r-graph-gallery.com/4-barplot-with-error-bar.html)
- Fancharts (which are super useful for time trends) (see https://gjabel.wordpress.com/2013/04/24/bank-of-england-fan-charts-in-r/)
- Regression estimates confidence intervals (https://rpubs.com/aaronsc32/regression-confidence-prediction-intervals)
- Regression prediction confidence intervals (https://cran.r-project.org/web/packages/dotwhisker/vignettes/dotwhisker-vignette.html)
- Heatmaps: See, for example, this Divvy bike heatmap (https://austinwehrwein.com/data-visualization/heatmaps-with-divvy-data/). Such an example reports quantities of interest and frequencies, so it is used to depict variability across time. A short tutorial for heatmaps can be found at: https://sebastianraschka.com/Articles/heatmaps_in_r.html