Module 3: Theory, hypotheses, and arguments

Overview

Module 3 builds on Module 2 by examining theory, hypotheses, and arguments. We distinguish between these concepts but also reflect on how they work together within a final research report: In particular, we focus on the linkage between theory (our answer / logic) and hypotheses (the empirical implications of our theory). We examine some common pitfalls that crop up whenever we find ourselves working with imperfect data as well as some strategies for avoiding them.

Objectives

  • Differentiate theory, hypotheses, and arguments.
    • Explain how these concepts work together within a research report.
  • Identify common pitfalls in linking theory and hypotheses and explain how we can avoid them.
  • Analyze the linkage between theory and evidence as it crops up in Lessing and Willis.
  • Recall and practice the basics of cleaning and transforming data using dplyr.
  • Reflect on how to give substantive and constructive peer feedback during the week’s team meeting.

Due

Mon. @ 8pm CST. Write 1-2 paragraphs of feedback on each of your team members’ initial research designs. Post your comments as an attachment beneath your team’s sub-heading at Ed Discussion. If you are the first member of your team to post, please create a new thread.
Thu. / Fri. @ 8pm CST. Post your Q & A about the week’s reading at Ed Discussion.

Theory, hypotheses, and arguments

Before diving into the week’s concepts, let’s briefly review some of the concepts that we examined in Module 2 on “Research areas, topics, and questions.” Toward that end, watch the video on “Recap of Module 2.”

Now that we have examined research questions, let’s turn our attention to theories, hypotheses, and arguments. Before you watch the videos below on this topic, read the Chapter by Bryans et al. on “Explaining the Social World.” As your read, think about your answers to the following questions. Doing so will prepare you to better engage the video content and, hopefully, assist you as your progress in your research:

  • What is theory? What are hypotheses? How do these concepts go hand-in-hand?
    • Think about your own research proposal: Did it have a theory (or just hypotheses)? What is your theory?
  • How did you develop your theory? Did you mainly use induction or deduction?
  • What assumptions do you make in your theory?
    *If your theory is correct, what do expect to observe in the world? In other words, what is the best evidence you could find that would tell you that your theory is correct?

Now watch the videos on “Theory, hypotheses, and arguments” and “Why theory?”

The next video aims to get you thinking critically about how we can effectively link theory to hypotheses: “If your theory is correct, what should we observe in the world?” Doing this well is not easy, in part because we rarely find perfect evidence in support of our theories.  When this happens, it’s easy to step into some common pitfalls rather than take the more appropriate step of gathering additional evidence or modifying our theory to ensure it is useful despite empirical limitations. The next video examines some of these common pitfalls as well as some strategies for avoiding them. Be prepared to pause the video throughout, as this video is meant to be a sort of exercise in “spotting the fallacy.”

You will now read “Legitimacy in Criminal Governance” by Lessing and Willis. I selected the article not only because it is one of the most interesting academic articles you will ever read but also because it exemplifies the meme that I presented in the video on “Why theory?” above. That is, Lessing and Willis help us to interpret/connect/make sense of some rich data that I think are extremely puzzling.

Before you read, think about your intuitions to the following questions:

  • How do you think criminal organizations (e.g., mafias and drug gangs) keep their members in line?
  • If your answer (theory) is correct, what would you expect to find if you could examine the internal working of one of these criminal organizations?

Now read the article. When you have finished, watch the video on “Anatomy of an argument.”

R exercise on tidy data and dplyr review

If you are completing the R exercises each week, be sure to download the exercise for Module 3 from Canvas. This week’s exercise is meant to help you brush up on your data cleaning and transforming skills using the dplyr package. While this is less fun than the webscraping we did last week, it is probably more useful, as most of you will have to spend some time cleaning, transforming, and merging any data that you obtain for use in your final research report.

Module 2: Research areas, topics, and questions

Overview

Module 2 examines research questions: in particular, the attributes of a good research question, and a general process we can follow to move from a broad research area to a more specific topic and, finally, a focused question. The module will ask you to reflect

Objectives

  • Grasp the attributes of a good research question
  • Grasp the process for developing a good research question and begin to implement it
  • Examine the development of a research question by engaging a real-world example
  • Examine and practice web scraping in R

Due this week

  • Thu and Fri @ 8pm CST: Ed discussion Q & A.
  • Fri @ 8pm CST: Proposed research design. Submit via the Assignments section at Canvas. Also post as an attachment beneath your team’s heading at Ed Discussion. NOTE: You should read each others’ proposals and prepare 1-2 paragraphs of feedback for each one. You will be required to post your written feedback to Ed Discussion by Mon. 4/12 @ 8pm (Week 3).

Some guidance on research designs

Your proposed research design should be a roughly 1-page, single-spaced document. It should do three things: (1) lay out a research question and justify it; (2) delineate a brief answer to the question; and (3) discuss the sort of data you will need in order to answer the question and why.

The readings and videos for this week are meant to help you to develop your research question and design. So, if you can, try to complete the reading and watch the videos relatively early in the week.

I want to preview a couple of key points from the reading and videos here as means of reassuring and guiding you. In particular:

  • Your research question and design will evolve. In fact, it will likely continue to evolve even as you conduct your analysis and write up your results toward the end of the quarter. (I am still constantly revising the design document for my own dissertation as it becomes a book!) This can be frustrating, but it is inevitable. You will be moving in and out of the literature and data surrounding your question in upcoming weeks, and in doing so, you will get a much better idea about what makes a “good” question as well as the sorts of questions you can examine given the available data.
  • For part 3, it is not necessary that you have specific data / datasets in mind. You can instead approach this as a thought experiment: Given your question and tentative answer, what are your ideal data and why? What sort of measures would you obtain? What would the units of analysis be? Would the data be cross-sectional, panel / time-series, or something else? Are the data likely to derive from an experiment, survey, or machine-learning algorithm, or are they likely to have been hand-coded by scholars? The more you think through these and related questions, the better. Thinking carefully about your ideal data now will help you to identify workable data later.

The Module

Attributes of good questions and developing your question

First read the chapter on “Beginning the research process” by Buttolph Johnson. As you read, think about your answers to the following questions. You don’t need to write your answers down, but thinking about them will help you to develop your own research question:

  • What are some attributes of a good research question?
  • How does one actually go about developing a good research question?
  • How does the literature review go hand in hand with the development of a good question?

Once you have completed the Buttolph Johnson chapter, watch the three videos below on “Research areas, topics, and questions”. Note that you will need to enlarge the videos in order to view them.

Anatomy of a research question

Now read the article by Albertus and Deming on “Branching out.” You should read the entire article. In your reading, you might try to implement some of the advice laid out by Dane’s chapter on “Reading and structuring research” in Module 1. In addition, in line with this week’s overarching topic, here are some questions to think about as you read. To the degree that there is time, we may discuss some of these questions during our team meetings this week:

  • What is the central question that the authors ask?
  • What have other scholars found in terms of answers to this question (or very similar questions)?
  • Given that the question is not really new, what contribution – if any – do you think that the authors make?
  • What sort of data would you want – in theory – in order to answer the question posed by the authors?
  • What data do the authors actually use and how does it differ from the ideal data that you described above?
  • How do the authors justify the data that they use?

Once you have read the article, watch the video on “Anatomy of a research question.”

R exercise on web scraping

Remember that you are not required to complete and submit the weekly R exercise for a grade. They are a voluntary tool for introducing you to some operations in R that you may find useful this quarter and/or in the future. This week’s exercise introduces you to one procedure for scraping data from the web pages written using HTML. I am not an expert on this topic, so any data scientists with experience in web scraping should feel free to post additional resources and advice at Ed Discussion.

Note that for this exercise, you will need to add SelectorGadget to your web browser. I use Chrome and have installed the SelectorGadget extension from Chrome’s Web Store (https://tinyurl.com/298y44yt). For other browsers, you can simply drag the SelectorGadget bookmarklet to your bookmarks toolbar. You can find the bookmarklet at selectorgadget.com.

I have included a very brief video below that introduces the selector tool as it applies to this week’s R exercise.

Module 1: Reading and structuring research

Overview

Module 1 begins by walking through the course Syllabus. It then kicks off our examination of the different components of good research design by laying out strategies for critically reading and structuring a quantitative social science research report. It then asks you to critically examine a summary report by the Chicago Million Dollar Blocks Project as well as use the Project’s data to create a basic plot using GGplot.

Objectives

  • Logisitics
    • Examine the course Syllabus
    • Enroll in weekly team meeting via email
  • Reflect on strategies for critically reading quantitative social science research
  • Describe the main components of a social science research report, their contents, and their purpose
  • Reflect on the linkage between theoretical claims and data analysis via examination of the Chicago Million Dollar Blocks Project.

Due

  • You should send an email to me with 3 times that you are available to regularly meet with a team of 3-5 students. See the Syllabus for details and available times. Email me by Thursday evening.
  • Ed Discussion Q&A are due at 8pm CST on Thursday (Q) and Friday (A) each week. This week, you only need to post one comment by Friday at 8pm. Your comment should center on the Chicago Million Dollar Blocks Project. See below.
  • The Module 1 exercise is due at 8pm CST on Friday.

Module 1

Syllabus

Watch the video on “Course Syllabus.” If you have questions about the Syllabus after watching the video, please post your question to Ed Discussion under “Course logistics” so that other students can view the answer.

Course Intro.

Course format

Assignments and grading

Final research report

Course policies

Course schedule

Reading and structuring research

How to read and structure research may seem relatively straightforward; it may even seem obvious. You are at Chicago, after all, in part because you are a good reader. You are also probably a decent writer and have some good intuition about how to do research. But quantitative social science research can be a strange form. It is thus critical that we become deeply familiar with the form — its structure, conventions, jargon, and so forth — so that we can read it critically and, ultimately, apply it to our own research. With this in mind, I encourage you to reflect on your answers to the following questions as you read the Dane and Buttolph Johnson chapters:

  • How does critical reading of quantitative research differ from reading, say, an ethnographic study?
  • What are the different components of a quantitative research report?
  • And more critically perhaps: What should each component do? What is its contribution to the report as a whole?

Now read “Reading a research report” by Dane and “The research report: An annotated example” by Buttolph Johnson.

The Chicago Million Dollar Blocks Project

In addition to examining and reflecting on the different components of quantitative social science research, we are going to examine lots of data and data visualizations this quarter. We will do so mainly via readings, brief videos, and weekly exercises in R. The aim here is to get you thinking about how to effectively support a theoretical using good data and and detailed data analysis.

Go to the Chicago Million Dollar Blocks Project (https://chicagosmilliondollarblocks.com/). Carefully read the web page and examine the data presented there.

The Project begins to exemplify what you will do this quarter: In particular, it uses high-quality data to support a claim about a high-interest and pressing social topic. But there are also areas in which the “report” shown at the Project’s web page could be improved. In this vein, think about your answers to the following questions as you study the page:

  • What is the aim of the web page? What is it trying to do exactly?
  • What is a “war on neighborhoods”?
  • What evidence exists of a war on neighborhoods?
  • What sort of statistic would be clear and compelling evidence in support of the Project’s main claim (as you understand it)?
  • Are you persuaded? If not, how might you improve the web page to make it more compelling?

For this week’s Q&A, post one way that you might try to improve the “report” shown at the Project’s web page. Post your comment to the thread that I have created at Ed Discussion by Friday evening at 8pm CST. The thread is under “Reading Q&A / Chicago Million Dollar Blocks Project.” You are not required to post an additional reply this week, but you may do so if you wish.

Once you have scrutinized the Project web page and posted at Ed Discussion, complete the Module 1 exercise, which uses data from the Chicago Million Dollar Blocks Project. Download the exercise from the “Module 1” folder located at the “Files/Module 1” section of Canvas. Also download the corresponding dataset. Complete the RMD file, render it to PDF, and upload your PDF to the “Assignments” section of Canvas.