Applied Econometrics Report

Term Paper: Replicate the Analysis and Findings of Published Work
Your assignment is to replicate the relevant analysis from one empirical journal article that you
find interesting, and then report your comparative results in the form of one term paper. I
emphasize now, and will continue to do so during the term, the need to work steadily throughout
the semester on this project. Some of the homework assignments may include submission of
preliminary materials and updates relevant to this effort.
March 18: Make Your Choice
Thorough description of your paper. For guidance, refer to “12 Steps to Understanding a
Quantitative Research Report”. You should also include a copy of the tables that you plan to
replicate. You need not replicate every table in the paper. Choose two or three tables that provide
evidence for the main point of the paper.
The paper must have been published in a peer-reviewed journal, and it must have the data that
goes with it (often journals have an online repository), or the data must readily available (eg., US
Census, Bureau of Transportation Statistics, etc.)
The paper must include some type of regression analysis. The topic of the paper does not have to
be related to economics, but it can’t be a simple one or two-sample test.
You should understand the main point, and the paper should interest you.
King, G. 2006. Publication, Publication. Political Science and Politics 39(1): 119-125.
http://gking.harvard.edu/files/abs/paperspub-abs.shtml
April 8: Summary statistics
This assignment insures that you have gathered, input, and cleaned the data for the replication.
You should submit the published table of summary statistics and your best effort at replicating
them. Please submit one or two paragraphs explaining how you are progressing.
April 15: Draft of report
You should have completed data analysis, and a rough draft of your results should be ready.
April 29: Finished report
Your finished report should be ready. Submit a paper copy in class, and email your report to me.
2
Inspiration for Replication Projects
Journal of Applied Econometrics … http://qed.econ.queensu.ca/jae/
Mike Watson’s Resources … http://www.princeton.edu/~mwatson/publi.html
The IQSS Dataverse … http://dvn.iq.harvard.edu/dvn/
IUCPSR … http://www.icpsr.umich.edu
Plos One … https://journals.plos.org/plosone/
Things to Think About
Here are some suggestions for how to approach your term paper. You need not address all of
these questions, but you must write a well-argued term paper rather than a bullet-point response
to these items. I will not accept the submission of raw output from statistics applications, and I
welcome creative and elegant visual displays of results and data.
Explain the economic problem or question that the original paper addresses. Can you explain the
paper in plain English to a non-economist? What is the dependent variable? What is the key
independent variable? What is the proposed causal process? What are some confounding factors
(e.g., other independent variables, reverse causality, or selection) that the original paper
addresses? How big is the estimated effect?
What is the unit of observation (e.g., countries, calendar quarters for a single country, countryyears for a panel of countries, sub-national regions such as states of the United States, persons, or
households)? What are other relevant dimensions of the data (e.g., frequency of the data or time
or geographic span)?
How did you gather the data, for example, downloaded the cleaned data from the author’s or
journal’s website or reconstructed the data from the description in the paper?
Compare your attempt at pure replication to the results (e.g., summary statistics and regression
output) in the original paper. Do your results differ in sign, size, or significance from those of the
original paper? If so, can you explain the difference (e.g., data source, data construction,
estimation procedure)? For example, you might consider overlooked confounding factors,
shortcomings in the estimation procedure, outliers, or limited external validity of the model. It’s
also possible that the original paper is less robust than your work! Compare the sign, size, and
significance of the results in the extension to those in the original paper.
3
12 Steps to Understanding Quantitative Research
adapted from William N. Evans, Notre Dame
1. Citation. Give the full citation for the study.
2. Purpose and general rationale. In broad terms, what is the purpose of the study,
and how does the author(s) make a case for its importance?
3. Fit and specific rationale. How does the topic of the study fit into the existing research
literature, and how does the author justify the investigation in the paper? What are the key
independent variable and the main dependent variable?
4. Participants. What is the unit of observation. Describe what or who was studied (give number
and characteristics) and how the sample was selected.
5. Context. Where did the study take place? Describe important characteristics.
6. Steps in sequence. What were the main procedures in the study? Describe or diagram in a
flowchart, showing order and any important relationships among the steps.
7. Data. What constitutes data (e,g., test scores, administrative data, survey or questionnaire
responses) and how were the data collected.
8. Analysis. What form of analysis of data was used? What statistical methods were employed?
How did the method address the speci_c questions it was designed to answer?
9. Results. What does the paper identify as the main results (products or findings produced by
the analysis of data)?
10. Conclusions. What does the paper assert about how the results in Step 9 respond to
the purpose(s) established in Step 2? How did the events and experiences of the
entire study contributed to that conclusion?
11. Cautions. What cautions does the author(s) raise about the study itself or about
interpreting the results? Add here any of your own reservations.
12. Discussion. What interesting facts or ideas did you learn from reading the report?
Include here anything that was of value, including: results, research designs and
methods, references, instruments, history, useful arguments, or personal inspiration.