Publication Checklist

• Introduction
  • Introduction presents a clear “gap”: an important uncertainty in our current knowledge of the paper topic
  • Introduction contextualizes our paper in relation to key themes in the literature
  • End of the introduction states our contributions and refers the reader to the appropriate sections
  • The introduction avoids getting too concrete. The introduction discusses the paper at the level of research questions, hypotheses, abstract variables, and relationships between them. Readers who have read some of the papers we cite can understand most of what’s presented in the Introduction without reading the rest of the paper.
• Methods
  • The paper includes a Methods section that describes the experimental methods for the paper
  • The Methods section explains how abstract concepts (presented in the introduction) were operationalized — which concrete variables did we measure, and why?
  • If we used data, the Methods section contains a Data subheading
    • For each dataset that we used, there is a paragraph in the Data subheading describing the data and how we used it. Specifically, …
      • the dataset paragraph describes the purpose of the dataset, why it was originally collected
      • the dataset paragraph describes why we chose to use this dataset
      • the dataset paragraph describes the contents of the dataset qualitatively (what kind of data is it)
      • the dataset paragraph describes the contents of the dataset quantitatively (how many features, how many observations)
      • the dataset paragraph is not too long — if there are more than two paragraphs of important information, the dataset paragraph refers the reader to further description in the Appendix
      • In the main body or Appendix, we describe any preprocessing steps that were applied to the data before use in the main experiments
    • For each major step in the experiment, there is a subsection or paragraph in the Methods section describing how this step was performed
      • The subsection describes the models or algorithms used.
      • Papers that introduce the models or algorithms are cited.
      • The subsection justifies the choice of models or algorithms.
      • The subsection describes important hyperparameter settings, and refers the reader to Appendix section listing the hyperparameters in detail if needed
• Results
  • The Results section presents the findings from our paper
  • For each research question or hypothesis in the Introduction, the Results section answers this question in terms of the operationalization from the Methods section. In other words, IF the reader believes in and understands the operationalization, then the Results section answers their questions.
• Figures
  • Publication has several figures
  • Each Figure meets quality criteria
    • Each Figure has legible font sizes
    • Each Figure has horizontal and vertical axis labels
    • Each Figure has a legend
    • Each Figure indicates its sample size
    • Each Figure uses a colorblind-accessible color palette
    • Each Figure has a caption
      • Figure caption explains all aspects of the visual encoding
    • Figure is referenced in the main body
      • Main body reference highlights key takeaways of the Figure for the reader
• Discussion
  • The Discussion lifts the reader from the concrete “ground floor” of the Results. The discussion comments on concrete findings, only to emphasize their implications at the level of the hypotheses/research questions from the Introduction.
• Related Work The goal of the related work section is to position our work as related but unique.
  • Related Work section explains in what way our paper connects with the existing literature, using specific comparisons.
  • Related Work section explains in what way our paper is different from the existing literature, using specific comparisons.
  • At least three other papers are cited in the Related Works section (yes, there are always at least three things to talk about here)
  • The contribution of each related work is stated clearly and positively, giving credit where credit is due
  • Important limitations of the related works that are addressed by our work are stated clearly
  • Unimportant limitations of the related works are not discussed (we don’t need to nitpick)
• Conclusion
  • The conclusion recapitulates the findings of the paper, at a similar level of abstraction as the Introduction