research_competencies

Intro

Getting started in research can be difficult. You’re learning a lot of new concepts. The goal for your project is not always clearly defined. These factors can make it hard to measure progress, or to come up with goals for yourself as a researcher. After six months working in a lab, you feel like you’ve progressed, but you can’t put your finger on how. Or maybe you feel that you haven’t progressed, but you don’t know how things could be different.

This guide aims to make that process (slightly) easier. This guide is a list of competencies – skills that researchers need. I’ve tried to organize the guide so that easier skills are presented first. That way, you can work on them as stepping stones towards a more capable version of yourself as a researcher (and maybe as a person!).

Take the guide with a big grain of salt – I am new to research and new to mentoring other researchers. That being said, I believe in what’s presented here, and I hope it is helpful to you.

A Skill Tree for Research

When you’re getting started in research, you might wonder where that journey will take you, and what you will need to do along the way. This is also true in role-playing games – some of my favorite video games solve this problem with a skill tree.

The skill tree gives you a preview of the ways that you could develop your character. As you accumulate experiences, you can spend experience points to unlock skills in particular areas.

In a skill tree, certain skills are prerequisites for others. You can’t do advanced, blacksmithing until you know basic blacksmithing and so on. One could argue that the same is true in research. It doesn’t make sense to try to write your first paper without having red several other papers first. It doesn’t make sense to present your own work until you’ve had the experience of presenting the work of other researchers (in a journal club for example).

The visualization below is my attempt to write down a skill tree for research, so that you can have an idea of how how you can develop yourself. This is certainly not set in stone. For many researchers, this just happens implicitly through mentorship with people who are further along in the skill tree, but I think there’s some value to thinking about it explicitly. I hope it’s useful to you.

Skill Areas

After you have a high level perspective on where you’re headed, you might be curious or even excited to know how you can develop specific skills. This requires going into a little bit more detail than what’s shown in the skill tree diagram.

Below, I’ve tried to list out some checks for competency, and some exercises that can help to build your competency in specific skill areas.

Those skill areas for research are (in no particular order):

1. Research Thinking
2. Good Habits
3. Writing code and using programming tools
4. Reading
5. Writing
6. Presenting

As of summer 2024, this checklist is still under construction. If you have ideas, feel free to share!

Research Thinking

Competencies:

1. Level 1

   • I can identify the independent and dependent variables in a research question or hypothesis.
     • Exercise: For each of some example papers, identify the hypotheses or research questions. For each research question, identify the independent and dependent variables.
   • For a research question in my areaa of focus, I can name multiple ways to operationalize the independent and dependent variables
   • Operationalize a toy research question

2. Level 2

   • Critically evaluate the validity of a study’s conclusions
   • Assess the pros and cons of different operational variables

3. Level 3

   • Design a simple experiment or study to test a hypothesis

Ways to Increase your Research Thinking Skills:

Habits

Competencies:

1. Memory Habits:

   1. Level 1

      • I have a capture system that allows me to record important facts and ideas at the times when I encounter them
      • I have a memory system that makes me confident I can recall important facts later, given sufficient time to study them

2. Ways to Develop your Memory Habits:

   • Try using a spaced repetition system on a topic you want to learn. I recommend Mochi for flashcards, but the specific platform is less important than the habit of using it.

Programming

Debugging

• I can use appropriate language to express the challenge I am facing in my code
• I can formulate my challenge into search queries that lead to solutions (e.g. from a search engine or ChatGPT)
• I can express multiple hypotheses for why my code is not working
• For each hypothesis, I can write down one or more tests to check if the hypothesis is true
  • For each test, I can write down the expected result
  • For each test, I can name reasons why the test may have returned misleading results
  • For each test, I can express what the test must do to yield convincing evidence for or against the hypothesis

Writing Code

1. Level 1

   • I can translate a simple mathematical expression into a Python function

2. Level 2

   • Exercise: Read the code for an existing research software project and explain its major components and their function
   • Navigating and understand a large pre-existing codebase
   • Write a single script
   • Debug a piece of code to fix a logical error
   • Implement basic data structures (e.g., lists, dictionaries) in code

Other Tools

1. Version Control

   I can …

   • Use a version control system (e.g., git) for a small project
   • Initialize a git repository
     • Commit changes to a git repository
     • Push changes to a remote repository
     • Pull changes from a remote repository
     • Resolve a merge conflict
     • Use a branching strategy for a small project
     • Use a branching strategy for a large project
     • Use a branching strategy for a collaborative project
     • Create a pull request

   Ways to increase your competence in version control:

   • https://gitimmersion.com/lab_01.html

2. Code Editor

   I use an editor that allows me to navigate a codebase by…

   • Searching for files by name
   • using “Go to Definition” on function or symbol names
   • searching for the occurence of phrases across the codebase
   • The above items are possible in most popular editors, including VSCode, PyCharm, Vim, Emacs, although installing plugins may be required.

   I use AI tools for code completion, like…

   • Microsoft CoPilot (VSCode plugin)
   • Cursor.sh (VSCode-like editor)
   • gptel for emacs

   Some might push back on this point, and there are good arguments that the initial learning curve is steep. However, I would recommend that you…

   • learn to use a modal editor (like Vim or Emacs), or configure modal keybindings in another editor (like the Vim plugin for VSCode).

Shell

• I use a shell that helps me avoid redundant work. For example…
  • zsh with autosuggestions and syntax highlighting plugins
  • shell_gpt for English-to-bash translations (warning: not guaranteed to do what you want, double-check outputs)
  • fzf to quickly find files and search previous commands in the terminal

Reading

Level 1

I can…

• Identify the hook in a research paper
• Identify author paraphrases of findings from previous work
• Identify the stated limitations of a research paper

Level 2

• Identify the relationships between the present work and previous research
• Identify unstated limitations of a research paper

Level 3

I can…

• Summarize the key findings of a research paper
• Explain in your own words how the quantitative results of a research paper lead to the qualitative interpretations
• Identify the research gap in a literature review

Quantity

1. Deep Reading

   • Read and took notes on 1 paper
   • Read and took notes on 10 papers
   • Read and took notes on 50 papers

2. Shallow Reading

   • Shallow read 1 paper
   • Shallow read 10 papers
   • Shallow read 50 papers

Writing

Level 0

• Install a citation manager (I highly recommend Zotero in general, or org-ref if you are already an Emacs user)
• Add a paper to a citation manager
• Cite a paper in a document using a citation manager

Exercise: Install Zotero, plus the Zotero browser extension for your favorite browser. Find a paper on ArXiV, and add the paper to your library using the browser extension.

Level 1

• Write a concise abstract for a research paper
  • Exercise: Find a research paper that you haven’t read before. Don’t look at the abstract! Download it, read it, and then write your own abstract. Compare your abstract to the original. You can also do this for the conclusion section.

Level 3

• Create a well-structured outline for a research proposal

Presenting

I can…

• Restate an audience question accurately in my own words
• Reply with a relevant “yes-and” to an audience comment
  • Catch a “softball” audience question and answer it successfully
  • Catch a “curveball” audience question and answer it successfully
  • Catch a “fastball” audience question and answer it successfully
  • Identify the flaws in a poorly-designed figure
  • Visual Presentation
    • Improve a poorly-designed figure
    • Design a clear and engaging slideshow for research findings
    • Outline a slide deck for a research presentation
    • Edit an existing slide to improve visual hierarchy
    • Edit a slide to improve conciseness
    • Use visual aids effectively in a presentation
    • Unpack a figure for an audience
    • Identify the main point of a figure
    • Describe the visual marks and channels used in a figure.
  • End-to-End
    • Conduct an engaging and informative presentation of another researcher’s paper
    • Conduct an engaging and informative presentation of my own work