Good Research Practices

Semester

Semester 1, 2023-2024

Type of course

Methodological and Practical Courses

Date

January 26, 2024

Location

Utrecht University

---

Duration

1 day

Maximum number of participants

30

ECTS

0.5 EC will be appointed for participation in the complete course

Staff

Daniel Lakens (TU/e)

Content, learning goals, preparation

Recent discussions in psychology have led researchers to acknowledge that there is room to improve our research practices. In this workshop, we will discuss state of the art statistical and methodological insights that will teach you how to draw better inferences from the scientific literature, and design more reliable and efficient experiments. I will explain why the scientific literature is biased, and how you can detect this bias. We will discuss why researchers contribute to bias in the scientific literature, and how these research practices conflict with the Code of Conduct for Research Integrity. We will reflect on the strange state of affairs where the code of conduct instructs us to abide by certain practices, but how we instead rely on cultural norms within our research community that determine how we do our job. How do we need to organize our science such that we are following the code of conduct? Which changes do we need to make to make sure our research is reliable, that we communicate our findings honestly, and that we hold each other accountable?

Please bring a laptop with R and RStudio installed. Read the VSNU code for research integrity. If you are interested in this topic, read the additional literature below.

Literature

Compulsory:

VSNU code of conduct for research integrity. 

Optional:

Fiedler, K., & Schwarz, N. (2015). Questionable Research Practices Revisited. Social Psychological and Personality Science, 1948550615612150. https://doi.org/10.1177/1948550615612150

Spellman, B. A. (2015). A Short (Personal) Future History of Revolution 2.0. Perspectives on Psychological Science, 10(6), 886–899. https://doi.org/10.1177/1745691615609918

Schedule

	Morning
10:00-11:15	Type 1 and Type 2 error control. Why error inflation quickly destroys the evidence in your data, and how to prevent you from fooling yourself. Practical Assignment: Error inflation when simulating optional stopping, and correcting error rates through sequential analysis.
11:30 – 12:30	Scientific Integrity in Practice. A short overview of why we know (psychological) science is biased. A comparison of current research practices with the VSNU scientific code of conduct.  Discussion: Where do we deviate from the Code of Conduct in Practice. What would be needed to adhere to the code of conduct? Bias detection: How can we detect simple mistakes in reported results. How can we catch these mistakes in our own work? Practical assignment: Statcheck
	Afternoon
13:30-15:00	What do real research lines look like? Explanation of why you will observe mixed results in the research lines you perform in your PhD. Practical assignment: Likelihood Mixed Results.
15:15-17:00	How do we design informative studies, both to show the presence as the absence of effects? How can we publish null results?  Discussion: How can we prevent publication bias? Practical assignment: Equivalence testing.