Consent confirmation (https://gdpr-info.eu/) before interviews Sweden, Brasil: strong on empirical studies
Make sure the definition of the variable is clear before asking it to the participants of the study (e.g., what does it mean “computer”?)
Negation has cognitive effort
Framing the question impact the outcome in empirical research
Types of validity
- construct validity: measurement bias, wrong reference theory
- internal validity: mortality, instrumentation bias, learning bias
- conclusion validity: lack of statistical significance, reliability of measurements, noise, randomness
- external validity: being able to generalize effectively, sample quality, external variables not considered (e.g., tiredness)
Causality and correlation
- spurious correlations may happen (https://www.tylervigen.com/spurious-correlations)
Triangulation
Use multiple methods/measurements to cross-validate the results
Cargo cult
Do not repeat/do something without really understanding it, just because everyone does it. Cargo cult reduce credibility and trustworthiness
Negative results are not non-results
A good data presentation can be useful to find logical errors or bias
How many variables: as much as it serves the purposes of the study
Controlled experiments
- Randomized controlled trial (RCT): treatment applied only to one subset of the population. Members are assigned randomly to the control group. Used in pharma (high reliability)
- Natural experiments (“quasi experiment”): population assignment cannot be controlled, but can be random and representative. Used often to study natural circumstances (e.g., correlation between cholera outbreak and patients residence)
Between groups: above In-group studies: often with few participants, split the group and test different condition for the same variable