Везде
New Data, New Statistics: from Reproducibility Crisis toward New Requirements to Data Analysis and Presentation in Social Sciences
Table of contents
Annotation Estimate Publication content
References Comments
Share
QR
Metrics
New Data, New Statistics: from Reproducibility Crisis toward New Requirements to Data Analysis and Presentation in Social Sciences
Annotation
PII
S013216250003163-2-1
DOI
10.31857/S013216250003163-2
Publication type
Article
Status
Published
Authors
I. Deviatko 
Affiliation:
National Research University Higher School of Economics
Institute of Sociology FCTAS RAS
Address: Russian Federation, Moscow
Edition
Issue 12
Pages
30-38
Abstract

The article analyzes main causes and consequences of the interdisciplinary crisis of the reproducibility and reliability of the results of scientific research that has unfolded in the social sciences in parallel with the «data revolution». This crisis is expressed not only in the growing concern of scientists about the reliability of research results and the possibilities to establish the practices securing the transparency of empirical data and the statistical software used for their analysis, but also in disputes on limitations of the routine approach to significance testing and feasibility of alternatives based on Bayesian approach. Some aspects of the relationship between theory and data-driven methods of searching for patterns in empirical data are briefly discussed in the context of describing a new approach to multimodel analysis aiming at evaluation of model robustness and model uncertainty.

Keywords
reproducibility crisis in social sciences, transparency of data and open data, publication, bias, null hypothesis significance testing and Bayesian approach, model robustness, data-driven approach and sociological theory
Acknowledgment
The paper is supported by the Russian Science Foundation, grant 17-78-20172
Received
26.12.2018
Date of publication
09.01.2019
Number of purchasers
10
Views
693
Readers community rating
0.0 (0 votes)
Cite   Download pdf

References

1. Deviatko I.F. (2016) From «Virtual Lab» to «Social Telescope»: Metaphors of Theoretical and Methodological Innovations in Online Research. In: Online-research in Russia: Trends and Prospects. Ed. by A.V. Shashkin, I.F. Deviatko, S.G. Davydov. Moscow: Online Market Intelldgence: 19–33. (In Russ.)

2. Deviatko I.F. (1991) TETRAD-methodology: A Completion of Procedural Episteme? Vestnik Akademii nauk SSSR [Herald of the Russian Academy of Sciences]. No. 2: 79–94. (In Russ.)

3. Kitchin R. (2017) Big Data, New Epistemologies and Paradigm Shifts. Sotsiologiya: metodologiya, metody, matematicheskoe modelirovanie (4M) [Sociology: Methodology, Methods, Mathematical Modeling (4M)]. No. 44: 111–152(In Russ.)

4. Schrodt Ph.A. (2016) Seven Deadly Sins of Contemporary Quantitative Political Analysis. Sotsiologiya: metodologiya, metody, matematicheskoe modelirovanie (4M) [Sociology: Methodology, Methods, Mathematical Modeling (4M)]. No. 43: 154–210 (In Russ.)

5. Appelbaum M., Kline R., Nezu A., Cooper H., Mayo-Wilson E., Rao S.M. (2018) Reporting Standards for Quantitative Research in Psychology: The APA Publications and Communications Board Task Force Report. American Psychologist. Vol. 73 (1): 3–25. DOI: 10.1037/amp0000191.

6. de Groot A.D. (1956/2014) The Meaning of «Significance» for Different Types of Research. [Transl. and annot. by E.-J.Wagenmakers et al.] Acta Psychologica. Vol. 148 (May): 188–194.

7. Doucouliagos, C., Stanley T.D. (2013) Are All Economic Facts Greatly Exaggerated? Theory Competition and Selectivity. Journal of Economic Surveys. No. 27(2): 316–39.

8. Gigerenzer G. (2004) Mindless Statistics. The Journal of Socio-Economics. Vol. 33(5): 587–606.

9. Glymour C., Scheines R., Spirtes P. and Kelly K. (1987) Discovering Causal Structure. San Diego, CA: Academic Press.

10. Graham J.H., Özener B. (2016) Fluctuating Asymmetry of Human Populations: A Review. Symmetry. Vol. 8(154): 1–36.

11. Haller H., Kraus S. (2002). Misinterpretations of Significance: A Problem Students Share with Their Teachers? Methods of Psychological Research. No. 7(1): 1–20.

12. Ioannidis J.P.A. (2005) Why Most Published Research Findings Are False. PLoS Med. No. 2(8): e124. DOI: 10.1371/journal.pmed.0020124.

13. Kerr N.L. (1998) HARKing: Hypothesizing after the Results Are Known. Personality and Social Psychology Review. Vol. 2(3): 196–217.

14. Lehrer J. (2010) The Truth Wears off: Is There Something Wrong with the Scientific Method? The New Yorker. December, 13.

15. Munafò M.R., Nosek B.A., Bishop D.V.M., Button K.S., Chambers C.D., Percie Du Sert N., Simonsohn U., Wagenmakers E.J., Ware J.J., Ioannidis J.P.A. (2017) A Manifesto for Reproducible Science. Nature Human Behaviour. Vol. 1(0021). DOI:10.1038/s41562-016-0021.

16. Nelson L., Simmons J.P., Simonsohn U. (2012) Let’s Publish Fewer Papers. Psychological Inquiry. Vol. 23(3): 291–293.

17. Nuijten M.B., Hartgerink C.H., van Assen M.A., Epskamp S., Wicherts J.M. The Prevalence of Statistical Reporting Errors in Psychology (1985–2013). Behavior Research Methods. Vol. 48: 1205–1226.

18. Oakes M. Statistical Inference: A Commentary for the Social and Behavioral Sciences. New York: Wiley, 1986.

19. Open Science Collaboration (Nosek B.A. et al.) (2015) Estimating the Reproducibility of Psychological Science. Science. 2015. No. 349(6251): aac4716. DOI: 10.1126/science.aac4716.

20. Rosenthal R. (1979) The «File Drawer Problem» and Tolerance for Null Results. Psychological Bulletin. Vol. 86(3): 838–641.

21. Rouder J.N., Speckman P.L., Sun D., Morey R.D., Iverson G. (2009) Bayesian t Tests for Accepting and Rejecting the Null Hypothesis. Psychonomic Bulletin & Review. No. 16(2): 225–237. DOI:10.3758/ PBR.16.2.225.

22. Selvin H.C. (1957) A Critique of Tests of Significance in Survey Research. American Sociological Review. Vol. 22(5): 519–527.

23. Stapel D.A., Lindenberg S. (2011) Coping with Chaos: How Disordered Contexts Promote Stereotyping and Discrimination. Science. No. 332(6026): 251–253.

24. Wagenmakers E-J., Verhagen J., Ly A., Bakker M., Lee M.D., Matzke D., Rouder J.N., Morey R.D. (2015). A Power Fallacy. Behavior Research Methods. No. 47(4): 913–917. DOI: 10.3758/s13428-014-0517-4.

25. Yong E. (2012) Replication Studies: Bad Copy. Nature. No. 485(7398): 298–300.

26. Young C. (2018) Model Uncertainty and the Crisis in Science. Socius: Sociological Research for a Dynamic World. Vol. 4: 1–7. DOI: 10.1177/2378023117737206.

Comments

No posts found

Write a review
Translate