RSOS royopensci Royal Society Open Science R. Soc. Open Sci. 2054-5703 The Royal Society 10.1098/rsos.221255 rsos221255 1001 205 Science, society and policy Review articles Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes http://orcid.org/0000-0002-3734-8006Madeleine PownallConceptualizationData curationSupervisionWriting – original draftWriting – review & editing 1, https://orcid.org/0000-0001-9000-8513Flávio AzevedoConceptualizationFormal analysisSupervisionWriting – original draftWriting – review & editingfalafla@gmail.com 2, Laura M. KönigFormal analysisMethodologyWriting – original draftWriting – review & editing 3, http://orcid.org/0000-0003-2522-8717Hannah R. SlackWriting – original draftWriting – review & editing 4, Thomas Rhys EvansInvestigationMethodologyMethodologyWriting – original draftWriting – original draftWriting – review & editingWriting – review & editing 5, 6, Zoe FlackInvestigationWriting – original draftWriting – review & editing 7, Sandra GrinschglInvestigationResourcesWriting – original draftWriting – review & editing 8, http://orcid.org/0000-0002-0540-3998Mahmoud M. ElsherifInvestigationMethodologyResourcesWriting – original draftWriting – review & editing 9, https://orcid.org/0000-0002-5406-2149Katie A. Gilligan-LeeWriting – review & editing 10, http://orcid.org/0000-0002-2976-3330Catia M. F. de OliveiraMethodologySoftwareWriting – original draftWriting – review & editing 11, http://orcid.org/0000-0003-1200-6672Biljana GjoneskaInvestigationMethodologyResourcesWriting – original draftWriting – review & editing 12, Tamara KalandadzeInvestigationWriting – original draftWriting – review & editing 13, Katherine ButtonInvestigationMethodologyWriting – review & editing 14, Sarah Ashcroft-JonesInvestigationWriting – original draft 15, http://orcid.org/0000-0002-6843-7116Jenny TerryInvestigation 16, http://orcid.org/0000-0003-3412-4311Nihan Albayrak-AydemirInvestigationResourcesWriting – original draftWriting – review & editing 17, 18, Filip DěchtěrenkoInvestigationWriting – review & editing 19, Shilaan AlzahawiInvestigationResourcesWriting – review & editing 20, http://orcid.org/0000-0002-1697-4198Bradley J. BakerResourcesWriting – original draftWriting – review & editing 21, http://orcid.org/0000-0002-7487-7898Merle-Marie PittelkowMethodologyWriting – review & editing 22, Lydia Riedl 23, http://orcid.org/0000-0002-9946-5953Kathleen SchmidtWriting – review & editing 24, http://orcid.org/0000-0002-5259-642XCharlotte R. PenningtonInvestigationResourcesWriting – original draftWriting – review & editing 25, http://orcid.org/0000-0003-3190-6772John J. ShawInvestigationResourcesWriting – original draftWriting – review & editing 26, Timo LükeInvestigationWriting – review & editing 27, Matthew C. MakelResourcesWriting – original draftWriting – review & editing 28, http://orcid.org/0000-0002-1331-6683Helena HartmannInvestigationWriting – review & editing 29, http://orcid.org/0000-0003-3569-931XMirela ZanevaInvestigationWriting – original draftWriting – review & editing 15, http://orcid.org/0000-0002-9369-6953Daniel WalkerWriting – review & editing 33, Steven VerheyenResourcesWriting – review & editing 30, Daniel CoxWriting – review & editing 31, Jennifer MattscheyMethodologyWriting – review & editing 17, Tom Gallagher-MitchellWriting – review & editing 32, Peter BranneyInvestigationWriting – review & editing 33, http://orcid.org/0000-0002-4219-6625Yanna WeisbergWriting – original draftWriting – review & editing 34, http://orcid.org/0000-0002-9870-3825Kamil IzydorczakWriting – original draftWriting – review & editing 35, Ali H. Al-HoorieInvestigationWriting – review & editing 36, Ann-Marie CreavenWriting – original draft 37, Suzanne L. K. StewartInvestigationWriting – original draft 38, Kai KrautterWriting – review & editing 39, Karen Matvienko-SikarInvestigationWriting – review & editing 40, Samuel J. WestwoodWriting – review & editing 41, http://orcid.org/0000-0001-5766-0489Patrícia ArriagaResourcesWriting – review & editing 42, Meng LiuMethodologyWriting – review & editing 43, Myriam A. BaumWriting – review & editing 39, Tobias WingenWriting – review & editing 44, Robert M. RossInvestigationResourcesWriting – review & editing 45, Aoife O'MahonyInvestigationResourcesWriting – review & editing 46, http://orcid.org/0000-0001-6211-8600Agata BochynskaWriting – review & editing 47, Michelle JamiesonWriting – original draft 48, Myrthe Vel Tromp 50, Siu Kit Yeung 51, Martin R. Vasilev 52, Amélie Gourdon-KanhukamweInvestigationWriting – original draft 53, 54, http://orcid.org/0000-0003-0066-8222Leticia MicheliResourcesWriting – original draft 55, Markus KonkolWriting – original draft 56, David MoreauWriting – original draftWriting – review & editing 57, James E. BartlettWriting – original draft 49, Kait ClarkWriting – original draft 58, Gwen BrekelmansWriting – original draft 59, Theofilos GkinopoulosWriting – original draft 60, http://orcid.org/0000-0001-9602-5015Samantha L. TylerWriting – review & editing 61, Jan Philipp RöerInvestigationWriting – original draft 62, Zlatomira G. IlchovskaInvestigationResourcesWriting – review & editing 9, Christopher R. MadanWriting – review & editing 4, Olly RobertsonWriting – review & editing 63, 64, http://orcid.org/0000-0002-5813-3303Bethan J. IleyInvestigationResourcesWriting – original draft 65, Samuel GuayResourcesSoftware 66, http://orcid.org/0000-0001-5059-6576Martina SladekovaWriting – original draft 16, 7, Shanu SadhwaniWriting – original draft 7 and FORRT 67 School of Psychology, University of Leeds, Leeds LS2 9JT, UK Department of Psychology, University of Cambridge, CB2 3EB, UK Faculty of Life Sciences: Food, Nutrition and Health, University of Bayreuth, 95447 Bayreuth, Germany School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK School of Human Sciences, University of Greenwich, London SE10 9LS, UK Centre for Workforce Development, Institute for Lifecourse Development, University of Greenwich, London SE10 9LS, UK School of Humanities and Social Science, University of Brighton, BN2 0JY, UK Institute of Psychology, University of Graz, 8010 Graz, Austria School of Psychology, University of Birmingham, B15 2TT, UK School of Psychology, University College Dublin, Dublin 4, DO4 V1WD, Ireland Department of Psychology, University of York, York YO10 5DD, USA Macedonian Academy of Sciences and Arts, North Macedonia, XCWR+GJM, 1000 Faculty of Teacher Education and Languages, Department of Education, ICT and Learning, Ostfold University College, 1757 Halden, Norway Department of Psychology, University of Bath, Bath BA2 7AY, UK Department of Experimental Psychology, University of Oxford, Oxford OX1 4BH18, UK School of Psychology, University of Sussex, Brighton BN1 9RH, UK School of Psychology and Counselling, the Open University, Milton Keynes MK7 6AA, UK Department of Psychological and Behavioural Science, London School of Economics and Political Science, UK Department of Mathematics, College of Polytechnics Jihlava, 1556/16, 586 01, Czech Republic Graduate School of Business, Stanford University, CA 94305, USA Department of Sport and Recreation Management, Temple University, PA 19122, USA Department of Psychology, University of Groningen, 9712 CP, Groningen, the Netherlands Department of Psychiatry and Psychotherapy, Philipps-University Marburg, D-35039 Marburg, Germany Department of Psychology, Ashland University, OH 44805, USA School of Psychology, Aston University, Birmingham B4 7ET, UK Division of Psychology, De Montfort University, Leicester LE1 9BH, UK Institute for Educational Research and Teacher Education, University of Graz, Graz, 8010 Graz, Austria School of Education, Johns Hopkins University, MD 21218, USA Department for Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna 1010, Austria Department of Psychology, Education and Child Studies, Erasmus University Rotterdam, Rotterdam 3000, The Netherlands Division of Neuroscience and Experimental Psychology, University of Manchester, Manchester M13 9PL, UK Department of Psychology, Liverpool Hope University, Liverpool L16 9JD, UK Department of Psychology, Faculty of Management, Law and Social Sciences, University of Bradford, Bradford BD7 1DP, UK Department of Psychology, Linfield University, Linfield, 503-883-2200, USA Faculty of Psychology in Wrocław, SWPS University of Social Sciences and Humanities, Wrocław 03-81536, Al Jubail 35819, Poland Jubail English Language and Preparatory Year Institute, Royal Commission for Jubail and Yanbu, Saudi Arabia Department of Psychology, University of Limerick, V94 T9PX, Ireland School of Psychology, University of Chester, Chester CH1 4BJ, UK Department of Psychology, Saarland University, 66123 Saarbrücken, Germany School of Public Health, University College Cork, Cork, T12 K8AF, Ireland Department of Psychology, School of Social Science, University of Westminster, London W1B 2HW, UK Iscte-Universty Institute of Lisbon, CIS-IUL, 1649-026 Lisboa, Portugal Faculty of Education, University of Cambridge, Cambridge CB2 1TN, UK Institute of General Practice and Family Medicine, University Hospital Bonn, University of Bonn, 53127 Bonn, Germany Department of Philosophy, Macquarie University, NSW 2109, Australia School of Psychology, Cardiff University, Cardiff CF10 3AT, UK University Library, University of Oslo, 0313 Oslo, Norway School of Social and Political Sciences, University of Glasgow, Glasgow G12 8QQ, UK School of Psychology and Neuroscience, University of Glasgow, Glasgow G12 8QQ, UK Department of Psychology, Leiden University, 2311 EZ Leiden, The Netherlands Department of Psychology, the Chinese University of Hong Kong, Hong Kong, SAR 100871, People's Republic of China Department of Psychology, Bournemouth University, Poole BH12 5BB, UK Department of Neuroimaging, IoPPN, King's College London, UK IGDORE, London, UK Department of Psychology III, University of Würzburg, 97070 Würzburg, Germany Faculty for Geo-Information Science and Earth Observation, University of Twente, 7522 NB, The Netherlands School of Psychology, University of Auckland, Auckland 1142, New Zealand Department of Social Sciences, University of the West of England, Bristol BS16 1QY, UK Department of Biological and Experimental Psychology, Queen Mary University of London, E1 4NS, UK South-East European Research Centre, Thessaloniki, Greece Department of Neuroscience, Psychology and Behaviour, University of Leicester, UK Department of Psychology, Witten/Herdecke University, Germany Departments of Psychiatry and Experimental Psychology, University of Oxford, UK School of Psychology, Keele University, Newcastle ST5 5BG, UK School of Psychology, Queen's University Belfast, Belfast BT7 1NN, UK Department of Psychology, University of Montreal, Canada Framework for Open and Reproducible Research Training

Electronic supplementary material is available online at https://doi.org/10.6084/m9.figshare.c.6641489.

 1752023May 17, 2023 52023 10 5221255 2792022September 27, 2022 2642023April 26, 2023 © 2023 The Authors. 2023 http://creativecommons.org/licenses/by/4.0/ http://creativecommons.org/licenses/by/4.0/ Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.

In recent years, the scientific community has called for improvements in the credibility, robustness and reproducibility of research, characterized by increased interest and promotion of open and transparent research practices. While progress has been positive, there is a lack of consideration about how this approach can be embedded into undergraduate and postgraduate research training. Specifically, a critical overview of the literature which investigates how integrating open and reproducible science may influence student outcomes is needed. In this paper, we provide the first critical review of literature surrounding the integration of open and reproducible scholarship into teaching and learning and its associated outcomes in students. Our review highlighted how embedding open and reproducible scholarship appears to be associated with (i) students' scientific literacies (i.e. students’ understanding of open research, consumption of science and the development of transferable skills); (ii) student engagement (i.e. motivation and engagement with learning, collaboration and engagement in open research) and (iii) students' attitudes towards science (i.e. trust in science and confidence in research findings). However, our review also identified a need for more robust and rigorous methods within pedagogical research, including more interventional and experimental evaluations of teaching practice. We discuss implications for teaching and learning scholarship.

 higher education, open research, open scholarship, open science, pedagogy, teaching