The resources on this page are intended to give you a ‘head start’ on tertiary education research, and are mapped to broad steps in the research journey.
Allied Health Scholarship of Teaching & Learning (SoTL) Seminar Series
The Allied Health Program (within the College of Sport, Health and Engineering, Victoria University) officially launched the inaugural Allied Health Scholarship of Teaching and Learning (SoTL) Seminar Series in October 2021.
This 12-part Seminar Series follows Victoria University Connected Learning's 'Steps in the Educational Research Process' and aims to increase academic staff:
- confidence, and perceived self-efficacy and readiness in undertaking SoTL
- involvement and collaboration in SoTL.
The Seminars provide an excellent opportunity to hear a snapshot of the presenter’s SoTL experiences (with respect to their allocated Seminar topic) and to connect with academics with teaching and research backgrounds who are affiliated with Allied Health, along with invited guests and collaborators.
The Seminar Series Leads (Dr Julia Gilmartin-Thomas, Professor Rebecca Lane, Mr Rees Thomas, Dr Daniel Loton and Mr Nicholas Tripodi) would like to invite you to attend an upcoming Seminar.
The Seminars are:
- 15 minutes in duration
- one Monday per month from 1pm to 1.15pm
- open to anyone with an interest in SoTL (internal or external to Victoria University).
If you are external to Victoria University, or if you have any queries about the Seminar Series, please email: [email protected]
Steps & resources in the educational research process
- Choose a topic and read prior research
- Devise research questions and project to address a knowledge gap
- Form a team, consider team roles, and HDR supervision
- Consider seeking funding for the project
- Connected Learning SoTL page, HERDSA SoTL modules
- Researching the Educational Experience @ VU, see ‘Quickly get a grasp of tertiary education research’
- Consult Educational Research Advisor
- Quality tertiary education Research Design
- Consider pre-existing VU institutional data
- Consider pre-registering the project, including questions/aims and hypotheses on the Open Science Framework
- Ethical approval
- Pre-register on Open Science Framework (OSF)
- Gain Data Insights approval
- Connected Learning ethical micro-project for Higher Education
- VUHREC Ethics process
- Introduction in Research Integrity Training – Arts and Humanities
- Check and obtain permissions to use any established or validated scales
- Create a recruitment / data collection plan
- If using institutional data, extract, filter and merge to form dataset for analysis
Submit a peer-reviewed journal article and/or conference
Tertiary education research methodologies
Designing a tertiary education (TE) research project involves consideration of research methodology: the specific approaches to collecting, analysing, and making sense of data that ultimately produce the key outcome - findings. To support TE research at VU, we’ve compiled sources that give an overview of research design and methodology common in the TE field.
Some methodological considerations are unique to certain research communities, while others are somewhat universal. Methods are often discussed within the broader traditions of quantitative research, that follows empiricism and uses numbers to understand a phenomenon; and qualitative research, which generally takes a constructivist approach to understand how people construe meaning in their experiences.
If you are new to research, or to TE research – don’t worry! Almost every conceivable methodology has been applied and published in TE. In other words, there is no one right way to do research, but a multiplicity of approaches. With an informed application grounded in prior research, with appropriate justification, the method adopted will help to make a meaningful contribution to knowledge.
Many analyses make use of specific software. Rstudio and R packages are commonly used for quantitative research, and is usually free to use for research. VU also has licenses for SPSS (descriptive and inferential statistics), AMOS (structural equation modelling) and NVivo (qualitative analysis), which can be found on the VU SoftwareCentre.
VU has several methodological training opportunities via the Research Professional Development calendar, including some on specific software. We also recommend signing up to the VU research info mailing list for further updates on researcher development (intranet sign-in required). Further enquiries can be directed to the Educational Research Advisor
Here is a list of specific methods applied in the higher education context:
General higher education research
Scholarship of Teaching and Learning (SotL) HERDSA online modules
Sampling - Robinson, O. C. (2014). Sampling in interview-based qualitative research: A theoretical and practical guide. Qualitative Research in Psychology, 11(1), 25-41., van Rijnsoever, F. J. (2017). (I can’t get no) saturation: a simulation and guidelines for sample sizes in qualitative research. PLoS One, 12(7), e0181689.
Auto-ethnography - Herrmann, A. F. (Ed.). (2020). The Routledge International Handbook of Organizational Autoethnography. Routledge. (available via library request)
Conversation analysis - Sidnell, J., & Stivers, T. (2013). The handbook of conversation analysis. Wiley-Blackwell.
Practice-based theory - Hofmann, R. (2020). Dialogues with data: generating theoretical insights from research on practice in higher education. In Theory and Method in Higher Education Research. Emerald Publishing Limited.
For studies using raters, observers or coders:
Inter-rater reliability - Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters. Advanced Analytics, LLC. (available on request)
Survey design - Fowler Jr, F. J., & Cosenza, C. (2009). Design and evaluation of survey questions. The SAGE handbook of applied social research methods, 375-412.
Validity and reliability of measures - Wagemaker, H (2020). Reliability and Validity of International Large-Scale Assessment. Springer.
Psychometrics: classical test theory, IRT, SET E-SEM - Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: optimal balance between goodness of fit and parsimony. Multivariate Behavioral Research, 55(1), 102-119., Carlson, J. E. (2020). Introduction to Item Response Theory Models and Applications. Routledge.
Power analysis - Nakagawa, S., & Foster, T. M. (2004). The case against retrospective statistical power analyses with an introduction to power analysis. Acta ethologica, 7(2), 103-108.
Mediation and moderation - Hayes, Andrew F. (2013). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. New York, NY: The
Multi-level modelling - Gorard, S. (2003). What is Multi–level Modelling For?. British Journal of Educational Studies, 51(1), 46-63.
Causal modelling in longitudinal data - Zyphur, M. J., Voelkle, M. C., Tay, L., Allison, P. D., Preacher, K. J., Zhang, Z., ... & Diener, E. (2020). From data to causes II: Comparing approaches to panel data analysis. Organizational Research Methods, 23(4), 688-716., Morgan, S. L., & Winship, C. (2015). Counterfactuals and causal inference. Cambridge University Press.
Randomised controlled trials - Matthews, J. N. (2006). Introduction to randomized controlled clinical trials. CRC Press.
Implementation science - Soicher, R. N., Becker-Blease, K. A., & Bostwick, K. C. (2020). Adapting implementation science for higher education research: the systematic study of implementing evidence-based practices in college classrooms. Cognitive Research: Principles and Implications, 5(1), 1-15.
Meta-analysis - Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87-122.
Mixed or multi-method
Charting usage - Munoz-Najar Galvez, S., Heiberger, R., & McFarland, D. (2020). Paradigm wars revisited: A cartography of graduate research in the field of education (1980–2010). American Educational Research Journal, 57(2), 612-652.
Quality tertiary education research reporting
Given the wide variety of methodological approaches and associate debates, it is critical to thoroughly report exactly how a research project was undertaken. The guiding principle is that the research publication should describe the method in sufficient detail for others to replicate the study, and hopefully arrive at the same conclusions (if the study is empirical). Thorough reporting will also increase the chances of acceptance during peer review.
There are several issues associated with Open Science, some are methodological, and others relate to the availability of the research outputs to the public. The two most important aspects to consider for your own TE projects are:
- pre-registration of hypotheses, which provides a public register of the hypotheses before data is collected or analysed (this prevents changing the hypothesis to fit statistical results later)
- sharing the de-identified dataset so others can re-analyse the data using different methods.
We strongly recommend you pre-register your project, including specific research questions, aims or hypotheses.
Comprehensive reporting of how a study was undertaken is a key issue, not only in TE research, but across all research fields. Many studies are not reported in sufficient detail. This is especially important given many studies in education are subsequently meta-analysed, for which detailed descriptions of the data and method are required.
Check the journal you are aiming for to see if they recommend a particular reporting checklist. The CONSORT (quantitative) or COREQ (qualitative) checklists can help in comprehensive reporting. There is also a mixed-methods appraisal tool for mixed methods research. Finally, the Healey (2019) paper discusses distinct writing styles in SoTL.
With the rise of research output metrics, there are also wider debates on how to assess quality in research. While these may be more relevant to institutions than individual researchers, it is worthwhile being cognizant of their development.
Finally, when your draft is complete, consider uploading your manuscript to a pre-print server. These enable you to share results prior to publication in a journal, to show the evolution of the work, and most journals support the use of pre-prints. EdArxIv is one option that is focused on education, but there are other non-specific pre-print servers also, like Springer’s In Review.