Categories
Digital Skills Digital Tools

Teaching and Assessing Laboratory Sessions Online

Teaching and Assessing Laboratory Sessions Online 

Nicholas Harmer and Alison Hill share how imaginative use of instructional videos and  assessment material enable students to complete their exams

Authors : Nicholas J. Harmer and Alison M. Hill

What exactly is the shift in culture and/or organisational practice that you wish to highlight?  

The COVID-19 pandemic provided a challenge to deliver and assess experimental learning. Many approaches have been used (recently reviewed by Kelly [1]). Online solutions have included the use of videos [2-4], simulations (off the shelf or created bespoke) [5-7], virtual reality [8], or live online streamlining [9,10] where teaching assistants or academic staff film themselves live carrying out the experiment. Another solution has been to send out practical kits to students for them to perform experiments at home [11]. Laboratory instruction has the twin purposes of gaining hands-on experience of performing techniques and processing and interpreting the data collected. To achieve the second goal, historical data sets [2,5,9,12] or data from teaching assistants/instructors [4,13] have been provided to students to process. Others have asked students to generate the data themselves from simulations [7]. We required a solution that would enable students to understand the theory behind the practical techniques, to see how these techniques were carried out, and to undertake data analysis. Our solution was to create laboratory videos [14] (figure 1) of the instructors performing experimental work using our own equipment in our teaching laboratory to replace the hands-on laboratory session. This was supplemented with interactive Learning Science LabSims [15] to consolidate the theory and steps of each technique. Finally, we created personalised data generated by modelling for data analysis.  

Figure 1: Our approach to teaching and assessing laboratory work involved the use of laboratory videos and generating unique data sets for students with associated answers for staff.

The assessed practical involves the separation of a mixture of three proteins (from a possible eight) using biochemical separation techniques. Students are asked to write an extended laboratory report worth 35% of their module score. While students were not able to collect their own data in 2020/21, we reasoned that if we supplied them with a data set, they could gain all the intended data processing skills. However, we wanted to ensure that the students submitted their own work, and not the result of collaboration where some students rely on the data interpretation of others in their group. Therefore, we adopted unique data sets.  Moving the experiment online allowed us to include an additional two proteins that are impractical to use in the laboratory. This allowed us to expand the number of combinations of proteins from the usual three to eight.

Historical data were used to develop a mathematical model in R [16] of the separation techniques. This strategy allowed us to create realistic data that were pre-screened to ensure it was interpretable, but also with some randomness introduced to provide realistic, unique data sets. The programme produced individual unique data files for students with numerical data and an image (figure 2A). Simultaneously, a corresponding answer file was generated with answers and plotted data to be used for marking (figure 2B) 

Figure 2. Example unique student (A) and staff (B) data packs for the laboratory report.

Students were provided with a laboratory schedule, a laboratory video, LabSims, their unique data set and a video explaining how to process their data and write up their report. A Padlet was used to collate questions ahead of two synchronous sessions where students’ questions about data processing and writing up their reports were answered.

2. What did ‘working well’ look like?  

All students used their own data files and there was no evidence of student collusion. We observed no difference (Kruskal-Wallis Test) in student attainment with previous years with the mean and median marks remaining the same, demonstrating the robustness of our approach (figure 3).  

Figure 3. Box and whisker plot of Laboratory report performance 2015/16-2020/21. Boxes show 25th-75th percentiles and the whiskers 10-90th percentiles. Data from all years were compared to the 2020/21 cohort using the Kruskal-Wallis test. Significance levels: ns, p > 0,05; *, 0.01 < p < 0.05. 

‘Lab simulations gave a taste of what the practical work would have been like, allowing us to understand how the theory is put into practice’. Student feedback 2021. 

There was much attention to detail in each of the filmed practical videos so that students were fully able to experience the experiments without having done them before. The practical videos were well received by the students and helped aid live sessions making them more succinct’. Dr Sariqa Wagley, Biosciences Digital Lead 

For the laboratory data, there was an initial investment in using historical data to generate a mathematical model of the experiment (4-6 hours for an academic with limited R experience to write the code). We gained the capability to include additional reagents that are too expensive or precious for laboratory use in the simulated data. This increased the number of permutations/possible answers. For the staff, the answer packs resulted in a reduction in marking time of the laboratory report as the data were processed in the answer packs. 

While many others have used laboratory videos and LabSims during the pandemic to replace laboratory work, our methodology of creating unique data sets is novel and highly effective. This approach can be scaled at no extra effort which would not be possible where each data set had to be created individually. 

In 2021/22, we have returned to the laboratory and students are once again able to collect their own data. The methodology was used to create unique data sets for students who missed the session through illness. The laboratory videos are now used for pre-lab preparation and during the laboratory session to show close-ups of the techniques which is much more effective than having the whole class clustered around a single experimental rig. We continue to use the LabSims for pre-lab preparation and consolidation of theory. Students are now coming into the laboratory with more confidence in the techniques.  

3. How could this practice be spread?  

We have presented the work at three conferences and written a paper for Journal of Chemical Education [17]. This paper includes annotated R scripts to make it easier for others to adopt this method. We have also published the R files at https://github.com/njharmer/Student-specific-data. We put out a press release and this was featured on the University news [18] and subsequently picked up by other news agencies [19-22].

We believe our method of generating unique data sets could be applied widely. This approach is suitable anywhere where there is a unique numerical answer (e.g., mathematics, economics, statistics, chemistry, biological sciences, physics, social sciences etc.). The R code can be readily adapted to incorporate images into the student documents, for example mathematical symbols or chemical diagrams. The same approach could easily be used in other programming languages that academics are familiar with.

References

1.Kelley, E. W., LAB theory, HLAB pedagogy, and review of laboratory learning in chemistry during the COVID-19 pandemic, J. Chem. Educ., 2021, 98, 2496-2517; DOI: 10.1021/acs.jchemed.1c00457.

2.Howitz, W. J., Thane, T. A, Frey, T. L., Wang, X. S., Gonzales, J. C., Tretbar, C. A., Seith, D. D., Saluga, S. L., Lam, S., Nguyen, M. M., Tieu, P., Link, R. D., and Edwards, K.D. Online in no time: Design and implementation of a remote learning first quarter general chemistry laboratory and second quarter organic chemistry laboratory. J. Chem. Educ., 2020, 97, 2624-2634; DOI: 10.1021/acs.jchemed,0c00895.

3.Cabalsa, J. M., and Abraham, L., Exploring biochemical reactions of proteins, carbohydrates, and lipids through a milk-based demonstration and an inquiry-based worksheet: A COVID-19 laboratory experience, J. Chem. Educ., 2020, 97, 2669-2677; DOI: 10.1021/acs.jchemed.0c00666 .

4.Donovan, W. J., The whiplash of a COVID-19 teaching pivot and the lessons learned for the future, J. Chem. Educ., 2020, 97, 2917-2921; DOI: 10.1021/acs.jchemed.0c00755.

5.Dukes III, A. D. Teaching an instrumental analysis laboratory course without instruments during the COVID_19 Pandemic. Chem. Educ., 2020, 97, 2967-2970; DOI: 10.1021/acs.jchemed.0c00648.

6.Mistry, N., and Shahid, N., Design and delivery of virtual inquiry-based organic chemistry experiments, Chem. Educ., 2021, 98, 2952-2958; DOI: 10.1021/acs.jchemed.1c00571.

7.Perri, M. J., Online data generation in quantitative analysis: Excel spreadsheets and an online HPLC simulator using a Jupyter notebook on the Chem Compute web site, Chem. Educ., 2020, 97, 2950-54; DOI: 10.1021/acs.jchemed.0c00565.

8.Ku, Y., Xu, Y., and Zhu, X., Designing and implementing VR2E2C, a virtual reality remote education for experimental chemistry system, J. Chem. Educ., 2021, 98, 2720-2725; DOI: 10,1021/acs.jchemed.1c00439.

9.George-Williams, S., Motion, A., Pullen, R., Rutledge, P. J., Schmid, S., and Wilkinson, S., Chemistry in the time of COVID-19: reflections on a very unusual semester, J. Chem. Educ., 2020, 97, 2928-2934.

10.Petillion, R. J., and McNeil, W. S., Student satisfaction with synchronous online organic chemistry laboratories: Prerecorded video vs livestream, J. Chem. Educ., 2021, 98, 2862-2869; DOI: 10.1021/acs.jchemed.1c00549.

11.Caruana, D. J., Salzmann, C. G., and Sella, A., Practical science at home in a pandemic world, Nature Chemistry, 2020, 12, 790-783; DOI: 10.1038/s41557-020-0543-z.

12.Valle-Suárez, R. M., Calederón-Mendoza, G. L., Lanza-Sorto, N. A., and Ponce-Rodríguez, Teaching instrumental analytical chemistry in the framework of COVID-19: Experiences and outlook, J. Chem. Educ., 2020, 97, 2723-2726; DOI: 10.1021/acs.jchemed.0c00707.

13.Donovan; Karayilan, M., Vakil, J., Fowler, D., Becker, M. L., and Cox, Jr, C. T., Zooming in on polymer chemistry and designing synthesis of high sulfur-content polymers for virtual undergraduate experiment, J. Chem. Ed., 2021, 98, 2062-2073; DOI: 10.1021/acsjchemed.1c00105.

14.Wren, A., Bitpod. Bitpod.co.uk (accessed 7 Jan 2022).

15.Learning Sciences, Learning Science laboratory simulations, https://learningscience.co.uk/labsims (accessed 7 Jan 2022).

16.R v.3.6.1 (R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ (accessed 7 Jan 2022) and RStudio v1.1.456 were used in this work. www.rstudio.com (accessed 7 Jan 2022); a cloudbased version is available: https://rstudio.cloud (accessed 7 Jan 2022).

17.Harmer, N. J., and Hill, A. M., ‘Unique Data sets and Bespoke Laboratory Videos: Teaching and Assessing of Experimental Methods and Data Analysis in a Pandemic’, 2021, J. Chem. Educ., 98: 4094-4100; DOI: 10.1021/acs.jchemed.1c00853.

18.University of Exeter, https://www.exeter.ac.uk/news/research/title_890418_en.html (date accessed 7 Jan 2022)

19.Mirage news, https://www.miragenews.com/unique-data-creates-fair-and-robust-online-exams-693956/ (date accessed 7 Jan 2022).

20.Phys.org, https://phys.org/news/2021-12-unique-fair-robust-online-exams.html (date accessed 7 Jan 2022).

21.Brogan, T., Education Technology, https://edtechnology.co.uk/e-learning/digital-learning-pioneers-develop-fair-and-robust-online-exam-method/ (date accessed 7 Jan 2022).

22.Abraham, J., Eminetra, https://eminetra.co.uk/digital-learning-pioneers-develop-fair-and-robust-online-test-methods/870121/ https://phys.org/news/2021-12-unique-fair-robust-online-exams.html (date accessed 7 Jan 2022).

Join the conversation at the Digital Culture Forum

By Nicholas J. Harmer

Assoc. Prof. in Biochemistry, Living Systems Institute, University of Exeter

Leave a Reply

Your email address will not be published. Required fields are marked *