Prof Ross Galloway, School of Physics & Astronomy, University of Edinburgh

Over the duration of a physics degree, students will answer hundreds (thousands?) of questions set for them by academic staff. But how often will they write assessment questions themselves? Student generated assessment questions are a form of student generated content where students come up with questions, not to ask of the teaching staff, but for each other to solve. It involves students engaging in the same activities as faculty do when setting questions: designing questions, choosing distractors, designing model answers, and giving feedback.

There are a number of motivations for student generated questions:

1. Generation of additional learning resources. This places students in the role of educational partners, where ‘crowd-sourcing’ can build a large bank of questions. Students can target areas they (not we) think are important for additional coverage. It gives students a sense of ‘ownership’, and helps strengthen students’ own understanding.
2. Opportunities for student creativity. There is lots of latent creativity in a class of students, but physics degrees typically have limited opportunities for students to express their creativity formally as part of the programme. It also allows high-performing students to stretch themselves and demonstrate their level of knowledge and skill without an upper boundary.
3. Developing students’ assessment literacy. Student generated questions help students to understand assessment structure; why do we structure questions the way we do? What are the limitations on question-setting? What makes a good or bad question? It also allows reflection on assessment-answering technique: what are these questions actually asking me to do? How do I de-code the question requirements? Finally, it promotes reflection on the feedback process: how do you get the point across in feedback? How do you interpret other people’s feedback?

PeerWise is a tool developed by Prof Paul Denny at the University of Auckland. It is a free, web-based multiple choice question repository, where the question bank is built by students. Students develop new questions with associated explanations, answer existing questions and rate them for quality and difficulty, take part in discussions, and can follow other authors.

Since generating their own assessment questions is typically an unfamiliar experience for students, it is important to set clear expectations for the type of questions to be generated. If left to their own devices, students often write questions based around factual recall, or simple conceptual understanding, of a kind that can be answered with a few moments’ thought. Such questions do serve a purpose, of course, but there is potential for students to develop much more sophisticated questions that involve extended problem solving, calculation, synthesis of ideas and critical evaluation: the multiple choice question format need be no barrier to such ambitious types of questions.

To assist students to make the most out of the PeerWise system, some form of scaffolding activity is very useful. This can take the form of a ‘worked example’ of setting a challenging multiple choice question, complete with discussion of plausible and effective distractor options, and a model solution. Students can work through the example in groups and share their ideas for how they would approach the task.

We have found that the first few submissions to the PeerWise question repository set the tone and expectations for subsequent student activity, so ‘seeding’ the repository with one or two high quality example questions is a good idea. Our experience has been that, when suitably prepared and motivated, students are able to come up with questions of very high quality that really test important physics ideas and reasoning skills. They often choose entertaining or light-hearted contexts, as can be seen in the example question in Figure 1.

We have also found that some form of course credit attached to use of PeerWise helps with student engagement, but that attempts to assess the ‘quality’ of student submissions to PeerWise often causes students discomfort and suppresses their creativity and enjoyment of the activity. We recommend a small amount of course credit be given simply for participating and submitting the required contributions: our experience is that the actual quality of student submissions (for the most part) remains high.

We have evaluated student use of PeerWise in a number of ways, and find that students are able to generate sophisticated questions [1] and that use of PeerWise is associated positively with attainment on other course assessments, even when controlling for prior student performance [2].

PeerWise is free, very quick and simple to set up, and lets your students engage deeply with course topics through expert-like behaviours; why not give it a try with your classes?

1. Simon P. Bates, Ross K. Galloway, Jonathan Riise, and Danny Homer, “Assessing the quality of a student-generated question repository”, Phys. Rev. ST Phys. Educ. Res. 10, 020105 (2014)
2. Alison E. Kay, Judy Hardy, and Ross K. Galloway. “Student use of PeerWise: A multi-institutional, multidisciplinary evaluation”, British Journal of Educational Technology 51 (2020)