Assessing the quality of complex student work in higher education, where the boundaries between right and wrong judgements are often blurred, is a demanding and inherently subjective process. Grades are intended to represent levels of competence and can have significant impacts on students’ educational and career opportunities, as well as for society’s ability to secure qualified professionals. It is therefore essential that grading decisions are fair and consistent. Although social moderation is widely used in higher education as a mechanism of quality assurance, little is known about what happens within these deliberative settings, often likened to a black box. This is problematic, because agreement on a grade does not reveal the evaluative judgements underpinning that agreement, including the depth of discussions, potential disagreements about the quality of the work, how criteria were applied, or how this unfolds across meetings. As a result, we cannot know whether social moderation serves its intended purpose of assuring quality.
In this project, I open the black box of social moderation to examine how evaluative judgements are shaped and enacted within these settings. This is explored through three analytical entry points across three articles: (1) what justifications examiners use and how these justifications shape their evaluative judgements (published), (2) how examiners organise the moderation process, and how different organisational approaches shape what becomes subject to moderation (work in progress), and (3) how assessment criteria are reconstructed across time and space within the assessment process – from early-stage meetings aimed at calibrating examiners for the upcoming judgements, to later-stage co-grading meetings in which final grades are determined (work in progress). The project adopts a sociomaterial perspective, understanding evaluative judgements as products of unfolding interactions between human and material elements – such as examiners, assessment criteria, and student texts – that together shape what becomes foregrounded and weighted, and what is downplayed, in the process of reaching agreement.
The bachelor’s thesis at The Norwegian Police University College is selected as the empirical entry point because it represents a form of complex student work that is commonly used in higher education. The examiner group is also complex, comprising professionals from policing, law, psychology, sociology, and criminology and offering a wide range of perspectives into the assessment of the bachelor’s theses. The rich dataset provides a robust foundation for examining how evaluative judgements are enacted in settings of social moderation and is likely to be relevant for other educational contexts beyond The Norwegian Police University College.
