How do you know if you have designed a "good" assessment task?

How do you know if you have designed a "good" assessment task? What characterizes a good assessment? The three films on this page will help you to reflect on how to design your own assessments as well as on which considerations can influence your decisions. The films on this page discuss:

Criteria for good assessment - Assessing Health Professional Education (John Norcini)
Engaged Feedback (Chris Rust and David Boud)
The teachers' dilemma (Lena Boman Enqvist, Karolinska Institutet)

Criteria for Good Assessment - Assessing Health Professional Education

The Ottawa Conference 2010 produced a white paper with set of criteria for good assessment that include: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability.

In this film Characteristics of good assessment (12 min) John Norcini, NASEM Health and Medicine Division, presents these criteria. These were presented for a workshop he was about to hold so please ignore that he says "seeing you in October" since this does not refer to us at Karolinska Institutet.

To summarize from Norcini here are the seven criteria he mentions for good assessment:

Validity: The examination measures / assesses what is stated in the learning objectives.
Reliability (Reproducibility): Different assessors will give an objective assessment of the students' work based on agreed criterion.
Equivalence: Different versions of an exam must be at the same level ie the same severity.
Educational effect: The examination should motivate the student to learn.
Catalytic effect: The examination provides feedback that stimulates learning.
Acceptability: Different stakeholders find the examination process and the results credible.
Feasability: The exam is practical and realistic.

Extra reading:

UME Medical Education Guide nr 4: "Making Thinking Visible - Assessment and assessment criteria" by Cormac McGrath, Karolinska Institutet, 2007
Assessment methods in medical education, John J. Norcini, Danette W. McKinley
Teaching and Teacher Education 23 (2007) 239–250
You can also read more in the article: Med Teach. 2011;33(3):206-14. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Norcini J1, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, Galbraith R, Hays R, Kent A, Perrott V, Roberts T.

Engaged Feedback

In the following film (3:35 min) Professor Chris Rust and Professor David Boud discuss the assessment and feedback principle of 'engaged feedback'.

They describe engagement as being about moving beyond the ineffective one-way communication of poor assessment, where assessment is a monologue of "input" to the student. Instead they advocate the feedback loop where there is a dialogue with the student. Engaged feedback is seen as being about getting students to truly interact with the quality of their work. As an instructor you need to find ways of engaging students with feedback by using self-assessment and peer-assessment, marking exercises or getting students to discuss criteria.

The teachers' dilemma

In this film (3:05min) Lena Engqvist Boman presents a report which outlines the Teachers' dilemma in designing assessment. This report is in Swedish and is written by Lena Engqvist Boman, Ingela Thorsson and Håkan Hult and is based on interviews with teachers at the nursing programme at Karolinska Institutet. The dilemmas raised include for example the question whether assessment should be an opportunity for learning? or for control? Lena summarizes the report in English but read the report in Swedish here: Lärares dilemman vid examination