Consumer Involvement

The literature review

To enhance the ADC’s understanding of the value of consumer involvement in the design and delivery of ADC assessments, the ADC commissioned The University of Sydney to undertake research titled Consumer involvement in the design and delivery of examinations and assessments for professionals and health professional education: a scoping review of literature. This research involved reviewing available evidence regarding the involvement of consumers in assessments designed to assess competence for the purpose of entry to the health professions.

The research found that consumers can contribute meaningfully to the design and delivery of assessments and examinations by using their own experiences in healthcare, particularly in the areas of practitioner professionalism and communication. The research also identified that best practice for consumer involvement in the design health practitioner assessments is yet to be explored.

Consumer involvement and assessments and examinations

The ADC wanted to use the findings of this important research to lead the way in designing a process to involve consumers in driving continuous improvement in assessments and examinations.

An initial workshop was held in November 2019 that brought together a diverse group of consumer representatives from across Australia. Participants were encouraged to draw on the extensive, lived experience as consumers of dental care and reflect on the concept of the value of good communication in healthcare. The workshop allowed the ADC to develop a comprehensive list of attributes and actions, which represent good communication in a dental setting. The five key themes include:

• ensuring consumers are seen as the centre of decision making,
• seeking out and respecting consumer views and priorities,
• involving consumers in decision making, and
• clearly, respectfully, and fully explaining the information a consumer needs to make an informed decision.

The next part of the process involved using the key themes to design a communication assessment rubric. Communication is assessed within the ADC practical examination, where examiners assess candidates according to defined assessment criteria. The five key themes were reshaped into assessment criteria forming a revised communication assessment rubric.

A second workshop was held in February 2020 which brought together the consumer representatives and current ADC practical examination examiners. Participants were asked to test the revised rubric by viewing a series of examination videos and acting as examiners, scoring the candidates shown in the videos. This workshop involved extensive training and calibration for consumers and examiners. This was an important step to ensure all participants held a shared understanding of the redesigned rubric and marking process.

Using borderline regression, a criterion-referenced standard-setting method, participants’ scores were analysed and a revised passing standard for communication was identified that aligned with the new communication marking rubric.

Overall, the engagement of consumers enabled the ADC to successfully redesign the communication assessment rubric and the passing standard for the communication cluster for the ADC practical examination that more accurately reflects communication from the perspective of a consumer.

Next steps

The new communication marking rubric and passing standard for the communication cluster were introduced to the practical examination for dental practitioners in 2021. We continue to monitor the relevance of the rubric and its ability to assess communication needs of health consumers.

The ADC would like to thank our representative consumers and examiners who helped us throughout the duration of this project. The ADC will continue to drive improvement across the organisation through consumer engagement in future projects.