3.3.12 Quality criteria

It seems that no consensus exists with regards to the standard of methodological rigor to apply. And that “no definitive evidence exists which demonstrates the reliability or validity of the technique” (Keeney, 2011⁹⁵, p. 104). This is partly due to the variety of the Delphi surveys and the constant evolutions in this field⁹¹.

We have not identified any checklists to assess the quality of a Delphi survey.

However, the following aspects of the survey could be assessed (adapted from Jillson¹⁰³ and Hasson⁹¹):

• Applicability of the method to the specific research problem
• The quality of the composition of the Delphi panel. Participants have to be carefully chosen in function of their expertise and position in the group.
• Design and administration of the questionnaire
• Feedback

A Delphi survey should be reviewed in terms of reliability, validity and trustworthiness to judge its worth⁹¹.