The Delphi Method in Research: How to Run a Consensus Study Step by Step (2026)

When a research question cannot be resolved through empirical measurement alone — because the evidence base is sparse, expert judgment is the primary currency, or a field needs to agree on definitions, priorities, or practice standards — the Delphi method in research offers a principled route to structured consensus. Developed at the RAND Corporation in the 1950s by Olaf Helmer and Norman Dalkey, the technique has since become standard practice across health sciences, education, policy, and engineering. This guide covers every decision point: panel design, round construction, consensus thresholds, variant selection, and transparent reporting under CREDES guidance.

What Is the Delphi Method?

The Delphi technique was formalised by Helmer and Dalkey at RAND in the early 1950s, with the canonical published description appearing in 1963. The method was designed to elicit and refine expert opinion on complex questions where controlled experiments are impractical and where group discussion risks social dominance effects — the tendency of vocal or high-status participants to suppress minority views. Its four defining characteristics have remained constant across six decades of application:

• Anonymity. Panellists never know who else is on the panel during rating rounds, eliminating status hierarchies and conformity pressure.
• Iteration. Multiple rounds are conducted, each building on the results of the last.
• Controlled feedback. Between rounds, the facilitator shares only the statistical summary of the group’s responses — median, IQR, frequency distribution — never attributed individual views.
• Statistical group response. Consensus is expressed as a mathematical aggregate rather than a verbal compromise.

These properties distinguish Delphi from focus groups (interactive, non-anonymous), nominal group technique (face-to-face ranking in a structured meeting), and ordinary surveys (which may repeat but do not converge toward consensus through controlled feedback).

When to Use Delphi vs Other Approaches

Delphi is appropriate when three conditions co-occur: the question resists definitive empirical resolution, expert judgment is a legitimate evidence source, and structured consensus is the desired research output. Common applications include:

• Developing clinical practice guidelines or minimum core outcome sets in health research
• Identifying competency frameworks and curriculum priorities in professional education
• Forecasting technological or policy developments over a 5–20 year horizon
• Establishing content validity indices for newly developed survey instruments
• Setting research priorities within a discipline or funding body

Delphi is not appropriate as a substitute for systematic review when an adequate empirical evidence base exists, nor as a qualitative method for eliciting lived experience — that role belongs to phenomenological interviews or grounded theory approaches. Researchers who wish to establish the construct or content validity of a measurement tool should pair Delphi findings with broader validation evidence; the complete framework is covered in the guide to construct, internal, and external validity in research design.

Panel Selection and Size

Panel composition is the most consequential methodological decision in a Delphi study. Expertise must be operationalised before recruitment — not assumed. Common operationalisations include years of clinical or professional experience in the domain, publication record in the relevant field, institutional seniority, or a scored checklist combining multiple criteria.

Panels should represent relevant heterogeneity, not merely seniority. A clinical guideline panel that includes only tertiary-centre specialists may produce consensus that does not translate to primary care. Deliberately recruiting across geography, institution type, professional role, and — in patient-relevant research — patient and carer representatives strengthens the transferability of final consensus statements.

Attrition between rounds is predictable: plan for 20–30% dropout and over-recruit accordingly. Panellists who do not complete all rounds are typically excluded from the final analysis, so documenting response rates per round and comparing the characteristics of completers and non-completers is standard, and required under CREDES.

Designing Your Rounds

Round 1: Open Exploration

Classical Delphi begins with a fully open Round 1 in which panellists respond to a broad, unstructured question — for example, “What criteria should determine priority access to a specialist pain clinic?” Responses are content-analysed thematically, condensed into discrete statements, and returned to the panel as the input for Round 2. This inductive phase ensures the consensus agenda emerges from the panellists rather than from the facilitator’s prior assumptions.

Researchers employing a modified Delphi approach often bypass Round 1 entirely, instead generating candidate statements from a prior systematic review, existing guidance documents, or qualitative interviews. This hybrid design accelerates the process but shifts agenda-setting power toward the research team. Transparency about item provenance is therefore essential in the methods section.

Rounds 2 and Beyond: Rating and Convergence

In subsequent rounds, panellists rate each statement on a numerical scale — most commonly a 9-point scale anchored at 1 (strongly disagree or completely inappropriate) and 9 (strongly agree or completely appropriate). After each round, the facilitator calculates the median and IQR for each item and returns this feedback alongside the panellist’s own prior rating. Panellists whose ratings fall outside the IQR are asked to provide a written rationale, which is shared with the group anonymously in the next round. This visibility of minority opinion is one of Delphi’s most important features: it prevents spurious convergence toward the median when genuine disagreement exists.

Most Delphi studies reach stability within two to three rating rounds. A third round is warranted when a substantial proportion of items have not reached the pre-specified threshold after Round 2, or when the IQR is narrowing but has not yet crossed it. Continuing beyond three rounds typically yields diminishing returns and risks participant attrition and fatigue.

Measuring Consensus

Consensus operationalisation is one of the most variable — and most criticised — aspects of Delphi methodology. Two primary approaches dominate the published literature:

IQR-based thresholds. On a 9-point scale, an IQR ≤ 2 is a widely used threshold for consensus, with IQR ≤ 1 indicating strong consensus. The threshold must be declared a priori in the protocol. Stricter thresholds improve the credibility of resulting statements but reduce the proportion of items reaching consensus, potentially producing a sparse final set.

Percentage-agreement thresholds. Some researchers define consensus as a specified percentage of responses falling within a pre-defined range — for example, ≥70% or ≥80% of panellists rating an item as 7–9 on a 9-point scale. This approach is intuitive but carries a subtle flaw: a bimodal distribution split between 1–3 and 7–9 can produce a low IQR while reflecting deep disagreement rather than genuine convergence. Reporting full frequency distributions, not just summary statistics, exposes this problem.

A rigorous analysis will report both metrics alongside distributions. Items reaching the threshold with a median in the disagreement zone (1–3) represent consensus for exclusion — substantively different from items where the IQR threshold is not met because opinion is genuinely divided. The appropriate outcome structure for Delphi reporting is tripartite: consensus in, consensus out, no consensus.

For studies using content analysis or thematic coding in Round 1, inter-rater agreement on item derivation should be assessed independently. The guide to inter-rater reliability and Cohen’s kappa covers the measurement approaches — weighted kappa, Krippendorff’s alpha — appropriate for this coding stage.

Delphi Variants: Classical, RAND/UCLA, and e-Delphi

Classical Delphi follows the four-feature structure above — open Round 1, iterative anonymous rating, controlled statistical feedback — with no face-to-face contact at any stage. It is the most methodologically transparent variant but the most demanding in terms of facilitation time and content-analysis expertise.

RAND/UCLA Appropriateness Method (Modified Delphi). Developed in the 1980s to guide clinical appropriateness decisions, this variant inserts a structured face-to-face or video discussion between two rating rounds. Panellists (typically capped at 18) meet after receiving Round 1 ratings, discuss areas of disagreement, then complete a second private rating round. The discussion element increases deliberative richness but introduces the social influence dynamics — authority gradients, eloquence effects, anchoring — that classical Delphi was designed to prevent. Researchers employing RAND/UCLA must explicitly acknowledge this trade-off in their methods section and design the moderation of the discussion session to minimise dominant voices.

e-Delphi. Online Delphi — delivery of all rounds via survey platforms — is now the dominant format across most research fields. It enables larger, geographically dispersed international panels; reduces coordination cost; supports rolling data collection; and allows flexible timing for participants across time zones. The choice of survey platform matters: the tool must support branching logic (routing panellists to a rationale text box only when their rating falls outside the group IQR), personalised feedback display, and secure, GDPR-compliant data storage. The considerations described in the comparison of survey tools for academic research — particularly around GDPR compliance, anonymisation, and data export formats — apply directly to e-Delphi platform selection.

Real-Time Delphi is a more recent innovation in which panellists access a shared platform simultaneously, observe live aggregated distributions, and update their ratings dynamically, collapsing iterative rounds into a single session. While efficient, real-time Delphi introduces social anchoring effects similar to group discussion; it is best suited to lower-stakes forecasting tasks rather than guideline development where methodological rigour is scrutinised.

Strengths and Limitations

The limitation that consensus does not imply correctness deserves particular emphasis. Delphi studies in medicine have occasionally produced recommendations later contradicted by randomised trial evidence. Researchers should explicitly position Delphi findings in relation to the available empirical evidence base, and address the epistemic status of consensus-based conclusions within the methodology chapter. For qualitatively grounded studies, the discussion of researcher positionality in reflexivity in qualitative research provides a useful parallel framework for examining how the facilitator’s choices shape the consensus agenda.

Reporting Your Delphi Study: CREDES Guidance

The Conducting and REporting DElphi Studies (CREDES) guidance provides the most widely adopted framework for transparent Delphi reporting in health and social science research. CREDES covers 26 items across six sections: rationale and objective; panel composition and recruitment; pilot testing; round-by-round procedures; consensus measurement; and discussion of limitations and generalisability. Studies in leading journals increasingly require adherence to CREDES or an equivalent reporting checklist as a condition of acceptance.

The items most commonly omitted in published Delphi studies are:

• A priori definition of the consensus threshold — declared before data collection, not selected post hoc to maximise consensus rates
• Response and retention rates per round, with demographic comparison of completers versus non-completers
• Full frequency distributions for each item, not only medians and IQRs
• Items reaching consensus for exclusion alongside those for inclusion
• Explicit rationale for stopping at the final round — threshold reached, no item movement, or predefined maximum
• Panel member characteristics sufficient to allow independent assessment of representativeness

Pre-registration of the Delphi protocol — including panel inclusion and exclusion criteria, consensus threshold, item generation procedure, and maximum round number — on a public repository such as OSF or PROSPERO is increasingly expected by high-impact journals. Pre-registration reduces post-hoc threshold manipulation and signals methodological rigour to peer reviewers. The 2024 publication of the DELPHI-STAR reporting extension offers the most comprehensive interdisciplinary guidance currently available and should be consulted alongside CREDES for studies intended for publication in systematic-review or methods journals.

Frequently Asked Questions