O'Reilly logo

Methods and Applications of Statistics in Clinical Trials, Volume 2: Planning, Analysis, and Inferential Methods by N. Balakrishnan

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 18

Interrater Reliability

Helena Chmura Kraemer

18.1 Definition

The classic model that underlies reliability stipulates that a rating of unit i, i =1, 2, … can be expressed as Xij = ξi + ij, where ξi is that unit’s “true value” (i.e., that free of rater error) and εij is error made by the jth independent rater sampled from the population of raters [1]. The interrater reliability coefficient is defined as:

equation

where σξ2 is the variance of the ξi in the population of interest and σX2 that of a single observation per subject. Thus, in a sense, reliability relates to the signal-to-noise ratio, where σξ2 relates to “signal” and σX2 combines “signal” and “noise.”

According to this definition, the reliability coefficient is zero if and only if subjects in the population are homogeneous in whatever X measures. This situation should almost never pertain when considering measures for use in randomized clinical trials. Consequently, testing the null hypothesis that ρ = 0 is virtually never of interest, although admittedly it is often observed in the research literature. Instead, the tasks of greatest interest to clinical research are (1) obtaining a confidence interval for ρ, (2) judging the adequacy of ρ, and (3) considering how to improve ρ.

18.2 The Importance of Reliability in Clinical ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required