Advanced search
833
Views
80
CrossRef citations to date
0
Altmetric
Original Articles

A Unified Approach for Assessing Agreement for Continuous and Categorical Data

, &
Pages 629-652
Received 10 Sep 2006
Accepted 08 Feb 2006
Published online: 05 Jul 2007
 

This paper proposes several Concordance Correlation Coefficient (CCC) indices to measure the agreement among k raters, with each rater having multiple (m) readings from each of the n subjects for continuous and categorical data. In addition, for normal data, this paper also proposes the coverage probability (CP) and total deviation index (TDI). Those indices are used to measure intra, inter and total agreement among all raters. Intra-rater indices are used to measure the agreement among the multiple readings from the same rater. Inter-rater indices are used to measure the agreement among different raters based on the average of multiple readings. Total-rater indices are used to measure the agreement among different raters based on individual readings. In addition to the agreement, the paper also assess intra, inter, and total precision and accuracy. Through a two-way mixed model, all CCC, precision and accuracy, TDI, and CP indices are expressed as functions of variance components, and GEE method is used to obtain the estimates and perform inferences for all the functions of variance components. Each of previous proposed approaches for assessing agreement becomes one of the special case of the proposed approach. For continuous data, when m approaches , the proposed estimates reduce to the agreement indices proposed by Barnhart et al. (2005 Barnhart , H. X. , Song , J. , Haber , M. J. ( 2005 ). Assessing intra, inter and total agreement with replicated readings . Statistics in Medicine 19 : 255270 .[Crossref], [Web of Science ®] [Google Scholar]). When m = 1, the proposed estimate reduces to the ICC proposed by Carrasco and Jover (2003 Carrasco , J. L. , Jover , L. ( 2003 ). Estimating the generalized concordance correlation coefficient through variance components . Biometrics 59 : 849858 .[Crossref], [PubMed], [Web of Science ®] [Google Scholar]). When m = 1, the proposed estimate also reduces to the OCCC proposed by Lin (1989 Lin , L. I.-K. ( 1989 ). A concordance correlation coefficient to evaluate reproducibility . Biometrics 45 : 255268 .[Crossref], [PubMed], [Web of Science ®] [Google Scholar]), King and Chinchilli (2001a King , T. S. , Chinchilli , V. M. ( 2001a ). A generalized concordance correlation coefficient for continuous and categorical data . Statistics in Medicine 20 : 21312147 .[Crossref], [PubMed], [Web of Science ®] [Google Scholar]) and Barnhart et al. (2002 Barnhart , H. X. , Haber , M. , Song , J. ( 2002 ). Overall concordance correlation coefficient for evaluating agreement among multiple observers . Biometrics 58 : 10201027 .[Crossref], [PubMed], [Web of Science ®] [Google Scholar]). When m = 1 and k = 2, the proposed estimate reduces to the original CCC proposed by Lin (1989 Lin , L. I.-K. ( 1989 ). A concordance correlation coefficient to evaluate reproducibility . Biometrics 45 : 255268 .[Crossref], [PubMed], [Web of Science ®] [Google Scholar]). For categorical data, when k = 2 and m = 1, the proposed estimate and its associated inference reduce to the kappa for binary data and weighted kappa with squared weight for ordinal data.

ACKNOWLEDGMENT

The research work for this article is supported by National Science Foundation (NSF) Grants DMS-0103727 and DMS-0603761, National Institutes of Health (NIH) Grant P50-AT00155 (jointly supported by National Center for Complementary and Alternative Medicine, the Office of Dietary Supplements, the Office of Research on Women's Health, and National Institute of General Medicine) and Astellas USA Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF and the NIH.

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access

  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
EUR 47.00 Add to cart

Purchase access via tokens

  • Choose from packages of 10, 20, and 30 tokens
  • Can use on articles across multiple libraries & subject collections
  • Article PDFs can be downloaded & printed
From EUR 400.00
per package
Learn more
* Local tax will be added as applicable
 

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.