The receiver operating characteristic (ROC) curve displays sensitivity versus 1-specificity over a set of thresholds. The area under the ROC curve (AUC) is a global scalar summary of this curve. In the context of time-dependent ROC methods, we are interested in global scalar measures that summarize sequences of time-dependent AUCs over time. The concordance probability is a candidate for such purposes. The concordance probability can provide a global assessment of the discrimination ability of a test for an event that occurs at random times and may be right censored. If the test adequately differentiates between subjects who survive longer times and those who survive shorter times, this will assist clinical decisions. In this context the concordance probability may support assessment of precision medicine tools based on prognostic biomarkers models for overall survival. Definitions of time-dependent sensitivity and specificity are reviewed. Some connections between such definitions and concordance measures are also reviewed and we establish new connections via new measures of global concordance. We explore the relationship between such measures and their corresponding time-dependent AUC. To illustrate these concepts, an application in the context of Alzheimer’s disease is presented.
Keywords: Time-dependent sensitivity and specificity, diagnostic test, censored survival times, inverse probability weighting, Alzheimer’s disease
Go to:
1. Introduction
An area of key interest for scientists is to measure accuracy over time of a marker or a test for an event, which occurs at random times and may be right censored. While there has been extensive work in the area of assessment of diagnostic tests with binary outcomes such as tests positive or negative for disease (no-disease) or for condition present (absent) outcomes, there is vast potential for research regarding assessment of tests for time-dependent outcomes. Clinical trials for individuals at the earliest stages of Alzheimer’s Disease (AD) are of great interest and establishing eligibility of a patient for a certain treatment, which is at the heart of precision medicine, is of key importance. Adequate biomarker or marker tests play a key role in this setting. Because observing the progression from normal cognition all the way to clinical AD dementia can take many years, it is not an efficient trial endpoint. Thus, there is interest in assessments that can help to make early distinctions between those with normal aging and those who are more likely to progress to AD dementia. The Clinical Dementia Rating (CDR) and CDR sum-of-boxes (CDR-SB) are useful for marking gradations of cognitive impairment along the spectrum from normal aging to AD dementia. The CDR considers six domains: memory, orientation, judgement and problem-solving, community affairs, home-and-hobbies, and personal care. Ratings in these six areas are used to generate a global CDR with scoring categories reflecting no impairment to mild dementia. However, CDR scores are based on methods ranging from clinical impression to lengthy interviews, limiting reliability and feasibility in many research environments. A new shorter interview and questionnaire for rating cognitive symptoms across the spectrum from minimal impairment to mild AD dementia was developed from a briefer CDR interview derived from longer legacy data interviews [1,2]. Participants had been followed for 14 years and completed annual clinical interviews and neuropsychological assessments to facilitate diagnosis of cognitive status. Time-to-event and covariates were recorded. Time-to-event corresponds to the time to progression to AD (specifically AD dementia) from a specific time origin. The censoring events were death, loss to follow-up, or development of non-AD dementia. In the context of AD, disease outcome is time-dependent and therefore tools for test assessment, such as ROC curve methodology, that vary as a function of time may be more relevant.
