Diagnostic tests are among the important means commonly used in clinical medicine. Before a new test can be used in clinical practice, it should be evaluated for clinical validity. Studies assessing the clinical validity of a test (also termed diagnostic accuracy studies) involve determining the test performance indices including the test sensitivity (Se) and specificity (Sp) (1). Other common performance indices are positive and negative predictive values, and likelihood ratios, which can be calculated based on the Se and Sp and the prevalence (pr) of the disease of interest (2, 3). To determine a test performance, its results should be evaluated against another test, the so-called reference standard (4). The reference standard can be a gold-standard test, i.e., a test with a Se and Sp of 1.0 (or 100%). The gold-standard test can thus correctly discriminate those with and without the disease or condition of interest. For a test with binary results, the outcome is clear – positive or negative. For tests with continuous results, however, we need to set a cut-off value to categorize the results into positive or negative (2). Compared to the gold standard, the obtained results can be categorized into true-positive (TP), true-negative (TN), false-positive (FP), and false-negative (FN) results (Table 1a). The tests Se and Sp are defined as follows (5):
Both the Se and Sp follow the binomial distribution. Then, the squared standard errors (SE2) for Se and Sp are:
The prevalence of the disease (π), is then:
Combining Eq. 1 and Eq. 3, we have:
where P(x) designates the probability of x. To evaluate the Se and Sp of a new test, it is common to compare its test results against those obtained from a gold-standard test. Nonetheless, the gold-standard test may not always be available. It either does not exist or is very difficult or expensive to perform for certain disease conditions (6). The question arise is that whether it is possible to calculate the Se and Sp of the new test based on the results obtained from its comparison with a non-perfect reference standard – a well-established (but not a gold-standard) test? This is not a new question, and several solutions has so far been proposed (1). Herein, I wish to propose an analytical method to address the question raised.
Stating the question
Suppose that we have a well-established test, say T1, with known Se and Sp (measured against a gold-standard test) of Se1 and Sp1 (Table 1a). Now, suppose that we have a new test, say T2, the results of which were compared against T1 (not against a gold standard), and that it had a Se and Sp (against T1) of Se2,1 and Sp2,1 (Table 1b). We wish to derive the Se and Sp of T2 (Se2 and Sp2), if it would have been tested against the gold standard (e.g., Table 1c).
The proposed solution
When we compare T2 against T1, the calculated prevalence, pr, is not really the true prevalence, π, as T1 is not a gold standard and thus would have FP and FN results. However, we can calculate the true prevalence, π, as follows (7):
where T + and T – represent positive and negative test results; and D + and D –, presence and absence of the disease, respectively. P(A|B) denotes the conditional probability of event A given event B.
Based on Eq. 6, we have:
Solving for Se2, gives:
Based on Eq. 7, we have:
Equations 9 and 11 are a system of two simultaneous equations. Substituting π from Eq. 5 and solving for Se2 and Sp2, yield:
Assuming that Se2 is a function of independent random variables pr, Se2,1, Sp2,1, and Sp1 (Eq. 12), using Eq. 13 and employing basic calculus, we have:
In the same way, assuming that Sp2 is a function of independent random variables pr, Se2,1, Sp2,1, and Se1 (Eq. 12), we have:
The SE for the Se and Sp of the tests can be calculated using Eq. 2.
It was shown that the test Se and Sp can be determined with acceptable accuracy even if the gold-standard test is not available. The Se and Sp of the new test (T2) derived by transforming the values obtained from its comparison with a non-gold-standard test (Se2,1 and Sp2,1) are acceptably close to the values if the test would have been compared with the gold-standard (Se2 and Sp2). The variances of the calculated Se2 and Sp2 (Eqs. 14 and 15) are higher than those you might obtain if you would have compared T2 directly against the gold standard, instead of T1. This is attributed to the uncertainty exist in the variables used for the calculation (Eq. 12). To examine the application of the technique proposed let us apply it to an example.
Suppose that in a validity study of 500 (arbitrary chosen) randomly selected people, a diagnostic test (let us call it T1) was tested against the gold standard (Table 1a), and that the test could correctly identify 85 of 100 diseased people, hence a Se (Se1) of 0.85, and 360 of 400 disease-free individuals, hence a Sp (Sp1) of 0.90 (Table 1a). The calculated SE2 for the Se1 and Sp1 are 1.3 × 10-3 and 2.3 × 10-4, respectively (using Eq. 2). Also, suppose that in a validity study on 600 (arbitrary chosen) randomly selected people, the results of a new diagnostic test, T2, was compared against T1 (Table 1b). Based on the information provided, the apparent prevalence, pr, is 0.25 (SE2 = 3.1 × 10-4). Using Eq. 5, the true prevalence (π) is:
Plugging in the values in equations 12, 14 and 15, estimations of Se2 and Sp2 are 0.95 (SE2 = 8.0 × 10-3; 95% confidence interval (CI): 0.77 to 1.00) and 0.80 (SE2 = 5.4 × 10-4; 95% CI: 0.75 to 0.85), respectively, which are compatible with the results if T2 would have been compared against the gold-standard test – 0.95 (SE2 = 5.9 × 10-4; 95% CI: 0.90 to 1.00) and 0.80 (SE2 = 5.0 × 10-4; 95% CI: 0.76 to 0.84), respectively (Table 1c). Note that the 95% CI of the calculated Se2 and Sp2 when they are derived through comparing the results with T1 is wider than those if they are directly compared against a gold-standard test.
In conclusion, it seems that this technique is useful, particularly where the gold-standard test is not readily available or is expensive. Further studies are needed to elaborate on the conditions of the validity study where the Se1 and Sp1 are estimated, the minimum number of data points examined, the probable effect of the prevalence of the disease or condition of interest on the choice of the reference test, among other things.