I’m writing from Philadelphia, at the annual meeting of the Society for Medical Decision Making. I’ve just had the pleasure of attending a great short course entitled “How to discuss evidence-based diagnosis with experienced clinicians (and avoid giving EBM at bad name)”, taught by Tom Newman and Mike Kohn from UCSF.
My goal in intending the course was to hear about more advanced issues in the evaluation of diagnostic test evidence that I could use to enhance my own teaching as the residents and faculty I teach become increasingly more skilled in the basics. The course did a great job of doing this. After reviewing basic concepts that readers of our book should be familiar with (sensitivity, specificity, predictive values, LR+, LR-, ROC curves), Mike and Tom delved into detailed discussion of continuous and multi-valued tests, focusing particularly on the calculation of interval likelihood ratios.
I’ll illustrate with an example from Medical Decision Making: A Physician’s Guide. In chapter 10, figure 10-3 shows the ROC curve for the 13C-urea breath test at 30 minutes for H. Pylori infection, based on data from a paper by Herold and Becker (BMC Gastroenterology 2002; 2). The curve is based on test values called “delta-δ”; here’s a partial table of test characteristics based loosely on that figure:
| Test value (“delta-δ”) | Sensitivity | Specificity |
|---|---|---|
| 0 | 1 | 0 |
| 0.5 | 0.999 | 0.03 |
| 1 | 0.99 | 0.20 |
| 3 | 0.95 | 0.80 |
| 10 | 0.80 | 0.95 |
| 25 | 0.35 | 0.99 |
In some papers (not Herold and Becker’s), tables like these are then used to compute the positive and negative likelihood ratio at different test values used as thresholds. Mike and Tom point out that this not only throws out a lot of useful information, but can even be misleading when test values are not normally distributed (for example, if both low and high values are problematic).
An interval likelihood ratio is a likelihood ratio for a test result between two test values. This is what you want, and here’s how it looks:
| Test value interval | LR(interval) | ||
|---|---|---|---|
| 0-0.5 | 0.03 | ||
| 0.5-1 | 0.05 | ||
| 1-3 | 0.07 | ||
| 3-10 | 1 | ||
| 10-25 | 11.3 | ||
| >25 | 35 |
If your test result was 0.7, you’d use the LR for the 0.5-1 interval; if your result was 15, you’d use the LR for the 10-25 interval. By constructing reasonable intervals, you get to take advantage of much more information than creating dichotomous thresholds. (How do you compute those LRs? It’s easy. The interval LR for the interval 1-3 is the absolute difference in sensitivity between delta-δ=1 and delta-δ=3 (0.99-0.95 = 0.04), divided by the absolute difference in specificity between delta-δ=1 and delta-δ=3 (0.80-0.20=0.60).
They also presented some very nice techniques for teaching physicians how to understand ROC curves, and several important nuances around evaluating the validity of diagnostic test studies.
The course was based on their forthcoming book, Evidence-Based Diagnosis, due out in 2009. It’s now on my “must buy” list.