ROC curves provide a simple, visual representation of medical diagnostic tests.
When a doctor delivers a diagnosis to a patient, his conclusion usually relies on tests -- and these tests are sometimes wrong. For instance, tests which rely on patients showing a constellation of symptoms may fail because not every patient displays every symptom. Doctors must choose which tests to use and apply them to get the largest number of correct diagnoses with the fewest mistakes. ROC curves are powerful graphical tools for analyzing and comparing the accuracy of different diagnostic tests.
Instructions
1. Create one 3-column table for each test you want to compare. Label the left column "cutoff," the middle column "true positive rate" and the right column "false positive rate."
2. Fill the tables with the correct true and false positive rates at each cutoff, using your data. For instance, one test for a disease may require a patient show some of the seven diagnostic criteria. At each cutoff -- three, four, five criteria -- the test will correctly diagnose some patients, whose result is a true positive. The test will also falsely diagnose some healthy patients as having the disease. These are false positives in your table.
3. Draw a square graph with units from 0 to 1 on the x and y axes. Label the x-axis "False Positive Rate" and the y-axis "True Positive Rate." Using your table, plot each (x,y) pair as a point. Use different colors for points from the different tests. Connect the points with a smooth, continuously sloping curve that begins at (0,0) and ends at (1,1) to graph each ROC curve.
4. Draw a straight diagonal line between the points (0,0) and (1,1). This line represents a worthless test which performs at chance diagnostic levels. If any ROC curve runs along or below this line, the test it represents is worthless -- no better than chance.
5. Compare the height of the ROC curves. If one ROC curve has a higher y-value at every x-value than another ROC curve, it is a strictly superior test. The higher y-values mean that this test is correctly diagnosing more patients while producing an equal number of false positives.
6. Compute the area under each ROC curve. To calculate the integral by hand, use the trapezoidal method or draw and count the number of squares that fit under the curve. Otherwise, use a computer or graphing calculator to integrate. The area under the ROC curve is a measurement of the test's "discrimination." A higher discrimination for one test than another means a more accurate test overall.
Tags: area under, diagnose some, diagnostic tests, each curve, each cutoff, false positive, false positives
