Reference standards, judges, and comparison subjects: roles for experts in evaluating system performance.
Medical informatics systems are often designed to perform at the level of human experts. Evaluation of the performance of these systems is often constrained by lack of reference standards, either because the appropriate response is not known or because no simple appropriate response exists. Even when performance can be assessed, it is not always clear whether the performance is sufficient or reasonable. These challenges can be addressed if an evaluator [...]
Author(s): Hripcsak, George, Wilcox, Adam
DOI: 10.1136/jamia.2002.0090001