If we are to accurately assess the potential advantages or disadvantages of any new technique or treatment, we must apply rigorous scientific standards and use validated instruments with comparisons to the contemporary gold standard.
However, a recent abstract by Tollefson et al highlights the problem of not performing adequate comparisons of new technologies to the contemporary gold standard prior to widespread incorporation into mainstream practice (see article, "Radical prostatectomy outcomes measured poorly by administrative data"). When robotic RP was introduced in the U.S. in the late 1990s, it was publicized with aggressive marketing and reports of significant benefits over open RP. Articles refuting these benefits followed, and one such study by Hu et al garnered significant media coverage after the authors reported that men undergoing minimally invasive RP were more likely to be diagnosed with erectile dysfunction and incontinence than men undergoing open RP (JAMA 2009; 302:1557-64).
The article by Tollefson et al reports-I believe accurately-differences in the use of administrative datasets and validated outcome measures to assess functional outcomes following RP. The authors compared self-reported validated questionnaire data from 562 men who underwent RP with administrative claims data for incontinence and ED. Not surprisingly, they found a poor correlation between the datasets. The authors call for the use of validated questionnaire data over administrative data, pointing out inaccuracies with administrative datasets.
I believe that this statement should apply more globally in urology. If we are to accurately assess the potential advantages or disadvantages of any new technique or treatment, we must apply rigorous scientific standards and use validated instruments with comparisons to the contemporary gold standard. Failure to apply these scientific methods will result in continued confusion for patients and practitioners and potentially prevent the use of these new techniques to improve the care of our patients and thus advance the field.