- Research article
- Open Access
Impact of a protein-based assay that predicts prostate cancer aggressiveness on urologists’ recommendations for active treatment or active surveillance: a randomized clinical utility trial
BMC Urology volume 17, Article number: 51 (2017)
Of the more than 1.1 million men diagnosed worldwide annually with prostate cancer, the majority have indolent tumors. Distinguishing between aggressive and indolent cancer is an important clinical challenge. The current approaches for assessing tumor aggressiveness are recognized as insufficient. A validated protein-based assay has been shown to predict tumor aggressiveness from prostate biopsy. The main objective of this study was to measure the clinical utility of this new assay in the management of early-stage prostate cancer.
One hundred twenty nine board-certified urologists were asked to participate in a randomized, two-arm experiment. We collected data over 2 rounds using simulated clinical cases administered via an online platform. The cases were all newly diagnosed Gleason 3 + 3 or 3 + 4 prostate camcer patients. Urologists in the intervention arm received a 15-min webinar on this protein-based assay and given assay test results for their simulated patients in round 2. Each case had a preferred recommendation of either active surveillance or active treatment. The measured outcome was rate of preferred recommendation, defined as urologists who recommended the proper treatment course. Analyses were done using difference-in-difference estimations.
Using multinomial logistical regression, urologists who were given the assay results were significantly more likely to choose the preferred recommendation (active surveillance or active treatment) compared to controls (p = 0.004). These urologists were also significantly more likely to involve their patients in the treatment decision compared to controls (p = 0.001).
By providing additional information to inform the physician’s treatment plan, a protein-based assay shows demonstrable clinical utility confirmed through a rigorous randomized controlled study design and regression analyses to test for effects.
Worldwide, prostate cancer is the most commonly diagnosed solid organ tumor and the second deadliest, with more than 1.1 million new cases in 2012 . Depending on emphasis of early detection and/or treatment, wide variation exists in mortality rates in various countries . Over recent decades, however, widespread use of prostate-specific antigen (PSA) testing has led to a pronounced shift toward identification of early-stage tumors, many of which are likely indolent and ultimately present little or no risk to the patient . Simultaneously, treatment advances in robotic surgery and advanced radiation therapy have opened up additional opportunities to aggressively treat localized tumors, which may subject some patients to unnecessary treatment and the risks. These advances are key drivers in the rising cost of prostate cancer treatment . Because of this, active surveillance is increasingly being recommended as a treatment option .
Distinguishing between aggressive and indolent cancer, and delivering the appropriate level of care to each group, is thus an important clinical and economic challenge. Active surveillance (AS) protocols, which emphasize identification of low-risk patients and monitoring of the tumor in lieu of aggressive treatment, have been incorporated into U.S. and European guidelines [6, 7], are linked with the highest quality-adjusted life expectancy , but appear to be underutilized , with significant variation in adoption that cannot be explained by clinical characteristics . The decision to pursue AS is a complex process where clinical risk stratification, physician recommendation, and patient preference all play roles [11,12,13,14]. In a recent review , the researchers noted that while a uniform approach to AS would be appealing, “current diagnostic and prognostic tools lack the precision needed to reliably monitor men… [who have] varying risks and preferences.”
Evaluation of appropriate use of AS and active treatment (AT) is plagued by subtle differences in patient presentation and differences in clinical treatment decisions. If, however, patient level variability could be controlled, the (expected high level of) variability in clinical decision making between AS and AT, defined as providing a clinical intervention [11, 15], could be understood.
Given the wide variation in prostate cancer treatment decisions, a key question is whether better diagnostic tests for assessing tumor aggressiveness leads to better treatment decisions and better clinical utility. Given the preponderance of low risk patients, diagnostics that increase confidence in risk assessment will also likely result in reduced health care spending. A new quantitative immunofluorescent protein-based assay (ProMark) that predicts tumor aggressiveness at biopsy has been described for patients with Gleason grades 3 + 3 or 3 + 4 prostate cancer, wherein aggressiveness is difficult to distinguish using existing clinical and pathological parameters .
The objective of this study is to measure the variability of treatment decisions of AT or AS and determine the impact of adding a protein-based risk assessment assay to current standard of care risk classification approaches in the management of early-stage prostate cancer. To overcome patient-level (case-mix) variability, this study uses validated case simulations of early stage prostate cancer, among a large cohort of urologists.
A “before-and-after” design was utilized in a longitudinal randomized controlled study of board-certified urologists practicing in the U.S. Urologists were asked to care for online simulated patients with underlying but undiagnosed prostate cancer, using Clinical Performance and Value (CPV®) vignettes via web-based interactive ‘patient visits.’ All patients had a Gleason Score of 3 + 3 = 6 or 3 + 4 = 7. CPVs are simulated cases wherein clinicians are asked to interview and examine patients and order investigational studies including biopsies and blood tests. Clinicians taking care of the simulated patient are provided with responses to all history and physical items and results for any tests or procedures they choose to order. They are asked to diagnose the patient and make a treatment plan based on their investigations.
Physicians were randomized into control or intervention study arms and completed three vignettes at baseline (Round 1) and another three 6 to 8 weeks later (Round 2). In Round 1, no urologist received any information about the protein-based assay test, and none of the vignettes included these results. Between Rounds 1 and 2, intervention group urologists were introduced to the protein-based assay via a 15-min informational video. Intervention participants were then provided with these results for each Round 2 vignette. Protein-based assay information and results were not made available to the control group. The study design was approved by Chesapeake IRB (Columbia, MD). All participants provided written consent to participate.
The protein-based assay
The protein-based assay is a novel 8-biomarker proteomic test, using quantitative multiplex immunofluorescence to measure protein levels on prostate biopsy tissue. The assay produces a prognostic risk score scaled from 0 to 100 and stratifies patients into low (0–33), intermediate (34–60) or high risk (61–100). These scores independently predict final disease pathology and assess disease aggressiveness . This test supplements current prostate cancer risk assessment methods, especially in cases where existing tools do not clearly delineate the appropriateness of AS or AT.
Eligibility and selection of physicians
Physicians had to (1) be currently practicing board-certified urologists, (2) have practiced (as a board-certified urologist) for greater than 2 or less than 30 years, (3) be English-speaking, (4) practice in a community/non-academic setting, (5) have ≥50 prostate cancer patients under care annually, (6) have Internet access, (7) have no prior experience with the protein-based assay test and (8) provide consent to participate in the study. Potential participants were contacted from a list of approximately 5100 practicing urologists who were randomly selected and invited to participate. Eligibility requirements, screening tools and study information presented during recruitment were identical for both groups. Eligible urologists were invited to participate in the study and 261 initially consented (Fig. 1). These were randomized into one of two arms: 138 in intervention and 123 in control. Of those initially randomized, 82 intervention and 69 control physicians participated at baseline, and 67 intervention and 62 control physicians completed the second round of CPVs. Statistical analysis yielded no significant differences between the physicians recruited in either of the two strategies nor among those who dropped out of the study or were lost to follow up.
Clinical Performance and value® vignettes
Treatment choice and treatment utility were measured at baseline (Round 1) and after 6–8 weeks (Round 2) using Clinical Performance and Value (CPV®) vignettes. CPVs are a validated means for assessing differences in clinical practice and inherent variation in care, independent of case-mix [17, 18]. All providers care for the same patients and patient types, eliminating patient variability or observed and unobserved patient heterogeneity from the analysis and allowing for whether a test changes clinical practice; both of which are difficult to completely overcome through chart review analysis.
The CPVs were designed around the evidence-base with clear appropriate recommendations for treatment courses. The vignettes simulated clinical encounters involving men presenting with suspected early stage prostate cancer, verifiable by ordering a biopsy. In each vignette, the urologists were asked to ‘care for the patient’ by answering open-ended questions regarding the clinical care they would provide. Responses were requested and scored in: taking a medical history, performing a physical examination, ordering appropriate diagnostic tests (including laboratory tests, imaging studies and procedures), determining a diagnosis, and outlining a treatment plan (i.e., AS or AT). Explicit scoring criteria were established prior to study administration and were derived from a literature review, National Comprehensive Cancer Network guidelines, and expert opinion. Completed vignettes were scored as a percentage of physician answers matching these evidence-based criteria.
Three groups of three (9 total) CPV vignettes, representing typical cases for practicing urologists, were written to evaluate the variability of AS versus AT and the impact of protein-based assay test results on patient management (Table 1). All nine cases were undiagnosed Gleason 3 + 3 and 3 + 4 prostate cancer patients, Stage T1c or T2a, specified by their activity level, an elevanted PSA and risk categories such that each case had a preferable treatment course that was either AS or AT (Table 2). The primary outcome measure was the appropriate recommendation according to the individual case.
Prior to Round 2, the intervention group was given information about the protein-based assay via a 15-min informational video that provided an overview of the test and interpretation of the scores. 87.5% of the intervention group watched the video webinar. The intervention group then received hypothetical protein-based assay scores and disease aggressiveness risk estimates for each patient case in the Round 2 vignettes.
The analysis determined how frequently did, urologists caring for Gleason 3 + 3 and 3 + 4 prostate cancer patients, recommend the preferred treatment pathway, and did this increase with the introduction of the protein-based assay? The preferred treatment pathway, defined on a case-by-case basis, was either AS or AT. Physician treatment was classified into four mutually exclusive categories: preferred treatment for the case; suboptimal treatment, defined as recommending AS when the case presentation and clinical guidelines indicated AT or recommending AT when AS was indicated; involving the patient in the treatment decision without making a recommendation for either AS or AT; and no cancer treatment recorded. If, for example, a urologist both recommended AS and involved the patient, then their response would be marked (depending on the case) as either preferred or suboptimal treatment.
Analyses used difference-in-difference estimations for intervention versus control in choosing the preferred treatment pathway. Multinomial multivariate regressions modeled choosing the preferred treatment pathways versus each of the other three possibilities for this variable (suboptimal, indeterminate, or no treatment). We included variables for round and the intervention arm. The interaction terms of round and arm measure the differential change in the odds ratio of a physician ordering the preferred treatment (compared to suboptimal treatment) after the intervention compared to Round 1. Physician and practice characteristics were included as control variables, consisting of: CPV case, physician age, prostate cancer patient load, overall urology patient load, in-practice access to robotic surgery capability, and proportion of patients covered by Medicare and Medicaid.
We also examined the impact of protein-based assay on the overall rates of AT, both in cases where surveillance was the preferred management pathway or where treatment was preferred. This analysis provided an additional perspective of the protein-based assay’s clinical utility and role in modifying treatment patterns in clinical use.
Analyses were performed in Stata 13.0 (College Station, TX). The sample size of physicians was sufficient to detect a difference of 10% with an alpha of 0.05 and a beta of 0.80.
Physician and practice characteristics
Participating urologists were typically in single specialty (86.3%), physician-owned (89.3%) group practices (85.9%). More than three-quarters (75.6%) have 11 or more years in practice post-fellowship and more than half (55.5%) have onsite robotic surgical capability. When looking at payor mix, these providers care for the same percentage of public versus private insurance. There were no significant differences between the two groups except a greater proportion of the intervention group reported seeing over 20 prostate cancer patients in a week (68.6%) versus the control group (46.8%) (Table 3).
We evaluated the physicians’ AS or AT treatment choice for each case. The choices recorded in the CPVs were categorized as 1) either having appropriately recommended either AS or AT based upon guidelines for that case, 2) incorrectly choosing, 3) presenting both options equally (AS and AT) to the patients (shared decision-making), or 4) no treatment specified (no AS, AT, or shared decision-making recommended). At baseline, we found that 19.7% of the participants chose the preferred treatment, 26.0% the suboptimal treatment, 23.6% left the choice to the patient, and 30.5% did not recommend one treatment or the other, with a nonsignificant difference between control and intervention physicians (p = 0.645). We observed that those urologists who did not specify either AS or AT treatment were more likely to misdiagnose the patient (p < 0.001) or not order a biopsy (p < 0.001) (data not shown).
In bivariate analyses across all nine cases, after the protein-based array was introduced in the intervention group, the optimal treatment was selected 29.1% by the intervention arm versus 21.6% for controls (Table 4). Intervention urologists saw a 6.9% greater increase in correct treatment than the controls (p = 0.001) over the two rounds of data collection. Similarly, suboptimal treatment choice declined by 10.8% in the intervention group, compared to the controls (p = 0.028). The percentage of urologists recommending that the patient choose, decreased in the intervention group by 4.3%.
In multinomial logistic regression, compared to controls, urologists in the intervention group were significantly more likely to recommend the preferred treatment (AS or AT) in Round 2, with an odds ratio of 2.84 (95% CI 1.39, 5.82) (p = 0.004). The multinomial logit accounts for the four different recommendations (preferred versus suboptimal versus patient choice versus no treatment recommendation), controlling for the individual case types and other variables of interest (Table 5).
The same model found that the introduction of a protein-based assay prompted urologists to involve their patients in treatment discussions more often than controls (OR 2.72; 95%CI 1.47, 5.05) (p = 0.001). Urologists treating more than 20 prostate cancer patients in a week were less likely to involve their patients in deciding the treatment for their cancer (p = 0.001).
Changes in use of active surveillance and treatment
We specifically evaluated urologists who made either an AS or AT recommendation in Round 1 (excluding those who would counsel their patients or those who did not provide a treatment plan), grouping the cases by whether a preferred AS (6 of the 9 cases developed) or AT (3 cases) strategy was preferable. Overall, for those six cases where AS is the preferable treatment, the percentage of providers recommending AT decreased by 28.9% more in the intervention group compared to controls (Fig. 2). This decrease is observed across all three case types.
For the 3 cases where AT was preferred, urologists in the intervention group increased the recommendation for AT by 18% in Round 2 compared to controls.
As understanding of prostate cancer disease progression increases, the evidence indicates over-treatment with many men undergoing unnecessary prostatectomies . The addition of a diagnostic tool to assess disease aggressiveness empowers clinicians to make more informed decisions on the recommended course of treatment.
This study confirms that care of prostate cancer patients, similar to many other disease states, is highly variable [18, 19]. Despite the variability in care, the results show that a protein-based assay had a positive impact on physician practice, improving treatment domain scores, and more specifically, moving patients from suboptimal to preferred recommendations, thereby providing the right treatment recommendation for the right patient.
Physicians provided with protein-based assay results changed their practice from “no recommendation” and “suboptimal recommendation” categories to either the preferable treatment pathways or engaging the patient in a conversation about their options. Importantly, this movement from the suboptimal to preferred recommendation categories by the intervention physicians maintained significance in the difference-in-differences model.
Limitations exist in this study. Since physicians responded to open-ended questions, we could not explore the counseling process for those who would present both AS and AT options to their patients. Thus, we could not determine their ultimate course of action. Likewise, a proportion of urologists did not identify the case as one of prostate cancer, and as a result, they recommended neither active treatment nor surveillance. We cannot know from these cases their treatment plan if they were presented with a prostate cancer patient.
A protein-based assay provided physicians with additional prognostic information that resulted in a change to more appropriate recommendations for management, particularly in those cases where biopsy findings did not indicate a clear and precise treatment pathway. The additional information encouraged physicians to change their treatment plans. Thus, this protein-based assay shows demonstrable clinical utility, confirmed through a rigorous randomized controlled study design and regression analyses testing for effects.
Clinical Performance and Value
Ferlay J, Soerjomataram I, Ervik M, et al. GLOBOCAN 2012 v 1.1. Cancer Incidence and Mortality Worldwide: IARC CancerBase No.11 Lyon, France. International Agency for Research on Cancer; 2014.
Center MM, Jemal A, Lortet-Tieulen J, et al. International variation in prostate cancer incidence and mortality rates. Eur Urol. 2012;61:1079–92.
Lu-Yao GL, Albertsen PC, Moore DF, et al. Outcomes of localized prostate cancer following conservative management. JAMA. 2009;302:1202–9.
Nguyen PL, Gu X, Lipsitz SR, et al. Cost implications of the rapid adoption of newer technologies for treating prostate cancer. J Clin Onc. 2011;29:1517–24.
Cooperberg MR, Carroll PR. Trends in management for patients with localized prostate cancer, 1990-2013. JAMA. 2015;314:80–2.
Mohler JL, Kantoff PW, Armstrong AJ, et al. Prostate cancer, version 2.2014. J Natl Compr Cancer Netw. 2014;12:686–718.
Heidenreich A, Bastian PJ, Bellmunt J, et al. EAU guidelines on prostate cancer. Part 1:screening, diagnosis, and local treatment with curative intent - update 2013. Eur Urol. 2014;65:124–37.
Hayes JH, Ollendorf DA, Pearson SD, et al. Active surveillance compared with initial treatment for men with low-risk prostate cancer: a decision analysis. JAMA. 2010;304:2373–80.
Overholser S, Nielsen M, Torkko K, et al. Active surveillance is an appropriate management strategy for a proportion of men diagnosed with prostate cancer by prostate specific antigen testing. J Urol. 2015;194:680–4.
Chamie K, Williams SB, Hu JC. Population-based assessment of determining treatments for prostate cancer. JAMA Onc. 2015;1:60–7.
Aizer AA, Paly JJ, Zietman AL, et al. Multidisciplinary care and pursuit of active surveillance in low-risk prostate cancer. J Clin Onc. 2012;30:3071–6.
Loeb S, Carter HB, Berndt SI, et al. Is repeat prostate biopsy associated with a greater risk of hospitalization? Data from SEER-Medicare. J Urol. 2013;189:867.
Xu J, Dailey RK, Eggly S, et al. Men's perspectives on selecting their prostate cancer treatment. J Nat Med Assoc. 2011;103:468–78.
Tosoian JJ, Carter HB, Lepor A, Loeb S. Active surveillance for prostate cancer: current evidence and contemporary state of practice. Nat Rev Urol. 2016;13:205–15.
Filson CP, Schroeck FR, Ye Z, et al. Variation in use of active surveillance among men undergoing expectant treatment for early stage prostate cancer. J Urol. 2004;192:75–81.
Blume-Jensen P, Berman DM, Rimm DL, et al. Development and clinical validation of an in situ biopsy-based multimarker assay for risk stratification in prostate cancer. Clin Canc Res. 2015;21:2591–600.
Peabody JW, Luck J, Glassman P, et al. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283:1715–22.
Peabody JW, Luck J, Glassman P, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Int Med. 2004;141:771–80.
Peabody JW, Strand V, Shimkhada R, et al. Impact of rheumatoid arthritis disease activity test on clinical practice. PLoS One. 2013;8:e63215.
QURE Healthcare received an unrestricted research grant from Metamark Genetics to support the costs of this study.
Availability of data and materials
The raw data underlying the results of this paper are available only from the corresponding author on reasonable request.
Ethics approval and consent to participate
The study design was approved by Chesapeake IRB (Columbia, MD) (Reference #01-MM-2014). All participants provided written consent to participate.
Consent for publication
Dr. Richie is Chief Medical Officer at Metamark Genetics, the company which developed the protein-based assay used in this study. Dr. Peabody developed the CPVs and is president of CPV Technologies, LLC, which owns the quality measurement tool used in this study. For the remaining authors, no competing interests were declared.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Peabody, J.W., DeMaria, L.M., Tamondong-Lachica, D. et al. Impact of a protein-based assay that predicts prostate cancer aggressiveness on urologists’ recommendations for active treatment or active surveillance: a randomized clinical utility trial. BMC Urol 17, 51 (2017). https://doi.org/10.1186/s12894-017-0243-1
- Proteomic biomarker
- Protein-based assay
- Gleason score
- Active surveillance
- Active treatment
- Simulated patients
- Evidence-based treatment