- Research article
- Open Access
- Open Peer Review
Radiotherapy for prostate cancer: DISCERN quality assessment of patient-oriented websites in 2018
BMC Urology volume 19, Article number: 42 (2019)
Prostate cancer is the most commonly diagnosed cancer in men. Radiotherapy represents one major treatment option in different therapeutic settings. As patients increasingly rely on internet-based medical information, we examined the quality of information on radiotherapy and prostate cancer in websites used by laypersons.
An Internet search from a patients` perspective was carried out using different search engines (Google, Yahoo and Bing, search terms: “prostate cancer” and “radiotherapy”). The quality of search results was analyzed with regard to the DISCERN score, HON code certification, the JAMA criteria and the ALEXA traffic rank.
In general, websites were of good quality. The highest quality was found for websites operated by charity organizations. No significant differences in results obtained via the above-mentioned tools were seen for the examined search engines, but Google revealed the most stable search results in terms of temporal changes.
Patients with prostate cancer can sufficiently inform themselves on general treatment options including radiotherapy on websites directed at laypersons. However, no simple strategy could identify high quality websites in general. For treating physicians, it is important to support patients in interpreting and ranking the vast quantity of information.
Prostate cancer is the most commonly diagnosed cancer in the male population worldwide . Radiotherapy for localized prostate cancer remains one of the mainstays among treatment approaches. Other fields of application are biochemical recurrences after prostatectomy, postoperative radiotherapy in high-risk situations and palliative settings e.g. painful bone metastases [2, 3]. As most patients do not have personal experiences with radiotherapy, a lack of information on risks and benefits is immanent. A recent Canadian survey revealed that more than 90% of men with newly diagnosed prostate cancer prefer to either play an active or collaborative role in treatment decision-making . Thus, gaining valid medical information before initiating treatment seems essential for these patients to improve their decision-making.
Today, a significant number of patients with prostate cancer [5, 6] and radiation oncology patients [7, 8] utilize the Internet to obtain medical information. For cancer patients in general, Castleton et al. reported Internet utilization in approximately two thirds of patients . However, the quality of online information for patients with prostate cancer is variable [6, 10,11,12]. For instance, Ilic et al. found the quality of websites on prostate cancer screening to be highly variable but mostly of poor standard . Similarly, others found shortcomings in quality and accuracy of Internet health information for the special topic of prostate cancer and proton therapy . Noteworthy, Shaverdian et al. showed that 39% of patients who selected the Internet as their primary information source reported their actual treatment experience to be worse than expected. Patients who had initially gathered medical information by consulting their urologists, radiation oncologist or even other patients reported a significantly better treatment experience . On the other hand, a recently published analysis revealed a high rating in quality, accessibility and usability of websites on prostate cancer in general .
To our knowledge, there is no current analysis of consumer health information on the Internet for patients with prostate cancer in need of a radiation therapy. In order to close that gap, our goal was to evaluate websites with the focus on prostate cancer and radiotherapy.
Website identification and selection
A complete list of analyzed and excluded websites is given in Additional file 1 of the appendix. Identification and selection of websites (English language) took place on August 22nd 2017 (9.00 a.m. CET). The keywords “prostate cancer” and “radiotherapy” were entered in the search-engines Google.com, Bing.com and Yahoo.com to emulate real user experience. A second search was carried out on September 9th 2017 (9.00 a.m. CET) to reveal possible temporal dynamics. The first 20 websites were subjected to further evaluation as it is known that patients rarely browse through further sites when searching the web [12,13,14]. To simulate a laypersons view, we excluded websites from further analysis when complete access was restricted by password, the websites only included personal experiences (e.g. blogs, videos) or contained limited information on radiotherapy (< one paragraph), as described by others [15, 16]. In detail, websites were excluded based on the following criteria: advertisement only (n = 2), a focus on clinical trials (n = 4), scientific articles (n = 20), newspaper articles (n = 2), video (n = 2), denied access (n = 2) and PowerPoint presentation (n = 1). The vast majority of excluded websites was found using the search engines Bing.com and Yahoo.com. For the Google search, only one website met the above exclusion criteria.
After the subtraction of duplicate websites (i.e. repeated appearance in different search engines and time points) a total number of 49 websites was subjected to further evaluation (n = 39 first search, n = 10 second search). Apart from HON code, JAMA benchmark criteria, ALEXA rank and DISCERN score, the country of origin and website operator were recorded for each website additionally. The quality of websites was evaluated by two investigators (SJ, LK) independently using the validated tools described below. Discrepancies were discussed and a consensus was reached.
The DISCERN tool was originally developed and validated in 1998 at the University of Oxford, UK. The aim was to analyze written medical information with 15 questions in terms of reliability and details on treatment (score of 1–5 for each question) [17, 18]. Overall quality was additionally rated by a 16th question in the DISCERN Plus score. As described by Nghiem et al. the results were rated from “excellent” to “very poor” . This methodology was used in a similar way in a previous published work of our study group . The individual DISCERN items used in our study are visualized in Table 1. Following DISCERN scoring, the respective websites were further analyzed by using HON code  and JAMA  (tools, as outlined by us in detail earlier . With the ALEXA traffic tool (https://www.alexa.com/) popularity and engagement characteristics of websites can be assessed. It is a measure of how often a website is frequented relative to all other sites on the web over the past three months.
We performed statistical analysis using GraphPad Prism 7 (GraphPad Software Inc., La Jolla, CA, USA) employing Spearman correlation, linear regression analysis and two-tailed non-parametric Wilcoxon-Mann-Whitney test for group comparisons as described in a previous published work of our study group in detail .
First, we analyzed the consistency of our data (i.e. exclusion rates, duplicate websites and temporal changes in search rank). Web search results obtained via Google seemed adequately directed at laypersons (one website excluded) and were highly consistent, as no duplicate results occurred and only a single new webpage was evident within the 20 hits in our second search after three weeks (Fig. 1A). In contrast, exclusion rates (n = 8 and 25) and the incidence of double hits (n = 8 and 10) and of novel search results/ranks (n = 10 and 14, Fig. 1B, C) were higher when Bing and Yahoo were employed, respectively.
Overall, the quality of information on websites about prostate cancer and radiotherapy was good, with a mean DISCERN Plus score of 55.1 ± 10.0, 51.8 ± 10.3 and 50.7 ± 10.1 for Google, Yahoo and Bing, respectively. The scores ranged from a minimum of 34 to a maximum of 74. 13% of all websites were rated as excellent, 31, 48 and 8% as good, fair and poor, respectively. No website was rated as very poor. Figure 2 shows the item-based results of the DISCERN score for the first search results on August 22nd 2017 of all three search engines (n = 39 websites in total). The items with the lowest scores were related to website content concerning sources/references, up-to-dateness and quality of life issues (question 4–6 and 13, Table 1). The overall quality of websites (DISCERN Plus score) was neither dependent on the choice of search engines nor did we observe a significant temporal change in quality (Fig. 3). The main categories of websites retrieved from the first search were charity/NGO sites (46%), followed by sponsored medical news sites (28%), hospital/university sites (20%) and governmental sites (6%). Websites operated by charity organizations had significantly higher DISCERN Plus scores (mean score: 55.5 ± 9.3) compared to hospital sites (mean score: 47.3 ± 9.6, p < 0.042) and medical news sites (mean score: 46.1 ± 6.1, p < 0.009), respectively. Most websites originated in the USA (46%) followed by the UK (33%), Australia/New Zealand (15%) and Canada (6%). The JAMA benchmark criteria were fulfilled for all four sections in 13%, for three, two and one section(s) in 13, 31 and 40%, respectively. One website did not fulfill any of the JAMA criteria. Only 13% of all websites were HON code certified. The median ALEXA global traffic rank was 62,375 with a minimum of five and a maximum of 4,710,605.
Neither the JAMA benchmark criteria, the HON code certification, the ALEXA traffic rank, the country of origin (except slightly superior DISCERN score for UK websites compared to USA websites, p < 0.033) nor the ranking within the search results showed a significant association with the respective DISCERN Plus score.
Of all analyzed websites, the vast majority had a clear focus on radiotherapy for prostate cancer; four websites provided only information for radiotherapy in general. All analyzed websites had a focus on curative teletherapy (i.e. definitive radiotherapy to the prostate or to prostatic bed in a postoperative or salvage setting), 76, 51 and 22% of all websites mentioned brachytherapy, active surveillance and palliative radiotherapy (i.e. radiotherapy to other sites than the prostate or prostatic bed, e.g. bone metastases), respectively. In 57% the procedure of radiotherapy starting from first preparation, daily treatment and aftercare was described in detail. Special radiation techniques like “hypo-fractionation”, “Intensity modulated Radiotherapy (IMRT)”, “Image guided Radiotherapy (IGRT)” and “proton therapy” were mentioned in 37, 72, 27 and 31% of all analyzed websites, respectively.
In recent years, the Internet has gained importance as a source for health-related information. Several study groups confirmed this fact in patients with oncological diseases in general [9, 23], as well as for patients with gynecologic cancers , breast cancer [19, 20], colorectal cancer , bladder cancer , laryngeal cancer  and prostate cancer  in particular. The increasing amount of data published within the last two years highlights its clinical relevance.
For prostate cancer in particular there is a high need for information because several treatment strategies exist. Radiotherapy represents a major component for the management of this oncologic entity. Thus, we performed an evaluation of websites addressing information on prostate cancer and radiotherapy. The first 20 websites were subjected to further evaluation as it is known that patients rarely browse through further sites when searching the web . Ngyuen et al. showed that 72% of the patients even restricted their search to 1–5 websites only . In a study of Borgmann et al. the list of evaluated websites was limited to 10 sites .
Overall, we found a good quality of information on analyzed websites according to the validated DISCERN Plus score (for Google search: mean DISCERN Plus score: 55.1 ± 10.0). Our results are in concordance with findings from studies analyzing website quality of other cancer entities, returning a mean DISCERN Plus score of 50 , 52 , 42  and 42 , respectively. Notwithstanding, the only other comparable study evaluating websites with general information on prostate cancer, obtained higher DISCERN Plus scores (mean 77) . Taken together, websites with information on prostate cancer seem to be of slightly better quality compared to websites covering other cancer entities. A potential reason why the web search by Borgmann et al. returned webpages of higher quality might be associated with the use of a more general and popular search term. This is reflected by a lower ALEXA traffic rank for prostate cancer sites in general (median ALEXA traffic rank in the study of Borgmann et al.: 2718) compared to our focus on radiotherapy and prostate cancer (median ALEXA traffic rank: 62375). In our study, webpage quality (DISCERN Plus score 2.14 ± 1.03) of less frequently visited websites (n = 14, ALEXA traffic rank < 62,375) was not significantly different from more frequently visited sites (n = 15, ALEXA traffic rank > 62,375, DISCERN Plus score 1.67 ± 0.98). We did not find an association of ALEXA traffic rank and superior DISCERN score (data not shown).
In our analysis only 13% of websites were HON code certified. Most other studies evaluating websites on cancer entities in general found higher certification rates ranging from 20 to 38% [12, 19, 20, 26, 29,30,31]. The vast majority of these studies showed that a higher HON code certification was not necessarily accompanied by a better content quality, as determined by the DISCERN Plus score. Borgmann et al. presumed a lower HON code certification frequency for websites on more specific topics, e.g. prostate cancer and radical prostatectomy. This phenomenon could also account for the low number of HON code certified websites in our study, which is in concordance with findings by the study group of Alkhateeb indicating 17% of HON code certification in their focused web search on prostate cancer and surgery .
The websites operated by charity organizations were of superior quality concerning the DISCERN score compared to hospital sites and sponsored medical news sites. In line with this finding, Liebl et al. also found websites of non-profit providers and self-help groups of superior quality compared with profit-driven websites for cancer patients in general . Moreover, an earlier analysis of our study group also revealed better results for hospital and NGO websites compared to sponsored medical news sites for webpages with information on breast cancer . Thus, currently non-commercial sites should be preferentially recommended to patients seeking information on prostate cancer and radiotherapy.
No significant differences in the DISCERN Plus score, JAMA benchmark criteria and HON code certification were found between the three search engines (data not shown). Search results obtained via Google were greatly directed at laypersons and showed a high grade of consistency (i.e. exclusion rates, duplicate websites and temporal changes in search rank), in contrast to the other search engines. These properties might render Google a reliable choice for laypersons searching for information on prostate cancer and radiotherapy.
Interestingly, a large percentage of analyzed websites provided information on modern radiotherapy techniques like IMRT (72%). Also, websites mentioned brachytherapy in the setting of curative treatment in 76% and the possibility of no treatment (active surveillance) in 51%. Despite the well-balanced information on modern treatment options for localized disease, we found a significant lack of information on palliative radiotherapy, which was only mentioned by 22% of websites.
One has to be aware that besides the influence of temporal changes, internet search results may vary based on e.g. the choice of search engine, search term, search date and country of origin settings within the search engine. However, within Google.com, no substantial differences in search results were observed using different search terms (e.g. “radiation” or “radiation therapy” instead of “radiotherapy”) or countries of origin (data not shown).
Websites with complete access restricted by password, websites with personal experiences only (e.g. blogs, videos) or with limited information on radiotherapy (< one paragraph), newspaper articles and PowerPoint presentations were excluded from further analysis. We are aware of the fact that those sources may also contain meaningful information for prostate cancer patients. However, the exclusion was executed to apply the above-mentioned tools in order to standardize evaluation and to preserve comparability. This was handled similar in other publications [15, 16].
Websites on radiotherapy and prostate cancer directed at laypersons have the potential to sufficiently inform patients about general treatment options. The fact that we were unable to find a simple strategy for the identification of high-quality websites (i.e. HON code certification, JAMA benchmark criteria, ALEXA ranking, different search engines or country of origin and DISCERN Plus score) emphasizes the responsibility of the treating physicians to interpret and rank the vast quantity of information.
Health On the Net Foundation.
Intensity modulated Radiotherapy.
Image guided Radiotherapy.
Journal of the American Medical Association.
Global Burden of Disease Cancer Collaboration, Fitzmaurice C, Allen C, Barber RM, Barregard L, Bhutta ZA, Brenner H, et al. Global, regional, and National Cancer Incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 32 Cancer groups, 1990 to 2015: a systematic analysis for the global burden of disease study. JAMA Oncol. 2017 Apr 1;3(4):524–48.
Cornford P, Bellmunt J, Bolla M, Briers E, De Santis M, Gross T, et al. EAU-ESTRO-SIOG guidelines on prostate Cancer. Part II: treatment of relapsing, metastatic, and castration-resistant prostate Cancer. Eur Urol. 2017 Apr;71(4):630–42.
Mottet N, Bellmunt J, Bolla M, Briers E, Cumberbatch MG, De Santis M, et al. EAU-ESTRO-SIOG guidelines on prostate Cancer. Part 1: screening, diagnosis, and local treatment with curative intent. Eur Urol. 2017 Apr;71(4):618–29.
Davison BJ, Breckon EN. Impact of health information-seeking behavior and personal factors on preferred role in treatment decision making in men with newly diagnosed prostate cancer. Cancer Nurs. 2012 Nov-Dec;35(6):411–8.
Smith RP, Devine P, Jones H, DeNittis A, Whittington R, Metz JM. Internet use by patients with prostate cancer undergoing radiotherapy. Urology. 2003 Aug;62(2):273–7.
Shaverdian N, Kishan AU, Veruttipong D, Demanes DJ, Kupelian P, McCloskey S, et al. Impact of the primary information source used for decision making on treatment perceptions and regret in prostate Cancer. Am J Clin Oncol. 2017 May 23.
Metz JM, Devine P, DeNittis A, Jones H, Hampshire M, Goldwein J, Whittington R. A multi-institutional study of internet utilization by radiation oncology patients. Int J Radiat Oncol Biol Phys. 2003 Jul 15;56(4):1201–5.
Adler J, Paelecke-Habermann Y, Jahn P, Landenberger M, Leplow B, Vordermark D. Patient information in radiation oncology: a cross-sectional pilot study using the EORTC QLQ-INFO26 module. Radiat Oncol. 2009;4:40.
Castleton K, Fong T, Wang-Gillam A, Waqar MA, Jeffe DB, Kehlenbrink L, Gao F, Govindan R. A survey of internet utilization among patients with cancer. Support Care Cancer. 2011 Aug;19(8):1183–90.
Ilic D, Risbridger G, Green S. Searching the internet for information on prostate cancer screening: an assessment of quality. Urology. 2004 Jul;64(1):112–6.
Shah A, Paly JJ, Efstathiou JA, Bekelman JE. Physician evaluation of internet health information on proton therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2013 Mar 15;85(4):e173–7.
Borgmann H, Wölm JH, Vallo S, Mager R, Huber J, Breyer J, et al. Prostate Cancer on the web-expedient tool for Patients' decision-making? J Cancer Educ. 2017 Mar;32(1):135–40.
Sacchetti P, Zvara P, Plante MK. The internet and patient education--resources and their reliability: focus on a select urologic topic. Urology. 1999;53(6):1117–20.
Nguyen SK, Ingledew PA. Tangled in the breast cancer web: an evaluation of the usage of web-based information resources by breast cancer patients. J Cancer Educ. 2013;28(4):662–8.
Weissenberger C, Jonassen S, Beranek-Chiu J, Neumann M, Müller D, Bartelt S, et al. Breast cancer: patient information needs reflected in English and German web sites. Br J Cancer. 2004;91(8):1482–7.
Perzel S, Huebner H, Rascher W, Menendez-Castro C, Hartner A, Fahlbusch FB. Searching the web: a survey on the quality of advice on postnatal sequelae of intrauterine growth restriction and the implication of developmental origins of health and disease. J Dev Orig Health Dis. 2017;22:1–9.
Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105–11.
Charnock D, Shepperd S. Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res. 2004;19(4):440–6.
Nghiem AZ, Mahmoud Y, Som R. Evaluating the quality of internet information for breast cancer. Breast. 2016;25:34–7.
Janssen S, Käsmann L, Fahlbusch FB, Rades D, Vordermark D. Side effects of radiotherapy in breast cancer patients : the internet as an information source. Strahlenther Onkol. 2017.
Boyer C, Selby M, Scherrer JR, Appel RD. The health on the net code of conduct for medical and health websites. Comput Biol Med. 1998;28(5):603–10.
Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the internet: Caveant lector et viewor--let the reader and viewer beware. JAMA. 1997;277(15):1244–5.
Mattsson S, Olsson EMG, Johansson B, Carlsson M. Health-related internet use in people with Cancer: results from a cross-sectional study in two outpatient clinics in Sweden. J Med Internet Res. 2017;19(5):e163.
McLeod J, Yu I, Ingledew PA. Peering into the deep: characterizing the internet search patterns of patients with gynecologic cancers. J Cancer Educ. 2017;32(1):85–90.
Grewal P, Alagaratnam S. The quality and readability of colorectal cancer information on the internet. Int J Surg. 2013;11(5):410–3.
Corfield JM, Lawrentschuk N. Health information quality on the internet for bladder cancer and urinary diversion: a multi-lingual analysis. Minerva Urol Nefrol. 2017.
Narwani V, Nalamada K, Lee M, Kothari P, Lakhani R. Readability and quality assessment of internet-based patient education materials related to laryngeal cancer. Head Neck. 2016;38(4):601–5.
Alsaiari A, Joury A, Aljuaid M, Wazzan M, Pines JM. The content and quality of health information on the internet for patients and families on adult kidney Cancer. J Cancer Educ. 2016.
Quinn EM, Corrigan MA, McHugh SM, Murphy D, O'Mullane J, Hill AD, Redmond HP. Breast cancer information on the internet: analysis of accessibility and accuracy. Breast. 2012;21(4):514–7.
Saeed F, Anderson I. Evaluating the quality and readability of internet information on Meningiomas. World Neurosurg. 2017 Jan;97:312–6.
Sobota A, Ozakinci G. The quality and readability of online consumer information about gynecologic cancer. Int J Gynecol Cancer. 2015;25(3):537–41.
Alkhateeb S, Lawrentschuk N. Consumerism and its impact on robotic-assisted radical prostatectomy. BJU Int. 2011 Dec;108(11):1874–8.
Liebl P, Seilacher E, Koester MJ, Stellamanns J, Zell J, Hübner J. What cancer patients find in the internet: the visibility of evidence-based patient information - analysis of information on German websites. Oncol Res Treat. 2015;38(5):212–8.
Availability of data and materials
All internet sites are provided in the appendix section.
Consent to publish
Ethics approval and consent to participate
Not applicable (no humans were included in this study).
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.