Skip to main content

Quality assessment and relevant clinical impact of randomized controlled trials on chronic prostatitis/chronic pelvic pain syndrome

Abstract

Objective

This study evaluated the quality of randomized controlled trials (RCTs) on chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS).

Methods

We searched PubMed, Web of Science, and Embase for RCTs (original articles) on CP/CPPS published from database establishment to 2021. The RCT quality assessment was performed using the Consolidated Standards of Reporting of Trials (CONSORT) statement and the improved Jadad scale.

Results

In total, 77 RCTs were included. According to the evaluation, 26 (33.77%) papers presented the description of the specific random methods, only 6 (7.79%) papers described the allocation concealment methods, and 26 (33.77%) articles referred to the “blind method”. Of the RCTs, 34 (44.16%) papers recorded the number of patients who withdrew from the study, and 67 (87.01%) papers reported adverse reactions. However, few reports mentioned the sample size calculation, clinical trial registration, or information about the relevant research programs and funding. In addition, 19 (24.68%) reports had Jadad scale scores of ≥ 4 points, and 58 (75.32%) reports had Jadad scale scores of ≤ 3 points.

Conclusion

To date, the quality of RCT reports on CP/CPPS needs to be further improved, and the results of the RCTs should be accepted and utilized cautiously. It is suggested that researchers should follow the CONSORT statement and the improved Jadad scale to standardize the design and implementation of RCTs to improve the quality of RCTs and provide reliable evidence for the treatment of CP/CPPS.

Peer Review reports

Introduction

Prostatitis is a common urinary system disease in adult men. In 1995, the US National Institutes of Health (NIH) reclassified prostatitis into types I (acute bacterial prostatitis), II (chronic bacterial prostatitis), III (chronic prostatitis/chronic pelvic pain syndrome [CP/CPPS]), and IV (asymptomatic inflammatory prostatitis [AIP]) [1]. CP/CPPS refers to a group of diseases in which symptoms such as pain or discomfort and abnormal urination appear in the pelvic area for more than 3 months [2, 3]. CP/CPPS accounts for more than 90% of cases of prostatitis, which is divided into inflammatory (NIH-IIIA) and non-inflammatory (NIH-IIIB) subtypes according to the white blood cell count in [4]. CP/CPPS seriously affects the quality of life of patients. Most patients are not satisfied with the treatment effect, and the incidence of CP/CPPS continues to increase [5]. In recent years, CP/CPPS has become a hot issue in urology and andrology research [6].

Randomized controlled trials (RCTs) provide the strongest recommendations in evidence-based medicine, making them the “gold standard” for evaluating the effectiveness and safety of medical interventions [7]. In recent years, many researchers have conducted a large number of CP/CPPS-related RCTs. The quality of the research and reports will directly affect the judgment of the intervention effect. However, evaluations of RCTs on CP/CPPS are lacking. Thus, this study assessed the quality of RCTs on CP/CPPS using the Consolidated Standards of Reporting of Trials (CONSORT) statement [8] and the improved Jadad scale [9] to propose directions for further studies.

Methods

Subjects of the analyses

RCTs on CP/CPPS published from the establishment of databases to December 31, 2021 were identified in PubMed, Web of Science, and Embase using the following search terms: “randomized,” “randomization,” and “randomly.” The search terms for CP/CPPS were prostatitis, prostatism, chronic prostatitis, chronic pelvic pain syndrome, abacterial prostatitis, nonbacterial prostatitis, and prostatodynia. Two reviewers independently searched for RCTs, whereas the different results retrieved by the two reviewers were adjusted and fine-tuned by a third reviewer.

Quality assessment method

Two reviewers analyzed the RCTs using two quality assessment tools, including the CONSORT statement and the improved Jadad scale. If the data evaluated by the two reviewers were inconsistent, adjustment was performed by a third reviewer.

CONSORT statement

The CONSORT statement is a clinical trial report specification formulated by the CONSORT working group. It provides authors with a standardized method to prepare trial results reports, prompts authors to complete transparent reports, and helps them to critically evaluate and explain their results. The CONSORT statement includes a 25-item checklist and a flowchart. The list includes six specifications: title and abstract, introduction (background and objectives), methods (trial design, participants, interventions, outcomes, sample size, randomisation: sequence generation, allocation concealment mechanism, implementation, blinding, and statistical methods), results (participant flow, recruitment, baseline data, numbers analysed, outcomes and estimation, ancillary analyses, and harms), discussion (limitations, generalisability, and interpretation), and other information (registration, trial protocol, and funding). The checklist focuses on how the reported experiment is designed, analyzed, and interpreted. The flowchart presents the progress of all participants in the experiment. The CONSORT statement applies to randomized parallel controlled trials, and strict compliance with the requirements of the items in the list will help authors write clear, complete, and transparent clinical trial reports. If there are deficiencies in the research, then they will be exposed by the transparent reports. Therefore, if an improperly designed experiment is implemented and a transparent report is required, researchers cannot pass the review process for publication of the paper without disclosing the deficiencies of the experiment. The CONSORT statement not only helps researchers to improve the level of clinical research design but also helps editors and peer reviewers to review expert manuscripts and readers rigorously evaluate published literature [10].

Jadad scale

The Jadad scale (also known as the Jadad score or Oxford quality scoring system) is a tool for independently evaluating the quality of clinical trial methodology [11]. The improved Jadad scale has a maximum score of seven points, with 1–3 points indicating low quality and 4–7 points indicating high quality. For random sequence generation, when using computer-generated random numbers or similar methods, two points each are awarded for each respective category. When a randomized trial is used but the method of random allocation is not described, one point is awarded for each respective category. When alternate allocation methods are used, such as odd and even numbers, zero points are awarded. For randomization concealment, two points are given when appropriate methods are used to prevent clinicians and subjects from predicting the allocation sequence, such as central or pharmacy control of the allocation plan or the use of containers with consistent serial numbers, on-site computer control, sealed opaque envelopes, or other methods. One point is given when the method of concealment is not clear, such as indicating the use of a random number table or another random allocation plan. Zero points are given when the method of concealment is inappropriate or not used. For blinding, two points are given when the blinding method is appropriate, such as using a completely consistent placebo tablet or a similar method. One point is given when the method is not clear; for example, the test is stated as a blind method, but the method is not described. Zero points are given when the method is not appropriate; for example, it is not appropriate to use double-blind or blind methods, such as comparison of tablets and injections. For withdrawal, one point is given when the number of withdrawals and the reason for withdrawal are described; otherwise, zero points are given.

Results

Document retrieval

In total, 1716 articles were obtained from the initial search. Of these, 1018 articles were obtained after the first screening, and 77 RCTs were finally included after the second screening. The specific literature retrieval process and results are presented in Fig. 1.

Fig. 1
figure 1

Flow chart of literature screening for RCTs of CP/CPPS. The number of final analysis RCTs of CP/CPPS, included and excluded articles, from the PubMed, Web of Science, and Embase databases

Basic characteristics of the included studies

The sample of these studies ranged from 17 to 324, with a median of 64, and only 12 (15.58%) of the studies described the process and method of sample size estimation. 21 (27.27%) were multicenter studies, with 2 to 34 research centers. The baseline information was consistent and comparable across studies. The RCTs all aimed to evaluate the effectiveness and safety of the interventions. Regarding the diagnostic criteria for CP/CPPS, 8 (10.39%) studies used the Chronic Prostatitis Collaborative Research Network sponsored by the National Institutes of Health; 4 (5.19%) studies used the European Association of Urology guidelines; 53 (68.83%) studies stated that the diagnosis was made based on history, physical examination, and laboratory tests; and 12 (15.59%) studies only mentioned patients with CP/CPPS, without specifying which diagnostic criteria were used. Of the interventions, 9 (11.69%) studies were alpha1 receptor blockers, including terazosine, doxazosin, and alfuzosin. 5 (6.49%) studies were pollen extracts, and other therapeutic agents included finasteride, palmetto, tamsulosin, silodosin, apremilast, tanezumab, duloxetine, pregabalin, rofecoxib, and zafirlukast, among others. 8 (10.39%) of the studies were extracorporeal shock wave therapy, and other physical rehabilitation treatments included transurethral needle ablation, transurethral microwave thermotherapy with urethral cooling, sono-electro-magnetic therapy, transcutaneous electrical nerve stimulation, repetitive transcranial magnetic stimulation, posterior tibial nerve stimulation, myofascial physical therapy, and cryotherapy, etc. Among the control groups, 44 (57.14%) studies used a placebo or sham treatment, and 33 (42.86%) studies used positive treatment.

Literature quality evaluation results based on the CONSORT statement

The evaluation of the quality of the literature in the CONSORT statement included the title and abstract, introduction, methods, results, conclusions, and other information. The number and percentage of articles for each topic are presented in Table 1.

Table 1 Assessment of the quality of RCTs of CP/CPPS based on the CONSORT statement

Title and abstract

Among the included 77 RCTs, 67 (87.01%) articles mentioned “randomized” and “controlled” in the title and abstract, and the other 10 (12.99%) articles did not specify the type of study in the title and abstract.

Randomization, blinding, and allocation hiding

Although the 77 included RCTs mentioned “randomized,” only 26 (33.77%) papers mentioned the specific randomization methods. Among them, 17 (33.77%) papers generated random assignment sequences by computer, and 9 (11.69%) papers used the order of patient visits as the grouping method. Only 26 (33.77%) articles mentioned the “blind method,” but they did not explain how the method was implemented. 6 (7.79%) studies mentioned allocation hiding.

Subjects and test procedures

36 (46.75%) articles mentioned the recruitment of subjects and the baseline situation of subjects. Conversely, only 6 (7.79%) articles used graphs to present the baseline data. Only 34 articles (44.16%) recorded the number of patients who withdrew from each group after random grouping. 16 (20.78%) articles used flowcharts to record the test process.

Adverse reactions

Of the 77 included RCTs, 67 (87.01%) studies reported whether adverse events occurred during the study, and none of the remaining literature stated whether adverse events occurred before or after the trial.

Statistical methods

Among the 77 included RCTs, all articles specifically described the statistical methods used to compare the primary and secondary outcome indicators of each group, and 15 (19.48%) articles described the methods of additional analysis.

Clinical registration, plan announcement, and funding description

Among the included studies, only 16 (20.78%) documents stated that they were registered on various registration platforms, only 3 (3.90%) mentioned the relevant trial schemes and 6 (7.79%) mentioned funding status.

Interpretation of results

Among the 77 included RCTs, 57 (74.03%) studies described the limitations and reasons for the trial in the discussion section. And 57 (74.03%) studies provided explanations that are consistent with the experimental results. Meanwhile, 61 (79.22%) articles provided negative results.

Literature quality evaluation results based on the improved Jadad scale

Among all 77 articles evaluated by the improved Jadad scale, only 19 (24.68%) had a score of ≥ 4. Meanwhile, 58 (75.32%) were low-quality studies (score ≤ 3), and little of them mentioned random hiding and execution. The number and percentage of articles for each score are presented in Table 2.

Table 2 Assessment of the quality of RCTs in the treatment of CP/CPPS based on improved Jadad scale scores

Discussion

According to the CONSORT statement and the improved Jadad scale, this study evaluated the literature on RCTs in the treatment of CP/CPPS. The results illustrated that the quality of the studies in this field needs to be improved in several aspects.

As we all know, irregular presentation of the title and abstract may influence the reader’s judgment of the literature and affect whether the literature can be identified by the searcher. Therefore, we recommend that the authors clearly label the title and abstract with “randomized” and “controlled” to indicate that the study was an RCT. However, this does not mean that the authors did not use randomization in their study. Appropriate sample size estimation is an important part of clinical trials [12]. The sample size must be sufficient to reliably answer the relevant questions raised by the research hypothesis and meet the accuracy and reliability of statistics; at the same time, the sample size should not be excessive to avoid waste. Estimation of the sample size is the basis for the credibility of the research results. Therefore, future clinical studies of CP/CPPS should pay attention to the estimation of sample size.

Although most studies did not explicitly state the method of randomization, this does not mean that these studies did not use randomization or that randomization occurred in error. Therefore, we further used the improved Jadad scale to better assess randomization, blinding, and allocation concealment. Whether the random sequence is reasonable directly affects the feasibility and credibility of clinical RCTs. Blinding is one of the important measures for controlling bias in RCTs. Implementation of a blinding method can effectively control the bias caused by human-made differences between groups [13]. Adopting the correct method design (such as the random method, blind method, and allocation concealment) is an important measure for reducing the bias of clinical research. An incorrect design may exaggerate research results.

A flow chart can intuitively record the collection of cases, the status of the test, and the number of withdrawals, but most researchers do not pay sufficient attention to the test process. Adverse event reporting is an important part of RCTs. Data on adverse reactions help readers to judge the safety of various therapies, and they can have a major impact on judging the acceptability and practicality of interventions. In addition, our results suggested that researchers do not pay enough attention to data statistics. Statistics, as the foundation of scientific research, is an important tool for medical scientific research [14]. The correct application of statistical methods directly affects the quality of medical scientific research and the level of scientific research papers.

Moreover, clinical registration, plan announcement, and funding description are conducive for readers to judge the transparency and reliability of CP/CPPS clinical research. Our results also illustrate that researchers’ awareness of clinical research registration is relatively weak at this stage. Clinical trial registration is a basic requirement for high-level journals to publish articles. The articles in the clinical registry are only available in the last few years. Future CP/CPPS clinical research should ensure that studies are appropriately registered. Inadequate discussion of study findings may lead to publication bias, thereby exaggerating the effects of interventions. These studies we analyzed are relatively well done in the process of discussion.

The CONSORT statement has a total of 37 entries for the 25-item checklist, and the number of entries met by these studies ranged from 6 to 25. If both the number of entries and the Jadad score were high, we considered these studies to be relatively reliable and their quality to be acceptable. The screening was performed with the condition of ≥ 20 entries and a Jadad score of ≥ 4. A total of 15 studies met the requirements, and we considered these to be more formal and scientifically reliable papers.

Through quality assessment, we found that some lower-quality RCT studies still managed to be published, which may include the following reasons. First, the treatments in these studies may be relatively new and the studies are still in the exploratory phase. Second, these studies were able to present scientific evidence that addressed some clinical questions or contradictions. Finally, although they do not follow a standardized protocol to improve the quality of the study, their data are realistic.

Generally speaking, the results of our study indicated that the quality of RCTs in the treatment of CP/CPPS needs to be improved, and care should be taken when accepting and using the results. These inadequacies included a lack of information on the nature of the experiment in the title, the estimated sample size, the absence of random grouping or unreasonable methods, a lack of blinding, the absence of graphs to represent the baseline data of subjects, no records on patient withdrawal and elimination, a lack of details about adverse events, and the absence of clinical registration. This missing information may cause readers to question the correctness and rationality of the relevant research design, making it difficult to promote the clinical application of treatments for CP/CPPS. When conducting future research, researchers should follow the internationally recognized CONSORT statement and use the improved Jadad scale to standardize the design and implementation of RCTs; correctly select randomization, blinding, and statistical methods; and pay attention to adverse reaction reports and clinical registration instructions. This strategy to improve the quality of RCTs will provide a more reliable basis for the promotion of clinical applications for the treatment of CP/CPPS.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

RCT:

Randomized controlled trial

CP/CPPS:

Chronic prostatitis/chronic pelvic pain syndrome

CONSORT:

Consolidated standards of reporting of trials

References

  1. Krieger JN, Nyberg L Jr, Nickel JC. NIH consensus definition and classification of prostatitis. JAMA. 1999;282(3):236–7.

    Article  CAS  Google Scholar 

  2. Franco JVA, Turk T, Jung JH, Xiao YT, Iakhno S, Tirapegui FI, et al. Pharmacological interventions for treating chronic prostatitis/chronic pelvic pain syndrome: a cochrane systematic review. BJU Int. 2020;125(4):490–6.

    Article  Google Scholar 

  3. Liu SJ, Gao QH, Deng YJ, Zen Y, Zhao M, Guo J. Knowledge domain and emerging trends in chronic prostatitis/chronic pelvic pain syndrome from 1970 to 2020: a scientometric analysis based on VOSviewer and CiteSpace. Ann Palliat Med. 2022;11(5):1714–24.

    Article  Google Scholar 

  4. Khan FU, Ihsan AU, Khan HU, Jana R, Wazir J, Khongorzul P, et al. Comprehensive overview of prostatitis. Biomed Pharmacother. 2017;94:1064–76.

    Article  Google Scholar 

  5. Doiron RC, Shoskes DA, Nickel JC. Male CP/CPPS: where do we stand? World J Urol. 2019;37(6):1015–22.

    Article  Google Scholar 

  6. Nickel JC. Chronic prostatitis/chronic pelvic pain syndrome: it is time to change our management and research strategy. BJU Int. 2020;125(4):479–80.

    Article  Google Scholar 

  7. Horwitz RI, Hayes-Conroy A, Caricchio R, Singer BH. From evidence based medicine to medicine based evidence. Am J Med. 2017;130(11):1246–50.

    Article  Google Scholar 

  8. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

    Article  Google Scholar 

  9. Li H, Chen C, Hu F, Wang J, Zhao Q, Gale RP, et al. Impact of corticosteroid therapy on outcomes of persons with SARS-CoV-2, SARS-CoV, or MERS-CoV infection: a systematic review and meta-analysis. Leukemia. 2020;34(6):1503–11.

    Article  CAS  Google Scholar 

  10. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.

    Article  Google Scholar 

  11. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.

    Article  CAS  Google Scholar 

  12. Eng J. Sample size estimation: how many individuals should be studied? Radiology. 2003;227(2):309–13.

    Article  Google Scholar 

  13. Liu J, Kjaergard LL, Gluud C. Misuse of randomization: a review of Chinese randomized trials of herbal medicines for chronic hepatitis B. Am J Chin Med. 2002;30(1):173–6.

    Article  Google Scholar 

  14. Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine–reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357(21):2189–94.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was supported by the National Natural Science Foundation of China (No. 82104880) and Scientific and Technological Innovation Project of China Academy of Chinese Medical Sciences (No. CI2021A02208).

Author information

Authors and Affiliations

Authors

Contributions

MZ: Conceptualization; Formal analysis; Writing-original draft; Writing- review and editing. Q-HG: Data curation; Formal analysis. S-JL: Data curation; Formal analysis. Y-JD: Data curation. HW: Writing-review and editing. W-XY: Conceptualization; Supervision; Writing-review and editing. JG: Conceptualization; Supervision. All authors read and approved the final manuscript

Corresponding author

Correspondence to Jun Guo.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no further competing/conflicting interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, M., Gao, QH., Liu, SJ. et al. Quality assessment and relevant clinical impact of randomized controlled trials on chronic prostatitis/chronic pelvic pain syndrome. BMC Urol 22, 122 (2022). https://doi.org/10.1186/s12894-022-01078-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12894-022-01078-5

Keywords