EAU, AUA and NICE Guidelines on Surgical and Minimally Invasive Treatment of Benign Prostate Hyperplasia: A Critical Appraisal of the Guidelines Using the AGREE-II Tool

Benign Prostatic Obstruction (BPO) due to Benign Prostatic Hyperplasia (BPH) is one of the most common urological diseases and often requires surgical treatment [1-3]. A number of less invasive procedures have come to light recently allowing us to pursue a more personalized approach. In this regard, various randomized controlled studies (RCTs), systematic reviews and meta-analyses have been conducted. Evidence-based data supporting the effectiveness of these procedures led to them being reflected in the recently updated (up to July 2020) clinical practice guidelines (CPGs).

There are a number of tools that can be used to assess the quality of the guidelines. One of the most validated and extensively used is the AGREE (Appraisal of Guidelines for Research and Evaluation) instrument. It was developed to address the issue of variability in the quality of the guidelines and in the development process and reporting [4, 5].

The purpose of the present study was to critically appraise the methodological rigour and transparency of CPGs issued by the leading organizations on the minimally-invasive treatment of BPO using the AGREE II instrument and to evaluate the comprehensiveness of surgical and minimally invasive BPH treatment methods as presented in the CPGs.

We included recent updates of full texts of clinical guidelines for surgical minimally invasive treatment of BPH. The guidelines by any other national associations or non-English guidelines were excluded. The largest English-speaking urological societies’ guidelines were used. In this analysis, current guidelines from the European Association of Urology (EUA) [6], American Urological Association (AUA) [7] and National Institute for Health and Care Excellence (NICE) [8] were included. We used AGREE-II tool (version 2, December 2017 update) to assess the quality of these guidelines.

AGREE-II instrument consists of 23 items that are scored on a 7-point scale ranging from 1 (strongly disagree, which means that no relevant information is provided) to 7 (strongly agree, which means that the quality of reporting is excellent). These 23 items are grouped into 6 domains [5].

Final domain scores were calculated by summing up all the item scores within the domain and by scaling the total as a percentage of the maximum possible score for that domain [5].

Four urologists (optimal according to AGREE II) with differing levels of expertise (1- <5 years, 2–3- 5–10 years and 4- >15 years) were trained to use the AGREE II rating system (the training tools are available online at www.agreetrust.org). All the participants practice outside the jurisdictions of the guidelines (EU, USA, UK). The participants independently appraised and scored each guideline. The scores were then expressed as standardized domain scores on a percent scale ranging from 0 to 100% (a percentage of the maximum possible score for each particular domain).

The performance in each domain was evaluated using a threshold of 60%. In addition, the total score (expressed as median and IQR) of the guidelines and the domains was calculated. Domain scores were categorized as good (≥80%), acceptable (60–79%), low (40–59%), or very low (<40%) [9, 10].

The EAU guideline [6] latest version was published in 2020 and is updated annually. Members of the Panel incorporated into the guidelines for ease of evidence acquisition and synthesis strictly systematic reviews with meta-analysis, RCTs, and prospective non-randomized comparative studies. All collected data are assessed and graded according to the GRADE approach. Based on these forms, the team determined how strong the rating was and assigned each recommendation a “strong” or “weak” rating.

The AUA guideline [7] latest version was published in 2019, an update of the previous one released in 2010. The evidence review team identified eligible RCTs, clinical controlled trials (CCTs), systematic reviews/meta-analyses, and observational studies. The team also used the Cochrane Collaboration’s tool for assessing risk of bias. Overall quality of evidence was evaluated using the GRADEpro software. The quality of evidence (graded from A–C) and Linking statement type (strong, moderate and conditional recommendations, clinical principle and expert opinion) are assigned to each recommendation.

The NICE [8] guideline was published in 2010, updated in 2015 and some additional files were released in 2019. The development group assessed the quality using the GRADE system and graded it according to NICE Guidelines Manual. The NICE development group does not assign strength rating or linking statement type, etc. They explain each recommendation by giving information about the values of different outcomes, clinical benefits and harms, economic considerations, quality of evidence (by discussing it) and some other considerations.

The AGREE-II quality scores for domain 1 ranged from 56.9 to 94.4% with a median score (IQR) of 66.7% (61.8–80.5%). AUA and NICE scores were >60% in the domain 1. However, the score of EAU was <60% with the lower scores in objectives description, health questions and study populations.

The AGREE II quality scores for domain 2 ranged from 25.0 to 61.1% with a median score (IQR) of 50.0% (37.5–55.6%). The included CPGs reported minimal information about patient’s preferences or target users of the guidelines and subsequently were assigned low scores.

The AGREE II quality scores for domain 3 ranged from 62.5 to 83.9% with a median score (IQR) of 65.1% (63.8–74.5%). All CPGs scores were >60% in the domain 3. However, the most common weaknesses across all CPGs were the following: the methods for formulating the recommendations and external review.

The AGREE II quality scores for domain 4 ranged from 79.2 to 91.7% with a median score (IQR) of 80.6% (79.9–86.2%). This domain was well-addressed in all the CPGs that were included. Recommendations were informative, specific, and key issues were easily identifiable. Moreover, the highest median score was in this domain (median 80.6%) with minimal variability.

The AGREE II quality scores for domain 5 ranged from 32.3 to 56.3% with a median score (IQR) of 33.3% (32.8–44.8%). The NICE guideline received the highest score. However, none of the CPGs received a score above 60% (EAU–32.3%; AUA–33.3%; NICE–56.3%). The information regarding barriers, facilitators, the monitoring and auditing of criteria was missing in most of the CPGs. This domain is characterized by the lowest median score of 33.3%, with a medium variability.

The AGREE II quality scores for domain 6 ranged from 60.4 to 79.2% with a median score (IQR) of 72.9% (66.6–76.1%). All the guidelines scored >60% and provided adequate information about competing interests. EAU guidelines include a conflict of interest disclosure for each panel member which states that no external sources of funding were involved. As for the AUA guidelines, there is also a conflict of interest disclosure which must be completed by each co-author. Where the NICE guidelines is concerned, a conflict of interest is listed not only for the panel members, but also for the studies which were assessed to develop the guidelines.

The difference between the appraisers’ score within each domain was not >2 points (at 7-point scale). The median score (IQR) of overall assessment is 83.3% (79.1–91.6%). The highest score was received by NICE guidelines (91.6%) and the lowest–EAU guidelines (79.1%). Overall, the NICE guideline scored highest on 5 out of the 6 domains, the EAU guideline scored lowest on 4 out of the 6 domains with the AGREE II instrument (Table 1).

All the included guidelines state bi- or monopolar TURP as the standard option for the treatment of men with BPH. Only the EAU guideline recommends this method for patients with prostate volume not >80 cm³. Beyond that, the associations recommend using transurethral incision of the prostate (TUIP) in case of small glands (<30 cm³).

The EAU, AUA and NICE guidelines agree on the use of the electro- or laser vaporization for BPO treatment. The EAU guidelines recommend laser vaporization primarily for patients with prostate volume 30–80 cm³ as an alternative to TURP. The same indications apply for bipolar transurethral vaporization (TUVP). The AUA guidelines do not mention any restrictions on vaporization by prostate size. The NICE working group reported that bipolar TUVP, transurethral vaporesection of the prostate (TUVRP) and laser vaporization techniques may be used only as a part of clinical trials in comparison with TURP. The only technique with a clear recommendation is vaporization with GreenLight XPS laser in non-high-risk patients (with low bleeding risk, prostate size <100 cm³ and without urinary retention).

In their guidelines, the Associations under consideration describe open SP as a treatment option for large prostates with minor differences. The EAU mentioned open SP when EEP was not available. The AUA suggests considering open, laparoscopic or robotic assisted SP, depending on the surgeon expertise. The NICE guideline stated open SP as an alternative to TURP, TUVP or HoLEP.

The EAU and AUA working groups offer enucleation for patients with prostate size >80 cm³. EAU stated that Holmium laser enucleation (HoLEP) is the current standard for large prostates and is an alternative to TURP or open SP. According to European recommendations, specialists can also use diode or thulium laser for enucleation, the latter being an alternative to HoLEP. The AUA recommends HoLEP and thulium laser enucleation (ThuLEP) as prostate size-independent procedures. The NICE guidelines are not very specific when it comes to recommendations regarding laser enucleation methods.

The EAU and AUA recommend laser treatment (enucleation or vaporization) for patients on anticoagulant/antiplatelet therapy. Similarly, the EAU recommends laser modalities: HoLEP, vapoenucleation and vaporesection with Tm:YAG laser and vaporization with 532-nm Potassium-Titanyl-Phosphate (KTP) or 532-nm Lithium Borat (LBO) lasers. The NICE team discusses the implementation of GreenLight XPS as a possible option but notices that there is insufficient high-quality evidence.

All associations offer PUL with some variations in indications, a permitted prostate size being one of them. The EAU recommends <70 cm³, the AUA– <80 cm³ and the NICE– <100 cm³. The AUA also points to the need for informing the patients about PUL’s inferiority to TURP. Where the NICE guideline is concerned, PUL is thought to be suitable for men older than 50.

The EAU and AUA offer waterjet ablation (aquablation) as a treatment option for men with prostate volume 30–80 cm³, but both guidelines highlight the lack of long-term follow up data. The NICE guideline recommends water jet ablation as 1 possible option. The association states that the evidence on efficacy is limited, that the procedure should only be done within clinical trials settings.

The EAU guideline recommends PAE for patients who want to consider minimally invasive treatment options despite less optimal objective outcomes compared with TURP. The AUA considers PAE only for clinical trials. The NICE indications are similar to the EAU.

Both the EAU and NICE make mention of intra-prostatic injections. Whereas the EAU discarded intraprostatic Botulinum toxin-A injection, the NICE guideline offers it only as a part of a RCT. The AUA does not discuss it.

The AUA team states that it is a suitable option for patients with a prostate volume <80 cm³, and for those who have already been informed about its safety and efficacy and want to preserve their sexual function. The NICE guideline points out that the procedure should only be done by specifically trained urologists. The EAU states that randomized controlled trials against a reference technique are needed to confirm the first results.

The NICE working group proposes using a prostatic urethral temporary implant only within clinical trials because the evidence on the safety and efficacy is limited in terms of its quantity and quality. The EAU and AUA do not discuss this procedure.

Besides the abovementioned methods, the AUA recommends using transurethral microwave thermotherapy (TUMT) only for patients who have been informed that surgical retreatment rates are higher compared to TURP. It also does not recommend transurethral needle ablation (TUNA). The NICE guideline does not offer TUMT, TUNA, high-intensity focused ultrasound (HIFU), transurethral ethanol ablation of the prostate (TEAP) as an alternative to TURP, TUVP or HoLEP. The EAU makes no mention of them.

To our knowledge, this study is the first to evaluate the development and reporting quality of CPG, as proposed by the AGREE-II tool [5]. Our review focused on the most recent updates of the international association guidelines [6-8]. The overall scores for the guidelines showed that all 3 guidelines are being rigorously developed and serve as examples of well-designed CPGs. The NICE scored best with 91.6%, whereas the EAU and AUA scored 79.1 and 83.3%, respectively.

The guidelines aim to improve the quality of patient care and reduce unnecessary differences in practice [11]. The main target audience for the guidelines are urologists, therefore in our study we purposefully included practicing urologists without any other specific qualification (e.g., in field of linguistics or data analysis) to appraise the guidelines. Amongst the 6 parameters assessed with AGREE II, the overall domain scores for clarity of presentation were the highest (80.6%) with minor variance between the 3 guidelines. This domain represents how the CPGs are dealing with the main goal–reporting effective treatment options for different patients. We believe that despite lower scores in other domains, a high score in this field signifies the clinical importance of all the guidelines (EAU–91.7%; AUA–80.6%; NICE–79.2%). The domains relating to editorial independence, scope and purpose yielded the second and third best scores respectively.

One of the least scored domains was “Applicability” with a score of 33.3% (EAU–32.3%; AUA–33.3%; NICE–56.3%). The latter is the anticipated finding as most of the guideline development groups did not indicate the barriers or facilitators to implementing guideline recommendations and did not provide ways to audit effective implementation. Being a secondary goal in guideline development, the applicability is an important point that should be considered by the Panels. However, the applicability of the guidelines is not limited to methods of implementation and as suggested by Speakman et al. [12] should include a discussion of how Panels evaluate new methods. The authors believe that this may result in minimizing the misuse under commercial pressure, particularly in private healthcare centers [12].

The domain stakeholder involvement was also ranked low with a score of 50.0% (EAU–25.0%; AUA–50.0%; NICE–61.1%). After the evaluation process, we found that most guidelines did not highlight the views and preferences of the target population and did not clearly define the target users. The NICE guidelines reported the broadest spectrum of Panel participants and were more precise in defining target users. The absence of precise data on stakeholder involvement should not be considered a major limitation to the guidelines, but it could significantly improve the applicability of the guidelines for patients.

The stakeholder involvement and applicability domains were also ranked lowest in the study of Xu et al. [13], who compared guidelines on α1-blockers and 5α-reductase inhibitors administration. The lack of target population views and implementation mechanisms were described by Gupta et al. [14] in assessing the prostate cancer guidelines (stakeholder involvement and applicability were also ranking lowest). Discussing guidelines on smoking cessation, Sharma et al. [15] assign 54.2 and 33.3% respectively for the rigour of development and editorial independence domains. Such consistency in CPGs design may signify another important limitation to modern medical practice–we are more prone to focus on the estimated efficacy and percent of complications rather than report the actual needs of the population.

The differences in surgical options among the included guidelines were insignificant. First, each association recommends a different range of methods. Such discrepancies could emerge because the working groups had used different search strategy and included different studies for evidence synthesis. The EAU selected strictly systematic reviews, RCTs and prospective non-randomized comparative studies. The AUA, in addition to the abovementioned, also included observational and clinical controlled trials. The NICE guidelines did not limit search strategy and used free-text terms. This differences in approaches have been previously mentioned by Speakman et al. [12] where they discussed how the EAU panel evaluates papers. Such a rigorous approach to selecting and assessing articles allowed EAU scored first in the domain of clarity of presentation. The second possible reason for the differences is the updating process. The EAU CPG are only being updated annually.

The EUA CPG 2021 have been released recently. No changes in methodology or editorial board comparing with the previous year guidelines took place, thus the newest guidelines obviously have the same score for domains 1–3 and 6. The main change in terms of surgical treatment was restructuration of the section: while previous year it was divided according to specific technologies, no they are grouped depending on the surgical technique as follows: resection, enucleation, vaporization, alternative ablative techniques. The same urologists ranked this new edition of EAU guidelines for domain 4 and 5 and got the same results as for the previous version.

The AGREE tool was elaborated in 2003 as an instrument for CPG evaluation [4] and has been amended recently [5, 16, 17]. Unfortunately, high AGREE II scores do not guarantee high quality of CPGs. Despite the presence of high and rather low ranked domains, the included guidelines are of high-quality and clearly reflected all the relevant information about BPH treatment. Notwithstanding, using many appraisal tools, AGREE II was shown to be a reliable assessment instrument with the highest potential [18].

This study has several limitations. Firstly, CPGs were appraised only by urologists. However, it should be borne in mind that it was a large team comprising both less- and highly experienced specialists. Secondly, the AGREE II tool is not an ideal instrument, but it remains one of the most reliable. The inability to distinguish significant differences between 2 similar scores is one of the major limitations of the AGREE II, and the only reliable threshold suggested is 60% distinguishing between low and acceptable levels.

The analyzed CPGs comprehensively highlight the minimally invasive and surgical treatment options for BPH. According to the AGREE II tool, the domains for clarity of presentation and editorial independence received the highest scores. The stakeholder involvement and applicability domains were ranked as the lowest. Improving the CPG in these domains may help to improve the clinical utility and applicability of CPGs.

The paper is exempt from ethical committee approval because this study is a literature review and did not involve humans or animals.

The authors have stated explicitly that there are no conflicts of interest in connection with this article.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

D. Enikeev: Data analysis, Protocol development. V. Misrai: Manuscript editing. E. Rijo: Manuscript editing. R. Sukhanov: Data collection, Manuscript editing. D. Chinenov: Data collection, Manuscript editing. M. Gazimiev: Data collection, Manuscript editing. M. Taratkin: Data analysis, Manuscript writing. C. Azilgareeva: Data analysis, Manuscript writing. A. Morozov: Data analysis, Manuscript editing. T.R.W. Herrmann: Manuscript editing. P. Glybochko: Protocol development, Manuscript editing.

Data are available on demand.