External validation of Ito’s nomogram and T.O.HO. scoring system in flexible ureterorenoscopy

Abstract

Purpose

This study aims to assess stone-free rates after flexible ureterorenoscopy (fURS) using the T.O.HO. (Tallness, Occupied lesion, Hounsfield unit evaluation) scoring system and Ito’s nomogram.

Materials and methods

In the study conducted with 484 patients, the following parameters were analyzed: age, sex, comorbidities, hospitalization, affected side, extracorporeal shock wave lithotripsy (ESWL) history, stone length, stone density, number of stones, location, and presence of hydronephrosis.

Results

Multivariate logistic regression analysis revealed that stone length, stone number, and lower pole stone location were associated with the prediction of stone-free status. The cut-off value for Ito’s score was determined to be 13.5 points, with an AUC of 0.792 (sensitivity, 0.609; specificity, 0.821) and a 95% confidence interval of (0.752–0.832) (Fig. 1). The cutoff for the T.O.HO. score was 6.5 points, with an AUC of 0.744 (sensitivity 0.738, specificity 0.602) and a 95% confidence interval of (0.699–0.789).

Conclusion

In conclusion, T.O.HO. scoring system and the Ito’s nomogram are promising tools to predict stone-free status (SFS) after fURS in preoperative evaluation. In addition, the success of scoring systems in predicting SFS preoperatively appears promising and offers a potentially valuable approach.

Introduction

Urinary stone disease (USD), is a prevalent condition worldwide, posing a significant public health issue. It is associated with a deterioration in the quality of life and loss of labor force [1]. There is a variety of options in the treatment of USD, depending on stone-related factors such as the size and the location of the stone as well as the patient’s comorbidities and urinary tract anatomy [2]. Advancements in optical technology and miniaturization have led to the development of new instruments in urinary stone treatment. In current practice, flexible ureterorenoscopy (fURS) is one of the first-choice modalities in the surgical treatment of USD, boasting high success and low complication rates [2, 3]. Stone-free rate (SFR), a primary determinant of success, is reported to be over 90% for fUR [4]. However, some stone and patient related factors might interfere with the success of this procedure [5]. Scoring systems evaluating these factors are helpful for preoperative counseling regarding surgical success and possible complications.

The first scoring system predicting the surgical success rate of fURS was described by Resorlu et al. in 2012 (RUSS) [6]. With the advancing imaging technology, practical scoring systems and nomograms aiming at higher predictive values have been developed. S.T.O.N.E., S-RESC, Ito’s, R.I.R.S. and T.O.HO. are some of the scoring systems developed for this purpose [7,8,9,10,11]. Recently, a new scoring system has also been described by Polat et al [12]. These scoring systems allow for more accurate interpretations of surgical complications and the likelihood of achieving stone-free status (SFS), enabling better patient counseling [6,7,8,9,10,11,12].

An ideal scoring system should include all variables that may affect the outcome and provide practical and objective measurements. Although predictive parameters have been demonstrated in previous studies, all scoring systems require validation. This study aimed to investigate Ito’s nomogram and the T.O.HO. scoring system, which have relatively fewer external validation studies and have been defined with high predictive values for SFR (AUC: 0.830, 0.870, respectively) [9, 11].

Patients and methods

Data collection and ethics statement

We retrospectively analyzed 801 patients who underwent fURS for kidney stones between October 2012 and March 2019. The present study protocol was reviewed and approved by the Institutional Review Board of Ankara City Hospital (approval number: E2-23-3590). Data for 236 patients were either inaccessible or unreliable, resulting in their exclusion from the study. In addition, 64 patients with a history of previous kidney stone surgery and 17 patients with renal abnormalities were excluded. The remaining 484 patients were included in the study. The patients’ age, sex, comorbidities, duration of hospitalization, history of extracorporeal shock wave lithotripsy (ESWL), stone characteristic, and presence of hydronephrosis were analyzed. Patients were diagnosed using preoperative non-contrast computed tomography (CT). Stone characteristics included lateralization (right-left), number (single or multiple), length (mm), location (lower-middle-upper pelvic complex), and density (Hounsfield Unit (HU)). The mean HU value of the stones was calculated by taking the average of the midpoint of the two regions in the slice where the maximum and minimum diameters of the stones were measured, along with the CT attenuation values at the central and outer edges of the stone. For patients with multiple stones, the stone diameter was taken as the sum of each stone and the mean HU value was taken as the mean HU value of each stone.

All procedures were initiated with a 9.5 F semi-rigid URS (Karl Storz, Tuttlingen, Germany) by entering the ureter through a guidewire (0.035 inches, Microvasive; Boston Scientific Corp., Natick, MA). Laser lithotripsy was performed using fURS (7.5 F; Karl Storz Flex-X2, Tuttlingen, Germany) and a 270–350 μm holmium laser (AMS; Sureflex). A ureteral access sheath was used to facilitate stone removal and reduce intrarenal pressure. Preoperative stenting was applied in cases of ureteral stenosis, severe renal colic despite analgesic use, and the presence of pyelonephritis. After the operation, a DJ or ureteral catheter was inserted depending on the patient’s clinical and operative status. Patients with no residual stones on radiograms obtained on postoperative day 1 were discharged, and the DJ stent was removed 2 weeks postoperatively. Patients with residual stones were scheduled for secondary surgery or ESWL.

Definition of stone-free status

Patients were defined as having SFS if no residual stone was detected on imaging (X-ray or CT) on postoperative 1st month.

Score analysis

Ito’s nomogram is calculated using 5 variables: stone volume (13/8/5/0 points), presence of stones in the lower pole of the kidney (5/0 points), the operator having more than 50 fURS experiences (3/0 points), number of stones (2/0 points), and degree of hydronephrosis (2/0 points). The total nomogram score (25 to 0 points) is obtained by summing the scores of these criteria. A high total score predicts successful fURS [9]. The T.O.HO. scoring system includes stone length (1/2/3/4/5 points), stone location (proximal ureter or renal pelvis: 1 point, middle and upper pole: 2 points, lower pole: 3 points) and stone density (1/2/3 points). Using this information, the score is calculated between 3 and 11. A high score indicates the likelihood of a failed fURS [11].

Score analyses were performed by two independent authors (SB and KC). In case of different measurements, the opinion of a third author (SY) was obtained.

Statistical analysis

Data analysis was performed using SPSS 22.0 software. The normal distribution of the data was assessed using the Shapiro-Wilk test. The Mann-Whitney U test was used to compare non-categorical data, and the chi-square test was used for categorical data. The backward logistic regression method and multivariate logistic regression analysis were employed to evaluate parameters that may be independent risk factors in predicting fURS success. To evaluate the Ito’s nomogram and T.O.HO. for predicting fURS success, a ROC curve was analyzed with a 95% confidence interval, and cases with a p value < 0.05 were considered statistically significant.

Results

A total of 484 patients were included in the study. Comorbidities and clinical characteristics of the patients are shown in Table 1. The mean age of the included patients was 45.8 ± 14.1years. 16.1% of the patients had a previous history of ESWL. A preoperative DJ catheter was placed for 26.7% of the patients. Localizations of the stones were 27.9% lower pole, 9.5% middle pole, 6.4% upper pole, 35.7% pelvis, and 20.5% in multiple calyces. A total of 268 patients had a single stone and 216 patients had multiple stones. 64.3% of the patients had hydronephrosis prior to surgery. There was no statistically significant difference between the stone-free and non-stone-free groups in terms of age (p = 0.72), comorbidity (p = 0.68), length of hospital stay (p = 0.82), lateralization (p = 0.85), preoperative stent placement (p = 0.67), and the presence of hydronephrosis (p = 0.15) respectively. Among the two groups, statistically significant differences were determined between stone density (p = 0.004), stone length (p < 0.001), ESWL history (p = 0.016), number of stones (p < 0.001), localization (p < 0.001), stone volume (p < 0.001), T.O.HO.( p < 0.001), Ito’s scores (p < 0.001) respectively (Table 1). According to multivariate regression analysis, stone length (OR:1.247; Cl:1.190–1.308; p < 0.001), T.O.HO. score (OR: 2.274; Cl: 1.89–2.735; p < 0.001), Ito’s nomogram (OR: 0.777; Cl: 0. 739-0.817; p < 0.001), number of stones (OR:3.430; Cl:2.177–5.405; p < 0.001), and lower pole location (OR:0.684; Cl:0.579–0.808; p < 0.001) were independent risk factors affecting the success of fURS (Table 2). ROC curves were plotted for Ito’s and T.O.HO. scores to assess efficacy. The cut-off value for Ito’s nomogram was 13.5 points and the AUC was 0.792 (sensitivity 0.609, specificity 0.821) with a 95% confidence interval (0.752–0.832) (Fig. 1). The cut-off for the T.O.HO. score was 6.5 points, and the AUC was 0.744 (sensitivity 0.738, specificity 0.602) with a 95% confidence interval (0.699–0.789) (Fig. 2).

Table 1 Characteristics of the entire cohort and subgroups according to stone-free status

Table 2 Multivariate logistic regression analysis of potential independent risk factors for postoperative success rate

Fig. 1

ROC curves for predicting success of retrograde intrarenal surgery based on Ito score

Fig. 2

ROC curves for predicting success of retrograde intrarenal surgery based on T.O.HO. score

Discussion

Although scoring systems primarily rely on stone characteristics such as size, number, and location in the collecting system, they have different predictive values due to the variety of other parameters they include. In the RUSS and R.I.R.S. scoring systems, anatomical features such as infundibulopelvic angle (IPA) and infundibular length were also evaluated using urography imaging [6, 10]. In Polat, R.I.R.S., and Ito’s scoring systems, not only the diameter but also the area and volume were taken into account for the size of the stone, aiming to measure the stone burden more precisely [12]. Stone density significantly affects stone fragmentation, and this parameter is included in the R.I.R.S., T.O.HO., S.T.O.N.E., and Polat scoring systems [7, 10,11,12] The S-RESC scoring system, initially described for predicting SFR after percutaneous nephrolithotomy (PNL), was later adapted for fURS [8]. It is based only on the location of the stone in the collecting system, without considering parameters like stone size, number, density, and presence of hydronephrosis. As a result, it has been criticized for its limited predictive value.

The parameters in the study by Resorlu et al. were reported to be independent predictors for SFS, but the AUC for the RUSS score was not mentioned [6]. In a subsequent external validation study, it was reported to have an AUC value of 0.735 [13]. The RUSS scoring system has many limitations. Urography is required for IPA measurement, and the measurements require experience, leading to an obstacle in achieving a standard measurement. In addition, although Resorlu et al. stated that they also evaluated patients with renal anomalies, only 2 patients in their study group had renal malformations. Therefore, this study does not present sufficient data to make judgements about patients with renal anomalies. Nevertheless, it was found to be an independent predictor of SFS in validation studies.

The S.T.O.N.E. scoring system, which can be applied relatively easily, evaluates stone characteristics and the degree of hydronephrosis, and has limited reliability with an AUC value of 0.63. The validity of this scoring system is limited because of the retrospective design of the study, and the small proportion of patients with high scores in the study group [7]. However, the external validation study conducted by Selmi et al. described high predictive values for the S.T.O.N.E. scoring system in terms of SFS (AUC: 0.725) [14].

The S-RESC scoring system does not include the characteristics of the stone, except its localization in the collecting system. When defined by Jung et al., it was reported to have a high predictive value for SFS (AUC: 0.806) [8]. However, it does not evaluate any parameters such as stone size, number, density, and presence of hydronephrosis. If stones of different sizes and densities are localized in the same region of the kidney, the S-RESC system would still propose the same results, which is why it has been criticized in many studies. In external validation studies; while Polat et al. reported that it had no predictive value (AUC: 0.582) [12]. Selmi et al. stated that it had a high predictive value for SFS (AUC: 0.755). However, in the regression analysis of the same study, it was seen that the S-RESC scoring system was not an independent predictor of SFS [14].

The R.I.R.S. scoring system, which includes all features of the stone and the parameters related to the anatomical structure of the kidney, was defined by Xiao et al. in 2017. It provided high predictive values for SFS in the internal validation (AUC 0.828 for the 1st postoperative day, 0.904 for the 1st postoperative month) [10]. However, external validation studies could not provide similar results [15]. This may probably be related to a lack of experience or lack of standardized measurements of IPA, infundibular length, or stone density.

Recently, Polat et al. reported a new nomogram in their study, in which the surface area of the stone, the number of stones, the density of the stone, and the localization of the stone in the collecting system are identified as independent predictors of SFS. It differs from other scoring systems in only evaluating large stones between 2 and 4 cm. A high predictive value was reported for SFS (AUC: 0.802) [12]. However, in the same study, it was not determined by regression analysis whether the nomogram was an independent predictor of SFS. This study had some limitations, such as being a retrospective study conducted with a small number of patients, and the exclusion of patients with musculoskeletal abnormalities and complex stones. Most importantly, it is frustrating why fURS was performed on patients who would be candidates for PNL as first-line treatment according to the guidelines. Treatment of large kidney stones with fURS may result in a decrease in SFR and reoperation may be required. Its reliability in < 2 cm stones is unclear. Larger external validation studies may shed light on these questions.

In 2015, Ito et al. evaluated the data of 310 patients who underwent fURS and developed a nomogram utilizing parameters such as stone volume, presence of lower pole stone, surgeon’s experience, number of stones, and presence of hydronephrosis, which they found to be independent predictors for SFS. This scoring system, called Ito’s nomogram, was proposed to have a high predictive value, with an AUC value of 0.87. Higher scores result in an increased possibility of achieving SFS [9]. This nomogram is the only scoring system that takes surgeon’s experience into consideration as an independent predictor of SFS. Stone density was also examined in the study, but it was not included in the nomogram as it was not detected as an independent predictor. The limitation of this nomogram is that it does not evaluate renal malformations. It showed a low predictive value for SFS in the external validation study (AUC: 0.658), and the regression analysis showed that Ito’s nomogram was not an independent predictor of SFS [12]. The result of this external validation study could be confusing, as the study group consisted of patients with large stones (2–4 cm), who were less likely to achieve SFS.

T.O.HO. is a practical scoring system that consists of 3 parameters. Patients were evaluated based on stone length, location, and density. Although fewer parameters are evaluated, they provide high predictive values, with an AUC of 0.83 [11]. The T.O.HO. scoring system also showed a high predictive value for SFS in its external validation study (AUC: 0.758). In the same study, it was observed that a higher predictive value could be achieved with the modified T.O.HO. scoring system, obtained by adding stone volume to the parameters (AUC: 0.821) [16]. If similar success rates are determined with external validation studies, it seems promising in the future as a simple and practical scoring system.

This study aimed to externally validate Ito’s nomogram and T.O.HO. scoring systems, which are highly reliable and can be easily applied using preoperative CT. We demonstrated that both scores are good predictors of SFR after fURS with high accuracy rates (AUC: 0.792, AUC: 0.744, respectively). Additionally, in our study, by regression analysis, stone size, number of stones, lower pole localization, as well as Ito’s nomogram and the T.O.HO. scoring system, were found to be independent predictors of SFS.

As well as achieving SFS, success of the surgery also requires completion without complications or with an acceptable level of complications. The biggest shortcoming of these scores is that none of the scoring systems described in this article have examined their predictive strength for surgical complications. In R.I.R.S., the authors stated that the score correlated with the duration of surgery and could therefore be used to predict postponement of surgery to a second session and prevent complications [10]. In an external validation study by Bozkurt et al., RUSS, R.I.R.S., and Ito’s nomogram were able to accurately predict complications [17].

Almost all scoring systems have been reported to have a high predictive value in the studies in which they were defined. However, external validation studies show that this is not the case, and even meta-analyses show that there is a serious heterogeneity in the results. It is emphasized that this heterogeneity is due to the lack of standardized measurements or inconsistencies in the definition of surgical success. When the studies thought to cause heterogeneity were excluded, the S.T.O.N.E. scoring system was found to have the highest AUC value with 0.771 compared to S-RESC, R.I.R.S., and RUSS systems (0.709, 0.704, and 0.669, respectively). In the same meta-analysis, homogeneity regarding Ito’s nomogram could not be achieved by the exclusion of studies. In this meta-analysis, it was reported in the conclusion section that no scoring system was superior to other scoring systems in pairwise analysis [15].

An ideal scoring system is expected to cover all factors affecting success and complications, while at the same time being easily applicable. In our study, we demonstrated that Ito’s nomogram and T.O.HO. scoring system provides high accuracy rates with a reasonable number of parameters that can easily be evaluated. Although these scores provide a prediction for surgical success, the final surgical decision should be based on the patient’s condition and expectations.

Due to limited data in our study, the relationship between complications could not be evaluated. The limitations of the study include the retrospective collection of data and the use of X-ray for residual stone control. In other centers, the use of computed tomography for residual stone control is essential for validation of the study.

Conclusion

Based on the results of our external validation, Ito’s nomogram and T.O.HO. scoring systems can predict fURS success with high accuracy rates. All scoring systems in the literature are far from ideal because they do not have standardized criteria in terms of both definition and evaluation. There is a need to develop scoring systems with a wider cohort, which can predict complications as well as surgical success.