The postoperative morbidity index: a quantitative weighing of postoperative complications applied to urological procedures

The concept of outcomes measurement was first described by Dr. E. Codman in the early 1900’s and has now become a national incentive set forth by the Centers for Medicare and Medicaid Services and other organizations [1]. Although there are many well-designed programs currently in place to monitor the quality and outcomes of care in certain specialties or institutions, they tend to be complex and therefore unlikely to be effectively implemented at a national or global level [2]. One of the major outcomes to report in any surgical field is post-operative complications. The Clavien complication grading system is a severity grading system developed by Clavien et al., and published in 1992. This complication grading system ranks complications based on the magnitude of the intervention(s) required for their treatment and whether the complications cause permanent injury or death [3]. In 2004, the Clavien complication grading system was modified to add more detail to the more serious complications, however, with it has come inconsistencies in the application of this grading system, e.g., non-uniform grade contraction [4].

As a result of these inconsistencies, the grading system was extensively modified (renamed Accordion Severity Grading System) in 2009. Specifically, the Accordion Severity Grading System added flexibility to the grading system by introducing an expandable classification, and clarity was improved by introducing rigorously defined qualitative terms [5]. However, to date, all complication severity grading systems, e.g., Memorial Sloan Kettering Severity Grading System and Accordion Severity Grading System [6]. are key short-term outcomes measures of operative procedures and lack robust quantitative measure of the severity of surgical complications, which would allow comparison between two health states.

Recently, severity weighting of the Accordion Severity Grading System led to a) correction of criteria for two of the higher grades of severity and b) severity scores with weight for each of the six severity levels, enabling for the first time a way to quantify complications. Specifically, the application of severity weights to Accordion Severity Grading System (postoperative morbidity index, PMI) was applied to data gathered from American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) program for the year 2007 and demonstrated that weighting of complications provided new insights into the burden contributed by specific types of complications [7, 8].

Straberg et al. evaluated and reported the feasibility of PMI, which applied to general surgery estimates quantitative morbidity scores after surgical procedures including laparoscopic colectomy, appendectomy, and pancreaticoduodenectomy, by taking into consideration validated weighted values in post-operative complications [9]. Based on these encouraging results from our general surgery colleagues, we report on the feasibility of applying Accordion Severity Grading System and PMI in a large urologic cohort (n = 654) that underwent 11 common urologic procedures. To date the expansion of PMI as an estimation of post-operative complication of urologic procedures has not been explored.

This study was approved by Orlando Health Inc. (Orlando, FL) Institutional Review Board with a waiver of consent. Orlando Health Inc. is a large healthcare system (> 1,000 beds) comprised of eight facilities in central Florida, affiliated with the University of Central Florida College of Medicine and Florida State University School of Medicine. Complications identified within the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) within the Department of Urology in Orlando Health Inc. were queried retrospectively to gather information regarding patient outcomes to urologic surgeries performed from January 1, 2011 to December 31, 2011. Based on the number of procedures performed in 2011, the location of the procedures (in-patient and outpatient) and the difficulty of the procedures, CPT codes associated with 11 diverse procedures covering a wide range of urologic procedures [e.g., insertion or replacement of inflatable penile prosthesis, nephroureterectomy, partial nephrectomy, percutaneous nephrostomy tube placement, radical cystectomy, radical prostatectomy, renal/ureteral/bladder extracorporeal shockwave lithotripsy (ESWL), transurethral destruction of bladder lesion, transurethral prostatectomy, transurethral removal of ureteral obstruction, and ureteral catheterization] were queried and included for analysis. All patients identified in each of the 11 procedures were evaluated (i.e., no patient was excluded from analysis). To establish the true PMI of a procedure in an institution one would expect that >25 patients per group would be needed although the study has not yet been done to determine the exact number.

Next, a weighted postoperative morbidity index (PMI) was calculated as previously described [8] (i.e., to calculate the PMI for each operative procedure, the weights of all the complications for all patients who underwent the corresponding procedure were summed and divided by the total number of patients undergoing that procedure). A PMI of 0 would indicate that no patient having the procedure had any postoperative complications, while on the other hand, and a PMI of 1.000 would indicate that every patient having the procedure suffered postoperative death. In order to analyze complication severity, the sum of severity weights for all patients having any complication after a procedure were divided by the total number of patients with complications in the group (i.e., the denominator was the number of patients having a complication after the procedure, rather than the total number of patients having the procedure). Descriptive statistics were performed in Excel 2007 (Microsoft Corp).

The volume of surgery over the past several decades has increased dramatically in all parts of the world, with an estimated 234 million operations performed annually, making safe delivery of surgical care a major public health concern [10]. From the early 1900’s, it has been a tenet of the surgical profession that the careful tracking and analysis of outcomes is essential to provide safe, high-quality care [1]. A simple, low-cost metric assessing post-operative complications, capable of providing rapid feedback to the surgical teams in any setting could therefore aid clinical care and quality improvement efforts.

The concept of severity weighting used to calculate the PMI is derived from utility weighting, which is the mathematical method of assigning value weights to multidimensional outcomes states to reflect overall impact [9]. The value of severity weights used in the current study comes from a well-validated study where 50 surgical experts were asked to evaluate and score 12 clinical vignettes [11]. Thus the PMI is an index, which might be most useful in detecting trends and serve as a point of reference in the surgical field. Considering this, the PMI numbers generated in the present study should be only taken as a starting point. For example, as shown in Additional file 1: Figure S1A, the PMI could be used to follow trends in complications for any particular urological procedure at the institutional level over months or years. As demonstrated in Table 3, when using the PMI, we are no longer simply analyzing the incidence rate of complications for a particular procedure, but we are also estimating the severity score of these respective complications. Furthermore, we can analyze the expected and actual severity grades of each complication that occurs. For instance, the post-operative complication rate of a transurethral prostatectomy was 35.63%, which could be further sub-classified into severity grades and compared to the complication grades with other procedures. This can be a valuable tool in standardizing practice or research, but also can be useful in properly counseling patients of their surgical risks. For example, one can advise that the severity of complications after a transurethral prostatectomy are approximately five times higher than the severity of a complication following a ureteral catheterization (PMI 0.10 vs. PMI 0.02, respectively).

It is evident that radical cystectomy at our institution had the most frequent complication rate and that most of the morbidity related to radical cystectomy was due to Grade 4 and Grade 5 complications, as shown in Additional file 1: Figure S1A. This high complication rate was noted by other investigators [12–14]. Specifically, DeNuzio and colleagues reported 415 complications in 302 patients undergoing cystectomy and classified these complications as Clavien type I (109 patients), II (220 patients), IIIa (45 patients), IIIb (22 patients), IV (11 patients) and V (8 patients) [14]. Furthermore, ureteral catheterization and ESWL had similar PMIs, but closer analysis shows that the burden of complications of ESWL came from Grade 4 complications, making it more severe than the burden from ureteral catheterization. Another important application of the PMI is to detect trends in deterioration or improvement in surgical outcomes, particularly after the institution of corrective measures or protocols. When analyzing this data and comparing with the curves in Additional file 1: Figure S1A, which displays the percentages of complications using the unweighted severity grades by procedure, similar PMI scores show completely different severity grade distributions. For example, when comparing nephroureterectomy and transurethral prostatectomy, both procedures have similar PMI scores (0.104 vs. 0.107), however transurethral prostatectomy has a greater number of grade 1 complications. In contrast, comparison of the curves in Additional file 2: Figure S1B, which shows the burden of weighted severity grades by procedure group, we can observe that most of the severity score for the transurethral prostatectomy is derived from grade 4 complications rather than from grade 1 as observed in the unweighted plot. In our series, we realized that almost all grade 4 complications in the transurethral prostatectomy were secondary to the need of a further resection of the prostate (data not shown). It would then be possible to further examine the indications for reoperation in our case series. Furthermore, this information can be used to compare surgeons, hospitals and procedures as well as initiate studies in order to determine the causes of such complications in that particular procedure. This technique of assessment then quality improvement could ultimately enhance the level of care provided to the urologic patient.

One limitation of this method is that it still lacks absolute objectivity in rating complications. For instance, there is no way to factor in whether the high reoperation rate after transurethral prostatectomy was due to the natural disease process of prostatic hypertrophy or to inadequate gland resection. Furthermore, we applied the severity weights of these ACS NSQIP 30-day morbidity derived from our general surgery colleagues. We believe these weights should be transferrable to the urologic patient, seeing that they are based on general medical/surgical tenets, but a follow-up study will validate these findings with urologic surgeons. Next, the patient population may have a higher comorbidity or some other factor that predisposes them to have a higher post-operative risk profile. These are important factors that must be factored into such a comprehensive system, however they are beyond the scope of this project. Thus, the technique utilized by our study of calculating PMI scores has limitations that must be considered before it can be applied in any institution or in any clinical situation. In concordance with Strasberg et al., we believe that a) the simplicity of the PMI makes it an easy tool to implement but at the same time, a tool lacking the ability to perform individual risk adjustment, b) the fact that only the most serious complication in each patient is considered in order to calculate the PMI may tend to lose certain information when a patient presents with multiple complications, c) the PMI might be less useful for detecting differences across urological care providers at any point in time and d) the application of the PMI should be adequate for the majority of procedures, except for that ones with unusual complication, for example in the area of transplantation where the death of a living donor must receive a special severity weight.

Based on the above results, quantitative severity weighing of post-operative complications of urologic procedures is feasible and may provide exceptionally informative data related to outcomes. As our national healthcare system continues to search for uniform yet applicable ways to measure and report quality care of the surgical patient, attention should be given to PMI.