The Net Promoter Score with Friends and Family Test applied to arthroscopic shoulder surgery
Article information
Abstract
Background
The Friends and Family Test (FFT) developed by the UK National Health Service evaluates whether patients are satisfied with a service provided, where improvements are needed, and how likely patients are to recommend the intervention. Calculated from the FFT, the Net Promoter Score (NPS) creates a recommendation metric for treatment. The primary aim of this prospective study is to evaluate NPS for arthroscopic subacromial decompression (ASD) and rotator cuff repair (RCR). Secondary aims are to postoperatively evaluate 1-year changes in patients’ Oxford Shoulder Scores (OSSs) in terms of the proportion of patients satisfied with their surgery and correlation with FFT.
Methods
During a 2-year period, all patients undergoing ASD or RCR completed questionnaires prospectively. Collected preoperatively and postoperatively at 1 year.
Results
NPSs were 31 for ASD (n=32) and 52 for RCR (n=39). OSSs increased by 4.3 and 6.9 for ASD and RCR, respectively (P<0.001). Overall, 75% of ASD and 77% of RCR patients were either “satisfied” or “very satisfied,” respectively, with procedure outcomes. Scores from FFT had a positive correlation with improvement in OSS and satisfaction scores among patients undergoing arthroscopic shoulder surgeries (P<0.001).
Conclusions
The current study shows positive NPS outcomes in patients with ASD and RCR. Scores from FFT correlate well with both satisfaction and OSS among patients. NPS can be an adjunct to traditional patient-reported outcome measures to provide global evaluation of patient experiences to aid in determining the clinical value of common procedures in shoulder orthopaedics.
Level of evidence
III.
INTRODUCTION
The use of arthroscopy in shoulder surgery was first described by Andren and Lundberg [1] in 1965 for the management of adhesive capsulitis. The use of arthroscopy has grown in popularity over the decades, allowing our greater understanding of previously unseen shoulder pathologies, as well as novel treatment approaches for conditions such as rotator cuff disease, impingement, and instability [2]. In England alone, a 91% increase in arthroscopic subacromial decompression (ASD) was seen in the 10-year period between 2007 and 2017, with 28,802 procedures (52 per 100,000 population) performed annually [3].
Despite its increasing popularity, ASD in particular has come under scrutiny in recent times with randomized controlled trials showing no evidence of effectiveness in comparison to conservative treatment. The “Can Shoulder Arthroscopy Work?” trial by Beard et al. [4], conducted on a large scale in 2018, compared ASD to diagnostic arthroscopy alone and no treatment, only finding modest improvement in surgery groups at levels below the threshold for being clinically significant. Those clinical trial results showed no difference in ASD versus diagnostic arthroscopy alone. The Finnish Subacromial Impingement Arthroscopy Controlled Trial, comparing the outcomes of surgery to exercise therapy alone, reports similar findings among patients in Finland [5].
In 2018 the National Health Service (NHS) in England commissioned a consultation pertaining to procedures they deemed as being of low clinical value, one of which was ASD for shoulder pain [6]. Following investigation, though not advising an outright discontinuation of ASD, the healthcare system recommends that ASD should be conducted under specific circumstances only with the aim to maximize value in NHS spending and reduce avoidable harm to patients. Certainly these are admirable goals, but the rationale behind the addition of ASD to NHS scrutiny has been questioned. Many of the studies used to justify this position rely on patient-reported outcome measures (PROMs), which in and of themselves are not the optimal way to evaluate the clinical value of a procedure. PROMs are primarily influenced by pain and psychological factors, rather than overall patient experience or ability to complete tasks following recovery [7].
Other methods of quantifying the value of a procedure have been sought. The Net Promoter Score (NPS) is a variant of the Friends and Family Test (FFT) developed by the NHS in 2013 to help determine whether patients are happy with a service provided or whether improvements to services are needed. The work of Hamilton et al. [8] first used the FFT to investigate patient satisfaction in total joint arthroplasty, showing high levels of satisfaction comparable to those of successful commercial organizations. That research reports factors including patient age, sex, comorbidities, and Oxford Hip or Knee Scores accounting for 95% of the variation in NPS. The authors recommend as a minimum a relevant clinical outcome score and patient experience measure for evaluation of treatment. No studies currently exist investigating the relationship of Oxford Shoulder Score (OSS) and NPS or satisfaction in patients.
The primary aim of this prospective study is to evaluate the NPS of patients with ASD and rotator cuff repair (RCR). Secondary aims are to postoperatively evaluate 1-year changes in OSSs, correlation with NPS, and the proportion of patients satisfied with their surgery.
METHODS
This study was approved by institutional review board, and all patients undergoing elective shoulder surgery under the care of a single consultant shoulder surgeon at the study center consented to prospective recording of outcome data.
Study Setting and Design
This was a single-center prospective cohort study set in a district general hospital, which is the sole regional provider of shoulder surgery services to a population of 360,000 during a 2-year period (January 2017 to December 2018). Patients were identified through clinical coding. Inclusion criteria were all patients undergoing ASD or RCR, who had outcome data recorded preoperatively and postoperatively at outpatient clinics. Patients were seen at 6 weeks and 6 months routinely, with further follow-up at 1 year in cases of any ongoing symptoms. Patients filled out paper questionnaires postoperatively at 1 year via postal surveys, and scores were electronically recorded. Patients who did not respond to the postal survey had a telephone survey. Patients who underwent shoulder arthroscopy for an indication other than ASD or RCR (e.g., adhesive capsulitis, labral tear, or diagnostic purposes), as well as patients who underwent revision procedures and patients who did not complete the survey were excluded.
Indications for Surgery
All patients were referred for shoulder pain and diagnosed by a consultant shoulder surgeon. Patients underwent a course of dedicated rotator cuff physiotherapy with or without steroid injection. If patients remained in significant pain despite these treatments, they were offered surgery. In cases where the clinical cause for pain was unclear, magnetic resonance imaging scans were used to confirm diagnoses.
Outcome Measures
The NPS was calculated from the FFT. Patients were asked “How likely are you to recommend this operation to friends and family if they needed similar care or treatment” in postoperative questionnaires at 1 year following each initial surgery (FFT; Supplementary Material 1). Possible responses were “extremely unlikely,” “unlikely,” “neither likely nor unlikely,” “likely,” and “extremely likely.” A validated five-point scoring system was used, yielding scores of 1–5, respectively [9]. Results were interpreted as scores of 5 classifying respondents as promoters, scores of 4 classifying respondents as passive, and scores of 3, 2, and 1 classifying respondents as detractors. The NPS was simply calculated from the percentage of promoters minus the percentage of detractors, with values thus ranging from –100 to +100. This approach is consistent with previous application of the NPS in orthopedic surgery [8]. Patient satisfaction was measured with the following question, “How satisfied are you with your operated shoulder?” Possible responses included “very dissatisfied,” “dissatisfied,” “neither satisfied or dissatisfied,” “satisfied,” and “very satisfied,” yielding scores of 1–5, respectively. Tasks to measure patients’ OSS (Supplementary Material 2) [10] were performed preoperatively and postoperatively at 1 year. Changes in scores postoperation were calculated in each patient.
Statistical Analysis
Data were confirmed as normally distributed with Kolmogorov-Smirnov tests and values were presented as mean with standard deviation. Comparisons in patient characteristics were conducted with chi-square tests and in OSS with unpaired t-tests. Additionally, chi-square tests were used to compare the FFT proportions of detractors, passives, and promoters. Pearson correlation coefficients were calculated between individuals’ FFT and respective changes in OSS postoperation and satisfaction scores. All statistical analyses were performed using statistical software IBM SPSS ver. 23.2 (IBM Corp.).
RESULTS
During the study period, 84 arthroscopic procedures were performed for patients undergoing ASD and RCR. Twelve patients underwent arthroscopy for other reasons and were excluded from the study. Pre- and postoperative surveys were available for 71 patients, comprising 32 ASD patients (45%) and 39 RCR patients (55%). No outcome data were available for 13 patients who underwent ASD or RCR. Mean patient age was 55.6 years (standard deviation [SD], 13.2 years; range 18–78 years) and there was a significant difference in mean age at operation with ASD patients at a mean age of 47.7 years and RCR patients at a mean age of 61.5 years (P<0.001). There was no significant difference in gender between the two groups insofar as ASD patients were 63% female and RCR patients were 54% female (P=0.643). Follow-up was 1 year for all 71 patients who completed the survey.
Primary Outcome
Across both procedures the overall NPS was +43 (41 promoters [FFT scores of 5], 19 passives [FFT scores of 4], and 11 detractors [FFT scores of 1–3]). The NPS for RCR was +52 (24 promoters [FFT scores of 5], 11 passives [FFT scores of 4], and 4 detractors [FFT scores of 1–3]) in comparison to a NPS of +31 for ASD (17 promoters [FFT scores of 5], 8 passives [FFT scores of 4], and 7 detractors [FFT scores of 1–3]). A breakdown of proportions of promoters, detractors, and passives for preoperative variables, operation, and satisfaction scores is summarized in Table 1.
Secondary Outcomes
The mean preoperative OSS for ASD was 35.3 (SD, 8.3) and the mean postoperative OSS for ASD was 40.5 (SD, 11.1), with a mean change of +5.3 (SD, 10.3; P=0.007). For RCR, the mean preoperative OSS was 35.9 (SD, 8.2) and the mean postoperative OSS was 42.6 (SD, 12.3), with a difference of +6.9 (SD, 12.3; P<0.001).
Satisfaction responses are summarized in Table 2. Overall, 75% of ASD patients and 77% of RCR patients were either “satisfied” or “very satisfied.” In ASD patients, FFT scores had a strong positive correlation with OSS values (r(30)=0.81, P<0.001) and a moderately positive correlation with satisfaction scores (r(30)=0.58, P<0.001). In RCR patients, FFT scores also had a strong positive correlation with OSS values (r(38)=0.79, P<0.001) and a moderate positive correlation with satisfaction scores (r(38)=0.56, P<0.001).
DISCUSSION
The current study showed the NPS of patients undergoing ASD to be 31 and the NPS of patients undergoing RCR to be 52, with 75% and 77% of patients undergoing these respective procedures reporting satisfaction with their experience. Postoperative changes in OSS were +4.3 for ASD and +6.9 for RCR, both statistically significant increases. The findings confirm that FFT is strongly correlated with OSS and moderately correlated with satisfaction among patients undergoing arthroscopic shoulder surgeries. This is the first study to investigate correlations between FFT and NPS in shoulder surgery.
The work of Hamilton et al. [8] investigated NPS values of 6,186 total joint arthroplasty patients, revealing a total NPS for hip arthroplasty of 71 and a total NPS for knee arthroplasty of 49. That research found a 40% correlation between overall patient satisfaction and patients being promoters of specific procedures. This suggests that, for patients, factors such as meeting preoperative expectations of surgery, achieving pain relief following surgery, and hospital experiences as a whole may explain the remaining influence.
The work of Stirling et al. investigated NPS values in patients experiencing four common hand surgeries. Scores were 83 for trigger finger release, 68 for carpal tunnel decompression, 62 for Dupuytren’s disease release, and 44 for excision of ganglia [11]. That research finds that NPS correlates with functional outcomes, overall hospital experiences, and levels of patient satisfaction. The four procedures investigated have been described as procedures of low clinical value; nevertheless, their demonstrated high NPS can be used as a further measure of determining value, rather than relying on PROMs alone. In light of this existing research, the current study can be used to show that RCR and ASD are indeed also procedures of relative clinical value based on patient reports of favorable NPS, satisfaction, and functional outcomes.
A valid measuring tool related to PROMs, the Single Assessment Numeric Evaluation (SANE) rates patient opinions from 1 to 100 by asking, “How would you rate your shoulder today as a percentage of normal (0% to 100% scale with 100% being normal)?” [12]. This score has been validated against the American Shoulder and Elbow Surgeons score for general shoulder pathologies and (relating more specifically to Rowe scores for instability), SANE scores have been shown to be responsive in re-testing after procedures [13]. The single question response is quick and easy for patients to report. It can reduce barriers to reporting outcomes such as patient understanding, clinician assessment, scoring discrepancies, and interpretation. The SANE test has been validated for shoulder function, but any correlation to patient satisfaction levels or overall hospital experiences has not been investigated. While SANE can be a useful adjunct when determining the value of a procedure, the measure should be thought of more as a PROM than a global measure of patient experience.
In the UK, NHS designation as a “procedure of low clinical value” has been previously applied to complimentary or alternative treatments. Now, however, the term has been extended to describe some elective surgeries with the aim to save costs by restricting availability [14]. The concern is that expert clinicians are not being appropriately involved in these decision processes, leading to incorrect labelling of procedures for short-term financial gain [15]. As a result, there is a risk of accruing a backlog of unmet needs that will have to be confronted at some point in the future, together with a legacy of serious long-term health problems for patients [16]. Insofar as patient experience is positively associated with safety and clinical effectiveness [17], the NPS should be used as a partial measure in determining the value of certain procedures.
Strengths of this study include the prospective collection of outcome data and high rate of follow-up. In order to achieve these parameters, patients were told in clinic that they would receive a postal survey at 1-year following their procedure, and were instructed in the importance of completion. The postal surveys were sent with pre-paid return envelopes. Some limitations of this study are single surgeon/single center-based data. In addition, the small sample size for both patient cohorts and lack of patient demographic data to help predict patient-reported scores are further limitations.
The current study shows favorable NPS in patients undergoing ASD and RCR. Furthermore, a high rate of satisfaction is seen in patients experiencing these procedures, together with clinically significant improvements in OSS. Thus, FFT scores are shown to correlate well with both satisfaction and OSS patients undergoing arthroscopic shoulder surgeries. The NPS can be an adjunct to traditional PROMs to provide a global evaluation of patient experiences to aid in determining the clinical value of common procedures in shoulder orthopaedics.
Notes
Author contributions
Conceptualization: JM, SS. Data curation: JM. Formal Analysis: JM. Funding acquisition: Investigation: JM. Methodology: JM. Project administration: SS. Supervision: SS. Writing – original draft: JM. Writing – review & editing: JM, SS.
Conflict of interest
None.
Funding
None.
Data availability
Contact the corresponding author for data availability.
Acknowledgments
None.
SUPPLEMENTARY MATERIALS
Supplementary materials can be found via https://doi.org/10.5397/cise.2022.01116.