Common questions about shoulder arthroplasty (SA) searched online by patients and the quality of this content are unknown. The purpose of this study is to uncover questions SA patients search online and determine types and quality of webpages encountered.
The “People also ask” section of Google Search was queried to return 900 questions and associated webpages for general, anatomic, and reverse SA. Questions and webpages were categorized using the Rothwell classification of questions and assessed for quality using the
According to Rothwell classification, the composition of questions was fact (54.0%), value (24.7%), and policy (21.3%). The most common webpage categories were medical practice (24.6%), academic (23.2%), and medical information sites (14.4%). Journal articles represented 8.9% of results. The average JAMA score for all webpages was 1.69. Journals had the highest average JAMA score (3.91), while medical practice sites had the lowest (0.89). The most common question was, “How long does it take to recover from shoulder replacement?”
The most common questions SA patients ask online involve specific postoperative activities and the timeline of recovery. Most information is from low-quality, non-peer-reviewed websites, highlighting the need for improvement in online resources. By understanding the questions patients are asking online, surgeons can tailor preoperative education to common patient concerns and improve postoperative outcomes.
IV.
Prior to surgery, orthopedic surgeons routinely counsel their patients on the risks, benefits, expectations, and rehabilitation associated with any proposed procedure. This understanding is necessary for informed consent on the part of patients, and studies have shown that appropriately aligned patient expectations are positively correlated with higher postoperative satisfaction [
Orthopedic patients seek answers to their surgical questions from many sources other than doctors. Today, most orthopedic patients use the internet to ask medical questions [
The purpose of this study is to investigate what questions shoulder arthroplasty patients search online and to determine the types and quality of webpages provided to patients from the top results of each query. By understanding what questions patients are asking online regarding shoulder arthroplasty, surgeons can improve communication in preoperative education and prepare patients for their experiences and outcomes prior to shoulder arthroplasty. These steps will, in turn, increase patient satisfaction and improve clinical outcomes.
This study was conducted using publicly available, de-identified data to protect individual privacy, and as such, did not require Institutional Review Board approval or informed consent. Therefore, no additional ethical clearance was necessary for this research.
The methods of our study were adapted for shoulder arthroplasty from the work of Shen et al. [
Search queries were performed on internet browser Google Chrome, version 90.0.4430.212. Searches were performed independently on May 13, 2021, by authors (MCK and KDC), with the search location set to the United States. To avoid the bias of personalized search results influenced by prior search history, searches were run on a newly installed application with no prior queries. The following steps were performed according to instructions found on support.google.com to ensure no personalized search bias was present [
For each search query, the “People also ask” tab was expanded until approximately 130 suggested searches appeared on the page. Each suggested question was paired with a single hyperlink to a webpage. The suggested questions and webpage hyperlinks were collected using automated Google Chrome extension Scraper, version 1.7. Questions that were clearly unrelated to the topic of shoulder arthroplasty were removed from the dataset. The final dataset was ultimately limited to the first 100 relevant “People also ask” questions per search string.
Based on previous studies in internet health information, each question was categorized using the Rothwell classification into one of three themes—fact, policy, or value [
Each website hyperlink was visited, and the website information source was categorized as academic, commercial, government, journal, lawyer/law-related, medical information site, medical practice, non-medical media site, or single-surgeon personal. Descriptions and examples of classifications for each website can be found in
Each website was scored on a four-point scale for quality of source according to the established
Cohen’s kappa coefficient was used to evaluate interobserver reliability of question classification and website classification. Pearson’s chi-square tests and Student t-tests were used to evaluate the results for significance. Statistical significance was set to P-values <0.05.
Nine hundred questions (comprising 300 questions for each of the three categories of total shoulder arthroplasty—general, anatomic, and reverse) were generated with nine hundred associated webpages. Cohen’s kappa coefficient for interrater reliability was found to be 0.94 for question categorization and 0.98 for website categorization.
Most questions were categorized as fact questions by the Rothwell classification (54.0%), followed by value (24.7%) and policy (21.3%) questions (
Of the 900 total websites analyzed in this study, most were categorized as medical practice (24.6%), followed by academic (23.2%), medical information (14.4%), commercial (9.3%), journal (8.9%), single-surgeon personal (8.0%), government (6.2%), non-medical media (4.9%), and lawyer/law-related (0.4%) sites (
The mean JAMA score for all 900 webpages was 1.69. The websites with the highest mean JAMA scores were journal websites (mean, 3.91) in comparison to all other websites (P<0.001). The websites with the lowest mean JAMA scores were medical practice websites (mean, 0.89) in comparison to all other websites (P<0.001). Means for the remaining categories were as follows: academic=1.36, commercial=1.32, government=1.91, lawyer/law-related=1.25, medical information sites=2.65, non-medical media sites=1.93, and single-surgeon practice=1.07.
There was a significant relationship between search category (general, anatomic, or reverse) and Rothwell question classification (fact, policy, or value). Pertinent values were χ2=27.80, df=4, and P<0.001. Most questions categorized herein as policy questions by the Rothwell classification (n=192) were retrieved from searches in reverse total shoulder arthroplasty (n=94, 49.0%), in comparison to general (n=52, 27.1%) or anatomic (n=46, 24.0%) searches. There was also a significant relationship between search category and question subclassification with pertinent values of χ2=40.83, df=18, and P=0.002. Questions categorized as pertaining to indications/management (n=148) were more frequently related to reverse searches (n=74, 50%) than general (n=42, 28.4%) or anatomic (n=32, 21.6%) searches (
The results of this study highlight the questions most commonly asked online by patients undergoing shoulder arthroplasty, free from the biases present in traditional clinical surveys. The key findings of our study are (1) the most commonly asked question is, “How long does it take to recover from shoulder surgery?”, (2) by Rothwell classification, the most common questions are those classified as fact (54.0%), (3) the most common subcategories of questions are related to specific activities (16.9%), followed by questions about pain (15.3%), (4) the majority of answers to commonly asked questions are found on webpages hosted by medical practices (24.6%), academic institutions (23.2%), and medical information sites (14.4%), and (5) overall, the majority of webpages score poorly according to JAMA benchmark criteria for source and quality (1.69), with journals scoring the highest (3.91).
Knowing what questions shoulder arthroplasty patients are seeking answers to online is an important aspect of shoulder surgeons’ practices, allowing for improved counseling and education prior to surgery. Most questions asked by patients on Google were fact-based questions (54.0%), with the majority of these relating to specific activities and timeline of recovery. Overall, the most popular suggested question was, “How long does it take to recover from shoulder replacement?” This question, or slight iterations thereof, was the top suggested question in five of nine search queries and was in the top three questions of all nine strings of queries. The most common subcategorization of questions was specific activities, representing 16.9% of all questions. This finding highlights patients’ concerns about their ability to perform and continue performing specific activities postoperatively. The activities in question varied, with most being related to basic activities of daily living such as bathing and dressing, but also included more personalized and high-functioning activities such as bowling, golfing, and swimming. This illustrates the vast differences in patients’ preoperative goals. Generally speaking, while some patients are simply looking to perform activities of daily living without pain, others are hoping to return to higher-level functionality and recreational athletics. Physicians can answer a range of questions by providing patients with basic information regarding expectations for postoperative daily activities, as well as by asking each patient what activities are important to them to customize their preoperative education. It is critically important that surgeons determine which activities a patient wishes or expects to return to postoperatively in order to best educate patients and set realistic expectations for them. Numerous studies have shown that expectations regarding functional outcomes are correlated with reported postoperative outcomes [
Medical information sites scored highly on JAMA benchmark criteria because they nearly always listed authorship and date of publication. Medical practice, single-surgeon practice, and academic webpages often did not cite authors, references, or date of publication, and thus scored poorly according to JAMA benchmark criteria. All categories besides journal sites scored poorly for disclosure criteria. The most linked webpages were medical practice and academic pages, comprising 24.6% and 23.2% of total searches, respectively. When combined with single-surgeon practice sites (8.0%), most webpages were hosted by organizations employing orthopedic surgeons. However, each of these three categories scored poorly on the JAMA benchmark criteria, earning scores of 0.89, 1.36, and 1.07, respectively. These informational webpages were likely authored or reviewed and/or editorialized by surgeons. Nevertheless, these sites rarely listed authors, publication dates, disclosures, or references. This highlights the need for improvement in informational publishing practices from orthopedic sources.
We observed that 14.4% of webpages belonged to medical information sites such as verywellhealth.com and healthline.com. These webpages host topical articles written by healthcare journalists and undergo editorial review by physicians. Such medical information sites score very well according to JAMA benchmark criteria, with an average score of 2.65, insofar as they nearly always list authors, publication dates, and references. Non-medical media sites such as news articles, social media, and Wikipedia articles accounted for only 4.9% of all webpages returned by Google Search. As social media is generally regarded as a poor source of information, it is encouraging to see that social media websites are rarely provided as educational resources by the Google algorithm. Commercial and lawyer/law-related webpages only accounted for 9.3% and 0.4% of total webpages, respectively. While it is reassuring that most webpages visited by patients are hosted by organizations employing physicians, the quality of these published resources would benefit from increased academic rigor and source transparency.
The internet has been shown by numerous studies to be a source of poor quality health information for patients with specific needs [
Questions about reverse shoulder replacements were more likely to be policy questions and subcategorized as indications/management according to our findings. Patients want to know what makes them a candidate for reverse shoulder replacement, as well as when and why reverse shoulders are used instead of anatomic shoulder replacements.
A recent study published by Sudah et al. [
A limitation of this study is that it relies on Google-generated questions that people also ask when seeking online information about shoulder arthroplasty. These questions do not come directly from known orthopedic patients, and it is impossible to confirm who is searching for these questions. It is assumed these are questions posed by patients. It is also impossible to know whether these questions are being searched preoperatively or postoperatively. While there may be some uncertainty in who is asking these questions, there is a notable advantage in seeing what questions are being asked by patients in the reassuring context of online anonymity. The “People also ask” function generates results based on search trends within individuals’ regions of search. In this study, the United States was set as the search country, thus results are less applicable to nationalities and patient populations outside of the United States. Similarly, searches were conducted in 2021; results will change over time as patient populations and available information changes. Another limitation of this study is that JAMA benchmark criteria are a better measure of transparency and publishing practices than a measure of the accuracy of content. This is a known limitation of the criteria and was acknowledged in its original publication [
The most common questions asked online by SA patients are related to performing specific postoperative activities and the timeline of recovery. The majority of information is provided by low quality, non-peer-reviewed websites. This study highlights the need for improvement in online resources available to patients undergoing SA. By understanding which questions patients have and the quality of online information provided in response, surgeons can improve preoperative education and postoperative outcomes for patients of all types of shoulder arthroplasty.
Conceptualization: JRM, NM. Data curation: MCK, KDC. Formal Analysis: KDC. Methodology: JRM. Project administration: JRM. Writing – original draft: JRM, KDC. Writing – review & editing: JRM, NM, DD, NSH, GPN, NNV, GEG.
None.
None.
Contact the corresponding author for data availability.
None.
Rothwell classification for general, anatomic, and reverse shoulder arthroplasty. Questions for each of the three search groups as categorized by Rothwell classification: fact, policy, and value.
Subcategorization of questions for general, anatomic, and reverse shoulder arthroplasty. Breakdown of questions into subcategorizations based on topical content.
Breakdown of websites classified based on host webpages.
Rothwell classifications, subcategorizations, and examples
Question categorization | |
---|---|
Fact | Ask whether something is true, and to what extent |
e.g., Where is the incision for shoulder replacement? | |
Specific activities | Can you throw a ball after shoulder replacement? |
Timeline of recovery | How long does it take to recover from shoulder replacement? |
Technical details | What muscles are cut during a total shoulder replacement? |
Restrictions | What can you not do after shoulder replacement? |
Cost | How much does Medicare pay for shoulder replacement? |
Policy | Ask whether a certain course of action should be taken to solve a problem |
e.g., Is shoulder replacement an inpatient or outpatient procedure? | |
Indications/management | Are there alternatives to shoulder replacement? |
Risks/complications | What is the chance of dying in surgery? |
Value | Ask for evaluation of an idea, object, or event |
e.g., What is the success rate of shoulder replacement? | |
Pain | How painful is total shoulder replacement? |
Longevity | How long does a shoulder replacement last? |
Evaluation of surgery | Are shoulder replacements successful? |
Website categorizations and examples
Website categorization | |
---|---|
Academic | Webpage hosted by an academic institution or organization |
e.g., |
|
Commercial | Webpage hosted by a for-profit company |
e.g., |
|
Journal | Academic journal publication, may be hosted by third party site |
e.g., |
|
Government | Governmental hosted webpage |
e.g., |
|
Lawyer | Single attorney, law firm, or legal advice webpage |
e.g., |
|
Medical information site | Company or organization for the purpose of medical information reviewed by medical professionals |
e.g., |
|
Medical practice | Medical or surgical practice of physicians |
e.g., |
|
Non-medical media site | Webpages not specializing in medical information such as general news and social media sites |
e.g., |
|
Single surgeon practice | Single surgeon practice or personal webpage |
e.g., |
JAMA benchmark criteria
JAMA benchmark criteria | |
---|---|
Authorship | Clearly identifiable author and contributors with affiliations and relevant credentials present |
Attribution | References and sources clearly listed with any copyright information disclosed |
Currency | Clearly identifiable posting date of any content as well as date of any revisions |
Disclosure | Website ownership clearly disclosed along with any sponsorship, advertising, underwriting, and financial support |
Each criteria received 1 point for a maximum of 4 points.
JAMA:
Question results breakdown by classification and subclassification
Category | General | Anatomic | Reverse | No. (%) |
---|---|---|---|---|
Fact | 179 | 175 | 132 | 486 (54.0) |
Specific activities | 58 | 46 | 48 | 152 (16.9) |
Timeline of recovery | 41 | 46 | 33 | 120 (13.3) |
Restrictions | 29 | 29 | 22 | 80 (8.9) |
Technical details | 42 | 47 | 24 | 113 (12.6) |
Cost | 9 | 7 | 5 | 21 (2.3) |
Policy | 53 | 47 | 92 | 192 (21.3) |
Indications/management | 42 | 32 | 74 | 148 (16.4) |
Risks/complications | 11 | 15 | 18 | 44 (4.9) |
Value | 68 | 78 | 76 | 222 (24.7) |
Pain | 41 | 45 | 52 | 138 (15.3) |
Longevity | 1 | 2 | 4 | 7 (0.8) |
Evaluation of surgery | 26 | 31 | 20 | 77 (8.6) |
Total | 300 | 300 | 300 | 900 |
Website results breakdown by classification
Website classification | General | Anatomic | Reverse |
---|---|---|---|
Academic | 70 | 82 | 57 |
Commercial | 18 | 22 | 44 |
Government | 25 | 18 | 13 |
Journal | 27 | 27 | 26 |
Lawyer | 2 | 1 | 1 |
Medical info site | 46 | 42 | 42 |
Medical practice | 74 | 65 | 82 |
Nonmedical media site | 14 | 15 | 15 |
Single surgeon personal | 24 | 28 | 20 |