Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 03.11.20 in Vol 9, No 4 (2020): Oct-Dec

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/23519, first published Aug 17, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Carotid Endarterectomy Versus Carotid Artery Stenting: Survey of the Quality, Readability, and Treatment Preference of Carotid Artery Disease Websites

    1Division of Vascular Surgery, University of Ottawa, Ottawa, ON, Canada

    2Division of Vascular Surgery, Queen's University, Kingston, ON, Canada

    Corresponding Author:

    Michael Yacob, MD, FRCSC

    Division of Vascular Surgery

    Queen's University

    76 Stuart St, Victory 3

    Kingston, ON, K7L 2V7

    Canada

    Phone: 1 6135482573

    Email: michaelyacob@gmail.com


    ABSTRACT

    Background: The internet is becoming increasingly more important in the new era of patient self-education. Carotid endarterectomy (CEA) and carotid artery stenting (CAS) are recognized interventions to treat patients with carotid artery stenosis. Using the Google search platform, patients encounter many websites with conflicting information, which are sometimes difficult to understand. This lack of accessibility creates uncertainty or bias toward interventions for carotid artery disease. The quality, readability, and treatment preference of carotid artery disease (CAD) websites have not yet been evaluated.

    Objective: This study aimed to explore the quality, readability, and treatment preference of CAD websites.

    Methods: We searched Google Canada for 10 CAD-related keywords. Returned links were assessed for publication date, medical specialty and industry affiliation, presence of randomized controlled trial data, differentiation by symptomatic status, and favored treatment. Website quality and readability were rated by the DISCERN instrument and Gunning Fog Index.

    Results: We identified 54 unique sites: 18 (33.3%) by medical societies or individual physicians, 11 (20.4%) by government organizations, 9 (16.7%) by laypersons, and 1 (1.9%) that was industry-sponsored. Of these sites, 26 (48.1%) distinguished symptomatic from asymptomatic CAD. A majority of sites overall (57.4%) and vascular-affiliated (72.7%) favored CEA. In contrast, radiology- and cardiology-affiliated sites demonstrated the highest proportion of sites favoring CAS, though they were equally likely to favor CEA. A large proportion (21/54, 38.9%) of sites received poor quality ratings (total DISCERN score <48), and the majority (41/54, 75.9%) required a reading level greater than a high school senior.

    Conclusions: CAD websites are often produced by government organizations, medical societies, or physicians, especially vascular surgeons. Sites ranged in quality, readability, and differentiation by symptomatic status. Google searches of CAD-related terms are more likely to yield sites favoring CEA. Future research should determine the extent of website influence on CAD patients’ treatment decisions.

    Interact J Med Res 2020;9(4):e23519

    doi:10.2196/23519

    KEYWORDS



    Introduction

    The internet is a popular source of information for Canadians seeking medical advice. According to Statistics Canada, Internet User Surveys revealed that 91.3% of Canadians used the internet [1], and 69.9% of home internet users searched for health information online [2]. With this trend toward health information acquisition online, carotid artery disease (CAD) poses a particular challenge for medical websites because there is no consensus on its optimal management [3,4]. Mixed interpretation of data, rapid evolution in technology and expertise, and pharmacotherapy improvements have led to inconsistent treatment guidelines and practice patterns across specialties and organizations [5]. Consequently, CAD patients searching for ways to treat their condition online may struggle to find a clear answer.

    The Carotid Revascularization Endarterectomy versus Stenting Trial (CREST) aimed to settle the carotid endarterectomy (CEA) versus carotid artery stenting (CAS) debate by eliminating confounding factors present in earlier CAD clinical trials. Strict CAS operator training requirements were implemented, standardized embolic protection devices were employed, and cardiac enzymes, electrocardiogram changes, and clinical presentations were routinely monitored. With its publication in 2010, CREST revealed no significant difference between CAS and CEA up to four years for the composite primary outcome: periprocedural stroke, myocardial infarction (MI), or death and subsequent ipsilateral stroke. The CEA group did demonstrate higher rates of MI and the CAS group higher rates of stroke. [6]. Ten-year CREST results published in March of 2016 yielded similar results that were sustained when the endpoints were stratified by symptomatic status, age, sex, or degree of stenosis [7]. Translation of CREST trial results into clinical practice was explored in a recent study by Otite et al [8]. Interestingly, the utilization of CAS increased post-CREST (2011-2014) compared with pre-CREST (2007-2010) for patients >70 years of age in the United States. Similarly, Hussain et al investigated the effects of clinical trial publications, including the CREST trial, on rates of carotid revascularization procedures in Ontario, Canada, between 2002-2014 [9]. In this period, CEA utilization decreased by 36%, while CAS increased by 72%. The CREST trial and the publication of other conflicting trials between 2006-2010 were associated with a decline in CEA rates [9]. The contradiction in clinical practice and trial results may be attributed to an interplay of multiple factors including, but not restricted to, differential interpretation of trial results, advances in CAS technology, availability and accessibility of physician providers and physician specialty.

    CEA has traditionally been the treatment of choice for patients with severe and/or symptomatic carotid stenosis. Nevertheless, with continued advances in best medical therapy (BMT) and the recent equivalent long-term CREST results, BMT and CAS challenge CEA as primary treatment modalities [10]. Variation in patient anatomy, age, comorbidities, surgical risks, use of embolization protection devices, and operator experience further complicate treatment decisions for individual patients. Since many patients today look to the internet for medical advice, we sought to identify the most easily accessible websites to patients, evaluate their quality, readability, and balance of information, and determine whether there was a preference for one treatment option.


    Methods

    Site Selection

    To generate a list of easily accessible CAD websites, we searched ten keywords commonly associated with CAD, such as “carotid stenosis,” “carotid stenting,” and “carotid endarterectomy” in Google Canada (Multimedia Appendix 1). Google searches yield approximately 10 unique links per page; the top ten websites returned by each search were recorded for a total of 100 websites. Of this sample, websites were excluded from content evaluation if they were repeats from a previous search. Repeat links were recorded to track how frequently a particular website appeared in similar searches. Of all the websites consulted, none was found to be a broken link, lack information regarding carotid disease, require a paid subscription, or be inaccessible.

    Demographic Information

    Demographic information collected from each site included (1) type of organization that created the website (specifically: physician/hospital/medical department, government organization, industry, layperson, other), (2) specialty affiliation of the website or its authors, (3) year of publication, (4) inclusion of randomized control data, and (5) differentiation between symptomatic and asymptomatic carotid disease for treatment preference.

    Assessment Tools

    Gunning Fog Index

    Each site was evaluated for readability using the Gunning Fog Index (GFI). The GFI estimates the years of formal education required to understand a passage based on the passage’s average sentence length and the number of complex words. For ease of use and avoidance of human error, an online calculator was used to calculate the GFI for each website [11]. If a website was divided into various webpages, we evaluated the summary overview webpage or the webpage that encompassed the bulk of the information contained in that site. Texts with near-universal understanding generally require a GFI score <8, indicative of an eighth-grade reading level; a score >12, equivalent to a high school senior reading level, is considered too difficult for the general populous [12].

    DISCERN Instrument

    Two researchers independently evaluated each site (SS, MY) for reliability and quality using the DISCERN instrument [13]. The DISCERN instrument consists of three sections totaling 16 questions, each with a rating scale from 1 (low, with serious or extensive shortcomings) to 5 (high, with minimal shortcomings). Section one evaluates each publication’s reliability based on various factors, including relevance, sources, and balance. Section two explores the quality of information concerning treatment options available. Section three consists of an overall rating of the publication based on sections one and two. Websites that received total DISCERN scores <48 were considered poor quality, as 48 represents an average score of <3 across each subsection.

    We deviated from the company’s instructions for section two (quality of information) in our use of the DISCERN instrument. Rather than evaluate each site according to how it addressed a single treatment choice, we evaluated each website according to how it addressed all three major treatment modalities for carotid stenosis: BMT, CEA, and CAS. Thus, if a website only addressed two out of the three main treatment options, that website automatically lost a minimum of one point for each question concerning the description, benefits, risks, and impact of the treatments. This method provided more consistency when comparing a site’s ability to honestly and thoroughly inform patients about all treatment options available for carotid disease.

    Site Preference

    An overall impression of preferred treatment was recorded for each website. Websites were considered to prefer CEA if they (1) stated that CEA was the standard of care; (2) started the discussion with CEA; (3) devoted far lengthier text to CEA without necessarily declaring that it was better; (4) and/or presented CAS as an alternative treatment intended for special circumstances only. These sites often described CEA as “older and effective,” “very safe,” “traditional,” and “durable.” Websites were considered to favor CEA and CAS equally if they devoted equal amounts of text to each treatment option and/or did not imply that one treatment was preferable to the other. Websites were considered to prefer CAS if they emphasized CAS as a newer, promising, less invasive option with a shorter hospital stay or devoted far lengthier text to CAS without necessarily declaring that it was better. Treatment preference was recorded as not applicable in websites that focused on transient ischemic attacks (TIAs) and strokes of various etiologies since these did not deal exclusively with carotid disease. For websites that recommended different therapies based on symptomatic status, preference was determined based on the recommendation for the symptomatic patient of average surgical risk.

    Statistical Analysis

    Statistical analysis was performed in Microsoft Excel. Interrater reliability for the DISCERN ratings was assessed using the Spearman correlation coefficient. Total DISCERN scores were averaged between the two evaluators. Differences between average total DISCERN and GFI scores for sites that preferred CEA, sites that preferred CAS, and sites that presented CEA and CAS equally were calculated using analysis of variance testing. Chi-square test of independence or fisher’s exact test as appropriate, was performed using an online calculator to determine whether there was a relationship between higher DISCERN scores and the presence of randomized controlled trial (RCT) data among the websites assessed. Spearman correlation coefficient was also employed to determine whether there was a correlation between DISCERN and GFI scores.


    Results

    Site Demographics

    A total of 54 unique CAD websites were identified using the search terms. Among these, 18 (33.3%) were produced by medical societies or individual physicians, 11 (20.4%) were produced by government organizations, 9 (16.7%) were produced by laypersons, and 1 (1.9%) was industry-sponsored (Table 1). Of the websites affiliated with or authored by a particular specialty/specialist, the three most common affiliations were vascular surgery (11 sites), neurology (7 sites), and internal medicine (6 sites) (Table 1). Of note, sites with multiple authors from different specialties were tallied multiple times in this category for each additional specialty represented among the authorship. We found that 44 (81.5%) websites were published after CREST, and 18 (33.3%) mentioned RCT data. Symptomatic was distinguished from asymptomatic disease on 26 sites (48.1%), 14 sites (25.9%) did not distinguish disease types, and 14 sites (25.9%) were excluded from this category because they were symptomatic stroke sites addressing various stroke etiologies (Table 1).

    Table 1. Website demographics.
    View this table

    DISCERN and GFI Data

    DISCERN scores from researchers SS and MY demonstrated strong interrater reliability (Spearman ρ=0.98). When averaged between the two researchers, DISCERN scores for all sites ranged from 28.5 to 76 out of a possible 80 points. A total of 21/54 websites (38.9%) received a poor-quality rating (total DISCERN score <48). There was no significant difference between average total DISCERN scores for sites that preferred CEA, sites that preferred CAS, and sites that presented CEA and CAS equally (P=.85). Sites with the ten highest DISCERN scores were more likely to contain RCT data than the remaining sites (P=.012). Specialty affiliation among sites with the top 10 DISCERN scores included neurology/neurosurgery (3 sites), vascular surgery (2 sites), and cardiology (2 sites). The remaining three sites did not state affiliation with any specialty, and two were Wikipedia pages.

    GFI readability scores ranged from 7.7 to 29.1, and 13 websites received a GFI score of <12, corresponding with a reading level at or below that of a high school senior. There were no statistically significant differences between average GFI scores for sites that preferred CEA, sites that preferred CAS, and sites that presented CEA and CAS equally (P=.99). GFI and DISCERN scores demonstrated a weakly positive correlation (Spearman ρ=0.34), indicating that websites containing a higher quality of information do not necessarily require a higher reading level.

    Treatment Preference

    Overall, most websites (31/54, 57%) demonstrated a preference for CEA, 8/54 (15%) presented CEA and CAS as equal treatment modalities, and 8/54 (15%) demonstrated a preference for CAS. Treatment preference was considered not applicable (N/A) in 7/54 (13%) of sites due to focus on stroke and TIA of various etiology rather than carotid disease alone. While recommended by most sites in conjunction with either CEA or CAS, best medical therapy was not cited as the best treatment modality alone in cases of average surgical risk with sufficiently severe carotid stenosis to warrant intervention. Among the ten sites with the highest DISCERN scores, CEA was preferred in six, CEA and CAS were presented equally in two, and CAS was preferred in two.

    Vascular surgery was the most common specialty affiliation, with 72.7% of vascular-affiliated sites favoring CEA, 9.1% presenting CEA and CAS equally, and 18.2% favoring CAS. Likewise, sites affiliated with neurology, neurosurgery, internal medicine, and family medicine more often demonstrated a preference for CEA (ranging from 50%-100% of websites affiliated with that specialty) than for CAS or no preference (Table 2). Websites affiliated with interventional radiology or cardiology demonstrated the highest proportion of sites favoring CAS, though they were equally likely to favor CEA (Table 2).

    Table 2. Website treatment preferences by medical specialty of the websites’ authors.
    View this table

    Among the 10 keywords searched, 7 yielded a majority of sites that favored CEA. Only one keyword—“carotid stenting”—yielded a majority of sites that favored CAS (Table 3). Of all 8 sites that demonstrated a preference for CAS, 7 appeared in the “carotid stenting” search, 1 appeared in the “carotid artery stenosis” search, and 1 appeared in the “mini stroke” search. The remaining 7 keywords yielded no sites that favored CAS. Two keywords—“TIA” and “mini stroke”—accounted for all the sites where treatment preference was deemed not applicable. Every keyword searched generated at least one site that presented CEA and CAS as equal treatment modalities (Table 3). Analysis of Google search trends dating back to 2009 revealed that “carotid stenting” was searched less frequently than “carotid endarterectomy” and “carotid surgery” (Figure 1).

    Table 3. Website treatment preference by keyword search.
    View this table
    Figure 1. Google search trends by keyword since 2009.
    View this figure

    Discussion

    Principal Findings

    High rates of reported information-seeking and use of web-based health technology places an onus on health care providers and educators to capitalize on these resources for disease education and management. Healthcare websites must be accessible, have high usability and reliability, and accommodate the average reading level of American adults, which is reportedly between the seventh- and eighth-grade [14,15]. To our knowledge, this is the first study to explore the quality, readability, and treatment preference of CAD websites. We found that CAD websites were often produced by medical societies or physicians (18/54, 33.3%) and government organizations (11/54, 20.4%). Websites ranged in quality and readability, and higher quality CAD websites did not necessarily require higher user reading levels. Treatment preference varied as a function of physician specialty, with vascular surgery-affiliated websites favoring CEA and interventional radiology and cardiology-affiliated websites favoring CAS.

    Consistent with current literature and guidelines, the majority of CAD websites demonstrated a preference for CEA. Abbott et al conducted a systematic review of guideline recommendations for the management of asymptomatic and symptomatic CAD published between 2008 and 2015. Of 28 guidelines with asymptomatic and 33 guidelines with symptomatic CAD procedural recommendations, 24 (86%) and 31 (94%) endorsed CEA for patients with average surgical risk. For symptomatic patients deemed higher surgical risk (due to comorbidities, unfavorable carotid anatomy, etc), CAS was endorsed by 27 guidelines (82%) [5]. Recently, Brott et al conducted a pooled analysis of individual patient-level data acquired from the four largest RCTs completed to date, assessing the relative efficacy of CAS versus CEA for treatment of symptomatic carotid stenosis. They showed that long-term outcomes continue to favor CEA. However, improvements in the periprocedural safety of CAS could provide similar outcomes of the two procedures in the future [16]. The majority of CAD websites reflect the treatment preference of medical practitioners for patients at average surgical risk.

    The majority of RCTs investigating CEA versus CAS found significant differences in perioperative outcomes, largely in symptomatic patients [17]. In our study, 48.1% (26/54) of CAD websites distinguished between symptomatic and asymptomatic CAD. Complication risks associated with CAE and CAS are higher in symptomatic than in asymptomatic patients [4,18]. Researchers have speculated that symptomatic carotid disease is associated with greater overall cardiovascular risk. Studies have also shown that annual stroke risk is lower for asymptomatic patients than symptomatic patients [18]. The benefit of CEA is greater among symptomatic compared [19-23] and remains the gold standard for this patient population [24,25]. Currently, clinical equipoise remains regarding the optimal management of asymptomatic CAD, as evidenced by the recent review article by Abbot et al and the response by Cambria et al, with the former advocating for optimal medical intervention as routine practice and the latter defending the use of mechanical intervention for asymptomatic patients [3,4]. Differentiation by symptomatic status is crucial in determining disease management, procedural risk, and subsequent treatment preference.

    Treatment preference for CAD websites varied as a function of physician specialty. Variation in patient treatment preferences is largely physician-driven, as the patients often depend on their physicians to prescribe appropriate treatment [26]. Provider enthusiasm for treatment recommendations may be driven by several factors, including availability, accessibility and operator experience, sociodemographic factors, and provider specialty.

    Wallaert et al examined the relationship between physician specialty and annual rates of CAS and CEA using Medicare claims from 2002 to 2010 [26]. Cardiologists performed the majority of CAS procedures, and regions with the highest proportion of cardiologists performed the most CAS procedures. Cardiologists and interventionalists have led efforts to extend CAS funding, while surgeons and neurologists have cautioned against expanding CAS approval outside of clinical registries and trials [26]. These findings are consistent with our findings that vascular surgery-affiliated websites favor CEA, and interventional radiology and cardiology-affiliated websites are more likely to favor CAS by comparison. Interestingly, a study by Keogh et al found that the number of available online CAS-related peer-reviewed sources is double the number of hospital- or health service-generated resources; the opposite is true for CEA [27]. Hospital and health service resources lend themselves to patient populations, which is reflected in the observation of higher readability of CEA than CAS resources [27]. The source of information—physician specialty, peer-reviewed sources, hospitals, or health services—influence website treatment preference in addition to readability based on the resource’s intended audience.

    It is recommended that patient health materials be written at or below the fourth to sixth-grade reading level by the American Medical Association (AMA), National Institutes of Health, and the Centers for Disease Control and Prevention [28]. If written below the sixth-grade level, material is considered easy to read; if written between the seventh and ninth grade levels, it is of average difficulty; and if written above the ninth grade level, it is difficult to read [14]. A minority of CAD websites included in our study (13/54, 24%) received a GFI score of less than twelve, corresponding with a reading level at or below that of a high school senior. A recent study also found that 99.5% of online cardiovascular disease-related health education materials recommended by the AMA were written above the fifth to sixth-grade level [29]. Our study found that the CAD websites’ readability was higher than recommended, which could have far-reaching implications for patients’ health literacy [29]. However, it is important to note that readability is only one element of literacy. The GFI may not reflect the reading level as it relies on the number of syllables in a word and the number of words in a sentence. Overall readability may be influenced by images, layout, design, and content organization [30]. As the internet is a growing resource for health information, it is critical to ensure web-based health resources are written at a level accessible to the general patient population.

    Additional limitations of this study must be taken into account. The literacy level of carotid stenosis patients may differ from that of the average American, and web-based resources written at a higher readability level may or may not be appropriate for this subgroup. While the DISCERN tool has demonstrated validity and reliability for evaluating the quality of online health information for treatment choices across conditions, there is subjectivity involved for certain rating criteria, which introduces interstudy variability when comparing studies and interrater variability within studies. However, our study demonstrated strong interrater reliability (Spearman ρ=0.98). Also, this study looked at static web-based delivery of health education. There is speculation that an interactive health education delivery approach, compared with a static one, may allow health material to be tailored to readers of a wide breadth of educational backgrounds. The literature has shown that adults with chronic illnesses have associated online health information use with behavior changes and decision-making [31]. Future research should assess how interactive web-based technologies, such as blogs and social networking sites, compared to websites, affect patient-provider communication.

    This study did not assess the usability and social reach of CAD websites. Future studies should evaluate how websites are engaging audiences. Many methods can be used to do so, including but not limited to (1) the LIDA online app to assess the usability of healthcare websites [27]; (2) global estimated website traffic over 30 days and over 3 months; and (3) counts of social bookmarking/networking links [31]. It is well known that patient preference for participation in health care varies greatly. Future research would also benefit from evaluating CAD patients’ decision-making preferences and preferences for online information regarding treatment options. The Health Information Wants Questionnaire collects data on the information and associated decision-making autonomy patients want in seven areas of health care [32].

    Conclusion

    CAD websites were most often affiliated with, or authored by, vascular surgeons, and CAD-related Google search terms were more likely to yield sites favoring CEA. Sites ranged in quality, readability, and differentiation by symptomatic status. Further research is needed to determine if website treatment preferences consistently and appropriately influence final treatment decisions by patients with carotid artery disease.

    Acknowledgments

    SS and MY completed data collection, data analyses, and drafting of the manuscript. AB was involved in drafting and editing the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    List of keywords used for Google Search.

    DOCX File , 12 KB

    References

    1. Statistics Canada. Table-0083-01 Internet use by province.   URL: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=2210008301 [accessed 2020-06-25]
    2. Statistics Canada. Internet use at home, by Internet activity and urban or rural distribution.   URL: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=2210006601 [accessed 2020-06-25]
    3. Abbott AL, Brunser AM, Giannoukas A, Harbaugh RE, Kleinig T, Lattanzi S, et al. Misconceptions regarding the adequacy of best medical intervention alone for asymptomatic carotid stenosis. J Vasc Surg 2020 Jan;71(1):257-269 [FREE Full text] [CrossRef] [Medline]
    4. Cambria RP, Conrad MF. Asymptomatic carotid stenosis: Revisionist history is usually wrong. J Vasc Surg 2020 Jan;71(1):2-4 [FREE Full text] [CrossRef] [Medline]
    5. Abbott AL, Paraskevas KI, Kakkos SK, Golledge J, Eckstein H, Diaz-Sandoval LJ, et al. Systematic Review of Guidelines for the Management of Asymptomatic and Symptomatic Carotid Stenosis. Stroke 2015 Nov;46(11):3288-3301. [CrossRef] [Medline]
    6. Mantese VA, Timaran CH, Chiu D, Begg RJ, Brott TG, CREST Investigators. The Carotid Revascularization Endarterectomy versus Stenting Trial (CREST): stenting versus carotid endarterectomy for carotid disease. Stroke 2010 Oct;41(10 Suppl):S31-S34 [FREE Full text] [CrossRef] [Medline]
    7. Brott T, Howard G, Roubin G, Meschia JF, Mackey A, Brooks W, CREST Investigators. Long-Term Results of Stenting versus Endarterectomy for Carotid-Artery Stenosis. N Engl J Med 2016 Mar 17;374(11):1021-1031 [FREE Full text] [CrossRef] [Medline]
    8. Otite FO, Khandelwal P, Malik AM, Chaturvedi S. National Patterns of Carotid Revascularization Before and After the Carotid Revascularization Endarterectomy vs Stenting Trial (CREST). JAMA Neurol 2018 Jan 01;75(1):51-57 [FREE Full text] [CrossRef] [Medline]
    9. Hussain MA, Mamdani M, Tu JV, Saposnik G, Khoushhal Z, Aljabri B, et al. Impact of Clinical Trial Results on the Temporal Trends of Carotid Endarterectomy and Stenting From 2002 to 2014. Stroke 2016 Dec;47(12):2923-2930. [CrossRef]
    10. Abbott AL. Medical (nonsurgical) intervention alone is now best for prevention of stroke associated with asymptomatic severe carotid stenosis: results of a systematic review and analysis. Stroke 2009 Oct;40(10):e573-e583. [CrossRef] [Medline]
    11. Automatic Readability Checker (Our Free Text Readability Consensus Calculator).   URL: https://www.readabilityformulas.com/free-readability-formula-tests.php [accessed 2020-07-01]
    12. The Gunning Fog Index (or FOG) Readability Formula.   URL: https://readabilityformulas.com/gunning-fog-readability-formula.php [accessed 2020-07-01]
    13. The DISCERN Instrument.   URL: http://www.discern.org.uk/discern_instrument.php [accessed 2020-07-01]
    14. Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care 2008 Oct;53(10):1310-1315 [FREE Full text] [Medline]
    15. Weis B. Health Literacy: A Manual for Clinicians.: American Medical Association; 2003.   URL: http://lib.ncfh.org/pdfs/6617.pdf [accessed 2020-07-01]
    16. Brott T, Calvet D, Howard G, Gregson J, Algra A, Becquemin JP, Carotid Stenosis Trialists' Collaboration. Long-term outcomes of stenting and endarterectomy for symptomatic carotid stenosis: a preplanned pooled analysis of individual patient data. Lancet Neurol 2019 Apr;18(4):348-356 [FREE Full text] [CrossRef] [Medline]
    17. Galyfos G, Sachsamanis G, Anastasiadou C, Sachmpazidis I, Kikiras K, Kastrisios G, et al. Carotid Endarterectomy versus Carotid Stenting or Best Medical Treatment in Asymptomatic Patients with Significant Carotid Stenosis: A meta-analysis. Cardiovasc Revasc Med 2019 May;20(5):413-423 [FREE Full text] [CrossRef] [Medline]
    18. Moore W, Voeks JH, Roubin GS, Clark WM, Howard VJ, Jones MR, CREST Investigators. Duration of asymptomatic status and outcomes following carotid endarterectomy and carotid artery stenting in the Carotid Revascularization Endarterectomy vs Stenting Trial. J Vasc Surg 2019 Jun;69(6):1797-1800 [FREE Full text] [CrossRef] [Medline]
    19. Halliday A, Mansfield A, Marro J, Peto C, Potter J, Thomas D, MRC Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group. Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. The Lancet 2004 May;363(9420):1491-1502. [CrossRef] [Medline]
    20. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995 May 10;273(18):1421-1428 [FREE Full text] [Medline]
    21. Hobson RW, Weiss DG, Fields WS, Goldstone J, Moore WS, Towne JB, et al. Efficacy of carotid endarterectomy for asymptomatic carotid stenosis. The Veterans Affairs Cooperative Study Group. N Engl J Med 1993 Jan 28;328(4):221-227 [FREE Full text] [CrossRef] [Medline]
    22. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial Effect of Carotid Endarterectomy in Symptomatic Patients with High-Grade Carotid Stenosis. N Engl J Med 1991 Aug 15;325(7):445-453 [FREE Full text] [CrossRef] [Medline]
    23. European Carotid Surgery Trialists' Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998 May 09;351(9113):1379-1387 [FREE Full text] [Medline]
    24. Brott TG, Halperin JL, Abbara S, Bacharach JM, Barr JD, Bush RL, American College of Cardiology, American Stroke Association, American Association of Neurological Surgeons, American College of Radiology, American American College of Radiology, Society of NeuroInterventional Surgery, Society for Vascular Medicine, Society for Vascular Surgery. 2011 ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease. A report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines, and the American Stroke Association, American Association of Neuroscience Nurses, American Association of Neurological Surgeons, American College of Radiology, American Society of Neuroradiology, Congress of Neurological Surgeons, Society of Atherosclerosis Imaging and Prevention, Society for Cardiovascular Angiography and Interventions, Society of Interventional Radiology, Society of NeuroInterventional Surgery, Society for Vascular Medicine, and Society for Vascular Surgery. Circulation 2011 Jul 26;124(4):e54-130. [CrossRef] [Medline]
    25. Ricotta JJ, Aburahma A, Ascher E, Eskandari M, Faries P, Lal BK, Society for Vascular Surgery. Updated Society for Vascular Surgery guidelines for management of extracranial carotid disease: executive summary. J Vasc Surg 2011 Sep;54(3):832-836 [FREE Full text] [CrossRef] [Medline]
    26. Wallaert JB, Nolan BW, Stone DH, Powell RJ, Brown JR, Cronenwett JL, et al. Physician specialty and variation in carotid revascularization technique selected for Medicare patients. J Vasc Surg 2016 Jan;63(1):89-97 [FREE Full text] [CrossRef] [Medline]
    27. Keogh C, McHugh S, Clarke Moloney M, Hannigan A, Healy D, Burke P, et al. Assessing the quality of online information for patients with carotid disease. Int J Surg 2014 Mar;12(3):205-208 [FREE Full text] [CrossRef] [Medline]
    28. Roberts H, Zhang D, Dyer GS. The Readability of AAOS Patient Education Materials: Evaluating the Progress Since 2008. J Bone Joint Surg Am 2016 Sep 07;98(17):e70. [CrossRef] [Medline]
    29. Ayyaswami V, Padmanabhan D, Patel M, Prabhu AV, Hansberry DR, Agarwal N, et al. A Readability Analysis of Online Cardiovascular Disease-Related Health Education Materials. Health Lit Res Pract 2019 Apr 01;3(2):e74-e80 [FREE Full text] [CrossRef] [Medline]
    30. Huang G, Fang CH, Agarwal N, Bhagat N, Eloy JA, Langer PD. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol 2015 Apr 01;133(4):449-454 [FREE Full text] [CrossRef] [Medline]
    31. Lam CG, Roter DL, Cohen KJ. Survey of quality, readability, and social reach of websites on osteosarcoma in adolescents. Patient Educ Couns 2013 Jan;90(1):82-87. [CrossRef] [Medline]
    32. Cajita MI, Whitehouse E, Budhathoki C, Hodgson N. Association between Internet use and decision-making preference in older adults. Gerontechnology 2016 Jan;14(2):97-104. [CrossRef] [Medline]


    Abbreviations

    AMA: American Medical Association
    BMT: best medical therapy
    CAD: coronary artery disease
    CAS: carotid artery stenting
    CEA: carotid endarterectomy
    CREST: Carotid Revascularization Endarterectomy versus Stenting Trial
    GFI: Gunning Fox Index
    MI: myocardial infarction
    RCT: randomized controlled trial
    TIA: transient ischemic attack


    Edited by G Eysenbach; submitted 17.08.20; peer-reviewed by A Bashir; comments to author 08.09.20; revised version received 28.09.20; accepted 02.10.20; published 03.11.20

    ©Shira Strauss, Michael Yacob, Apoorva Bhandari, Prasad Jetty. Originally published in the Interactive Journal of Medical Research (http://www.i-jmr.org/), 03.11.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.