Many countries around the world include obstetric ultrasound examinations as part of routine maternity care. One of the investigations possible with the ultrasound examination involves assessment of the anatomy of fetuses, with the purpose to detect or rule out fetal anomalies [1]. The most common types of fetal anomalies are congenital heart defects, defined as malformations of the heart and great vessels [2]. The incidence of congenital heart defects is approximately 8 in 1000 live births [2], with an incidence of 2.5 to 3.0 for the most severe, which require specialist cardiologic care [3]. Between the years 2000 and 2005, roughly 1 in 5 cases of all congenital heart defects were detected before birth [4]. However, the rates of prenatal diagnoses have increased in recent years [5-7], and studies report that over 40% of the most severe defects are prenatally diagnosed today [2,4,7].

Expectant parents view the obstetric ultrasound examination as an important step toward parenthood [8-10] but are often unprepared for adverse news [11-13] and experience an emotional crisis when faced with a prenatal diagnosis [11,14]. Depending on legislation and availability of induced abortion, the diagnosis may require a decision to continue or terminate the pregnancy. Expectant parents faced with the diagnosis experience this decision as difficult [11,15]. The decision involves various informational [11,16,17] considerations such as the process of induced abortion and what to expect of the postnatal situation if the pregnancy is continued [17]. Consequently, these persons experience a great need for information and highly value that which is offered during consultations with specialist health professionals [17,18]. However, the psychological distress experienced in connection to the diagnosis negatively influences comprehension and retention of the information offered [12,17], and research suggests that these individuals use the Web for supplemental information [16,17,19].

Patients view the Web as an important source of health-related information [20] and commonly use it for such purposes [20,21]. During pregnancy, women frequently use it to search for information about a variety of topics such as fetal development, pregnancy complications, and antenatal care [22]. As a large source of potentially highly accessible information [23,24], it holds promise to promote patient equity and empowerment [23,25-27]. However, studies indicate that consumers experience various difficulties when using the Web for health-related information [28,29]. Further complicating the situation, there is a lack of overarching systematic activities that aim to assess and control the quality of the available Web-based information [23], resulting in a risk of contact with information of poor quality. Combined, these issues call attention to the disorganized state of the Web [24]. The informational difficulties [11,15-17] and ethical dilemmas [30] related to decision making regarding whether to continue or terminate the pregnancy highlight the need to explore the relevance, level of details, suitability, and information about treatment choices in patient information websites. Moreover, research indicates that users of Web-based information place high importance on website appearance [31-33], raising questions concerning whether the design of these websites correspond to the preferences of the intended consumers.

As a step toward ensuring high-quality patient information materials, research suggests a need for developers to involve the intended consumers in the stages of production [34]. However, studies investigating the quality of Web-based information often use researchers or health professionals as assessors, raising questions about its applicability in clinical settings. In light of the identified differences in perspectives between researchers, health professionals, and patients [35,36], more research that employs new methods to investigate the perspectives of the actual consumers of the information is needed. No study has yet been published that explores consumers’ perspectives on websites about congenital heart defects.

The overall aim of this study was to explore the quality of patient information websites about congenital heart defects, from the perspectives of individuals with personal experience of a prenatal diagnosis of congenital heart defect in the fetus. Specifically, we set out to address the following research questions:

In Sweden, all expectant parents are offered a routine ultrasound examination at approximately 18 weeks of gestation. One of the purposes of the examination is to assess the fetal anatomy, to detect or rule out possible fetal anomalies. Swedish law permits termination of pregnancy up to 18 completed weeks of gestation. At later gestations, approval must be granted by the National Board of Health and Welfare. Few pregnancies are terminated after 22 completed weeks.

Through mixed-methods design with quantitative and qualitative approaches, we aspired to utilize the strengths and to offset the weaknesses of each approach [37]. Mixed-methods designs are presented as promising strategies to optimize assessments of Web-based health-related information [38].

The two most popular search engines on the Web, Google and Bing [39], were used to search for Swedish patient information websites about congenital heart defects. Swedish search terms for heart defect (hjärtfel) and fetus heart defect (foster hjärtfel) were used in searches during May 2015. In total, this resulted in 291,990 hits. We performed the searches in the incognito mode of our Web browser, to minimize influence from previous search patterns. The first 20 hits of each search were screened for inclusion by the first author. In total, 80 hits were screened; nine hits were deemed irrelevant, as they did not include any information about congenital heart defects. Relevant hits that included information about congenital heart defects leading to newspaper websites (n=19), communities or blogs (n=9), websites specifically produced for health professionals (n=5), scientific papers (n=3), and password-protected websites (n=1) were excluded. After correcting for duplicate hits (n=24), 10 Swedish patient information websites about congenital heart defects were included (Figure 1).

The first author decided which search terms to use. After the assessments, we asked the assessors, with experience of a prenatal diagnosis, which search terms they preferred to use when searching for patient information websites about congenital heart defects in connection to the diagnosis. The most commonly reported words in these search terms were heart defect (n=29 times mentioned) and fetus (n=11 times mentioned) (Multimedia Appendix 1).

The included websites were affiliated with charities and private organizations (n=3); government, hospitals, and clinics (n=3); independent information websites (n=2); and pharmaceutical companies (n=2). Of the included websites, 9 appeared in the first 10 hits of the searches, and 8 were identified in more than one of the searches (Multimedia Appendices 2 and 3).

Assessors were purposefully recruited [40] to strive for variation with regard to country of birth, educational level, and age. They were recruited from 2 units for fetal cardiology in Sweden. Through her clinical network, the last author was responsible for the recruitment of assessors with continued pregnancy. In total, 8 potential assessors were asked to participate; 2 declined because of fear of rekindled painful memories. When contacted for further information, 2 females declined because (1) she felt like it was enough with her partner participating in the study (n=1 potential assessor) and (2) lack of time (n=1 potential assessor). Thus, 2 females and 2 males with continued pregnancy, assessed the websites. The first author was responsible for the recruitment of persons with terminated pregnancy. Potential assessors were identified when they took part in a previous interview study that used consecutive clinical recruitment [17]. In total, 8 potential assessors were asked to participate; 1 male declined because of lack of interest. When contacted for further information, 1 female and 1 male declined because of lack of time. Thus, 3 females and 2 males with terminated pregnancy assessed the websites.

Assessors with continued pregnancy were parents of children with congenital heart defects who were 2 (n=2 assessors) and 3 (n=2 assessors) years old. Assessors’ ages ranged between 23 and 43 years. Regarding country of birth, 3 assessors with continued pregnancy were born outside Sweden, and all with terminated pregnancy were born in Sweden. Of the assessors, 2 had senior high school and 7 had university/college as highest educational level (Multimedia Appendix 4). Assessors with terminated pregnancy had undergone the procedure 20 (n=1 assessor), 24 (n=2 assessors), 27 (n=1 assessor), and 38 (n=1 assessor) weeks before assessments. Of the assessors, 4 with continued (n=2 assessors) and terminated (n=2 assessors) pregnancy were couples, and so experienced the same prenatal diagnosis. Assessors with continued pregnancy had experience of ventricular septal defect (n=2 assessors) or a combination of transposition of the great arteries, ventricular septal defect, and pulmonary stenosis (n=2 assessors). Assessors with terminated pregnancy had experience of atrioventricular septal defect with associated trisomy 21 (n=2 assessors), aortic stenosis (n=2 assessors), or a combination of Ebstein’s anomaly and multiple structural malformations (n=1 assessor).

Inspecting all of the assessments (n=90), the highest proportion of the lowest score (1) was found for treatment choices (n=35, 39% of total assessment scores). By contrast, the highest proportion of the highest score (5) was found for relevance (n=16, 18% of total assessment scores); see Figure 2.

For all assessors, the interrater reliability ranged between 0.18 and 0.40 for the 6 questions in the quality assessment tool. Interrater reliability ranged between 0.36 and 0.66 for assessors with continued pregnancy and between 0.29 and 0.42 for assessors with terminated pregnancy. With the exception of treatment choices (median=2.0), all questions had a median total score of 3.0, representing moderate quality (Table 2). There were no significant differences between median scores of the assessments from assessors with continued and terminated pregnancy. Figure 3 presents the distributions of the assessment scores among the assessors with continued and terminated pregnancy.

Figure 4 presents the distributions of the assessment scores for each of the included websites. No website had a median score of 5.0, representing high quality, for any of the questions in the quality assessment tool. Median scores of 1.0, representing low quality, were found in questions about treatment choices (n=4 websites), details (n=2 websites), suitability (n=1 website), and overall quality (n=1 website). Website 10 had median scores of 1.0 for 4 of the 6 questions.

Qualitative analysis generated six categories: (1) advertisements, (2) comprehensiveness, (3) design, (4) illustrations and pictures, (5) language, and (6) trustworthiness.

This study aimed to explore the quality of patient information websites about congenital heart defects and to provide new insights into these sources from the perspectives of the intended consumers. To the extent of our knowledge, this study is the first to use individuals with experience of a prenatal diagnosis as assessors to explore the quality of information websites about congenital heart defects. Overall, the investigated quality criteria were quantitatively assessed as either partially or not fulfilled. Various issues were described during discussions, indicating a need for further improvement of Web-based information about congenital heart defects. The suitability was assessed as neither unsuitable nor suitable, and the assessors raised issues regarding untrustworthy information belittling regarding the consequences of living with a severe heart defect. All of these aspects are known risks with Web-based information [23].

Parents of children with congenital heart defects want information about appropriate websites in connection to a prenatal diagnosis [17,44], but a previous study suggests that few cardiologists provide these recommendations [45]. According to the assessments in this study, some websites are particularly poor choices as sources for supplemental Web-based information. For example, one website affiliated with a clinic, identified in three searches and placed high in the search rank, had significantly lower assessments across five of the six quality criteria. Combined, the quantitative and qualitative findings indicate a need for health professionals to acknowledge the existing quality deficits and varied quality of available websites and bring this subject up for discussion during consultations with individuals who plan to use it for supplemental information. These efforts are even more important in light of the reports that consumers of health information use suboptimal search strategies [28], are unsuccessful at finding satisfactory Web-based information about health-related topics [46], and rarely discuss Web-based information with health professionals [22]. However, we acknowledge that health professionals may lack the time needed to feel adequately updated about available Web-based sources [24,47] and experience challenges when consulting patients who read Web-based information [47,48]. To adequately meet these issues, there seems to be a need for systematic efforts from overarching institutions in charge of publishing such recommendations.

Our findings indicate a lack of Web-based information about pregnancy termination and a bias of information toward pregnancy continuation in websites about congenital heart defects. It is crucial that individuals faced with a prenatal diagnosis reach informed decisions on whether to continue or terminate the pregnancy. Research suggests that persons faced with a prenatal diagnosis need information about the option to terminate the pregnancy and use the Web for supplemental information [17]. However, few English websites about congenital heart defects contain this type of information, which the findings of this study also confirm for Swedish settings [41]. Preparatory information is a key aspect to quality abortion care [49], and previous studies report that patients have unanswered questions and feel unprepared for the abortion process [50-52]. It is of great concern that patients experience information about pregnancy termination as insufficient and high-blown, both from health professionals and from Web-based sources [17,50]. Moreover, previous studies indicate that websites about induced abortions contain inaccurate, misleading [53,54], unsuitable, and low-quality information [55], calling attention to the need for patients to cautiously use and interpret Web-based information about these topics. In light of our findings, health professionals must consider these aspects when consulting individuals who decide on pregnancy termination and must make efforts to ensure that patients come into contact with sufficient and relevant high-quality Web-based information. The findings illustrate the importance of coordinated informational approaches between units for fetal cardiology and fetal medicine.

The complex language and unnecessary medical terminology described by the assessors raises concerns regarding the readability of these sources. Readability levels above that recommended to the general, nonmedical audience have been observed among websites about noninvasive prenatal testing [56] and other pregnancy-related topics [57,58], echoing our findings. Illustrations are possible pedagogic tools to overcome readability issues and are especially important when offering information to individuals with low health literacy [59], that is, the degree to which individuals have the capacity to obtain, process, and understand health information [60]. Assessors appreciated clear and easy-to-read illustrations, described as beneficial to promote comprehension. However, they also pointed toward the use of complex and irrelevant illustrations, further illustrating the problems related to poor readability. In light of these findings, we recommend the use of supplemental high-quality illustrations when offering information about congenital heart defects, but efforts need to be made to ensure that they are understandable to the intended consumers and have a clear purpose. Moreover, website developers need to take steps to improve the readability of their Web-based information about congenital heart defects.

The strength of this study is that we used mixed-methods design [38] to quantitatively investigate a set of variables while still remaining open to other perspectives with an inductive qualitative approach [61]. Assessors with personal experience assessed the websites, with experience of both continued and terminated pregnancy following a prenatal diagnosis. Typically, studies investigating the quality of information websites about reproductive health issues use a limited number of researchers or health professionals as assessors, frequently between 1 and 4 assessors [41,53,56,62-64]. In light of the identified differences in perspectives between researchers, health professionals, and patients [35,36], we argue that by using 9 laypersons with personal experience as assessors, our study provides insight that may more closely correspond to the views and preferences of the intended consumers of the patient information.

We used both female and male assessors with experience of either continuation or termination of pregnancy, which implies that the assessments include views and preferences of laypersons with various backgrounds and experiences. However, considering the variations in severity, prognosis, and treatment of the possible heart defects that can be prenatally diagnosed today [65,66], we acknowledge that the assessment of our assessors may differ from laypersons with experiences of other prenatal diagnoses. The interrater reliability ranged between 0.18 and 0.66, with higher coefficients of concordance for assessors who shared similar pregnancy outcome. This indicates at least fair to moderate concordance between the assessors used in this study [42]. Moreover, we did not use any validated instruments to assess the websites, as these include an extensive number of items [38]. We urge a need for development of less extensive alternatives, suitable for the purpose of quality assessment that uses laypersons as assessors.

We did not collect any information concerning the literacy levels of the assessors. It is possible that their levels were above or below the population norms among the intended consumers. Consequently, we cannot make any claims regarding their literacy levels and their representativeness. The results need to be interpreted with this in mind.

We wanted the focus group discussions to reflect the topics of importance for the assessors and not be colored by our preconceptions as researchers. Thus, no discussion guide was used to guide the discussions, and the assessors were encouraged to freely discuss aspects perceived as relevant to them. We argue that this approach was an appropriate way to stay close to the perspectives of the assessors.

Swedish websites were included, an important aspect to consider in the context of prenatal diagnosis, which differs in routines and legislation between countries. However, previous assessments of English websites with a health professional or researcher as an assessor indicate that the identified problems in this study transcend various settings and are similarly perceived by professionals and laypersons [41]. It is important to bear in mind that consumers rarely search beyond the first 10 hits in searches performed in search engines [28]. Consequently, we argue that strengths of this study are that we screened the first 20 hits and performed the searches in the most commonly used search engines [39]. Moreover, 24 of the 80 screened hits were duplicates, indicating that we achieved saturation in the inclusion of available websites. The first author chose the search terms. Consequently, it is possible that these do not fully correspond to those used by the intended consumers. However, after the data collection, we asked the assessors which search terms they preferred to use when searching for patient information websites about congenital heart defects in connection to the diagnosis. The answers confirmed the used search terms (Multimedia Appendix 1), indicating adequate representativeness in relation to those search terms that are used among the intended consumers. Moreover, the first author screened the hits and decided which hits to include as patient information websites. It is possible that laypersons would include other hits.

We encourage researchers to conduct studies that investigate how to improve the quality of Web-based information about congenital heart defects. Additionally, more research is needed to explore how individuals with low educational levels, individuals with low health literacy, and immigrants experience these websites.

The results indicate that website developers must ensure that sufficient information about treatment choices are included, with balanced information concerning pregnancy termination and possible consequences of living with a severe heart defect. Efforts should be taken to include statistics, as well as information about postnatal care and quality of life. If previous cases of persons faced with prenatal diagnosis are presented, make sure to include examples of cases with continued and terminated pregnancies. Clean and easily navigated designs, without advertisements, are important aspects when developing and maintaining websites. Clear and easy-to-read illustrations of the anatomy of the normal cardiovascular system and specific heart defects are desired among the intended consumers and should be included in Web-based patient information. Details should always be provided concerning references, authorship, and date of production. Website developers should strive toward continuous updates of the website and clearly state when it has been updated.

From the perspectives of the intended consumers, patient information websites about congenital heart defects do not fulfill quality criteria concerning information about treatment choices and only partly fulfill quality criteria concerning appearance, details, relevance, suitability, and overall quality. There are differences between existing websites, indicating variations in the Web-based landscape. The findings indicate that these websites include inappropriate advertisements, biased information, poor illustrations, complex language, and have issues with trustworthiness. Improvement is needed to meet the needs among the intended consumers for unbiased, complete, and trustworthy high-quality patient information websites. Combined, the findings suggest that existing patient information websites about congenital heart defects are, to a large extent, inadequate tools for supplemental information following a prenatal diagnosis.

When counseling expectant parents faced with a prenatal diagnosis of congenital heart defect in the fetus, we encourage health professionals to initiate discussions about their intentions to use the Web. Professionals should inform these persons about the varied quality in the Web-based landscape and offer recommendations for appropriate websites that include unbiased high-quality supplemental information. We encourage website developers to avoid (1) using inappropriate advertisements and pictures, (2) publishing information that the intended consumers may consider belittling regarding the impact of living with a severe heart defect, and (3) using complex illustrations without a clear purpose. Furthermore, we encourage them to (1) include information about pregnancy termination, (2) write information in a readable and understandable language, without the use of unnecessary medical terminology, (3) include stories from persons with lived experience of continuation and termination of pregnancy, (4) continuously update their websites with accurate information written by specialist health professionals, and (4) include a list of references.