Published on 21.07.16 in Vol 5, No 3 (2016): Jul-Sept
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/5651, first published Feb 24, 2016.
Evaluation of Web-Based Consumer Medication Information: Content and Usability of 4 Australian Websites
Background: Medication is the most common intervention in health care, and written medication information can affect consumers’ medication-related behavior. Research has shown that a large proportion of Australians search for medication information on the Internet.
Objective: To evaluate the medication information content, based on consumer medication information needs, and usability of 4 Australian health websites: Better Health Channel, myDr, healthdirect, and NPS MedicineWise .
Methods: To assess website content, the most common consumer medication information needs were identified using (1) medication queries to the healthdirect helpline (a telephone helpline available across most of Australia) and (2) the most frequently used medications in Australia. The most frequently used medications were extracted from Australian government statistics on use of subsidized medicines in the community and the National Census of Medicines Use. Each website was assessed to determine whether it covered or partially covered information and advice about these medications. To assess website usability, 16 consumers participated in user testing wherein they were required to locate 2 pieces of medication information on each website. Brief semistructured interviews were also conducted with participants to gauge their opinions of the websites.
Results: Information on prescription medication was more comprehensively covered on all websites (3 of 4 websites covered 100% of information) than nonprescription medication (websites covered 0%-67% of information). Most websites relied on consumer medicines information leaflets to convey prescription medication information to consumers. Information about prescription medication classes was less comprehensive, with no website providing all information examined about antibiotics and antidepressants. Participants (n=16) were able to locate medication information on websites in most cases (accuracy ranged from 84% to 91%). However, a number of usability issues relating to website navigation and information display were identified. For example, websites not allowing combinations of search terms to be entered in search boxes and continuous blocks of text without subheadings.
Conclusions: Of the 4 Australian health information websites tested, none provided consumers with comprehensive medication information on both prescription and nonprescription medications in a user-friendly way. Using data on consumer information needs and user testing to guide medication information content and website design is a useful approach to inform consumer website development.
Interact J Med Res 2016;5(3):e21
- consumer health information;
- health communication;
- prescription drugs;
- nonprescription drugs;
- drug information service;
- usability testing
Medication is the most common intervention in health care . A 2010 survey of 12,262 consumers revealed that approximately 80% of Australians sought health information on the Internet, and of these individuals, approximately 70% sought information on medication [ ]. Focus groups with Australian consumers showed that consumers viewed the Internet as an important source of medication information, but also that consumers varied in their search and appraisal skills [ ]. Examining written medication information available on the Internet is of value as this information has the potential to affect consumers’ medication taking behavior and satisfaction [ , ].
Evaluations of Web-based health information have typically utilized instruments with various criteria  covering technical details (eg, disclosure of authorship and sponsorship, provision of references) [ ], design features (eg, layout, speed), readability (eg, Flesch Reading Ease, Simple Measure of Gobbledygook (SMOG)) [ , ], accuracy, and completeness of information. However, many of the criteria lack validity and reliability and have certain gaps [ , - ]. For example, a 2002 systematic review of the criteria used to evaluate health websites identified user testing as a neglected area [ ]. More recently, user testing has been used to assess websites in a number of studies and has highlighted the importance of this approach in usability evaluation [ , ]. For example, 1 study used 4 rounds of user testing to improve an Internet-based hemophilia self-management tool for adolescents [ ]. Another study used user testing and identified the need for websites to take user age into account in their design [ ].
Although there have been numerous evaluations of Web-based health information [, ], fewer studies have evaluated Web-based medication information [ , , - ]. Given the importance of Web-based medication information to consumers [ ], this study aimed to evaluate the medication information on 4 frequently used Australian websites. The evaluation took a unique approach by evaluating both website content and usability and by being guided by data on consumer medication information needs.
A mixed-method approach was used in this study comprising (1) an assessment of consumer medication information needs, (2) a website content evaluation using the consumer information needs, and (3) user testing of websites (including qualitative interviews).
Identification of Consumer Medication Information Needs
Consumer medication information needs were determined by examining the most frequently used medications in Australia and the most frequent consumer medication queries made to the healthdirect helpline. The top 5 most commonly used prescription and nonprescription medications in Australia were extracted from the Pharmaceutical Benefits Scheme (PBS)  and the National Census of Medicines Use, respectively [ ]. The top 5 prescription medications, by defined daily dose/1000 population/day, were atorvastatin, rosuvastatin, perindopril, irbesartan, and candesartan [ ]. The top 5 nonprescription medications (excluding paracetamol) used by survey respondents in the past month were fish oil supplements (26.1% of respondents), aspirin (21.5% of respondents), glucosamine (17.5% of respondents), calcium (12.3% of respondents), and cholecalciferol or vitamin D (11.4% of respondents) [ ]. Paracetamol (used by 42.9% of respondents in the last month) was excluded from this list as it was the subject of the most frequent consumer medication query made to the healthdirect helpline (see below).
The consumer medication queries made to the healthdirect helpline in November 2014 were extracted and reviewed to determine the medication therapeutic class that was the subject of the call and the query regarding the medication (eg, what to do if a dose was missed). The healthdirect helpline is a free 24-hour health advice telephone line that covers approximately 56% of Australia’s population. The telephone service receives between 60,000 and 70,000 calls per month, with medication queries the most frequent clinical issue discussed . In November 2014, the most frequent medication classes (and specific medication within each class) were analgesics (paracetamol, ibuprofen, and paracetamol and codeine), antibiotics (amoxicillin), antidepressants (sertraline), antihistamines (promethazine), and anticoagulants and antithrombotic agents (warfarin). The most common queries regarding each of these medication classes varied. However, the top 3 queries for all types of medications were (1) how to take a medication (how much to take, what to do if a dose is missed, and what to do in an overdose), (2) medication interactions, and (3) medication side effects.
Identification of Websites
We aimed to identify websites for the evaluation that consumers would most frequently encounter when using the Google search engine for medication queries. Search terms related to the top medications and queries outlined above were entered into Google. The 4 most frequently generated Australian health websites providing consumer medication advice were selected. Websites specific to a condition, a medication, or a population group were excluded (eg, beyondblue, a website targeting mental health ; Royal Children’s Hospital Melbourne targeting pediatric patients [ ]). The 4 websites included in our evaluation were Better Health Channel [ ], myDr [ ], healthdirect [ ], and NPS MedicineWise [ ].
Assessment of Website Content
First, each website’s content on the prescription and nonprescription medications identified from the PBS and National Census of Medicines Use was assessed with respect to the extent to which it covered the 3 most frequent medication queries to the healthdirect helpline (ie, how to take a medication, interactions, and side effects). Additionally, whether each website had an information page about the therapeutic classes of these medications was examined. For example, for atorvastatin, websites were evaluated on the extent to which they covered information on how to take the medication (how much to take, what do to if a dose is missed, and what to do in an overdose), interactions, and side effects as well as whether there was a general page on hypolipidemic medications.
Second, each website’s content was assessed with respect to whether it covered the most frequent medication classes that were the subject of calls to the healthdirect helpline and each class’s most frequent queries. Whether each website included information on the most frequent medication within each class was also examined. For example, when providing information on antibiotics, whether the website covered missed doses, interactions, and stopping an antibiotic were assessed, along with whether there was any specific information on amoxicillin. For analgesics and antipyretics, information on paracetamol, ibuprofen, and paracetamol and codeine was examined, as these were by far the most frequent medication types queried and these medicines are available over the counter (nonprescription).
A coding system was applied to indicate the extent to which information on each medication query was available on the websites. Two investigators (MZR and LR) initially tested the coding system to ensure it was suitable and reliable (ie, produced the same code when applied independently by 2 reviewers). Subsequently, a single investigator (LR) coded the queries on all websites. Each query was coded as either covered (C), that is, the information provided on the website was comprehensive enough to fully answer the query; partially covered (PC), that is, there was information related to the query, but it did not answer the query specifically; referred (R), that is, the website referred users to another site that answered the query; and not covered (and not referred; NC). By way of example, for the query “what medications does aspirin interact with?” a website was coded as covering the information if it provided a list of medications with which aspirin interacts but partially covered if it only stated that aspirin interacted with some medications and asked the user to seek advice from a health professional.
Website content assessment was conducted in January 2015.
We developed scenarios for testing based on the medication calls made to the healthdirect helpline outlined above. The most frequently queried medications were combined with the most frequently asked questions to create the scenarios. The scenarios consisted of 8 questions, all of which had answers available on the test websites (). Thus the scenarios sought to test the ease and speed with which users were able to find information that was contained on the websites.
|Sequence||Website 1||Website 2||Website 3||Website 4|
|1||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|2||Better Health Channel||healthdirect||NPS MedicineWise||myDr|
|3||myDr||Better Health Channel||healthdirect||NPS MedicineWise|
|4||NPS MedicineWise||myDr||Better Health Channel||healthdirect|
Consumers who were unfamiliar with the target websites took part in user testing. To recruit participants, posters were displayed at Macquarie University, Sydney campus, Australia. Participants received a complimentary lunch for taking part.
Participants were observed by a single investigator (MTB) while performing the 8 scenario tasks using the websites. To complete each task, the user was required to answer each medication-related query by locating relevant information on a website. Tasks were completed in a fixed order but the order of website use varied between subjects to minimize any learning effects, with each participant randomly allocated to 1 of 4 sequences listed in. Thus, as there were 8 scenarios, each participant used each website twice to locate a piece of information. User testing was conducted in January and February 2015.
The variables collected by the observer during each scenario were time taken to locate the desired medication information; number of screens required to locate the piece of medication information; number of new searches a user performed (ie, new entries into a search box); the user’s search method (eg, whether he or she used the search box or browsed subheadings); whether the user was successful in completing the task (ie, answered the question correctly); and any obvious negative affect (eg, frustration).
After completion of 4 scenarios (on 2 websites) participants took a short break and were asked to comment on the 2 websites they had just used. They were asked to indicate which website they preferred and why, to describe good and bad features of the websites, and to comment on the layout of information on the screen and on how understandable the website content was. Participants then completed the remaining 4 scenarios and were interviewed about the 2 additional websites. Finally, participants were asked to indicate which of the 4 websites was their preferred website and why and to describe an ideal website for locating medication information.
Nonparametric Friedman tests were used to detect the differences across the websites on time taken to locate information, number of screens required, and number of new searches. A generalized estimating equation approach, with consideration of the correlation of measurements from the same participant, was used to compare the websites on proportion of tasks successfully completed. Results were considered significant when P ≤.05.
Assessment of Website Content
shows website coverage of the most commonly used prescription medications in Australia. shows the total percentage of website coverage of the most commonly used prescription medications in Australia. Of the four websites, 3 (NPS MedicineWise, myDr, and Better Health Channel) covered each of the queries related to the medication, and the healthdirect website referred consumers to other sources for the information. All websites had a general information page on hypolipidemic and antihypertensive medications.
shows website coverage of the most commonly used nonprescription medications in Australia. shows the total percentage of website coverage of information on the most frequently used nonprescription medication in Australia. Compared with prescription medication, information on nonprescription medication was less comprehensive. Whereas all common queries related to these medications were covered by multiple websites for aspirin, calcium, and vitamin D, no website covered all queries on fish oil supplements and glucosamine. Three of the four websites covered general information on both anticoagulants/antithrombotic agents, and complimentary medicines.
|Prescription medication information||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|How to take ita||Rb||Cc||C||C|
|How to take ita||R||C||C||C|
|How to take ita||R||C||C||C|
|How to take ita||R||C||C||C|
|How to take ita||R||C||C||C|
aHow to take it includes how much to take, what to do if a dose is missed, and what to do in an overdose. Inclusion of all 3 resulted in a rating of C; if only 1 or 2 items were covered, then a rating of PC (partially covered) was given.
bR: referred to an external site.
dCovered on a general page about statins, which mentions atorvastatin and rosuvastatin.
|Nonprescription medication information||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|How to take ita||Rb||PCc||Cd||C|
|Fish oil supplements|
|How to take ita||R||PC||NCe||PC|
|How to take ita||R||PC||NC||NC|
|How to take ita||R||PC||C||C|
|Cholecalciferol (vitamin D)|
|How to take ita||R||PC||PC||C|
aHow to take it includes how much to take, what do to if a dose is missed, and what to do in an overdose. Inclusion of all 3 resulted in a rating of C; if only 1 or 2 items were covered then a rating of PC was given.
bR: referred to an external site.
cPC: partially covered.
eNC: not covered (and not referred).
shows website coverage of the most common queries to the healthdirect helpline related to analgesics and antipyretics. shows the total percentage of website coverage of queries to the healthdirect helpline related to simple analgesics and antipyretics. Each query was covered by at least one website for each medication, except for paracetamol and codeine, where no website contained information on whether it is safe to take another dose of paracetamol and codeine after vomiting. No website covered all the queries for paracetamol, and only 1 site (myDr) covered all the queries for ibuprofen.
shows the websites’ coverage of the queries that the healthdirect helpline has received related to antibiotics, antidepressants, antihistamines, and anticoagulants and antithrombotic agents. shows website coverage of general information queries to the healthdirect helpline related to antibiotics, antidepressants, antihistamines, and anticoagulants and antithrombotic agents. shows the total percentage of website coverage of queries to the healthdirect helpline related to antibiotics, antidepressants, antihistamines, and anticoagulants and antithrombotic agents. General information on each medication class was available from at least one website, except for antihistamines, with no website providing general information on this drug class. Information on missed doses of antidepressants was not covered by the websites and that on missed doses of antibiotics was only partially covered by 1 website (NPS MedicineWise). The most common queries related to antibiotics, antidepressants, and antihistamines were not all covered by any one site.
|Queries for analgesics and antipyretics||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|How much to take (children)||PCa||PC||Cb||PC|
|How much to take (children)||PC||PC||C||C|
|Paracetamol and codeine|
|Vomited after taking, is it safe to take another dose||NC||NC||NC||NC|
aPC: partially covered.
cNC: not covered (and not referred).
dR: referred to an external site.
|Queries||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|How long to take it for||R||Cd||C||NC|
|Starting and stopping||PC||C||C||C|
|How much to take (children)||R||NC||NC||NC|
|Use in breastfeeding||R||Ce||NC||NC|
|Anticoagulants, antithrombotic agents|
aR: referred to an external site.
bPC: partially covered.
cNC: not covered (and not referred).
eInformation on use of antihistamines while breastfeeding was on a page about medication use in breastfeeding.
|Queries||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|Specific information on amoxicillin?||No||Yes||Yes||Yes|
|Specific information on sertraline?||No||Yes||Yes||Yes|
|Specific information on promethazine?||No||Yes||Yes||Yes|
|Anticoagulants, antithrombotic agents|
|Specific information on warfarin?||Yesa||Yes||Yes||No|
aInformation provided on a page about stroke treatment.
|Total coverage (%)||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
A total of 16 consumers were recruited for user testing. The median age was 27 years (range 18-66) and 8 out of 16 participants (50%) were male. When asked how frequently they used the Internet, all participants reported that they used the Internet “multiple times a day.” When asked how often they used the Internet to find medication information, typical responses were “never” (n=7), “rarely” (n=4), and “once per month” (n=2).
shows results of user testing. The number of screens viewed while completing scenarios using NPS MedicineWise and myDr were fewer than those viewed when using healthdirect and Better Health Channel (Friedman's χ23=9.02, P=.03). However, there was no evidence to indicate differences in the time taken to complete scenarios (Friedman's χ23=5.47, P=.14), the number of new searches performed (Friedman's χ23=3.04, P=.39), or in the accuracy with which participants completed the scenarios (χ23=1.34, P=.72).
Of the 16 participants, 11 participants (69%) indicated NPS MedicineWise was their preferred website, 4 participants (25%) said myDr, and 1 participant preferred healthdirect to the other websites.
|Key results||healthdirect||NPS MedicineWise||myDr||Better Health Channel|
|Median time taken to complete a scenario (range)a||3 minb 37 sc|
(1 min 13s to 16 min 3s)
|2 min 56s |
(1 min 12s to 8 min 56s)
|2 min 41s |
(49s to 9 min 16s)
|4 min 34s |
(1 min 47s to 12 min 31s)
|Median number of screens viewed (range)||10 (3-34)||7 (2-17)||7 (3-15)||10 (5-17)|
|Median number of new searches (range)a||2 (0-7)||0 (0-4)||2 (0-6)||2 (0-9)|
a No evidence to indicate differences between websites.
b min: minute.
c s: second.
Overall, it appeared to be more difficult for participants to locate the appropriate page of information for each scenario than to find the relevant piece of information on a page. In 84 of the 128 (65.6%) scenarios observed, participants began the scenario by entering a keyword or keywords into the home page search box. In 32 of the 128 scenarios (25.0%), participants looked for the website’s medicine page before searching for a particular medication name. When trying to locate a piece of information on a webpage, more than half the participants (n=10) used a keyboard shortcut (ie, control-F) to find a keyword (eg, pregnancy) on the page, whereas the remaining participants scrolled through the information.
Participant interviews provided further information on their perceptions of the usability of websites. All participants reported that information displayed on websites was presented at the right level of difficultly, although some suggested that content was understandable because they were students or researchers:
I think there would be a not insignificant proportion of the population that would struggle with it, because they would baulk at the terminology that’s used.
The inability to search using combinations of search terms (eg, Panadeine Forte AND pregnancy) was identified to be a negative aspect of websites by participants:
Yeah, so if it would work more like Google where I would type in the keywords of what I was looking for then that would be much easier. That was something that I noticed all the websites didn’t do.
Users also reported that being directed to PDF versions of consumer medicines information (CMI) leaflets was problematic:
It’s really wordy and the format of it, because it’s set up, to me, it’s set up like a physical pamphlet, so if I had that in my hands, that’s fine but on the screen, the three column thing with the same format and the font of a physical pamphlet doesn’t work...it makes skimming much harder because, I mean, I could do it but it just took me longer.
A large number of menus and drop-down menus on the home page resulted in participants using the search box rather than browsing the website:
Yeah, so it’s very all over the place really. You really don’t know where to start with that one so you’re almost forced to go to search this side.
The listing of medications only by generic names or brand names, not both, was identified as a barrier to finding relevant information, as users were not necessarily familiar with both terms:
It can be confusing with generic versus brand names, because I know one, when that I was initially looking for Zoloft I looked at that list of anti-depressants and I think they only had the generic names so I didn’t see Zoloft on that list, so I think it’s important to have both written.
Websites presenting medication information in a separate location from health information was perceived as a problem because users generally viewed medication information as a subset of health information, not as a separate category of information:
I think it should be more integrated because I think that that’s logically how people think. They see health as being the generic term and medicine a subset within.
The large number of results being generated from search-box queries was also considered a barrier to locating information:
It’s like when you’re searching, it kind of gives you every possible result rather than the one you probably want, the common one. Well, I don’t know, I mean, it’s hard to get that balance between only throwing up a few common ones, and the person could miss out on what they want to see, or throwing up everything and the person just gets, like, what is all this?
Participants viewed features of websites that broke up large amounts of text (eg, subheadings, highlighting, or hyperlinked subheadings) as helpful:
Subheadings are very good. Especially when you know what you are looking for.
I think that in terms of NPS [MedicineWise] there was more bold so I found it easier to read because then I would just skim and if the bold didn’t apply then I would just ignore the regular font.
Bullet points are good. I mean, you don't want massive slabs of information that you need to search through.
You don’t want to sit there reading through it all. Having those little jumping links is helpful if you are looking for a particular bit of information.
Additionally, auto-completion of search terms in search boxes was reported to be a positive feature of websites:
I like that if you search something there are suggestions for what you are searching.
Several limitations were identified in the medication information available on 4 Australian health websites in relation to both content and usability. Although detailed information on specific prescription medication was provided, information on nonprescription medication and medication classes was less comprehensive. Several website features affected how quickly and easily users were able to locate medication information.
Information on the most common prescription medications and most frequent medication queries made to the healthdirect helpline were covered or referred by all 4 websites. The healthdirect website was the only website to refer consumers to other websites. This is because the healthdirect website acts as a portal site that directs consumers to other sources of reliable health information. The majority of the prescription medication information was available through CMI leaflets, either embedded into webpages (NPS MedicineWise, myDr) or as a link to a PDF file (Better Health Channel). CMI content is regulated by the Australian Government and is prepared by and the responsibility of pharmaceutical companies . Although CMI leaflets provide consumers with the full scope of information on a specific medication, the amount of information they present may be overwhelming for consumers [ , ]. Furthermore, CMI leaflets have been criticized for not promoting medication adherence because they include only limited information on the benefits of taking medications [ ]. Thus, although it is not feasible for a website to develop its own content on every prescription medication, the inclusion of general pages on medication classes may be an opportunity to provide consumers with more concise information than CMI leaflets, including content on the benefits of taking a medication [ ]. In this study, we found the general information pages on 2 prescription medication classes, antibiotics and antidepressants, to be limited in their scope on all websites for answering frequent consumer medication queries.
The way CMI content is displayed on the websites also appeared to be problematic. The inclusion of a link to a PDF version of the “paper” leaflet (as on Better Health Channel), which includes three columns of text, was not viewed favorably by consumers. This was primarily because the layout required users to continuously scroll up and down to read the text. Consumers preferred CMI content to be embedded into the webpage in a single column, as done on NPS MedicineWise and myDr. Interestingly, an assessment of consumer needs in relation to printed CMI leaflets also noted that consumers preferred a single-column layout .
Information on nonprescription medication was less comprehensively covered on the websites than prescription medication. The nonprescription medication or medication classes examined in this study included paracetamol, ibuprofen, paracetamol and codeine (formulations with codeine≤12 mg/unit), antihistamines, aspirin, fish oil supplements, glucosamine, calcium, and cholecalciferol. Nonprescription medications do not require a CMI leaflet according to Australian regulations. Instructions for use typically appear in or on the packaging. However, as is evident from the large volume of calls made to the healthdirect helpline about these medications, consumers may not always read, keep, or understand packaging instructions, or all the required information may not be provided on packaging instructions.
Of the nonprescription medicines examined in this study, 5 were complementary medicines. Complementary medicines are a subset of nonprescription medicines that can be defined as herbal, natural, or alternative medicines and include vitamins, minerals, herbs, and nutritional supplements. Australian studies estimate that 50% of complementary medicine users also take conventional medicines  and more than half of these consumers do not report complementary medicine use to their doctor [ , ]. Of complementary medicine users, 75% are unaware that the products are not tested for quality and safety by the Australian Therapeutic Goods Administration [ ]. Yet, our results showed complementary medicines were the least comprehensively covered by the websites evaluated. There appears to be a significant gap in information available to consumers to make informed decisions about their use of these products. This is particularly salient because the quality of Web-based information on complementary medicines is limited [ ].
Although the content of websites is important, it is also crucial for information to be easily located. Usability issues related to both website navigation (ie, locating the correct page) and information display (ie, locating information on a page) were identified in this study. A key navigation issue was that websites did not allow users to search using multiple keywords, as is typically the case in search boxes. This caused users to become extremely frustrated and resulted in delays. Information layout was important for locating content on a page, with participants preferring text to be broken up using subheadings, highlighting, or bullet points. These features are in line with those identified in a previous assessment of consumer needs related to printed CMI leaflets . The NPS MedicineWise website was preferred by the majority of user testing participants. The layout of the NPS MedicineWise website was looked upon favorably by participants and was most likely the reason participants were required to navigate through fewer screens to locate information on this website compared with the other websites.
This study had a number of limitations. The assessment of consumer medication information needs was based on calls to a national health helpline and the most commonly used prescription and nonprescription medications. Consumer queries from other sources, such as health professionals (physicians and pharmacists), were not captured. Additionally, there may be important medication safety issues not recognized by consumers but for which there is limited information, and our methods would not have captured these. The accuracy or the readability level of the medication information on the websites was not evaluated as part of this study. However, a recent study assessing the readability of 251 Australian health webpages found that their readability was above the average Australian levels of reading . Thus, clearly, this is also an important consideration for website design. Lastly, the number of participants used for user testing, although likely large enough to detect most issues [ ], limited our ability to detect statistically significant differences between websites for indicators tested. Despite these limitations, the study presents an innovative approach to the evaluation of medication information on websites and identified medication information gaps not previously recognized. Addressing these gaps may improve the safe use of medicines in the community.
This study applied a unique approach, guided by consumer medication information needs, to assess the content and usability of medication information on 4 Australian websites. Several gaps were identified with respect to website content, and several usability issues were identified with respect to navigation and information presentation. Results showed that the 4 Australian websites tested did not provide consumers with comprehensive medication information on both prescription and nonprescription medications in a user-friendly way. Additional content (eg, on nonprescription medication) and some simple redesign of content (eg, single-column text with bullet points) would improve both the content and usability of widely used Australian websites.
This research formed part of a project, supported by funding from Healthdirect Australia, evaluating Healthdirect’s telephony and digital medication services. MB and MR are employed at Healthdirect Australia. MB and MR contributed to the study design and revisions of the manuscript but were not involved in data analysis and interpretation.
MTB, MB, MR, JIW, MZR, AT, and LL contributed to the study design. LL analyzed medication query call data. MZR and MTB designed the user testing scenarios and website content assessment. LR assessed websites for medication information content. MTB conducted user testing and analyzed qualitative user testing data. LL analyzed quantitative user testing data. MTB, JIW, MZR, AT, LL, and LR contributed to the analysis and interpretation of the data. All authors reviewed the manuscript for content and approved the final version for publication.
Conflicts of Interest
- Roughead L, Semple S, Rosenfeld E. Sydney: Australian Commission on Safety and Quality in Health Care. 2013. Literature Review: Medication Safety in Australia URL: http://www.safetyandquality.gov.au/wp-content/uploads/2013/08/Literature-Review-Medication-Safety-in-Australia-2013.pdf [WebCite Cache]
- McDaid D, Park A. BUPA Health Pulse. London: BUPA, 2011; 2010. Online Health: Untangling the Web - Evidence from the BUPA Health Pulse 2010 International Healthcare Survey URL: https://www.bupa.com.au/staticfiles/Bupa/HealthAndWellness/MediaFiles/PDF/LSE_Report_Online_Health.pdf [WebCite Cache]
- Peterson G, Aslani P, Williams KA. How do consumers search for and appraise information on medicines on the Internet? A qualitative study using focus groups. J Med Internet Res 2003 Dec 19;5(4):e33 [FREE Full text] [CrossRef] [Medline]
- Koo MM, Krass I, Aslani P. Factors influencing consumer use of written drug information. Ann Pharmacother 2003 Feb;37(2):259-267. [Medline]
- Hamrosi KK, Aslani P, Raynor DK. Beyond needs and expectations: identifying the barriers and facilitators to written medicine information provision and use in Australia. Health Expect 2014 Apr;17(2):220-231. [CrossRef] [Medline]
- Eysenbach G, Powell J, Kuss O, Sa E. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002;287(20):2691-2700. [Medline]
- Charnock D, Shepperd S. Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res 2004 Aug;19(4):440-446 [FREE Full text] [CrossRef] [Medline]
- Cheng C, Dunn M. Health literacy and the Internet: a study on the readability of Australian online health information. Aust N Z J Public Health 2015 Aug;39(4):309-314. [CrossRef] [Medline]
- Nasser S, Mullan J, Bajorek B. Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. Australas Med J 2012;5(3):194-203 [FREE Full text] [CrossRef] [Medline]
- Gagliardi A, Jadad AR. Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ 2002 Mar 9;324(7337):569-573 [FREE Full text] [Medline]
- Bernstam EV, Sagaram S, Walji M, Johnson CW, Meric-Bernstam F. Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed? Int J Med Inform 2005 Aug;74(7-8):675-683. [CrossRef] [Medline]
- Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res 2015;17(5):e112 [FREE Full text] [CrossRef] [Medline]
- Breakey VR, Warias AV, Ignas DM, White M, Blanchette VS, Stinson JN. The value of usability testing for Internet-based adolescent self-management interventions: “Managing Hemophilia Online”. BMC Med Inform Decis Mak 2013;13:113 [FREE Full text] [CrossRef] [Medline]
- Pak R, Price MM, Thatcher J. Age-sensitive design of online health information: comparative usability study. J Med Internet Res 2009;11(4):e45 [FREE Full text] [CrossRef] [Medline]
- Cumbo A, Agre P, Dougherty J, Callery M, Tetzlaff L, Pirone J, et al. Online cancer patient education: evaluating usability and content. Cancer Pract 2002;10(3):155-161. [Medline]
- Ghoshal M, Walji MF. Quality of medication information available on retail pharmacy Web sites. Res Social Adm Pharm 2006 Dec;2(4):479-498. [CrossRef] [Medline]
- Gondim AP, Weyne DP, Ferreira BS. Quality of health and medication information on Brazilian websites. Einstein (Sao Paulo) 2012;10(3):335-341 [FREE Full text] [Medline]
- Kim KY, Metzger A, Wigle PR, Choe PJ. Evaluation of online consumer medication information. Res Social Adm Pharm 2011 Jun;7(2):202-207. [CrossRef] [Medline]
- Lin H, Ku C, Li J, Tan AC, Chou C. A nationwide evaluation on electronic medication-related information provided by hospital websites. J Eval Clin Pract 2013 Apr;19(2):304-310. [CrossRef] [Medline]
- Markman M. Presentation of chemotherapy options for cervix cancer on cancer-related Internet sites. J Womens Health (Larchmt) 2009 Jun;18(6):827-829. [CrossRef] [Medline]
- Mira JJ, Lorenzo S, Pérez-Jover V, Navarro I, Martín de Rosales AM, Lara C. Assessment of the quality of medication information for patients in Spain. Expert Opin Drug Saf 2013 Jan;12(1):9-18. [CrossRef] [Medline]
- Morgan M, Montagne M. Drugs on the Internet, part II: antidepressant medication web sites. Subst Use Misuse 2011;46(13):1628-1641. [CrossRef] [Medline]
- Lee K, Hoti K, Hughes JD, Emmerton L. Dr Google and the consumer: a qualitative study exploring the navigational needs and online health information-seeking behaviors of consumers with chronic health conditions. J Med Internet Res 2014;16(12):e262 [FREE Full text] [CrossRef] [Medline]
- Australian Prescriber. 2013. Top 10 drugs URL: https://www.nps.org.au/attachments/814567b00a6cd21f685241a01c1f1ac46f999299/store/5b53c34ffb2dd079a4454ec73ba9d03cd352e280c73292e66d8a134676df/top-10-drugs-6-211.pdf.pdf [accessed 2016-07-04] [WebCite Cache]
- Morgan TK, Williamson M, Pirotta M, Stewart K, Myers SP, Barnes J. A national census of medicines use: a 24-hour snapshot of Australians aged 50 years and older. Med J Aust 2012 Jan 16;196(1):50-53. [Medline]
- Healthdirect A. Heatlhdirect Australia Biannual Report January-June. 2014. URL: http://media.healthdirect.org.au/publications/HDA-BiannualReport-B1-2014-FA-FINAL.pdf [WebCite Cache]
- beyondblue. 2016. URL: https://www.beyondblue.org.au/ [WebCite Cache]
- The Royal Children's Hospital Melbourne. Kids Health Info - Fact Sheets. Kids Health Info - Fact Sheets URL: http://www.rch.org.au/kidsinfo/ [WebCite Cache]
- Better Health Channel. 2016. URL: http://www.betterhealth.vic.gov.au/ [WebCite Cache]
- myDr. 2016. URL: http://www.mydr.com.au/ [WebCite Cache]
- healthdirect. 2016. URL: http://www.healthdirect.gov.au/ [accessed 2016-02-15] [WebCite Cache]
- NPS MedicineWise. 2016. URL: http://www.nps.org.au/ [WebCite Cache]
- Aslani P, Hamrosi K, Feletto E, Raynor D, Knapp P, Huges J, et al. Investigating Consumer Medicine Information (I-CMI) Project: Australian Government Department of Health and Ageing; The Pharmacy Guild of Australia. URL: http://6cpa.com.au/wp-content/uploads/Investigating-Consumer-Medicine-Information-I-CMI-Project-Full-Final-Report-Part-1.pdf [WebCite Cache]
- MacLennan AH, Myers SP, Taylor AW. The continuing use of complementary and alternative medicine in South Australia: costs and beliefs in 2004. Med J Aust 2006 Jan 2;184(1):27-31. [Medline]
- Xue CC, Zhang AL, Lin V, Da CC, Story DF. Complementary and alternative medicine use in Australia: a national population-based survey. J Altern Complement Med 2007;13(6):643-650. [CrossRef] [Medline]
- Walji M, Sagaram S, Sagaram D, Meric-Bernstam F, Johnson C, Mirza NQ, et al. Efficacy of quality criteria to identify potentially harmful information: a cross-sectional survey of complementary and alternative medicine web sites. J Med Internet Res 2004 Jun 29;6(2):e21 [FREE Full text] [CrossRef] [Medline]
- Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003 Aug;35(3):379-383. [Medline]
|CMI: consumer medicines information|
|PBS: Pharmaceutical Benefits Scheme|
Edited by G Eysenbach; submitted 24.02.16; peer-reviewed by JJ Mira, S Lorenzo; comments to author 10.03.16; revised version received 16.05.16; accepted 24.05.16; published 21.07.16
©Magdalena Z Raban, Amina Tariq, Lauren Richardson, Mary Byrne, Maureen Robinson, Ling Li, Johanna I Westbrook, Melissa T Baysari. Originally published in the Interactive Journal of Medical Research (http://www.i-jmr.org/), 21.07.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.