This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.i-jmr.org/, as well as this copyright and license information must be included.
A minimum standard based upon consensus decision making recommends a core set of tinnitus-specific health complaints (outcome domains) that should be assessed and reported in all clinical trials as this enables comparisons to be made across studies as well as data pooling for meta-analysis.
This study aimed to further clarify how the outcome domain concepts should be defined for 5 of the core set: tinnitus intrusiveness, sense of control, acceptance of tinnitus, concentration, and ability to ignore. This step requires a clear and fully elaborated definition for each outcome domain, moving from an abstract or a vague concept to an operationalized and measurable health-related construct, so that a suitable measurement instrument can then be identified.
A series of 5 focus group–style semistructured discussions were conducted via a Web-based discussion forum, each open for 2 weeks and ending with a vote. The participants included 148 tinnitus experts who completed a preceding e-Delphi survey that had generated the original set of minimum standards. The participants were health care users living with tinnitus, health care professionals, clinical researchers, commercial representatives, and funders.
The Web discussions led to a revision of all 5 original plain language definitions that had been used in the preceding e-Delphi survey. Each revised definition was voted by 8 to 53 participants and reached the prespecified threshold of 70% consensus for all except tinnitus intrusiveness. Although a single definition was not agreed upon for tinnitus intrusiveness, the majority of participants shared the view that the concept should be sufficiently broad to encapsulate a range of subdomains. The examples included tinnitus awareness, unpleasantness, and impact on different aspects of everyday life. Thematic analysis of the 5 Web-based discussion threads gave important insights into expert interpretations of each core outcome domain, generating an operationalized and measurable health construct in each case.
The qualitative data gathered during the Web-based discussion forum provided an important in-depth understanding of the health concepts that had raised a debate during earlier face-to-face meetings. The descriptive summaries and definitions provide sufficient operationalization of those concepts to proceed to the second stage of core outcome set development that is to identify and evaluate suitable measurement instruments. This study supports the use of Web-based peer discussion forums in defining health concepts.
Chronic subjective tinnitus is a condition characterized by a persistent auditory sensation (eg, ringing, whistling, hissing, and buzzing) experienced only by the individual, with no corresponding external sound or source. The characteristics and impacts of tinnitus are highly variable from person to person [
The development of core outcome sets (COSs) can tackle this issue by establishing a common standard and minimum set of recommended core outcomes for use in clinical trials of a specific condition or intervention type as well as for use in other types of research and clinical audit [
The research reported in this paper is part of a longer-term program by the Core Outcomes Measures in Tinnitus (COMiT) initiative that aims to establish a COS for clinical trials assessing interventions for chronic subjective tinnitus in adults [
Graphic illustrating the Core Outcomes Measures in Tinnitus: International Delphi recommendations for core outcome domain sets for each family of interventions widely available for chronic subjective tinnitus in adults. Core outcome domains highlighted in bold are those considered in detail within this paper and represent 5 out of the 9 distinct domains.
In accordance with the COMET handbook [
This study used some important new knowledge relating to the 5 core outcome domains of interest that had been gathered during the previous stakeholder discussions [
Currently, there are no formal guidelines to assist COS developers on how to further conceptualize and define outcome domains [
Although the uptake of Web discussion forums as a platform for COS development is somewhat in its infancy, they have been successfully applied to evaluate the face validity of a new patient-reported outcome measure of treatment response in vitiligo [
In summary, the aim of this study was to specify and define the 5 least well-defined core outcome domains recommended for clinical trials evaluating the effect of sound-, psychology-, and drug-based tinnitus interventions. The 5 core outcome domains in need of discussion were (1) tinnitus intrusiveness, (2) sense of control, (3) acceptance of tinnitus, (4) concentration, and (5) ability to ignore, and these were to be explored using a moderated Web discussion forum with representative stakeholders from the COMiT’ID study. The goals were to establish agreement on a single plain language definition describing each of the core outcome domains and to gain a more in-depth understanding of each concept that would then indicate what sort of questions would need to be asked when assessing each outcome.
This qualitative study used a series of 5 focus group–style discussions conducted via a Web discussion forum. The Web discussion forum was chosen as a practical and cost-efficient research method for engaging with a large and geographically distributed sample of participants, which could not be achieved by face-to-face methods.
Each Web discussion focused on a single core outcome domain that included
This study was conducted under a substantial amendment to the ethical approval originally granted for the COMiT’ID electronic Delphi and consensus studies by the West Midlands—Solihull Research Ethics Committee and Health Research Authority (reference 17/WM/0095, March 2017). This amendment was approved on September 18, 2017.
All registered COMiT’ID participants were invited by email to join the Web discussion forum. We had taken a number of steps to safeguard the relevant expertise (and hence representativeness) of these participants, and the details are published elsewhere [
The invitation and reminder emails contained a link to the discussion forum website [
Eligible participants included members of the public with lived experience of tinnitus, health care practitioners, clinical researchers, and commercial representatives or funders. All the participants were targeted using a purposive sampling approach and had signed a self-declaration statement confirming that they met our eligibility criteria for having expertise on tinnitus. For full details see previous studies by Hall et al [
The website for the discussion forum was developed in partnership with the Tinnitus Hub [
The Tinnitus Talk platform offered a number of positive design features well suited to the research aims and encouraging active engagement in the discussion. The participants could be individually distinguished, but their anonymity was preserved through the use of a pseudonym rather than their true name. The posts were automatically ordered chronologically, which allowed the discussion to be read as a conversation. A direct reply feature quoted the original post, avoiding the need to scroll back and forth through the discussion, and sent a notification to the author of the original post, nudging them to return to the forum and encouraging a natural flow back and forth, similar to face-to-face conversation. The participants could not alter their responses after posting but could add further comments to clarify or change their opinions.
Instructional videos were created to improve usability regardless of experience in Web discussion forums and technical ability. An introductory video on the homepage [
Overall, 3 informational threads were open throughout the study: (1) an
In addition, 5 further threads were used for each of the 5 focus group–style discussions, one for each core outcome domain (
Timeline illustrating the design of the Web discussion forum, with dates indicating the duration of each discussion topic. The Core Outcome Measures in Tinnitus International electronic Delphi survey was in its final (voting) stage at the launch of the forum. FAQ: frequently asked questions; Feb: February; Mar: March; Tech: technology.
All 5 discussion threads started with a reminder of the plain language definition for the outcome domain given during the electronic Delphi survey [
The moderator then gave a brief executive summary of the discussion and proposed a final revised plain language definition. Wherever possible, the revised plain language definition was proposed, refined, and supported by participants during the discussion. Where this was not possible, the moderator developed a revised plain language definition based on the key themes raised during the discussion. These key themes were identified by a preliminary qualitative analysis of the discussion content conducted by the independent moderator as the discussion was unfolding.
Participants were asked to cast their vote according to 4 options: (1) agree with both the summary and definition, (2) agree with the summary but not the definition, (3) agree with definition but not the summary, and (4) disagree with both the summary and definition. Those who disagreed were asked to explain their reason and to recommend any changes. The Web platform used for the forum restricted voting to holding 1 per discussion thread. So, in cases where there was more than one plain language definition, participants were asked to choose their preferred definition. Any disagreements with the summary were collected in the form of written viewpoints instead of an actual vote. Consistent with the preceding electronic Delphi survey, at least 70% agreement across respondents was considered the threshold for accepting the summaries and definitions [
Each discussion was led by an independent moderator (AH) who had experience in leading focus groups and had not been involved in earlier stages of COMiT initiative work or experienced tinnitus herself. However, she did undergo a period of familiarization with the work conducted to date, including listening to recordings of the face-to-face consensus meetings. The moderating style was flexible, becoming more or less active depending on the degree of participant engagement. The aim was to foster a natural conversation style between participants as would happen during a face-to-face focus group. The most desirable style of conversation was one in which participants clearly explained and responded to one another’s personal perspectives to reach a shared understanding. Posting of monologues or isolated messages to the moderator was discouraged. To promote this desired style of discussion, the moderator posed carefully worded (ie, nonleading) questions to encourage participants to elaborate and to give specific examples where more detail would be useful. The moderator also regularly reemphasized the key questions and topics to cover, reinforcing and thanking participants for their contributions and bringing them into conversation with one another on occasions when they had shared either similar or contrasting perspectives. When disagreements or tension arose, the moderator reminded participants of the purpose and context of the discussion forum and the ground rules.
The moderator also maintained a high degree of contact with participants to prompt and remind them of the structure and next stages of the procedure. For example, reminder emails were sent to highlight the next discussion thread opening and to encourage voting before a discussion thread closed. In response to participant feedback, the moderator updated the first post in each discussion thread so that it summarized each key question asked in the forum, with hyperlinks to the corresponding post. The intention was to ease the burden on participants, allowing them to make informed contributions without necessarily needing to read every single post in the thread.
The first stage of the qualitative analysis relating to
The moderator (AH) prepared each executive summary to address 3 aspects of the discussion. One aspect of the discussion concerned key concepts that were discussed and new themes that emerged. The moderator determined what was key based upon themes that were most often mentioned and talked about by participants, and what appeared to be most relevant to the study aim. The second aspect concerned any recurring themes or strongly dissenting concerns that might necessitate a revision to the plain language definition. The third aspect concerned views where a concept for one core outcome domain seemed to converge with another concept that had previously been set aside from further discussion (either during the electronic Delphi survey or consensus meeting) [
In addition to the moderator-led executive summary, thematic analysis was applied to a download of the entire 5 discussion threads conducted by 2 analysts (AH and MK). MK provided an independent perspective as he was naïve to the project. The 2 analysts independently examined each discussion thread separately and, in the order, that they took place. Methods for thematic analysis followed Braun and Clarke’s 6-stage framework [
Of the 627 individuals invited, 251 registered for the Web discussion forum, leading to a recruitment rate of 40.0% (251/627). Of these 251 participants, 119 submitted one or more posts to the discussion forum. Henceforth, these are referred to as
Similar degrees of activity were sustained across the 5 discussion threads (
Participant activity across the 5 discussion threads evidenced by the total number of posts, number of unique individuals who submitted posts (discussants), and the number of unique individuals who voted (voters). Note that the number of posts includes those submitted by the moderator.
Participants (type) | Number of participants in each discussion thread | ||||
Tinnitus intrusiveness | Sense of control | Acceptance of tinnitus | Concentration | Ability to ignore | |
Posts | 160 | 133 | 147 | 142 | 145 |
Discussants | 49 | 44 | 54 | 49 | 47 |
Voters | 20 | 33 | 53 | 8 | 44 |
The level of activity on the Web discussion forum is plotted in
Graph showing the number of user registrations and forum posts over the 8 weeks, from the initial invitation email to closure of the final discussion thread. The hashtag symbol (#) indicates the first mass invitation email, and asterisk symbols (*) indicate subsequent reminder emails. Reminders generally coincided near the start and midway through each discussion thread but were required less toward the end of the study. Apr: April; Feb: February; Mar: March.
The executive summaries created from each Web discussion forum are presented in
In the preceding consensus meetings, what constitutes
It was a recurring theme during both the consensus meetings discussing sound-based and psychology-based interventions that
The plain language definitions for each of the final core outcome domains, and percentage of voters who supported the revised plain language definition where this was changed during the course of the Web discussion forum.
Outcome domain | Original plain language definition | Revised plain language definition | Number of agreements among voters, n (%) | Subdomains |
Tinnitus intrusiveness (n=20)a | Noticing the sound of tinnitus is there and it is invading your life or your personal space |
The extent to which tinnitus invades your life, stresses you in daily situations and prevents you from doing things you want to do The unacceptable and unwelcome interference of internal head and body noise heard only by the individual Being acutely aware of the sounds of tinnitus, feeling that it is invading your life or your personal space, changing your thoughts or actions and negatively impacting on your life |
7 (35) 3 (15) 4 (20)b |
Tinnitus awareness Tinnitus unpleasantness Impact on individual activities Impact on social life Impact on relationships Impact on work (maybe) Annoyance (maybe) |
Sense of control (n=33)a | Whether or not you feel you have a choice in how to manage the impact of tinnitus and feelings caused by tinnitus |
Feeling that you have effective options for managing the impacts of and feelings caused by tinnitus, through an understanding of your condition, learned strategies, and/or available resources |
27 (82) |
Coping |
Acceptance of tinnitus (n=53)a | Recognizing that tinnitus is a part of your life without having a negative reaction to it |
Recognizing that tinnitus is a part of your life and staying neutral toward it in both thoughts and actions |
44 (83) |
N/Ac |
Concentration (n=8) | Ability to keep your attention focused |
The ability to keep your attention focused on whatever you wish |
8 (100) |
Impact on work (maybe) |
Ability to ignore (n=44) | Ability to continue as normal as if tinnitus were not there |
The ability to continue as if tinnitus were not there |
35 (80) |
Conversations Listening |
aNumber of voters.
bSynthesized from comments made by 4 (20) voters who could not choose between the given definitions.
cNot available.
Similarly, the Web discussion forum gave greater clarity on the distinction between
The themes emerging from the discussion thread under each core outcome domain are reported in this section (but in no particular order).
Overall, 2 major themes emerged from the discussion thread: one highlighting the importance of the negative impact of tinnitus on everyday functioning and another highlighting the strength of its negative emotional impact.
A prominent debate concerned whether the concept of
The course of the discussion was guided by the ultimate objective of measuring
I know it’s possible to have noticeable but not disabling tinnitus... A treatment should aim to combat the negative effects. Begs the academic question of whether a treatment might work for people for whom tinnitus was never a problem, just something they noticed.
Consistent with this, discussants voiced support for
Another prominent theme concerned the emotional reaction to the intrusiveness of tinnitus. Discussants emphasized the potential for extreme suffering by describing how insufferable, intolerable, and invasive tinnitus could be. Different definitions of intrusiveness were drawn upon to help understand the depth of the concept, particularly exploring its personification. For example, 1 discussant wrote:
Intrusiveness itself is the inability to keep something unwelcomed be it physical or non-physical such as a thought, from infiltrating your mind/personal space without your permission. Tinnitus is like a burglar that enters your home, holds you hostage, but instead of stealing your belongings, it robs you of your sense of peace. It is therefore, intrusive and dominating.
Consistent with this, discussants voiced support for the inclusion of
I am content with the definition of tinnitus intrusiveness covering all the impacts, unpleasantness and awareness. As I am writing I can hear my tinnitus very clearly. It is not unpleasant as such, but a nuisance I could do without. As I have already said at first my tinnitus was always unpleasant and at its worst I had to really concentrate on not giving way to panic. Those days have gone and I hope they never return.
To conclude, there was no majority agreement on a revised definition, but
Overall, 3 major themes emerged from the discussion thread: 1 highlighting the importance of autonomy in how an individual manages his/her own tinnitus, 1 highlighting the associated sense of empowerment, and 1 highlighting the importance of actively adopting management strategies.
The concept of free agency was related to the
A sense of free agency promotes self-efficacy, and so, one emerging theme emphasized the emotional consequence of feeling in control and being able to make personal choices. One example is the post: “When you are able to manage the impact/response to some extent and continue to enjoy life (by focusing on what you can do and reframing what you can’t) you overcome the helplessness and achieve some sense of control.” This positive sense of self was described by discussants as self-confidence, empowerment, and self-efficacy, whereas the converse was described as being at the mercy of tinnitus, leading to despair.
Another emerging theme emphasized that
To conclude, discussants considered that
Overall, 3 major themes emerged from the discussion thread: 1 highlighting the importance of handling negative reactions, 1 highlighting reconciliation of one’s own identity, and 1 emphasizing the ongoing struggle to maintain acceptance.
The original plain language definition stipulated that
During the discussion of
Some discussants had very strong negative reactions to this concept, demonstrating hostility and frustration toward the idea of being told to
To conclude, the revised definition of
Overall, 3 major themes emerged from the discussion thread: 1 emphasizing the importance of intentional control, 1 highlighting the unavoidable prominence of tinnitus, and 1 drawing attention to the resulting cognitive effort and mental fatigue required when concentrating.
One emerging theme was the importance of being able to control your
Discussion of
The concept of
I would consider a sound-based treatment successful if it restored, even partially, my ability to immerse myself in a task and for this to feel less of an effort than it is now. Ideally, the treatment should reduce the occurrence of cognitive tiredness that makes sustained concentration difficult.
To conclude, the revised definition of
Overall, 2 major themes emerged from the discussion thread: one debating whether the
A key debate focused on whether the concept referred to any change in the tinnitus percept (ie, making the noise easier to ignore) or in the individual’s capability (ie, making the person better at ignoring tinnitus). Some discussants expressed no strong preference: “Either outcome/effect (change in tinnitus or in person) would be good.” However other discussants agreed that a reasonable expectation for sound-based treatments, the only intervention type that this outcome domain is recommended for, is to improve
Concerns were voiced about negative connotations of the word
To conclude, discussants seemed most supportive of an understanding of the
The 5 Web discussion forums brought together a self-selected subset of survey participants, including health care users and professionals with experience of tinnitus. Participants took part in semistructured discussions of 5 complex concepts relating to patient-reported tinnitus-specific complaints that had been voted during a preceding e-survey. The qualitative data collected during these discussion threads provide an important in-depth understanding of each health-related concept, which had not been possible hitherto. The descriptive summaries and revised definitions also provide clarification on aspects of similarity and distinctiveness between core outcome domains. These findings are informative for identifying outcome instruments that putatively assess these concepts and for evaluating their content validity [
Despite the paucity of qualitative data from people with tinnitus [
An increasing number of social and health science researchers are recognizing the internet as a rich source of information. A Web discussion forum facilitates participation by any number of individuals in a way that is not constrained by geographical location or time zone. It offers a rapid and easy way to engage with a large number of participants whilst being more flexible and cost-effective than conventional face-to-face methods. Furthermore, the data are already transcribed, and so they are less likely to contain errors and are immediately ready to analyze (see the study by Ferrante et al [
A major strength of using such a virtual environment is that it allows investigators to conduct real-time qualitative analyses as part of an iterative process in which the participants are actively involved in determining the meaning and significance of findings and where the moderators are able to consolidate, clarify, and resolve any misunderstandings for the purposes of concept definition. The Web format seemed to provide a suitable space that enabled participants to reflect and share ideas about word choice and semantics. With the exception of
Imagine not being part of this forum, but being told by your doctor that some new treatment leads to your being able to ignore the tinnitus, only to find out that it doesn’t for you, and then only worked in the research because the definition was engineered in a particular way... I suppose what I’m saying is that ability to ignore (and the other definitions) have to reflect what most tinnitus sufferers would think if they heard that phrase from their doctor or therapist.
In this way, discussants were not just research participants, they also played an important role in shaping the research
One of the positive themes emerging from the Web forum discussion was the therapeutic benefit of participation. Several discussants living with tinnitus thanked the moderator and research team, expressing a sense of reward in having taken part and describing how it had been personally enlightening and therapeutic. One said: “This is better than I felt 2 years ago, and I must admit that this academic research group with fellow sufferers has been a part of that improvement.” Another said: “Lastly I just wanted to say thanks to ’Manager’ and to COMIT'ID and everyone else for all of this. It’s been really good/therapeutic for me to have been a part of it, and to be able to hear all the experiences of everyone else.” Although Web discussion forums are commonplace for peer support groups [
I don’t think anyone is nit-picking. The purpose of these discussions is to better understand what the core outcomes mean to everyone and reach a better agreement about their definitions. Everyone’s views are valid, equally important to hear and worthy of respect. The whole point of the research is to achieve as broad a consensus as possible and this can only be achieved by exploring and discussing where and how our views align and where they differ.
A mix of health care users and professionals should also help to avoid some of the potential for bias in the design and interpretation of the study if carried out only by a particular stakeholder group (eg, health care users) [
A potential limitation of using Web forums for data collection is that participants need to be computer literate and able to communicate adequately in written English. This may limit the population somewhat and bias self-selection toward those who are more health literate. For this particular study design, it is possible that opening and closing discussion threads in sequence could have resulted in participants who joined later on in the 6-week process missing the opportunity to share their viewpoints on earlier outcome domains. It is also possible that within a different context or procedure, more themes could emerge as it was not possible to ascertain whether data saturation was reached by the forum discussions [
During preparation for the electronic Delphi survey, the COMiT team, with input from health care users, had made a decision to narrowly define the outcome domains, removing broad concepts that were reflected in a number of more narrowly focused outcome domains [
Content validity refers to a number of key attributes of a measurement instrument, namely, how relevant the items are for the construct and target population of interest, how comprehensively those items reflect the construct, how comprehensible the instrument is, and whether it is understood by patients as intended [
Our experience leads us to strongly advocate the use of qualitative methods to ensure concepts are defined to support clear and consistent interpretation by all end users and agreed upon before looking to map outcome domains to measurement instruments. The vast range of different interpretations held for the same domains became apparent during the study, and some major decisions were made as to how the core outcome domains should be conceptualized, defined, and distinguished going forward. Any COS development study following the recommendations of COMET [
Supplementary analysis of consensus meeting discussion on the 4 core outcome domains that were not part of the Web discussion forum.
Instructional Video 1. Hosted on the homepage, walking participants through how to register and create an account.
Instructional Video 2. Hosted on the homepage once logged in, walking participants through how to post, comment on others posts, and adjust their notification settings.
Moderator’s semistructured plans for each discussion thread, including general template of prompts, questions, and posts and tailored discussion packs for each core outcome domain.
Executive summaries and revised plain language definitions for each core outcome domain, including voting results for each discussion thread.
Core Outcome Measures in Effectiveness Trials
Core Outcomes Measures in Tinnitus
Core Outcomes Measures in Tinnitus: International Delphi
core outcome set
National Institute for Health Research
This paper presents independent research supported by the National Institute for Health Research (NIHR) and part funded by an NIHR Senior Investigator award (grant reference NF-SI-0617-10030). The views expressed in this paper are those of the authors and not necessarily those of the National Health Service, the NIHR, or the Department of Health and Social Care. The COMiT'ID initiative acknowledges the support of the NIHR Clinical Research Network in participant recruitment.
None declared.