Co-creation has emerged as a powerful approach for fostering collaboration and innovation across various disciplines [1]. Co-creation has the potential to produce new or improved tailored practices and solutions, which address complex challenges and generate meaningful outcomes for a defined need [2,3]. In health care, co-creation is thought of as a potential way to bridge the bench-to-bedside divide [4,5] by developing interventions that are more acceptable and contextually appropriate, thereby potentially enhancing their effectiveness and impact on health outcomes [6]. It can be used to address complex problems in public health (eg, the obesity epidemic, persisting poverty, or food insecurity), which are particularly resistant to resolution [1,4]. Additionally, co-creation has been widely used in health care at a rapidly expanding rate [7].

At the core of co-creation is the active engagement of multiple stakeholders in a collective intelligence process to collaboratively design and implement projects, processes, services, and solutions [7-9]. Rooted in the potential for tapping into diverse perspectives, skills, expertise, and experiences, co-creation stands as a dynamic approach to fostering innovation and stakeholder engagement [10] and an effective means to avoid top-down approaches or mere stakeholder consultations [5].

There is a growing recognition of co-creation as a valuable methodology, yet, to date, there is no common practice or standardized approach [1,11]. Co-creation has increasing traction in terms of involving stakeholders, but the extent to which this creates a fundamental change in practices is still unclear, and there is a strong potential that it will remain tokenistic [12]. Major challenges remain and need to be confronted to make co-creation trustworthy and unlock its full potential [1,2,11-13].

A well-documented challenge in co-creation research is its fragmentation. Slattery et al [14] describe the literature as complex, contradictory, and poorly synthesized, noting that inconsistent terminology makes it difficult to retrieve and understand relevant studies. Agnello et al [1] further highlight this fragmentation, attributing it to inconsistencies in terminology and inadequate cross-referencing and knowledge sharing among co-creation researchers. Smith et al [15] found that researchers operationalize co-creation in various ways, while Grindell et al [16] emphasize the interchangeable and often ambiguous use of the co-creation, co-design, and coproduction terms. These issues hinder the identification of best practices and the establishment of robust evidence for co-creation.

Research methods provide a structured and systematic approach to gathering and analyzing data, ensuring that findings are valid, reliable, and generalizable. Detailed insight into these processes is essential to truly understand and assess the quality of research [14,15,17-19]. Furthermore, well-documented research methods enable others to replicate studies and build on findings, which is crucial for advancing knowledge in any field [17]. Using appropriate methods allows researchers to control potential biases that might affect their results, making their findings more trustworthy and credible [20].

However, co-creation has been plagued with poor reporting of the steps involved in the process, in particular the methods used. Smith et al [15] urged the research community to improve reporting by clearly and consistently documenting methods. Similarly, Slattery et al [14] highlighted the need for better reporting on methods used in co-creation as a foundation for understanding its effectiveness and cost. An et al [13] also discovered limited reporting of methods in the co-creation literature and Lee et al [21] drew attention to the difficulty in evaluating co-creation due to a lack of systematic comprehension of methods. This challenge is also documented by Durugbo and Pawar [22], who highlighted the absence of a detailed description of the methods used to facilitate the interaction between the convener and the cocreators.

More than any other aspects of research, methods have the potential for adaptation or reuse across different contexts, research questions, and disciplines [6,17,23]. Consequently, poor reporting or restricted access to reliable methods creates inefficiencies and can slow progress. Given the numerous benefits of co-creation, it is surprising that there are no clear recommendations regarding methods appropriate for ensuring accurate, impactful, and trustworthy co-creation. The transferability of these methods suggests they could adapt the co-creation process to various contexts and research questions. However, poor reporting and limited access to trustworthy evidence about co-creation methods pose significant barriers, making it difficult to contextualize research, evaluate co-creation, replicate studies, or scale up co-creation research [1,14,15]. This lack of clarity also limits researchers’ ability to build on existing knowledge and apply best practices [24].

While some studies have highlighted individual methods for co-creation, a comprehensive inventory or systematic overview of the full range of methods used in co-creation is lacking. Researchers striving to apply co-creation often lack the time to systematically search, retrieve, review, and compare all the available literature to develop a thorough and critical sense of the varied methods [1,25]. Therefore, a systematic methods overview could increase clarity and enhance collective understanding of which methods have been used in co-creation [25,26].

As Slattery et al [14] noted, there is considerable value in a greater synthesis and differentiation of the co-creation literature, highlighting that methodological and theoretical barriers in co-creation often prevent systematic comparison using traditional means. Consequently, this study aims to address this gap using a nontraditional means, by systematically inventorying and analyzing methods used in co-creation. By providing a comprehensive overview and analysis, this research will lay the groundwork for developing a robust taxonomy and compendium of co-creation methods in future research.

We expanded on the definition of participatory methods by Vaughn et al [27] to establish a definition of methods used in co-creation: “Co-creation methods encompass a diverse range of tools, activities, approaches, and techniques strategically employed across the entirety of the co-creation process. These methods serve various purposes, including but not limited to data collection, facilitation, recruitment, reflection, data analysis, and dissemination, allowing for flexibility in achieving diverse objectives.”

We defined co-creation according to Agnello et al [1]: “Co-creation is any act of collective creativity that involves a broad range of relevant and affected actors in creative problem-solving that aims to produce a desired outcome.”

To analyze co-creation methods, we calculated method co-occurrence to understand how methods are combined in co-creation projects. The results were visualized using a Sankey diagram to illustrate the relationships between methods. We also conducted a bibliometric analysis of the included literature to investigate methodological trends and fields represented in the literature. Additionally, we examined the overlap between methods reported in academic literature and gray literature to determine if practitioners engage in co-creation similarly to those documented in academic research.

To understand how methods were combined, we analyzed co-occurrence in the academic literature by examining which methods appeared together in titles or abstracts. We used Python to map method names and count their co-occurrences, ensuring accuracy by marking and skipping already identified combinations. RAWGraphs (version 2.0; DensityDesign Research Lab) [31] was used to create Sankey diagrams that visualized these co-occurrences, removing method names with no co-occurrence. For ease of visualization, methods were categorized into three groups: (1) qualitative; (2) participatory; and (3) quantitative, mixed, and ethnographic. Separate Sankey diagrams for each group were produced to simplify and clarify the visualization of method combinations and their prevalence in co-creation research.

To gain insights into the methodological paradigms and fields within the academic literature, we performed a bibliometric analysis using VOSviewer (The Centre for Science and Technology Studies) [32]. This tool helped us construct and visualize bibliometric networks to assess the prevalence of different methodologies and research areas. By analyzing the source landscape and methodologies, we aimed to identify dominant fields and approaches represented in the included academic literature. Detailed steps for this analysis using VOSviewer are provided in Multimedia Appendix 2.

To determine if co-creation practices differ between academic and gray literature, we compared methods from both sources using a Venn diagram. We identified overlaps and unique methods by comparing the 2 lists of methods in Microsoft Excel using Conditional Formatting. Methods were categorized into three groups: (1) exclusively in academic literature, (2) exclusively in gray literature, and (3) found in both. We calculated the number and percentage of methods in each group and visualized these findings in a Venn diagram. This comparative analysis highlighted similarities and differences in co-creation approaches between academic and practitioner-focused literature.

We assessed the relative presence of methods in the different types of literature by examining their frequency of occurrence. In academic sources, the frequencies ranged from 349 to 1, reflecting the varying specificity of method names. For example, “interview” appeared in 124 papers, while the more specific “Lego serious play” method was found in only 1 paper. Examples of the most and least frequent methods in academic literature are listed in Table 2, with the full set of methods and their frequencies provided in Multimedia Appendix 4.

In terms of gray literature, frequencies were manually tallied based on the extraction tables, ranging from 12 to 1. Examples of the most and least frequent methods sourced from gray literature are listed in Table 3, with the full set of methods and their relative frequencies provided in Multimedia Appendix 5.

Of the 252 methods extracted from academic literature, 91.3% (n=230) co-occurred within the same title or abstract. The resulting Sankey diagram visualizing this co-occurrence is too large to include in this manuscript and is therefore presented as an open-access figure on Zenodo [33], allowing readers to delve into the detailed co-occurrences and gain a deeper understanding of methods used in co-creation research. The full dataset of co-occurrence is provided in Multimedia Appendix 6. For clarity, the methods were categorized into three groups: (1) qualitative methods; (2) participatory methods; and (3) quantitative, mixed, and ethnographic methods, which are each visualized in separate Sankey diagrams.

The co-occurrence analysis of the qualitative methods, shown in Figure 2, features 22 source methods (qualitative) on the left and 149 target methods (multiple types) on the right side of the Sankey diagram. This analysis revealed that focus group, one of the most frequently used methods, often co-occurs with other qualitative methods like group discussion, interviews, and in-depth interviews, as well as qualitative analysis such as content analysis and thematic analysis. Additionally, focus group commonly co-occurs with participatory methods like prototyping, photo voice, storytelling, and various ethnographic methods, including participant observation, field notes, and narrative.

The co-occurrence analysis of the participatory methods, visualized in Figure 3, includes 42 source methods (participatory) on the left and 132 target methods (multiple types) on the right side of the Sankey diagram. This analysis highlighted that some of the most frequently used methods, such as photo voice, and prototyping, often co-occur with other participatory methods like experience prototyping, concept mapping, and participatory mapping. Photo voice is also commonly linked to narrative and group discussion, and a mix of ethnographic and qualitative methods, including thematic analysis, field observation, and various types of interviews. Additionally, deliberative workshops frequently co-occur with other deliberative or participatory methods such as user committee, participatory budgeting, fuzzy cognitive mapping, and collective reflection.

The co-occurrence analysis of quantitative, mixed, and ethnographic methods is presented in Figure 4, with 24 source methods (quantitative, mixed, and ethnographic) on the left and 156 target methods (multiple types) on the right side of the Sankey diagram. This analysis showed that the most frequently used method, survey, often co-occurs with other high-frequency methods like questionnaire and narrative. Survey is also strongly linked to qualitative methods such as focus groups, group discussions, various types of interviews, and qualitative analysis methods. The ethnographic method, field notes, is associated with a combination of ethnographic and participatory methods, including participatory observation, participatory reflection, and group model building.

The VOSviewer co-occurrence analysis of keywords related to methodologies in the title and abstracts of the academic literature is visualized as a network map in Figure 5. This map includes 34 keywords, interconnected a total of 385 times, and organized by VOSviewer into 4 clusters [32]. Cluster 1 (in yellow) includes terms related to participatory research, community-based participatory research, and community engagement or participation. Cluster 2 (in teal) focuses on various forms of co-creation, value co-creation, and design methodology approaches. Cluster 3 (in pink) includes coproduction, co-design, public and patient involvement, and user involvement. Cluster 4 (in blue) groups together different forms of participatory action research and action research. The main terms across these 4 clusters are co-creation, coproduction, co-design, participatory action research, and participatory research. Further details about these clusters are provided in Table 4.

The VOSviewer co-occurrence analysis of the keywords related to fields in the title and abstracts of papers in the database is visualized as a network map in Figure 6. This map includes 150 keywords, interconnected 181 times and organized by VOSviewer into 48 clusters [32]. Focusing on the 8 most prominent clusters, which contain at least 5 keywords each, cluster 1 (in red) includes health service research, community health services, medical education, occupational therapy, primary health care, and social work. Cluster 2 (in blue) covers health promotion, health policy, and school health services. Cluster 3 (in green) includes dementia, emergency nursing, home care services, home nursing, long-term care, and terminal care. Cluster 4 (in purple) features palliative care, emergency medical services, and hospital emergency services. Cluster 5 (in orange) focuses on women’s health, maternal welfare, rural health, and supported employment. Cluster 6 (in yellow) includes humanities, pediatric hospitals, organizational innovation, quality health care, and vaccinations. Cluster 7 (in brown) is centered on health and nursing education. Cluster 8 (in teal) covers family health, child development disorders, learning disorders, and self-evaluation programs. Full details on these clusters are provided in Table 5.

This study extracted a total of 956 methods used in co-creation, which were then categorized into 3 distinct categories based on their sources. Of these, 16% (155/956) were found exclusively in academic literature, 74% (704/956) were sourced solely from gray literature, and 10% (97/956) were identified in both academic literature and gray literature. The comparison of these categories is visualized in Figure 7, and a complete list of methods per source type is provided in Multimedia Appendix 7.

This study is the first systematic methods overview of co-creation, identifying and delving into both academic and gray literature to extract 956 methods from 2627 sources, analyzing these methods and their interrelationships within and across source types. This study can facilitate the identification of methods suitable for co-creation endeavors, while also highlighting discrepancies between co-creation in research and nonacademic contexts. By offering a comprehensive and reliable snapshot of co-creation and its associated methods, this research enhances transparency regarding how co-creation is executed in both research and practice in the health research fields. The study serves as a foundational resource, providing an evidence-based and systematic inventory of methods, as well as shedding light on which methods are commonly used in conjunction with one another across the health research fields. This innovative approach to conducting a systematic methods overview enables the comprehensive analysis of a vast amount of literature and methods, offering valuable insights into the landscape of co-creation practices and methods globally and across disciplines.

A notable strength of this study lies in its utilization of methods sourced from a published and high-quality database of co-creation literature spanning from 1970 to 2022. This literature represents the largest known dataset of methods used in co-creation. This study serves as a foundational work that can be used to elucidate the interrelationships between different co-creation methods and co-creation processes. This study adopts an unbiased approach by providing a comprehensive overview of current co-creation practices, without prescribing specific methods or recommendations. By providing an overview of current co-creation practices, this study establishes a platform for further research and analysis into the various ways in which people design and execute co-creation. Furthermore, this study reveals a stark difference between academic and practitioner approaches to co-creation, highlighting the importance of understanding and bridging these disparities.

In the realm of research methodologies, each type has its unique characteristics and applications. Recently, Messiha et al [11] discussed how naïve realism (a singular objective reality) often favors quantitative methods, while relativism (multiple realities exist) tends to favor qualitative approaches. For instance, approaches such as qualitative, quantitative, and ethnographic studies offer different lenses through which to explore phenomena. Qualitative methods delve into the subjective experiences and perspectives of individuals, capturing rich narratives and deep insights; valuing methodological pluralism and diversity rather than universal truths [34]. Quantitative methods, on the other hand, focus on numerical data and statistical analysis to uncover patterns and relationships to uncover universal truths about reality. Finally, participatory methods emphasize collaboration between researchers and stakeholders; unlike traditional research methods, which prioritize objectivity, participatory methods prioritize engagement and collective decision-making [27]. Furthermore, Messiha et al [35] found that applying methodological principles of critical realism (recognizes independent reality and acknowledges multiple perspectives), namely using multiple method types to understand complex phenomena, enriched the evidence base for co-creation in public health research.

This study reveals that co-creation represents a fusion of various types of methods, and well-known methods for analysis, such as thematic analysis, are combined with qualitative methods like interviews. Co-creation combines methods for qualitative inquiry and analysis, quantitative analysis, ethnographic observation, and participatory collaboration. This amalgamation of methods offers great potential for generating innovative solutions, fostering stakeholder engagement, and addressing complex challenges. However, the convergence of multiple methods also introduces challenges, including potential divergences in processes and outcomes between co-creation projects. Overall, the mixed nature of co-creation presents both opportunities and challenges.

While analyzing the data within this study, a clear gap exists between the practices reported in academic literature and those used by practitioners in nonacademic settings. This divergence may stem from various factors, including differences in reporting styles, publication platforms, and the emphasis placed on outcomes versus process documentation. Academic publications often prioritize the reporting of research outcomes and their impact, allocating limited space for detailing the intricacies of the co-creation process. Conversely, practitioners may opt to disseminate their co-creation endeavors through alternative channels, such as reports, case studies, or web-based platforms, where they have more flexibility to document their methods. However, this discrepancy in reporting practices poses a significant challenge for the scientific community, which is well documented by various researchers who are calling for researchers to transparently document the entire process of co-creation, including the methods [6,14,15]. Without detailed documentation of each method used in the co-creation process, replication and validation of findings become challenging. Furthermore, the lack of transparency in documenting co-creation methods hinders knowledge sharing and collaboration among researchers and practitioners. This gap underscores the importance of adopting standardized reporting guidelines and best practices for documenting co-creation, regardless of the publication platform.

One notable observation is the prevalence of co-creation methods outside of academic literature. While academic research may provide valuable insights into academic rigor, and application of theory, it appears that the majority of the co-creation methods are published on nonacademic platforms. This raises questions about the transparency and documentation of co-creation processes. Efforts to bridge the divide between academic and practitioner perspectives on co-creation are essential for advancing knowledge, promoting collaboration, and maximizing the impact of co-creation initiatives. By fostering greater transparency, documentation, and knowledge sharing, researchers and practitioners can enhance the rigor and reproducibility of co-creation.

Grindell et al [16] underscore the distinctiveness of collaborative approaches like co-creation, co-design, and coproduction, contrasting them with traditional applied health research methodologies; emphasizing that these coapproaches are designed to foster meaningful engagement with a broader range of cocreators, including those not typically involved in research. However, the network analysis of the methodologies in the final dataset reveals a different story. While coapproaches and participatory research methodologies are prevalent, the actual methods used in these projects fall short of truly embracing collaborative engagement. This highlights a disconnect between the intended methodological goals and the practical application of collaborative methods.

Our findings align with recent work by Slattery et al [14] and Smith et al [15], who reported similar trends in the types of methods being used. Slattery et al [14] observed that the most frequently used methods were focus groups, interviews, and surveys, with less frequent use of methods like citizen juries and voting. Smith et al [15] also found that commonly reported methods included interviews, focus group discussions, surveys, discussion meetings, and workshops. This consistency across studies is encouraging, as it underscores a broader trend in the field and validates our findings about the dominant reliance on certain qualitative methods.

Louise and Annette [12] highlight the significance of using participatory methods, emphasizing their potential applicability to intervention development and their capacity to transcend traditional boundaries. When executed effectively, participatory methods offer numerous benefits, including genuine stakeholder involvement, a holistic understanding of multiple perspectives, an iterative and investigative approach, and a commitment to enacting meaningful change for those directly affected by the outcomes [12]. Maenhout et al [6] also found that co-creation is feasible with adolescents with intellectual disabilities if the correct methods are selected, specifically creative methods. Furthermore, studies favoring creative methods found that design and participatory approaches effectively engaged cocreators with their emotions, and abilities, and retained their involvement throughout the process [16,36]. Prototyping methods were found to be valuable for translating knowledge into tangible objects, while visual design methods facilitated the rapid communication of ideas in an accessible manner. The utilization of creative methods was observed to promote a shared understanding of the problem and identify critical needs, thus addressing power differentials and fostering a sense of ownership among stakeholders [16].

Grindell et al [16] illustrated how researchers rely on qualitative research methods (such as focus groups, observations, and interviews) when other, more creative methods, can achieve the same aim (such as role-playing, personas, and user journeys). These findings underscore the importance of using participatory and creative approaches in co-creation processes to build trust, confidence, and collaborative solutions. Considering the documented benefits of using these types of methods, this study reveals that the way people are currently cocreating is leaning heavily on qualitative methods. While qualitative methods provide rich insights into individual subjective experiences and perspectives, incorporating diverse approaches such as creative or participatory methods could further enhance stakeholder engagement, understanding, and problem-solving in co-creation processes.

There is a growing concern regarding participation fatigue in co-creation, alongside the risk of heightened inequity when only specific demographic groups engage, potentially leading to disengagement [37,38]. Amundrud et al [38] stress that co-creation should empower individuals to participate actively as citizens with the chance to genuinely influence outcomes. Various methods, including focus groups, one-on-one discussions, and creative workshops, are utilized for involvement and engagement. However, patients increasingly advocate for greater autonomy in determining the extent and nature of their involvement in research processes. In contexts like coproduction and participatory action research, traditional distinctions between researchers and participants are challenged, resulting in blurred boundaries and a reassessment of power dynamics [12]. This study reveals that the multi-method approach to co-creation primarily integrates various qualitative methods, while more creative or participatory methods are often found in nonacademic gray literature. Consequently, academics appear to rely solely on qualitative methods and rarely draw from nonacademic sources for guidance and inspiration. This heavy reliance on qualitative methods may increase the risk of disengagement, reassessment of power dynamics, or participation fatigue.

This systematic inventory of methods used in co-creation provides an interesting dataset that should be mined, for instance, to make further distinctions between the methods and their intended purpose and impact. We intend to use it to develop a taxonomy facilitating more consistent and better reporting, which other reviews highlighted as an important gap [14,15]. In turn, this could facilitate a more robust evaluation of the impact of co-creation, which is fundamental for the progression of this methodology [15,16]. Co-creation researchers could make use of such a taxonomy or apply our novel artificial intelligence and data science approach, to extract additional information about the methods used in co-creation, mirroring the approach of the Human Behavior Project for behavior change science [39]. This approach could help identify specific clusters of methods or sequences of methods that are particularly effective in certain contexts, for certain purposes, and in certain phases of the co-creation process. The data generated in this study also raises interesting questions about how and when collective intelligence and creative methods are best placed or used, and how scientific practices might have to change to provide room for effective collective creativity based on evidence.

Although we achieved our objectives by conducting a systematic overview of methods, our approach has some limitations including single screening of the gray literature and the lack of full-text screening of the academic literature. However, we recognize that this work was conducted in a peer-reviewed, precurated database of co-creation literature, and the complexity of the screening of gray literature was not high enough to necessitate double screening and extraction.

This study has yielded the first systematic overview of methods utilized in co-creation, intending to provide researchers and practitioners with a comprehensive understanding of methods used in co-creation to date. By increasing awareness of methods used in co-creation, we seek to unlock the full potential of co-creation and contribute to its advancement as a transformative methodology for research and practice. The analysis of methods sourced from both academic and gray literature revealed a rich array of methods used in co-creation, spanning participatory, qualitative, quantitative, mixed, and ethnographic approaches. This diversity underscores the versatility of methods, which can adapt to varying study objectives, target groups, contexts, and other influencing factors. However, it also highlights the need for more detailed guidance on method selection, method grouping, and application to ensure co-creation remains effective and meaningful.

This systematic exploration of co-creation methods offers valuable insights for individuals currently involved in co-creation, as well as those aspiring to participate in it. By illuminating the diverse landscape of co-creation methods, this study aims to enable researchers and practitioners to make informed decisions and enhance methodological rigor and innovation in co-creation. Through continued research and collaboration, we can further advance co-creation as a dynamic and impactful approach for addressing some of the most pressing complex or wicked public health challenges.