We ground our analysis in the socialization, externalization, combination, and internalization model of knowledge creation by Nonaka [20], informed by a narrative literature review across health care analytics maturity, workforce turnover, and natural language processing, with gray literature assessed using the authority, accuracy, coverage, objectivity, date, and significance checklist [21]. Gray literature was retained only when no peer-reviewed equivalent existed or when it provided unique industry data. The model describes organizational knowledge as emerging through a continuous cycle of 4 conversion modes:

In a healthy organization, these 4 modes form a self-reinforcing spiral: tacit insights become documented, documentation becomes systematized, and systematized knowledge is internalized by new members who then generate fresh tacit insights [20]. When any mode breaks down, the spiral stalls. In health care analytics, the breakdown is at the very first stage.

The framework’s 3 pillars name the organizational capabilities at stake in the “triple threat”: analytics maturity, workforce agility, and technical enablement. This structure aligns with established models across health care informatics and knowledge management (Table 1).

The HIMSS AMAM provides organizational benchmarks but does not address workforce knowledge retention. The data, information, knowledge, and wisdom hierarchy explains progression from raw data to actionable insights but does not account for institutional memory loss. The 3-pillar framework synthesizes these perspectives, positioning workforce dynamics as the critical enabler connecting data access (analytics maturity) with organizational wisdom (knowledge preservation) [13,20].

In the model by Nonaka, socialization is the foundational conversion mode: the primary channel through which newcomers absorb tacit context that formal training cannot convey [20]. Socialization depends on 2 preconditions: sustained interaction and sufficient temporal overlap between knowledge holders and receivers [22]. It is, in effect, an apprenticeship model requiring years of shared practice.

In the current health care environment, this mechanism has collapsed. The leadership and operational turnover documented above reveals an “apprenticeship window” shorter than the knowledge transfer cycle it must support: 30% of new employees depart within their first year [23], whereas specialized informatics roles require 18 to 24 months to reach fluency [17]. The arithmetic is unforgiving: by the time a new analyst has absorbed enough tacit context to be productive, their mentor may already be gone, and the new analyst is themselves halfway through an average tenure.

High turnover rates fracture the social networks required for mentorship [7,8]. The resulting knowledge loss is compounding: each departure removes a node from the organization’s informal knowledge network, making subsequent socialization attempts less effective because fewer experienced practitioners remain to serve as mentors [24]. Socialization is no longer a viable strategy for resilience.

To survive, organizations must shift reliance from socialization to externalization: converting tacit knowledge into explicit, durable artifacts [25]. However, traditional externalization (writing wikis, maintaining data dictionaries, and composing runbooks) suffers from 2 critical weaknesses. First, it is passive: it requires overworked staff to interrupt their workflow and perform a separate documentation task [26]. In a high-burnout environment where organizations face persistent attrition and talent shortages [27], this discretionary documentation is the first casualty. Second, it is low fidelity: the act of writing down tacit knowledge inevitably loses nuance, context, and the conditional logic that makes institutional knowledge valuable [22]. The result is documentation that exists but does not adequately capture what the departing expert actually knew.

We propose a form of active externalization: one that captures tacit knowledge as a by-product of the daily analytics workflow rather than a separate documentation burden. The mechanism is a new sociotechnical artifact: the validated query triple (see Multimedia Appendix 1 for worked examples). This artifact consists of (1) natural language intent (the clinical business question, eg, “Hypertension readmissions excluding planned transfers”), (2) executable SQL (the technical implementation), and (3) rationale metadata (the “why” behind the logic, eg, “Excluding status 02 per Centers for Medicare and Medicaid Services (CMS) 2025 rule”).

By capturing these 3 components during the act of analytics, we transform the ephemeral work of query generation into a permanent institutional asset [28].

The HITL-KG architecture (Figure 1) functions as a governance forcing function. It inserts a mandatory validation step into the analytics workflow, preventing laundered hallucinations while capturing expert knowledge.

A clinical user’s natural language query (step 1) is translated to SQL and run against the data warehouse (steps 2-3); an expert validation gate (step 4) either returns confirmed insights (steps 5-6) or loops back for correction. Confirmed query triples enter organizational memory (step 7; dashed line in Figure 1), which informs future queries and curates the knowledge base (step 8), closing a continuous learning loop in which best practices evolve rather than remain static.

Figure 2 summarizes the corresponding 6-step validated query cycle, tracing queries from clinical intent to expert validation into durable organizational memory.

A natural language question (step 1) yields a candidate SQL (step 2) that the analyst validates against the system’s explained logic (step 3), looping back if incorrect. Validated triples are stored in organizational memory (step 4) matched by future queries (step 5) and persist across staff turnover (step 6). The dashed arrows in Figure 2 mark the pillar outcomes: storage advances analytics maturity, persistence stabilizes workforce agility, and retrieval increases technical enablement.

The cycle proceeds from query generation, in which a conversational AI system translates the user’s natural language question into candidate SQL [30], to expert validation, the critical moment of externalization where the domain expert confirms or corrects the AI’s interpretation and thereby captures tacit knowledge [31,32]. The validated triple is hashed and stored in organizational memory [33], then retrieved ahead of future probabilistic generation [34]. Multimedia Appendix 1 details each step alongside worked examples.

Organizations have addressed institutional memory loss through several strategies, each with limitations that HITL-KG overcomes.

Code-based semantic layers (eg, data build tool and LookML) encode business logic in version-controlled repositories but suffer from “schema rot,” where electronic medical record (EMR) data models change frequently and maintenance exceeds high-turnover teams’ capacity, misaligning the layer and the underlying data [35]. HITL-KG’s validated query triples share the version control principle but add the rationale metadata that semantic layers lack: not just what a query does but why.

Traditional knowledge management (wikis, data dictionaries, and runbooks) relies on passive capture, where users must stop working to document, which reduces participation and produces inaccurate records under cognitive load [16,26]. Such documentation might suffice for a small, stable query set, but real workloads are neither: more than half of analytical queries recur in large production workloads [36], yet even small databases log hundreds to thousands of distinct query strings [37], so a fixed set of documented queries captures neither the large recurring core nor the evolving ad hoc tail. HITL-KG instead implements active capture across both regimes: the query itself is the documentation, validated at the point of use [28].

Unsupervised AI querying removes human oversight entirely; current NL2SQL accuracy makes this unsafe for clinical analytics [38]. HITL-KG occupies the middle ground: AI generates, humans validate.

The evidence presented below supports the existence and severity of the problem that HITL-KG addresses and the maturity of the technologies it requires. It does not constitute empirical validation of the framework itself, which remains a theoretically grounded, testable proposition whose empirical evaluation is the subject of future work.

The HITL-KG framework is supported by 3 pillars of empirical evidence synthesized from the literature cited throughout this paper.

Analytics maturity describes an organization’s progression in using data and quantitative models for fact-based decisions [39], a construct operationalized by staged maturity models that extend beyond the HIMSS AMAM [40,41]. Health care maturity remains chronically low. Assessments reveal that only 26 organizations worldwide achieved AMAM stage 6 and 13 reached stage 7 by late 2024 [1,2]. Because AMAM assessment is voluntary and self-selected, these figures likely overrepresent analytics-committed organizations, so low maturity may be even more widespread than the counts suggest [42,43]. The corollary is that the vast majority of organizations remain at the lower stages, characterized by fragmented data and limited predictive capabilities [44]. However, maturity is not merely an IT metric; it is a clinical safety predictor. In HIMSS’s companion, the EMR adoption model, which measures the EMR adoption underlying analytics capability, levels 6 and 7 correlate with 3.25 times higher odds of better Leapfrog Hospital Safety Grades [45]. Low maturity creates a “low maturity trap” where data quality issues (such as the 39%-71% missing data rates in cancer databases [46]) remain uncorrected because the experts who understand the context are leaving.

Critically, low maturity is not simply a status; it is a self-reinforcing trap. Organizations at stages 0 to 3 lack the automated monitoring and data governance infrastructure needed to detect their own deficiencies, so poor data quality goes unrecognized. The clinical consequences are measurable: one Medicare accountable care organization that implemented analytics to overcome electronic health record fragmentation reduced readmission rates from 24% to 17.8% and achieved US $1.6 million in cost savings [47]. However, data interoperability remains a leading obstacle to analytics adoption [3,4]. Barriers including employee resistance to change and lack of organizational readiness further stall data-driven initiatives [48,49]. The trap thus compounds: each year of delayed investment widens the gap between what the organization could know and what it does know.

Workforce agility is a workforce’s capacity for proactivity, adaptability, and resilience under change [50,51]. In health care analytics, the evidence below documents the instability that systematically erodes this capacity. The cost of turnover in informatics is higher than in standard IT. Knowledge loss can cost up to 3 times the annual salary [24,52]. In clinical informatics specifically, replacement costs range from US $50,000 to US $230,000 per departing employee, with a loss of 39 staff members in a single department in 1 year incurring a total cost of up to US $8.97 million [53]. With 30% of new employees leaving within their first year [23], health care IT professionals spend a limited portion of their employment at full productivity as specialized roles require 18 to 24 months to reach fluency [17]. This “revolving door” prevents the accumulation of the “collective knowledge structures” required for complex task performance [13].

The workforce crisis operates at multiple reinforcing levels, as detailed in the Theoretical Grounding: Socialization, Externalization, Combination, and Internalization and the Unstable Workforce section: leadership churn, operational shortages, and foundational attrition collectively prevent the sustained mentorship that socialization requires. This multilevel instability ensures that the temporal overlap between experienced practitioners and newcomers rarely materializes.

Technical enablement denotes the technologies and practices that let nonspecialist domain users access and analyze data without heavy reliance on central IT [54,55]. NL2SQL technology has matured along a clear accuracy gradient: general-purpose models achieve approximately 65% execution accuracy on health care–specific benchmarks such as MIMICSQL [56], domain-adapted systems such as MedT5SQL reach 80% accuracy [57], and architecturally specialized approaches combining large language models (LLMs) with structured knowledge representations achieve 94% accuracy on the same benchmarks [58]. While insufficient for unsupervised clinical deployment [38], this gradient demonstrates that the generation engine required for HITL-KG is viable within a human-validated workflow. Notably, high benchmark accuracy does not guarantee real-world generalization: evaluation on more challenging dataset splits reveals accuracy drops from 92% to 28% [59], reinforcing the necessity of the human-in-the-loop architecture. Health care–specific natural language interfaces such as Criteria2Query achieve fully automated cohort query formulation in 1.22 seconds per criterion [60], with over 80% of clinical users indicating willingness to adopt such tools.

However, NL2SQL is an enabler, not a solution in itself. Its deeper significance lies in the organizational prerequisites it demands. For an AI system to translate a natural language question into a correct SQL query, the organization must first establish validated data models, explicit business logic definitions, and codified domain terminology [3,4]. This reframes NL2SQL from a convenience tool into a catalyst for the kind of systematic knowledge externalization that the HITL-KG framework requires. The interface bridges the semantic gap between clinical experts and technical schema, allowing nontechnical domain experts to interact with data alongside broader modernization efforts [61-64]. The very difficulty of making NL2SQL work correctly thus becomes a governance opportunity: it compels organizations to surface and formalize tacit knowledge that would otherwise depart with its experts.

To operationalize the framework, we propose a 3-pillar assessment rubric (Table 2) that enables health care organizations to evaluate their current position across each pillar and identify compounding vulnerabilities.

As noted, the maturity models and knowledge management frameworks in Table 1 assume linear progression by a stable workforce [1]. In practice, an organization that reaches AMAM stage 5 but regresses to stage 3 after the departure of 2 senior analysts has not failed to mature; it has failed to be resilient. A recent systematic review of organizational resilience measurement in health care found no consensus on what to measure [65]. Only 4 instruments have been developed specifically for health care, and only 2 have been validated [66]. The 3-pillar assessment rubric addresses this gap by organizing evidence-based indicators around the domains where institutional amnesia operates, measuring not just where an organization stands but also how vulnerable it is.

Each indicator is scored as being of low, medium, or high strength using evidence-based anchors from the literature reviewed in the The Evidence Base: 3 Pillars section. Organizations scoring predominantly as “low strength” across multiple pillars face the self-reinforcing degradation cycle that the framework identifies as the central threat. Table 2 summarizes the 9 indicators; the full rubric with scoring anchors and evidence can be found in Multimedia Appendix 2.

The “schema coupling” indicator can be operationalized through continuous analytic integration: treating validated query triples as software assets within a continuous integration and continuous delivery pipeline that detects data and schema drift [35,67,68]. When a data warehouse schema is updated, the system automatically reruns stored queries and flags failures, transforming institutional memory into a living test suite [69,70].

The rubric complements rather than replaces the AMAM. Where the AMAM measures the sophistication of analytical capabilities at a point in time, the 3-pillar assessment rubric reveals how vulnerable those capabilities are to the compounding effects of turnover, low maturity, and technical barriers. Used together, they provide a 2D view of analytics health.