← Back to Publications Archive

Building Trustworthy AI for Health: What Clinicians Told Us

Introduction: From Theory to Practice

Over the past seven months, we have traced the intellectual journey of ethical AI in healthcare. We've made bias visible, built readiness frameworks, opened black boxes with transparency tools, mapped regulatory futures, explored the human work of adoption, and looked ahead to emerging ethical questions at the edge. Each piece has asked: What does ethical AI look like in theory?

March is the moment we answer a deeper question: What does it look like in practice?

This month, we share what we heard in conversations with clinicians, healthcare operations specialists, and digital health innovators. These were qualitative interviews, not a clinical study, and they are not results from a hospital deployment. We also participated in the Synapse Life Science competition, where we worked with university students and mentors to test our thinking. We gathered direct feedback from emergency department leaders, family physicians, rural healthcare teams, Indigenous health advocates, and hospital quality officers, all wrestling with the same challenge: How do we deploy AI in ways that feel trustworthy, locally grounded, and genuinely fair?

Their answers surprised us. Their wisdom will surprise you too.

The Real Question Clinicians Ask

When we first approached clinicians with ethical AI solutions, we expected them to ask about algorithms, metrics, and fairness indices.

They asked something different: "Show me the bias in my own data first."

One emergency physician we interviewed captured this perfectly. As they described it to us, clinicians are open to ethical AI but want hard evidence of bias in their own setting before changing practice or adopting tools. They don't want principles. They want proof.

This simple insight reframed everything we were building.

It meant our platform needed to start with retrospective analytics - evidence-first mode. Before recommending any intervention, our Data Diversity & Fairness Auditor (DDFA) would need to answer a very specific question: Is there measurable variation in care across patient groups in your setting, and is it clinically meaningful enough to warrant action?

Equally important, it needed to be able to report "no evidence of bias found" with the same confidence it reports problems. Bias detection isn't a fishing expedition. It's an investigation.

Where Bias Actually Shows Up: What We Heard From Clinicians

Across our interviews with emergency physicians, family medicine leaders, rural care teams, and Indigenous health advocates, a clear picture emerged: bias is multi-layered, contextspecific, and often invisible without direct evidence

Pediatric Emergency Care: The Subjectivity Problem

The clinicians we interviewed noted that objective algorithms - like sepsis early-warning tools in EHR - appear neutral on the surface. But they can be fed by biased sensors (e.g., oxygen saturation readings that are less accurate in darker skin) or incomplete social determinants of health data (Cross et al., 2024; Ghassemi et al., 2021).

More concerning is what happens with subjective assessments. Pain is explicitly acknowledged as bias-prone in pediatric care. Stoic versus expressive children are treated differently. Staff judgments strongly influence pain management decisions. A child's apparent comfort level - shaped by cultural background, family norms, and previous medical experiences - can directly affect whether they receive appropriate analgesia.

The insight: Objective algorithms need skeptical scrutiny. Subjective assessments need structured oversight.

Rural and Northern Communities: Geography as Structural Bias

Geography emerged as a powerful driver of inequity in clinical care. Northern communities often employ a "fire hose" approach - everyone with fever gets IV antibiotics and broadspectrum coverage - because retrieval times can be an hour or more by medevac.

Wound care and amputation patterns have been traced not only to clinical judgment but to distance from limb-salvage centers. Bias, resources, and geography intertwine in ways that standard algorithms cannot untangle without explicit attention to regional context.

The insight: Bias in rural settings is not just about clinician judgment. It's embedded in resource allocation, transport infrastructure, and the structural reality of geography.

Family Medicine: The Data Fragmentation Problem

Family medicine leaders in urban and suburban settings, highlighted how data gaps force clinicians to fly blind. Fragmented records, poor histories, and incomplete social context mean clinicians rely on personal heuristics - and biases - more than they would with complete information.

Add cultural norms, language barriers, and patient perceptions shaped by previous experiences, and bias becomes a conversation between incomplete data and unconscious assumption (Dankwa-Mullan et al., 2024; Cross et al., 2024).

Frequent emergency room users and complex patients are known informally, which can skew triage and treatment decisions. But without structured data, those patterns remain anecdotal

The insight: Fairness isn't just about correcting biased algorithms. It's about giving clinicians better data so their judgment is less dependent on bias.

Indigenous Youth: Data Sovereignty and Cultural Safety

Advocates and practitioners working with Indigenous communities and BIPOC clinicians brought the most sobering perspective: bias isn't just clinical; it's structural and historical.

Longstanding data extraction without community benefit has created deep distrust of healthcare research and technology. Indigenous wellness measures - like clinically and culturally validated tools for Indigenous youth - face sustainability challenges partly because data infrastructure and governance remain in the hands of mainstream institutions.

Bias emerges when mainstream systems ignore Indigenous definitions of wellness, context, and sovereignty. It emerges when datasets are too small or misinterpreted without cultural expertise. And it persists when technology is imposed without co-design.

The insight: For ethical AI to work in Indigenous and BIPOC contexts, data sovereignty, cultural safety, and co-design are non-negotiable.

What Clinicians Actually Want: Evidence, Simplicity, Trust

From these conversations, five consistent themes emerged about what makes ethical AI actually useful in practice:

1. Evidence First, Advocacy Second

Clinicians want hard data from their own setting before changing practice. In effect, clinicians told us that if you can show them, in their own data, that certain groups are systematically under- or over-served, they will listen and consider changing practice.

This suggests a staged approach: analytics first, co-interpretation with clinicians, then, much later, possible point-of-care interventions (a longer-term roadmap direction, not available today). Don't ask clinicians to change practice based on external research. Show them their own patterns.

2. Low Cognitive Load, High Clinical Relevance

Workflows are fragile. Any ethical AI solution must be quiet, embedded, and seamless, surfacing insights at the right moment - when a pain score is charted, when a sepsis alert fires, when a patient is triaged.

(These are design directions for a possible future version. Today's DDFA is retrospective and does not surface prompts in the clinical workflow.) Any future prompts must be explained in plain clinical language: "Patients like this in your setting have 2× higher rates of missed sepsis" rather than obscure fairness indices or demographic parity scores.

3. Local Validation, Not Generic Scores

Clinicians across specialties rejected one-size-fits-all fairness metrics. Bias in pediatric emergency care looks different from bias in rural primary care, which looks different from bias in mental health or maternal health.

DDFA must support intersectional, population-specific analysis - age, race, language, geography, cultural context - not generic fairness scores applied across settings.

4. Governance, Privacy, and Transparency as Enablers

There is strong sensitivity to privacy, security, and - especially around AI summarization of clinical notes and patient portals. But clinicians also understand that organizational AI governance committees and fairness policies are not obstacles; they are partners.

Institutional examples of strong AI governance support were flagged as a positive enabler of ethical AI adoption, not a barrier.

5. Multiple Personas, Multiple Pathways

Organizations need different solutions for different stakeholders. Hospital executives want readiness assessments and governance frameworks. Quality officers want bias evidence and equity dashboards. Clinicians want point-of-care prompts. IT teams want systems that integrate with existing EHRs

One platform, multiple entry points.

High-Yield Use Cases: Where Ethical AI Wins

From our interviews, several priority application areas emerged as both clinically compelling and technically feasible:

Sepsis and Acute Deterioration

Tools that monitor vital sign trends over time and trigger rechecks - especially for at-risk groups or patients languishing in waiting rooms—are seen as valuable. Bias mitigation here is less about race/gender directly and more about ensuring no group systematically deteriorates unnoticed.

Pain Assessment and Management

Pain is one of the clearest first use cases. A DDFA that flags when a patient's reported pain scores and interventions are consistently misaligned - high scores, low analgesia - across subgroups is seen as both realistic and impactful. This resonates across pediatrics, family medicine, and emergency care.

Emergency Department and Primary Care Triage Equity

Using intersectional analysis - age, race, language, rural versus urban - to show who gets triaged up or down, who waits longest, and who is more likely to be sent home versus admitted resonates with ER and family physicians. Medium-sized hospitals with diverse populations were described as promising places to learn.

Post-Hoc Equity Audits and Morbidity-Mortality Reviews

Shifting bias work from anecdote to structured institutional learning - creating evidence of disparity patterns for morbidity-mortality reviews or teaching purposes - makes change more sustainable and embeds fairness into how organizations learn.

Indigenous Wellness and Data Sovereignty

Applying fairness-audit capabilities to Indigenous wellness tools under Indigenous governance and co-design is a powerful, high-impact niche. This requires humility, patience, and genuine partnership.

What We Learned at Synapse

In early 2026, Synod Intellicare participated in the Synapse Life Science competition, a prestigious initiative focused on identifying and scaling solutions that accelerate digital health innovation, market validation, and commercialization expertise

Working with university students, we crafted a viable commercialization plan and attended the showcase. More importantly, we gathered insights and connections that deepened our understanding of the ecosystem we serve.

Key learnings from Synapse:

  • Clinician champions matter more than general awareness. Early adopter physicians and nurses who can share success stories are invaluable
  • Governance and organizational readiness are competitive advantages, not friction. Healthcare organizations desperate for structure around AI governance will embrace platforms that make this tangible.
  • Regional collaborations with diverse populations may show value faster. Rural and urban hospitals with high clinician engagement could be ideal starting points.
  • Academia is a partner, not a competitor. University partnerships for research, validation, and publications build credibility and scale impact

From Validation to Partnerships: Academic Co-Development

Beyond Synapse, we are working with academic collaborators, including the Connected Minds research collaboration with York University and Université de Sherbrooke, on research into fair and reliable AI in healthcare. This is early-stage research, and no results have been published.

These partnerships matter because they:

  • Ground our work in evidence. Peer-reviewed research, once completed, can build credibility with healthcare leaders and regulators.
  • Expand our reach. Academic collaborators bring networks of clinical sites, patient populations, and research infrastructure.
  • Ensure continuous improvement. Independent research reveals gaps, informs product iterations, and keeps us honest about limitations.
  • **Create a template for scale. **As we move into growth phase, academic partnerships may become a model for how other health systems evaluate our approach.

The March Conversation: What We Learned

Our clinical and strategic validation has crystallized a few critical insights that are reshaping our roadmap:

Insight 1: Evidence-First Mode is Non-Negotiable

We're building DDFA's retrospective analytics capabilities as a primary entry point. Organizations should be able to upload de-identified data, run fairness analyses, and get clear answers about disparity patterns before deciding to implement real-time interventions.

This evidence-first approach builds trust. It respects clinician skepticism. And it aligns with how healthcare actually changes, slowly, with local data, and with buy-in from practitioners.

Insight 2: Clinically Grounded Use Cases Beat Generic Fairness Messaging

Pain management. Sepsis detection. Triage equity. Indigenous wellness. These specific, highimpact domains resonate more than broad "ethical AI everywhere" messaging.

Our marketing, partnerships, and product roadmap are now organized around these use cases. We intend to study DDFA in one domain before expanding. No outcomes data exist yet.

Insight 3: Governance-First Organizations Are Your Allies

We initially worried that hospital governance committees and compliance teams would slow adoption. We were wrong.

Hospitals with strong AI governance are desperate for tools that make fairness tangible and auditable. We hope they will be natural early collaborators, because we are trying to solve a real governance problem.

This shifts our go-to-market strategy. We lead with HAIQ maturity assessment and governance enablement, then layer in DDFA and clinical dashboards.

Insight 4: Indigenous and BIPOC Partnerships Are Differentiators, Not Afterthoughts

Collaborations around Indigenous health measures, BIPOC clinician wellbeing, and equitycentered data sovereignty position Synod not as compliance tech but as equity-first innovation.

These partnerships require different commercial terms (shared governance, co-design, benefit-sharing) and longer timelines. But they also create defensible moats and deep market relationships.

Insight 5: Multiple Personas, Multiple Entry Points

We are no longer building one platform. We are building an ecosystem:

  • For hospital executives: HAIQ maturity assessment + governance readiness
  • For quality officers: DDFA bias detection + equity dashboards
  • For clinicians: Plain-language findings (point-of-care prompts are on our longer-term roadmap, not available today)
  • For researchers: Structured fairness audits enabling publications
  • For Indigenous partners: Co-designed tools with data sovereignty

Each persona has a different entry point, operational goal, and success metric.

What Happened to Our March Timeline

In March 2026 we published a list of planned milestones for April through Q3 2026. Several of those dates have now passed and our plans have changed, so we have removed the list rather than leave it reading like completed work. We have not completed a hospital deployment or pilot, and we have no outcomes data to report. Our current scope and roadmap are described on our Products page.

Editor's note: this article was first published in March 2026. Our positioning was updated in October 2026. The Data Diversity and Fairness Auditor (DDFA) is a team-operated retrospective workflow that analyzes de-identified structured data and existing model outputs supplied under appropriate institutional approvals. It does not currently connect to live EHRs, ingest images or unstructured records, certify legal compliance, or provide point-of-care alerts.

A Closing Reflection: Clinicians Know What Fair Looks Like

One theme kept emerging across all our conversations: clinicians know what fair looks like at their bedside. They see inequity in real time. They feel the weight of it.

What they lacked was a way to measure it, discuss it systematically, and act on it without overwhelming already-stretched teams.

That's what we're building.

Not a perfect fairness algorithm. Not a compliance checkbox. Not a theoretical framework.

A practical, clinically grounded tool that says: Here's what your data shows. Here's who is affected. Here's what you might do about it.

And then trusts clinicians to decide.

Because in the end, ethical AI isn't built by engineers alone. It's built by clinicians willing to look at their own data honestly, acknowledge what they see, and work with their teams to do better.

We are honored to be part of that work. Fair Care, For Everyone, Together.



References: Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS Digital Health, 3(11), e0000651.

Dankwa-Mullan, I., et al. (2024). Health equity and ethical considerations in using artificial intelligence for public health. Preventing Chronic Disease, 21, E47.

Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 3(11), e745–e750.

World Health Organization. (2021). Ethics and governance of artificial intelligence for health. World Health Organization.




💼 Follow us on LinkedIn: Synod Intellicare

✖️ Follow us on X: Synod Intellicare

🔍 Request a Demo: Request a Demo

📥 Subscribe to Our Newsletter: Stay In The Know

Audit Your Clinical AI Systems

Connect with our team to request a 30-Day Preliminary Disparity Report.

Book a Strategy Call →