Practitioner Voices in Ethical AI • August 2026
I have been thinking about a woman I will call Marlene for more than twenty years. She came in for care on a winter Tuesday, a Haitian-Canadian woman in her late fifties who worked two jobs and hadn’t had a family physician in three years because she couldn’t find one accepting new patients in her part of the city. She sat in the waiting room quietly before being seen. She rated her chest tightness as a six out of ten in a voice so practised at not making a fuss that the trainee beside me wrote “moderate discomfort, appears stable.” Her chart had almost nothing in it, a sparse medication list, two prior visits for unrelated complaints, no continuity of care. Clinical culture had given us all a framework for reading distress, and that framework was built on a body of knowledge that, for reasons of history and exclusion, did not include her. Her pain presentation, shaped by language, by culture, by the particular way her community had learned to describe suffering to people in positions of authority, did not fit the template. So the template missed her. She survived. I spent years afterward thinking about what we had nearly taught ourselves to miss, and whether we had ever really stopped teaching it.
Healthcare has never been confused about who this patient is. She appears in every data set we have ever produced. The Canadian Institute for Health Information (CIHI, 2024) has documented persistent gaps in health outcomes along lines of income, geography, immigration status, and race across Canadian health systems. We have known for decades which patients wait longest before triage, whose pain is recorded lower than what they actually report, who goes home without a follow-up appointment, and who comes back by ambulance three days later. I have sat in quality improvement meetings where the data, read carefully, told a clear story about which patients were being consistently underserved, which neighbourhoods, which languages, which communities, and watched those patterns get absorbed into next year’s improvement plan without the structural change that would have made a difference. Institutional knowledge of inequity, without institutional action, is a governance failure. It leaves a paper trail. And as it turns out, the machines are now reading it.
Now consider what happens when a clinical AI tool enters that system without fairness auditing as a precondition of deployment. In a 2024 American Hospital Association survey, 52.2% of U.S. hospitals used machine-learning-based predictive AI integrated with their EHR. Generative AI is following the same curve: 31.5% of hospitals were using it by 2024, with a further 24.7% planning to adopt within the year (Everson, Nong and Richwine, 2025). Most of those tools were trained on historical clinical data, data generated by institutions that have, for decades, systematically underserved the communities I am describing. When a clinical AI is trained on that data, it does not inherit a balanced picture of clinical risk. It inherits a distorted one. It learns that patients from certain communities have a particular profile, sparse documentation, lower recorded pain scores, earlier discharges, minimal follow-up, and it learns these patterns not as failures of the system, but as characteristics of the patient.
Obermeyer et al. (2019) demonstrated in Science that a widely used commercial health algorithm systematically underestimated Black patient illness severity, not because it was designed to discriminate, but because it was trained on cost as a proxy for need, and Black patients at equivalent illness levels had historically received less care. The algorithm did not introduce bias. It gave institutional credibility to the bias already present, formalized it, and deployed it faster than any human practitioner ever could. When a nurse is trained to defer to clinical decision support, and that tool was built on the same distorted data her institution has always produced, the bias that could have been named and corrected by an empowered clinician becomes instead the validated output of a trusted system.
The financial case is unambiguous, though for me the human cost precedes it. Every unnecessary readmission is a person who went home without what they needed. Every delayed diagnosis is hours, sometimes days, taken from someone’s recovery. The patients who drive the highest readmission rates are disproportionately from low-income, racialized, and marginalized communities, precisely the patients whose clinical risk is most likely to be underestimated by a model trained on biased historical data. Under the EU Artificial Intelligence Act, AI systems used for diagnosis, clinical decision support, triage, and patient monitoring are classified as high-risk and must undergo ongoing risk management, post-market monitoring, and oversight to identify and mitigate safety and bias issues (European Parliament, 2024). Healthcare executives deploying AI clinical decision tools without documented fairness auditing are taking on growing regulatory exposure. The awareness is there. The governance structure to act on it is not yet.
What I have learned over twenty years of managing clinical operations is that the things practitioners actually do consistently are the things embedded in professional expectation, not policy documents, not aspirational frameworks, but standards: the way infection control is a standard, the way informed consent is a standard. When something becomes professionally non-negotiable, it changes what practitioners are trained to see, willing to name, and prepared to act on. That is the institutional logic behind the Synod Certified Fairness (SCF) program now in development, a certification prerequisite designed for healthcare institutions to build into AI procurement before a tool ever enters a clinical environment. Not a vendor promise made in a sales presentation. A condition of entry: evidence that the tool has been tested for performance differences across demographic groups, and that those differences are documented in a form that governance committees and clinicians can act on. Clinicians I have worked with are ready to do the right thing, when the right information reaches them at the right moment.
Synod’s Data Diversity and Fairness Auditor (DDFA) is a team-operated retrospective workflow. It analyzes de-identified structured data and existing model outputs supplied under appropriate institutional approvals, giving institutions a documented baseline of their AI tools’ fairness performance. It does not currently connect to live EHRs or provide point-of-care alerts; those are on our 2027 roadmap. Synod is also participating in the Connected Minds research collaboration, Towards Fair and Reliable Large Language Models in Healthcare, with York University and the Université de Sherbrooke, with a paediatric emergency physician serving as clinical champion.
The nurses who trained over the last thirty years learned to advocate at the bedside. They learned to speak up when an assessment felt wrong, to escalate when clinical instinct told them the chart was not telling the whole story. The nurses entering the workforce now need to learn to advocate in a world where the recommendation at the bedside may come from an algorithm trained on the same inequities they are trying to correct. And we have an obligation, now, while these tools are still being shaped and their norms still being set, to demand that the institutions deploying them require fairness evidence the same way they require clinical evidence. If you would not introduce a medication without evidence of safety across the relevant patient population, do not introduce a clinical AI without fairness evidence. The invisible patient is still there. She has always been in the data. The question is whether the tools we are now adopting will finally see her, or whether they will learn, at scale and at speed, to look away.
An Invitation
- If you are a health technology innovator who wants to build trust into your platform from the ground up, not as a checkbox but as a genuine commitment, we want to work with you.
- If you are a healthcare AI vendor or a health system leader committed to procurement that reflects your organization’s values around equity and patient safety, we want to be your partner.
- If you want to know where your organization stands on ethical AI maturity right now, before the regulatory pressure arrives, the Ethical AI Maturity Assessment takes less than fifteen minutes.
👉 Take the Ethical AI Maturity Assessment: Ethical AI Maturity Assessment (EAMA)
👉 Become a Design Partner: Contact Us
About the Author
Lorna Baker is a retired registered nurse who served as Satellite and Hub Manager at an inner-city community health centre, with more than 20 years of experience managing clinicians in primary care settings serving low-income and racialized communities across Canada. She writes as part of Synod IntelliCare’s Practitioner Voices in Ethical AI series.
References
- Canadian Institute for Health Information. (2024). Preventable hospitalizations and health system performance. CIHI. https://www.cihi.ca
- European Parliament. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union.
- Everson, J., Nong, P., & Richwine, C. (2025). Uptake of generative AI integrated with electronic health records in US hospitals. JAMA Network Open, 8(12), e2549463. https://doi.org/10.1001/jamanetworkopen.2025.49463
- Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342