The Roadmap to Go-Live - Piloting, Training, and Achieving CATR

The final push in July required every ounce of determination I'd built up over the previous months. As I sat in clinic on that sweltering July morning, reviewing my notes from the previous day's AI clinic session with our Data Protection Officer and the ICB's Clinical Safety Officer, I knew we were at the critical juncture. The Technical Architecture Group approval loomed, and with it, our chance to finally achieve Clinical Assessment of Technical Readiness sign-off.

The AI Clinic Session: Questions That Mattered

Working with ICB's Data Protection Officer and the ICB's Clinical Safety Officer, we went through the key requirements systematically. The focus was on ensuring we understood and could implement the safety measures outlined in the ICB documentation.

Key areas covered included:

• •Patient consent workflows - ensuring patients understood they were interacting with AI-assisted documentation
• •Verification processes - confirming our two-tier system of real-time flagging and mandatory physician review
• •The fundamental principle from the Implementation Guidelines: "Users must review all generated documentation to confirm its accuracy before capturing it in their electronic medical records"
• •Understanding that as stated in the guidelines, "Users are professionally and legally liable for the accuracy of the patients' clinical record"

Structuring Our Pilot: A Methodical Approach

The NCL ICB Implementation Guidelines recommended that practices conduct a pilot "which would allow a gradual adoption while assessing the tool's accuracy and performance." The guidance suggested starting with a small, manageable scope: maybe one or two GPs using the tool for a period of time and then allowing practices to refine processes and ensure the AI system meets clinical needs before expanding its use.

Key requirements from the Implementation Guidelines included:

• •Staff training with detailed record keeping
• •Testing microphone quality before and during use
• •Ensuring quiet consultation environments free from distractions
• •Patient notification about AI use
• •Evaluating AI performance with diverse patient groups (varying accents, non-native English speakers, individuals with speech impediments, young children, elderly, and patients with learning difficulties)
• •Thorough review of all AI-generated documentation before transferring to patient records
• •For non-integrated systems, confirming patient identification before transferring notes

The Technical Challenges We Discovered

The Implementation Guidelines emphasised critical technical requirements. Microphone quality checks were mandatory. Clinicians needed to conduct regular checks on their microphones to ensure proper functioning before and during AI platform use. The Practice Manager was responsible for ensuring robust network/Wi-Fi and supporting infrastructure.

The guidelines specifically noted that consultation environments must be quiet and free from distractions to facilitate accurate dictation and processing.

The accent recognition challenge was explicitly acknowledged in the ICB documentation. Practices were required to evaluate and give feedback on the AI Scribe's performance with diverse patient groups, including those with varying accents, non-native English speakers, individuals with speech impediments, young children, the elderly, and patients with learning difficulties.

Training Materials and Colleague Preparation

I developed a comprehensive training package for my colleagues, knowing that user adoption would make or break our implementation. The training covered three critical areas: technical operation, clinical integration, and troubleshooting.

The technical training included hands-on practice with microphone positioning, system startup and shutdown procedures, and real-time error recognition.

Clinical integration training focused on workflow adaptation. How do you maintain eye contact with patients while monitoring AI transcription? How do you handle system failures gracefully? I developed role-playing scenarios where colleagues practised explaining AI assistance to sceptical patients and managing technical interruptions during consultations.

Achieving CATR Sign-Off: The Moment of Truth

The Clinical Authority to Release represented the culmination of our preparation. The CATR template provided by NCL ICB required sign-off from multiple stakeholders.

The CATR document confirmed that:

• •Clinical Risk Management Plan was confirmed and implemented
• •Hazard identification was completed
• •Risk assessment showed all identified risks reduced to acceptable levels
• •Post-deployment monitoring plans were in place
• •Clinical Safety Case Report and Hazard Log were reviewed
• •Information Governance compliance was achieved (DSPT, UK GDPR, DPIA)
• •Technical assessment was completed by ICB IT Team including cyber assessment

As the CATR template stated: "Based on the evidence provided and the clinical safety documentation reviewed, we authorise the release of Heidi AI for deployment. All foreseeable data protection and clinical risks have been addressed to an acceptable level."

The significance of CATR sign-off cannot be overstated. It represented official recognition that our AI implementation met clinical safety standards. The template emphasised that "each practice holds clinical accountability for the deployment of any AI" and that "clinicians need to be aware of the clinical risk of AI applications being implemented."

Our pilot revealed that while AI can enhance clinical practice significantly, it demands rigorous preparation, continuous monitoring, and unwavering commitment to patient safety. The road to go-live taught us that the most sophisticated technology is only as good as the humans who implement, monitor, and ultimately take responsibility for it.

---

This is Part 5 of a 6-part series documenting the implementation of Heidi AI Scribe in NHS primary care. ← Part 4: The MHRA "Bomb" | Part 6: Lessons Learned and the Future →