
Implementation Guide: Manage prescription refill requests end-to-end within approved protocols
Step-by-step implementation guide for deploying AI to manage prescription refill requests end-to-end within approved protocols for Healthcare clients.
Hardware Procurement
FortiGate 40F Next-Generation Firewall
$350–$450 per unit (MSP cost) / $700–$900 suggested resale with installation
HIPAA-compliant UTM firewall for practices with up to 25 users. Provides IPS, application control, web filtering, antivirus, and VPN. Required to secure the network perimeter for all PHI traffic between the AI agent cloud, EHR APIs, and practice endpoints. Supports centralized management via FortiCloud for MSP remote administration.
FortiGate 40F Unified Threat Protection (UTP) 1-Year Subscription
$250–$350/year (MSP cost) / $450–$600 suggested resale
Subscription bundle including FortiGuard IPS, antivirus, web filtering, and application control services for the FortiGate 40F. Required for ongoing threat protection of the practice network handling PHI.
FortiGate 60F Next-Generation Firewall
$600–$800 per unit (MSP cost) / $1,100–$1,500 suggested resale with installation
HIPAA-compliant UTM firewall for mid-size practices with 25–50 users. SD-WAN capable for practices with multiple ISP connections. Use instead of 40F for larger deployments. Only procure one firewall model per practice — choose 40F or 60F based on practice size.
FortiGate 60F Unified Threat Protection (UTP) 1-Year Subscription
$350–$500/year (MSP cost) / $600–$850 suggested resale
Subscription bundle for FortiGate 60F. Only needed if 60F is selected instead of 40F.
Yealink T54W IP Phone
$120 per unit (MSP cost) / $180–$220 suggested resale per unit
SIP desk phones for front-desk and clinical staff to monitor AI agent activity and handle escalated calls. Quantity of 4 is typical for a small-to-mid practice (reception, nurse station, provider office, back office). Adjust quantity based on practice staffing.
Jabra Evolve2 75 Wireless Headset
$200–$280 per unit (MSP cost) / $300–$380 suggested resale per unit
Noise-canceling wireless headsets for clinical staff who handle escalated refill calls from the AI agent. Active noise cancellation is essential in busy clinical environments. Quantity of 2 covers primary nurse/medical assistant plus backup.
Datto SIRIS 5 2TB BCDR Appliance
MSP-exclusive pricing via Datto partner program (~$200–$400/mo all-in) / $400–$700/mo suggested resale
HIPAA-compliant backup and disaster recovery appliance for on-premise EHR data and local configuration files. Provides encrypted local and cloud backup with instant virtualization. Required for practices running any on-premise EHR components (e.g., eClinicalWorks server). May be omitted for fully cloud-native EHR practices (athenahealth, DrChrono) if existing backup is adequate.
Software Procurement
Retell AI Voice Agent Platform
$0.07–$0.11/minute standard; ~$0.05/minute at enterprise volume (>$3K/mo); $10 free credits to start. Typical practice: $300–$800/month based on call volume. Resell at $500–$1,200/month.
Primary AI voice agent platform for handling inbound patient refill calls. No-code builder for creating refill conversation flows. Handles patient identity verification, medication name recognition via NLP, refill details collection, and protocol-based decision routing. Supports Twilio, Vonage, and direct SIP trunk connectivity. BAA available, SOC 2 certified.
$149/month base (SMS) + $30/user/month (VoIP) + ~$99/month AI agent add-on. Typical practice total: $350–$700/month. Resell at $550–$1,100/month.
Alternative to Retell AI — all-in-one patient communication platform with built-in AI agent capabilities for SMS, voice, and chatbot. Better suited for practices wanting a single platform for all patient communications, not just refill automation. 500+ integrations including major EHRs. HIPAA-compliant with BAA.
Starter plan for solo practices; Growth/Business for multi-provider. Approximately $200–$800/month depending on tier and workflow volume. Annual billing saves ~33%. Resell at $350–$1,200/month.
No-code healthcare workflow orchestration platform. Connects the AI voice agent to the EHR via pre-built connectors for athenahealth, ModMed, Elation Health, DrChrono, and others. Enforces clinical refill protocol rules (drug class validation, refill count checks, controlled substance routing). HIPAA and SOC 2 Type 2 compliant with BAA included on all paid plans.
Twilio Programmable Voice + Phone Numbers
$1/month per phone number + $0.0085/minute inbound + $0.014/minute outbound. Typical practice: $50–$200/month. Include in managed service bundle.
Telephony infrastructure connecting the AI voice agent (Retell AI) to the practice's phone system. Provides dedicated phone numbers for the AI refill line, call routing, and SIP connectivity. Retell AI natively integrates with Twilio for voice transport.
NordLayer Business VPN / ZTNA
$8–$11/user/month. Typical practice (10 users): $80–$110/month. Resell at $120–$170/month.
HIPAA-ready cloud VPN and zero-trust network access for securing remote connections to the AI management dashboard, EHR system, and Keragon configuration portal. Provides MFA, role-based access, encrypted tunnels, and activity logging. Alternative: use Fortinet ZTNA if already deploying FortiGate with FortiClient.
Compliancy Group – The Guard HIPAA Compliance Platform
$300–$500/month depending on practice size. Resell at $450–$800/month as part of compliance-as-a-service bundle.
HIPAA compliance management platform providing risk assessment templates, BAA tracking, policy documentation, incident management, and audit trail. Required to document the HIPAA risk assessment before deploying AI agents that handle PHI. Generates the required Security Risk Assessment (SRA) for Meaningful Use and MIPS compliance.
Surescripts E-Prescribing Network (via EHR)
Typically included in EHR subscription; no separate MSP procurement needed
National e-prescribing network that routes approved prescriptions to pharmacies. Integration is handled through the practice's existing EHR e-prescribing module. The AI agent does not connect directly to Surescripts — it creates the refill task in the EHR, and the provider's approval triggers the Surescripts transmission through the EHR's existing e-prescribing workflow.
Prerequisites
- Active Electronic Health Record (EHR) system with API access enabled — FHIR R4 preferred; supported EHRs include athenahealth, Epic, eClinicalWorks, NextGen, DrChrono, ModMed, or Open Dental. Confirm API tier/plan with EHR vendor.
- E-prescribing module active and connected to Surescripts network within the EHR — verify the practice can currently send electronic prescriptions to pharmacies
- Business-grade internet connection with minimum 50 Mbps symmetric bandwidth (100 Mbps+ recommended); low latency (<100ms) to cloud endpoints required for real-time voice AI
- Existing phone system capable of SIP trunking, or willingness to migrate to a SIP-compatible system. If using legacy analog/PRI lines, plan for SIP trunk migration or parallel deployment via Twilio
- Signed Business Associate Agreements (BAAs) with all existing cloud vendors handling PHI (EHR vendor, cloud backup provider, email provider, etc.)
- Clinical champion identified — a licensed physician or clinical director who will define and approve the refill protocol rules (approved medication classes, refill limits, controlled substance exclusions, escalation criteria)
- Current prescription refill workflow documented — staff must provide current-state process documentation including who handles refills, average daily volume, common medication classes, and known pain points
- Practice administrator or compliance officer available to participate in HIPAA risk assessment and sign off on new PHI data flows
- Windows 10/11 or macOS workstations with modern web browser (Chrome, Edge, Firefox) for staff dashboard access — no special endpoint software required
- Active domain name and SSL certificate for any patient-facing web chat widget deployment
- Practice's DEA registration current and EPCS (Electronic Prescribing for Controlled Substances) enabled if the practice handles any Schedule II–V medications — controlled substances will be excluded from AI automation but the system must correctly identify and escalate them
Installation Steps
...
Step 1: Conduct HIPAA Security Risk Assessment and Compliance Documentation
Before deploying any AI agent that handles PHI, you must complete a comprehensive HIPAA Security Risk Assessment (SRA). This is both a regulatory requirement and a liability shield for the MSP. Use the Compliancy Group platform (or equivalent) to: (1) document all current PHI data flows, (2) identify new data flows introduced by the AI agent (patient voice data, refill request data, medication information), (3) assess risks to confidentiality, integrity, and availability, (4) document mitigation controls, and (5) generate the formal SRA report. Additionally, prepare BAAs for all new vendors: Retell AI (or Emitrr), Keragon, Twilio, and NordLayer. Each vendor offers BAAs — request and execute them before any PHI touches their systems.
The 2025 HIPAA Security Rule update eliminates the distinction between 'required' and 'addressable' specifications — all are now required. Encryption of ePHI at rest and in transit is mandatory with very limited exceptions. Do NOT proceed to Step 2 until all BAAs are signed and the SRA is documented. This protects both the practice and the MSP from regulatory liability.
Step 2: Deploy and Configure HIPAA-Compliant Network Infrastructure
Install and configure the Fortinet FortiGate firewall to secure the practice network. This establishes the security perimeter for all PHI traffic between the practice, AI cloud services, and EHR APIs. Configure network segmentation to isolate clinical systems from guest/IoT traffic. Enable IPS, application control, web filtering, and SSL inspection (with healthcare-specific exceptions for EHR traffic). Set up FortiCloud for centralized MSP monitoring.
# Initial FortiGate setup (console access via USB or mgmt port at 192.168.1.99)
config system interface
edit port1
set alias 'WAN'
set mode dhcp
set allowaccess ping https
next
edit port2
set alias 'LAN-Clinical'
set ip 10.10.10.1/24
set allowaccess ping https ssh
next
edit port3
set alias 'LAN-Guest'
set ip 10.10.20.1/24
set allowaccess ping
next
end
# Enable HIPAA-relevant security profiles
config antivirus profile
edit 'hipaa-av'
set scan-mode full
config http
set av-scan enable
end
next
end
config ips sensor
edit 'hipaa-ips'
config entries
edit 1
set status enable
set log enable
set log-packet enable
set severity medium high critical
next
end
next
end
# Create firewall policy: Clinical LAN → Internet (for EHR/AI traffic)
config firewall policy
edit 1
set name 'Clinical-to-Cloud'
set srcintf port2
set dstintf port1
set srcaddr all
set dstaddr all
set action accept
set schedule always
set service ALL
set av-profile 'hipaa-av'
set ips-sensor 'hipaa-ips'
set ssl-ssh-profile certificate-inspection
set logtraffic all
set nat enable
next
end
# Enable FortiCloud management for MSP remote access
config system fortiguard
set forticloud-account-id '<MSP_FORTICLOUD_ID>'
end
exec forticloud activate
# Configure NTP for accurate audit logging
config system ntp
set ntpsync enable
set type custom
config ntpserver
edit 1
set server time.nist.gov
next
end
end
# Enable logging to FortiCloud / FortiAnalyzer
config log fortianalyzer setting
set status enable
set server '<FORTIANALYZER_IP_OR_CLOUD>'
endChoose FortiGate 40F for practices ≤25 users or FortiGate 60F for 25–50 users. Ensure the UTP subscription is activated before configuring security profiles. For SSL deep inspection, add exceptions for known EHR endpoints (e.g., athenahealth API endpoints) to avoid breaking clinical workflows. Configure Wi-Fi access points on separate VLANs — clinical SSID on the 10.10.10.0/24 network, guest on 10.10.20.0/24 with WPA3 Enterprise for clinical.
Step 3: Set Up Twilio Telephony Infrastructure
Create and configure the Twilio account that will serve as the telephony backbone for the AI voice agent. Provision phone numbers, configure SIP trunking if integrating with the practice's existing PBX, and set up the connection to Retell AI. The AI refill line can either be a new dedicated number or can be integrated into the practice's existing phone tree via SIP trunk forwarding.
<Response>
<Say voice='Polly.Joanna'>We're sorry, our automated refill system is temporarily unavailable. Please hold while we transfer you to our staff.</Say>
<Dial>+1PRACTICEPHONE</Dial>
</Response>Twilio's HIPAA-eligible services require a signed BAA and a paid account — the free trial does not qualify. Ensure the Twilio account is configured under the practice's billing (or MSP's master account with sub-accounts per client). If the practice wants the AI agent reachable from their existing main phone number, configure the existing PBX to route refill-related menu options to the Twilio number via SIP trunk rather than replacing the main number.
Step 4: Configure Retell AI Voice Agent Platform
Set up the Retell AI account, configure the AI voice agent for prescription refill intake, define the conversation flow, connect to Twilio for telephony, and configure the webhook to Keragon for workflow orchestration. This is the patient-facing AI layer that handles the actual phone conversation.
// Webhook configuration (Step 8)
{
"event": "call_ended",
"url": "https://api.keragon.com/v1/webhooks/<WORKFLOW_ID>/trigger",
"method": "POST",
"headers": {
"Authorization": "Bearer <KERAGON_API_KEY>",
"Content-Type": "application/json"
}
}
// Payload includes: call transcript, extracted variables
// (patient name, DOB, medication, pharmacy, etc.)
// Function parameters (Step 9 — verify_patient)
{
"first_name": "string",
"last_name": "string",
"date_of_birth": "string"
}Retell AI charges per minute of conversation ($0.07–$0.11/min standard). A typical refill call lasts 2–3 minutes, so expect $0.14–$0.33 per refill call. At 200 refills/week, monthly Retell cost is approximately $500–$1,150. The Knowledge Base feature costs $0.005/minute additional and $8/month per KB after the first 10. Enable call recording for HIPAA audit compliance but ensure recordings are encrypted and set auto-deletion policies per practice retention requirements.
Step 5: Define Clinical Refill Protocols with Practice Clinical Champion
Work with the practice's physician or clinical director to define the exact rules governing which refill requests the AI agent can process autonomously (up to the provider approval step) and which must be immediately escalated to clinical staff. This is NOT a technology step — it is a clinical governance step that directly determines the AI agent's decision logic. Document everything in writing with clinician sign-off.
# === REFILL PROTOCOL DOCUMENTATION TEMPLATE ===
# PRACTICE: [Practice Name]
# CLINICAL SPONSOR: [Physician Name, License #]
# DATE: [Date]
# VERSION: 1.0
# SECTION 1: APPROVED MEDICATION CLASSES FOR AI PROCESSING
# (AI can collect refill request and route to provider queue)
# - Antihypertensives (ACE inhibitors, ARBs, beta-blockers, CCBs, diuretics)
# - Statins and lipid-lowering agents
# - Oral diabetes medications (metformin, sulfonylureas, DPP-4 inhibitors)
# - Thyroid medications (levothyroxine)
# - Proton pump inhibitors
# - Antihistamines (prescription-strength)
# - Topical medications (dermatologic)
# - Oral contraceptives (if applicable)
# - Inhaled corticosteroids / bronchodilators (stable asthma/COPD)
# [ADD/REMOVE per clinical sponsor direction]
# SECTION 2: EXCLUDED MEDICATION CLASSES (Immediate human escalation)
# - ALL Schedule II controlled substances (opioids, stimulants, etc.)
# - ALL Schedule III-V controlled substances (benzodiazepines, etc.)
# - Anticoagulants (warfarin, DOACs) — require INR/lab review
# - Immunosuppressants
# - Antipsychotics / mood stabilizers
# - Chemotherapy agents
# - Any medication requiring lab monitoring before refill
# [ADD/REMOVE per clinical sponsor direction]
# SECTION 3: REFILL ELIGIBILITY RULES
# - Patient must be active in the practice (seen within last 12 months)
# - Medication must have remaining refills on the existing prescription
# - If no remaining refills: escalate to provider for new Rx decision
# - Patient must not have an overdue annual physical (if > 14 months since last visit)
# - Maximum of [3] refill requests per medication before requiring office visit
# SECTION 4: ESCALATION CRITERIA
# - Any adverse reaction reported by patient → immediate nurse escalation
# - Dose change request → provider review required
# - New medication request → provider review required
# - Patient confusion or distress detected by AI → transfer to live staff
# - Controlled substance request → transfer to clinical staff
# SECTION 5: SIGN-OFF
# Clinical Sponsor Signature: __________________
# Date: __________________This is the most important step in the entire implementation. The AI agent's behavior is only as good as the clinical protocols it enforces. Spend adequate time (plan for 2–4 hours of clinician time across 1–2 sessions) to get this right. The protocol document must be version-controlled and updated whenever clinical policies change. Keep a copy in the compliance documentation system. Dental practices typically have a much smaller refill scope (antibiotics, pain management, fluoride) — adjust the protocol template accordingly.
Step 6: Build Keragon Workflow Automation Pipelines
Configure Keragon as the workflow orchestration layer that sits between the AI voice agent (Retell AI) and the EHR. Keragon receives structured data from the AI agent via webhook, applies clinical protocol rules, performs EHR lookups/writes via FHIR APIs, and routes the refill request to the appropriate queue. Build three core workflows: (1) Patient Verification, (2) Refill Eligibility Check & Routing, and (3) Escalation Handler.
Workflow 1: Patient Verification
GET /fhir/r4/Patient?family={last_name}&given={first_name}&birthdate={dob}Workflow 2: Refill Eligibility Check & EHR Task Creation
GET /fhir/r4/MedicationRequest?patient={patient_id}&status=activePOST /fhir/r4/Task
{
"resourceType": "Task",
"status": "requested",
"intent": "order",
"description": "AI-processed refill request: [MedName] [Dose] for [PatientName]",
"for": { "reference": "Patient/{patient_id}" },
"requester": { "display": "AI Refill Agent" },
"input": [
{ "type": { "text": "medication" }, "valueString": "{med_name}" },
{ "type": { "text": "pharmacy" }, "valueString": "{pharmacy_name}" },
{ "type": { "text": "call_recording_id" }, "valueString": "{recording_id}" }
]
}Workflow 3: Escalation Handler
Keragon's no-code interface allows building these workflows visually with drag-and-drop steps. The FHIR API calls shown above are illustrative — exact endpoints and payloads vary by EHR. For athenahealth, use their proprietary REST API (not FHIR) for task creation: POST /v1/{practiceid}/appointments or custom task endpoints. Test each workflow with sample data before connecting to the live AI agent. Keragon's audit logging satisfies HIPAA's requirement for tracking all PHI access and actions.
Step 7: Configure EHR API Integration and Test Connectivity
Establish the actual API connection between Keragon and the practice's EHR system. This involves registering an API application with the EHR vendor, obtaining credentials, configuring OAuth2 authentication, and testing basic CRUD operations against patient and medication resources. This is typically the longest step in the implementation.
# === For athenahealth (most common SMB EHR) ===
# 1. Register API application at https://developer.athenahealth.com
# Create developer account → New Application → Select API version (v1)
# Application Type: Backend Service
# Requested Scopes: patient.read, medicationrequest.read, task.write
# Note Practice ID: provided by practice administrator
# 2. Obtain OAuth2 credentials
# Client ID: <provided after app approval>
# Client Secret: <provided after app approval>
# Token Endpoint: https://api.athenahealth.com/oauth2/v1/token
# 3. Test authentication
curl -X POST 'https://api.athenahealth.com/oauth2/v1/token' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id=<CLIENT_ID>&client_secret=<CLIENT_SECRET>&scope=athena/service/Athenahealth.MDP.*'
# Expected response: { 'access_token': '<TOKEN>', 'token_type': 'bearer', 'expires_in': 3600 }
# 4. Test patient search
curl -X GET 'https://api.athenahealth.com/v1/<PRACTICE_ID>/patients?firstname=John&lastname=Smith&dob=01/15/1980' \
-H 'Authorization: Bearer <ACCESS_TOKEN>'
# 5. Test medication list retrieval
curl -X GET 'https://api.athenahealth.com/v1/<PRACTICE_ID>/patients/<PATIENT_ID>/medications' \
-H 'Authorization: Bearer <ACCESS_TOKEN>'
# 6. Test task/message creation (for refill routing)
curl -X POST 'https://api.athenahealth.com/v1/<PRACTICE_ID>/patients/<PATIENT_ID>/messages' \
-H 'Authorization: Bearer <ACCESS_TOKEN>' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'messagetype=REFILL_REQUEST&subject=AI+Refill+Request&messagetext=Refill+requested+for+Lisinopril+10mg'
# === For Epic (FHIR R4) ===
# 1. Register at https://appmarket.epic.com (App Orchard)
# Create Backend System application
# Request FHIR R4 scopes: patient/*.read, MedicationRequest.read, Task.write
# 2. Generate JWT for Epic backend authentication
# Epic uses SMART Backend Services (JWT-based OAuth2)
# Generate RSA key pair and register public key with Epic
openssl genrsa -out epic_private_key.pem 2048
openssl rsa -in epic_private_key.pem -pubout -out epic_public_key.pem
# 3. Test FHIR Patient search
curl -X GET 'https://<EPIC_FHIR_BASE>/api/FHIR/R4/Patient?family=Smith&given=John&birthdate=1980-01-15' \
-H 'Authorization: Bearer <ACCESS_TOKEN>'
# 4. Enter credentials into Keragon
# Navigate to: Keragon → Integrations → [EHR] → Configure → Enter credentials
# Test connection within Keragon interfaceEHR API registration and approval can take 1–4 weeks depending on the vendor. athenahealth is typically fastest (1–2 weeks) for existing athenahealth practices. Epic App Orchard approval can take 2–6 weeks and may require additional security review. For eClinicalWorks with limited API access, consider using Keragon's HL7 v2 interface engine or Rhapsody integration engine as middleware. Start this step early in the project timeline — it is the critical path item.
Step 8: Deploy VPN/ZTNA and Secure Remote Access
Configure NordLayer (or Fortinet ZTNA) to provide secure, HIPAA-compliant remote access for MSP technicians and authorized practice staff to manage the AI platform dashboards, Keragon workflows, and EHR API configurations. This ensures all administrative access to PHI-adjacent systems is encrypted, authenticated, and logged.
- Option A: NordLayer Configuration
- Create NordLayer organization at https://nordlayer.com
- Add practice as a team/organization unit
- Configure MFA requirement (mandatory for HIPAA)
- Navigate to: Settings → Security → Multi-Factor Authentication → Require for all users
- Create dedicated gateways for practice network access
- Navigate to: Gateways → Create Gateway → Select region closest to practice
- Assign static IP if needed for EHR API IP whitelisting
- Configure access policies
- Navigate to: Access Control → Create Policy
- Policy 1: 'MSP Admins' → Allow access to: FortiGate mgmt, Keragon dashboard, Retell dashboard
- Policy 2: 'Clinical Staff' → Allow access to: EHR system only
- Policy 3: 'Practice Admin' → Allow access to: Keragon dashboard (read-only), AI reports
- Deploy NordLayer client to authorized devices
- Download from: https://nordlayer.com/download/
- Configure with organization credentials
- Verify MFA enrollment for each user
# Configure FortiClient with ZTNA tags (if using FortiGate)
config user forticlient-ems
edit 1
set name 'Practice-EMS'
set server '<FORTICLIENT_EMS_IP>'
next
end
config firewall policy
edit 100
set name 'ZTNA-MSP-Admin'
set srcintf 'ssl.root'
set dstintf port2
set srcaddr 'ZTNA-Tag-MSP'
set dstaddr 'AI-Management-Servers'
set action accept
set schedule always
set utm-status enable
set logtraffic all
next
endNordLayer is simpler to deploy and manage across multiple client sites — recommended for MSPs managing many practices. Fortinet ZTNA is more tightly integrated if you are already deploying FortiGate + FortiClient ecosystem. Whichever you choose, ensure MFA is enforced on ALL administrative access to systems handling PHI. Document all authorized users and review access quarterly as part of HIPAA compliance.
Step 9: End-to-End Integration Testing with Synthetic Data
Before going live, conduct thorough testing using synthetic (fake) patient data to verify every component of the system works correctly end-to-end: AI agent answers calls, correctly identifies refill requests, verifies patient identity, checks medication eligibility, routes to the EHR, and handles escalations. Create a test plan covering all protocol scenarios.
- Test Patient 1 — Happy Path: Name: Test Patient Alpha | DOB: 01/01/1970 | Active Rx: Lisinopril 10mg, 2 refills remaining, last visit 3 months ago
- Test Patient 2 — No Remaining Refills: Name: Test Patient Beta | DOB: 02/02/1975 | Active Rx: Metformin 500mg, 0 refills remaining
- Test Patient 3 — Controlled Substance: Name: Test Patient Gamma | DOB: 03/03/1980 | Active Rx: Alprazolam 0.5mg (Schedule IV)
- Test Patient 4 — Overdue Visit: Name: Test Patient Delta | DOB: 04/04/1965 | Active Rx: Atorvastatin 20mg, 3 refills remaining, last visit 15 months ago
- Test Patient 5 — Not a Patient: Name: Test Patient Epsilon | DOB: 05/05/1990 | Not in EHR system
Do NOT test with real patient data until the system has passed all synthetic data tests and received clinical sign-off. Create a formal test results document with pass/fail for each scenario — this becomes part of the compliance documentation. Plan for at least 2–3 rounds of testing as issues are discovered and resolved. Common issues: voice agent misrecognizing medication names (update knowledge base), EHR API field mapping errors (adjust Keragon workflow), and SMS delivery failures (verify Twilio phone number configuration).
Step 10: Conduct Pilot Launch with Limited Patient Population
After successful synthetic testing, launch the system with a controlled subset of real patients. Select 20–50 patients with upcoming refill needs (provider identifies from EHR) and route their refill calls through the AI agent while maintaining the traditional phone line as backup. Monitor intensively for 2–4 weeks.
The pilot phase is essential — do not skip it. Even after thorough synthetic testing, real patient interactions will reveal issues with medication name pronunciation, patient communication styles, and edge cases not covered in testing. During the pilot, have a clinical staff member review EVERY AI-processed refill request before it reaches the provider queue. This human safety net can be removed once confidence is established (typically after 2 weeks of error-free operation). Document all pilot findings for the compliance file.
Step 11: Full Production Deployment and Staff Training
After a successful pilot period with no critical issues, expand the AI refill agent to handle all incoming refill requests. Configure the practice phone system to route all refill calls through the AI agent as the primary handler. Conduct staff training for all clinical and administrative personnel who will interact with the system.
Schedule full deployment for a Tuesday or Wednesday — avoid Mondays (highest call volume) and Fridays (staff availability issues). Have MSP technician on-site for the first 2 days of full deployment to handle any issues in real-time. Prepare a rollback plan: if critical issues arise, the IVR can be updated within minutes to route refill calls back to the front desk while issues are resolved.
Custom AI Components
Rx Refill Voice Agent System Prompt
Type: prompt The core system prompt that defines the AI voice agent's persona, behavior, clinical protocol adherence, and conversation flow for prescription refill intake calls. This prompt is configured within the Retell AI platform's agent settings and governs all patient interactions.
Implementation:
Rx Refill Voice Agent System Prompt
Patient Identity Verification Function
Type: integration A real-time function called by the Retell AI agent during the conversation to verify the patient's identity against EHR records. Returns verification status so the agent can proceed or escalate. Implemented as a Keragon workflow triggered via webhook. Implementation:
# KERAGON WORKFLOW: Patient Identity Verification
# Trigger: Retell AI custom function webhook call
# Response: Synchronous JSON response within 5 seconds
# Workflow Name: verify_patient_identity
# Trigger Type: Webhook (synchronous)
# Endpoint: https://api.keragon.com/v1/webhooks/{workflow_id}/trigger
# --- INPUT SCHEMA ---
# {
# "first_name": "string",
# "last_name": "string",
# "date_of_birth": "string (MM/DD/YYYY)",
# "phone_last_four": "string (optional)"
# }
# --- WORKFLOW STEPS ---
# Step 1: Normalize Input
# Action: Transform data
# - Trim whitespace from all fields
# - Convert first_name and last_name to proper case
# - Parse date_of_birth into ISO format (YYYY-MM-DD)
# - Remove any non-numeric characters from phone_last_four
# Step 2: Query EHR Patient Search API
# Action: HTTP Request (via EHR connector)
# --- athenahealth example ---
# Method: GET
# URL: https://api.athenahealth.com/v1/{practice_id}/patients
# Query Parameters:
# firstname: {normalized_first_name}
# lastname: {normalized_last_name}
# dob: {dob_in_MM/DD/YYYY}
# show: patientid,firstname,lastname,dob,mobilephone,lastappointmentdate
# Headers:
# Authorization: Bearer {ehr_access_token}
# --- Epic FHIR R4 example ---
# Method: GET
# URL: https://{epic_fhir_base}/api/FHIR/R4/Patient
# Query Parameters:
# family: {normalized_last_name}
# given: {normalized_first_name}
# birthdate: {dob_in_YYYY-MM-DD}
# Headers:
# Authorization: Bearer {ehr_access_token}
# Step 3: Evaluate Results
# Action: Conditional Logic
# Condition A: No results returned
# → Return: { "verified": false, "reason": "no_match" }
# Condition B: Multiple results returned
# → If phone_last_four provided, filter by phone number match
# → If still multiple: Return { "verified": false, "reason": "multiple_matches" }
# Condition C: Single result returned
# → Extract patient_id, last_appointment_date
# → Calculate months_since_last_visit
# → Return: {
# "verified": true,
# "patient_id": "{ehr_patient_id}",
# "patient_first_name": "{first_name}",
# "last_visit_date": "{last_appointment_date}",
# "months_since_last_visit": {calculated_months},
# "active_patient": {months_since_last_visit <= 18}
# }
# Step 4: Audit Logging
# Action: Log to audit trail
# Log entry: {
# "event": "patient_verification",
# "timestamp": "{current_utc_timestamp}",
# "input_name": "{first_name} {last_name}",
# "input_dob": "{date_of_birth}",
# "result": "{verified: true/false}",
# "patient_id": "{ehr_patient_id or null}",
# "source": "retell_ai_agent"
# }
# --- OUTPUT SCHEMA ---
# Success: { "verified": true, "patient_id": "12345", "patient_first_name": "John", "last_visit_date": "2024-11-15", "months_since_last_visit": 3, "active_patient": true }
# Failure: { "verified": false, "reason": "no_match" | "multiple_matches" }
# --- TIMEOUT HANDLING ---
# If EHR API does not respond within 4 seconds:
# Return: { "verified": false, "reason": "timeout" }
# Agent will transfer to human staff
# --- ERROR HANDLING ---
# If EHR API returns error (401, 403, 500, etc.):
# Return: { "verified": false, "reason": "system_error" }
# Send alert to MSP monitoring: "EHR API error in patient verification - {error_code}"Refill Eligibility Engine Workflow
Type: workflow The core decision engine that processes completed refill calls, applies clinical protocol rules, checks medication eligibility against the EHR, and either creates a refill task for provider approval or triggers escalation. This is the brain of the autonomous agent system.
Implementation:
# KERAGON WORKFLOW: Refill Eligibility Check and EHR Task Creation
# Trigger: Retell AI post-call webhook (call_ended event)
# Processing: Asynchronous (patient is no longer on the line)
# Workflow Name: process_refill_request
# Trigger Type: Webhook (asynchronous)
# Endpoint: https://api.keragon.com/v1/webhooks/{workflow_id}/trigger
# --- INPUT SCHEMA (from Retell AI call_ended webhook) ---
# {
# "call_id": "string",
# "call_recording_url": "string",
# "call_duration_seconds": integer,
# "call_status": "completed" | "transferred" | "failed",
# "extracted_data": {
# "patient_id": "string (from verification step)",
# "patient_name": "string",
# "patient_phone": "string",
# "medication_name": "string",
# "medication_dose": "string or null",
# "pharmacy_name": "string",
# "pharmacy_location": "string or null",
# "patient_notes": "string or null",
# "adverse_effects_reported": boolean,
# "controlled_substance_detected": boolean,
# "was_transferred": boolean,
# "transfer_reason": "string or null"
# }
# }
# --- WORKFLOW STEPS ---
# Step 1: Pre-filter
# Action: Conditional check
# If call_status != "completed" OR was_transferred == true:
# → Log: "Call {call_id} was transferred/incomplete. No automated processing."
# → End workflow
# If patient_id is null or empty:
# → Log: "Call {call_id} has no verified patient. Manual review needed."
# → Trigger escalation with reason: "unverified_patient"
# → End workflow
# Step 2: Medication Classification
# Action: Lookup against approved medication list
# Data source: Practice-specific approved medication JSON (stored in Keragon variables)
# APPROVED_MEDICATIONS_LIST (configure per practice based on Step 5 protocol doc):
# {
# "approved_classes": [
# { "class": "ACE Inhibitors", "medications": ["lisinopril", "enalapril", "ramipril", "benazepril", "captopril", "fosinopril", "quinapril"] },
# { "class": "ARBs", "medications": ["losartan", "valsartan", "irbesartan", "olmesartan", "telmisartan", "candesartan"] },
# { "class": "Beta Blockers", "medications": ["metoprolol", "atenolol", "propranolol", "carvedilol", "bisoprolol", "nebivolol"] },
# { "class": "CCBs", "medications": ["amlodipine", "nifedipine", "diltiazem", "verapamil", "felodipine"] },
# { "class": "Diuretics", "medications": ["hydrochlorothiazide", "chlorthalidone", "furosemide", "spironolactone", "triamterene"] },
# { "class": "Statins", "medications": ["atorvastatin", "rosuvastatin", "simvastatin", "pravastatin", "lovastatin", "pitavastatin", "fluvastatin"] },
# { "class": "Oral Diabetes", "medications": ["metformin", "glipizide", "glyburide", "glimepiride", "sitagliptin", "saxagliptin", "linagliptin", "empagliflozin", "dapagliflozin", "canagliflozin", "pioglitazone"] },
# { "class": "Thyroid", "medications": ["levothyroxine", "liothyronine", "synthroid", "armour thyroid"] },
# { "class": "PPIs", "medications": ["omeprazole", "pantoprazole", "esomeprazole", "lansoprazole", "rabeprazole", "dexlansoprazole"] },
# { "class": "Inhaled Respiratory", "medications": ["albuterol", "fluticasone", "budesonide", "montelukast", "tiotropium", "ipratropium", "salmeterol", "formoterol"] },
# { "class": "Oral Contraceptives", "medications": ["norgestimate-ethinyl estradiol", "levonorgestrel-ethinyl estradiol", "drospirenone-ethinyl estradiol", "norethindrone"] },
# { "class": "Antihistamines Rx", "medications": ["hydroxyzine", "cetirizine", "levocetirizine", "desloratadine"] }
# ]
# }
# Normalize medication_name: lowercase, trim, remove brand name suffixes
# Fuzzy match against approved list (Levenshtein distance <= 2 for typo tolerance)
# Result: { "is_approved": boolean, "matched_class": "string or null", "confidence": float }
# If NOT approved and NOT controlled (may be unknown/unlisted):
# → Trigger escalation with reason: "medication_not_on_approved_list"
# → End workflow
# Step 3: Query EHR for Active Prescriptions
# Action: HTTP Request via EHR connector
# --- athenahealth ---
# GET /v1/{practice_id}/patients/{patient_id}/medications
# Filter: active medications only
# --- Epic FHIR ---
# GET /api/FHIR/R4/MedicationRequest?patient={patient_id}&status=active
# Parse response to find matching medication
# Match criteria: medication name (fuzzy match) AND dose (if provided by patient)
# Extract: prescription_id, remaining_refills, last_fill_date, prescribing_provider
# Step 4: Eligibility Rules Engine
# Action: Sequential rule evaluation
# Rule 1: Active prescription exists?
# If NO matching active Rx found:
# → Escalate: "no_active_prescription"
# → SMS to patient: "We could not find an active prescription matching your request. Our clinical staff will follow up with you."
# → End
# Rule 2: Refills remaining?
# If remaining_refills == 0:
# → Create EHR task with flag: "NEW_PRESCRIPTION_NEEDED"
# → SMS to patient: "Your prescription for {medication} has no remaining refills. Your provider will review and determine if a new prescription can be issued. This may require an office visit."
# → End
# Rule 3: Last visit within required window?
# Query last appointment date (from verification step or separate API call)
# If months_since_last_visit > 12 (configurable per practice):
# → Create EHR task with flag: "OVERDUE_VISIT_REFILL_REQUEST"
# → SMS to patient: "Your refill request has been submitted, but your records show it has been over 12 months since your last visit. Your provider may require an appointment before approving this refill."
# → Continue (don't block, but flag)
# Rule 4: Refill frequency check
# If last_fill_date < 21 days ago (for 30-day supply) or < 75 days ago (for 90-day supply):
# → Escalate: "too_early_refill" (possible diversion or error)
# → SMS to patient: "It appears this medication was recently filled. Our clinical team will review your request."
# → End
# Step 5: Create EHR Refill Task (All rules passed)
# Action: HTTP Request via EHR connector
# --- athenahealth ---
# POST /v1/{practice_id}/patients/{patient_id}/documents/prescription
# Body: {
# "documentsubclass": "PRESCRIPTION_REFILL_REQUEST",
# "internalnote": "AI REFILL AGENT REQUEST\nMedication: {medication_name} {medication_dose}\nPharmacy: {pharmacy_name} {pharmacy_location}\nPatient Notes: {patient_notes}\nCall Recording: {call_recording_url}\nProcessed: {timestamp}\nAll eligibility checks PASSED.",
# "providerid": "{prescribing_provider_id}"
# }
# --- Generic FHIR R4 ---
# POST /api/FHIR/R4/Task
# Body: {
# "resourceType": "Task",
# "status": "requested",
# "intent": "order",
# "priority": "routine",
# "code": { "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/task-code", "code": "refill" }] },
# "description": "AI-Processed Refill: {medication_name} {medication_dose}",
# "for": { "reference": "Patient/{patient_id}" },
# "requester": { "display": "AI Refill Agent v1.0" },
# "owner": { "reference": "Practitioner/{prescribing_provider_id}" },
# "input": [
# { "type": { "text": "medication" }, "valueString": "{medication_name} {medication_dose}" },
# { "type": { "text": "pharmacy" }, "valueString": "{pharmacy_name} - {pharmacy_location}" },
# { "type": { "text": "patient_notes" }, "valueString": "{patient_notes}" },
# { "type": { "text": "call_recording" }, "valueString": "{call_recording_url}" },
# { "type": { "text": "eligibility_flags" }, "valueString": "{comma_separated_flags}" }
# ]
# }
# Step 6: Patient Confirmation SMS
# Action: Send SMS via Twilio (through Keragon Twilio connector)
# To: {patient_phone}
# Body: "{PRACTICE_NAME}: Your refill request for {medication_name} has been submitted to your provider for approval. Once approved, it will be sent to {pharmacy_name}. Expected processing time: {TURNAROUND_TIME}. Do not reply to this message."
# Note: Keep SMS HIPAA-safe — minimal PHI, medication name is acceptable as patient initiated the request
# Step 7: Audit Log Entry
# Action: Write to audit log
# {
# "event": "refill_request_processed",
# "timestamp": "{utc_timestamp}",
# "call_id": "{call_id}",
# "patient_id": "{patient_id}",
# "medication": "{medication_name}",
# "eligibility_result": "approved" | "escalated",
# "escalation_reason": "{reason or null}",
# "ehr_task_id": "{created_task_id}",
# "sms_sent": true/false,
# "processing_duration_ms": {elapsed_time}
# }Controlled Substance Detection Module
Type: skill A reference data module containing comprehensive lists of controlled substance names (brand and generic) across all DEA schedules (II-V). Used by both the voice agent prompt and the Keragon workflow to detect controlled substances and trigger immediate escalation. Maintained as a versioned JSON file.
Implementation:
# CONTROLLED SUBSTANCE REFERENCE DATA
# Version: 1.0
# Last Updated: 2025-01-15
# Source: DEA Controlled Substances Schedules
# Usage: Import into Retell AI Knowledge Base AND Keragon workflow variable
{
"version": "1.0",
"last_updated": "2025-01-15",
"schedules": {
"II": {
"description": "High potential for abuse; severe dependence",
"action": "IMMEDIATE_ESCALATION",
"medications": [
{"generic": "oxycodone", "brands": ["oxycontin", "roxicodone", "xtampza"]},
{"generic": "hydrocodone", "brands": ["vicodin", "norco", "lortab", "zohydro"]},
{"generic": "oxycodone/acetaminophen", "brands": ["percocet", "endocet"]},
{"generic": "morphine", "brands": ["ms contin", "kadian", "avinza"]},
{"generic": "fentanyl", "brands": ["duragesic", "subsys", "actiq", "abstral"]},
{"generic": "hydromorphone", "brands": ["dilaudid", "exalgo"]},
{"generic": "methadone", "brands": ["dolophine", "methadose"]},
{"generic": "meperidine", "brands": ["demerol"]},
{"generic": "amphetamine/dextroamphetamine", "brands": ["adderall", "adderall xr"]},
{"generic": "methylphenidate", "brands": ["ritalin", "concerta", "daytrana", "quillivant"]},
{"generic": "lisdexamfetamine", "brands": ["vyvanse"]},
{"generic": "dextroamphetamine", "brands": ["dexedrine", "zenzedi"]},
{"generic": "methamphetamine", "brands": ["desoxyn"]},
{"generic": "tapentadol", "brands": ["nucynta"]},
{"generic": "cocaine", "brands": []},
{"generic": "pentobarbital", "brands": ["nembutal"]},
{"generic": "secobarbital", "brands": ["seconal"]}
]
},
"III": {
"description": "Moderate potential for abuse; moderate dependence",
"action": "IMMEDIATE_ESCALATION",
"medications": [
{"generic": "acetaminophen/codeine", "brands": ["tylenol 3", "tylenol with codeine"]},
{"generic": "buprenorphine", "brands": ["subutex"]},
{"generic": "buprenorphine/naloxone", "brands": ["suboxone", "zubsolv"]},
{"generic": "ketamine", "brands": ["ketalar"]},
{"generic": "testosterone", "brands": ["androgel", "testim", "depo-testosterone", "axiron"]},
{"generic": "nandrolone", "brands": ["deca-durabolin"]},
{"generic": "oxandrolone", "brands": ["oxandrin"]},
{"generic": "benzphetamine", "brands": ["didrex"]},
{"generic": "phendimetrazine", "brands": ["bontril"]}
]
},
"IV": {
"description": "Low potential for abuse; limited dependence",
"action": "IMMEDIATE_ESCALATION",
"medications": [
{"generic": "alprazolam", "brands": ["xanax", "xanax xr"]},
{"generic": "diazepam", "brands": ["valium"]},
{"generic": "clonazepam", "brands": ["klonopin"]},
{"generic": "lorazepam", "brands": ["ativan"]},
{"generic": "temazepam", "brands": ["restoril"]},
{"generic": "triazolam", "brands": ["halcion"]},
{"generic": "midazolam", "brands": ["versed"]},
{"generic": "zolpidem", "brands": ["ambien", "ambien cr", "edluar", "intermezzo"]},
{"generic": "eszopiclone", "brands": ["lunesta"]},
{"generic": "zaleplon", "brands": ["sonata"]},
{"generic": "tramadol", "brands": ["ultram", "ultracet", "conzip"]},
{"generic": "carisoprodol", "brands": ["soma"]},
{"generic": "phenobarbital", "brands": ["luminal"]},
{"generic": "modafinil", "brands": ["provigil"]},
{"generic": "armodafinil", "brands": ["nuvigil"]},
{"generic": "phentermine", "brands": ["adipex-p", "lomaira"]},
{"generic": "suvorexant", "brands": ["belsomra"]},
{"generic": "lemborexant", "brands": ["dayvigo"]}
]
},
"V": {
"description": "Lowest potential for abuse",
"action": "IMMEDIATE_ESCALATION",
"medications": [
{"generic": "pregabalin", "brands": ["lyrica", "lyrica cr"]},
{"generic": "lacosamide", "brands": ["vimpat"]},
{"generic": "brivaracetam", "brands": ["briviact"]},
{"generic": "cenobamate", "brands": ["xcopri"]},
{"generic": "codeine cough syrup", "brands": ["promethazine with codeine", "robitussin ac"]},
{"generic": "diphenoxylate/atropine", "brands": ["lomotil"]}
]
}
},
"matching_rules": {
"case_sensitive": false,
"match_type": "fuzzy",
"max_levenshtein_distance": 2,
"match_against": ["generic", "brands"],
"partial_match": true,
"notes": "Match any substring — e.g., patient saying 'my xanax' should match 'xanax'. Also match common misspellings and phonetic variations."
}
}Escalation Handler Workflow
Type: workflow Handles all escalation scenarios: controlled substance requests, adverse effects, unverified patients, unapproved medications, too-early refills, and system errors. Routes to the appropriate clinical staff member and creates documentation in the EHR. Implementation:
# Routes escalated requests to clinical staff with full context
# KERAGON WORKFLOW: Escalation Handler
# Trigger: Called by Refill Eligibility Engine or directly by Retell AI transfer event
# Purpose: Route escalated requests to clinical staff with full context
# Workflow Name: handle_escalation
# Trigger Type: Internal workflow call OR Webhook
# --- INPUT SCHEMA ---
# {
# "escalation_type": "controlled_substance" | "adverse_effects" | "unverified_patient" | "unapproved_medication" | "no_active_prescription" | "too_early_refill" | "overdue_visit" | "patient_requested_human" | "system_error" | "ai_comprehension_failure",
# "call_id": "string",
# "call_recording_url": "string",
# "patient_id": "string or null",
# "patient_name": "string",
# "patient_phone": "string",
# "medication_name": "string or null",
# "additional_context": "string or null",
# "timestamp": "ISO 8601 string"
# }
# --- ESCALATION ROUTING MATRIX ---
# Configure per practice:
# ROUTING_CONFIG = {
# "controlled_substance": {
# "priority": "high",
# "route_to": "clinical_nurse",
# "notification_method": ["sms", "ehr_task"],
# "sla_minutes": 60,
# "ehr_task_priority": "urgent"
# },
# "adverse_effects": {
# "priority": "urgent",
# "route_to": "provider",
# "notification_method": ["sms", "ehr_task", "email"],
# "sla_minutes": 30,
# "ehr_task_priority": "urgent"
# },
# "unverified_patient": {
# "priority": "normal",
# "route_to": "front_desk",
# "notification_method": ["ehr_task"],
# "sla_minutes": 240,
# "ehr_task_priority": "routine"
# },
# "unapproved_medication": {
# "priority": "normal",
# "route_to": "clinical_nurse",
# "notification_method": ["ehr_task"],
# "sla_minutes": 120,
# "ehr_task_priority": "routine"
# },
# "no_active_prescription": {
# "priority": "normal",
# "route_to": "provider",
# "notification_method": ["ehr_task"],
# "sla_minutes": 240,
# "ehr_task_priority": "routine"
# },
# "too_early_refill": {
# "priority": "high",
# "route_to": "clinical_nurse",
# "notification_method": ["sms", "ehr_task"],
# "sla_minutes": 60,
# "ehr_task_priority": "urgent"
# },
# "overdue_visit": {
# "priority": "normal",
# "route_to": "clinical_nurse",
# "notification_method": ["ehr_task"],
# "sla_minutes": 240,
# "ehr_task_priority": "routine"
# },
# "patient_requested_human": {
# "priority": "normal",
# "route_to": "front_desk",
# "notification_method": ["none_realtime_transfer"],
# "sla_minutes": 0,
# "ehr_task_priority": null
# },
# "system_error": {
# "priority": "critical",
# "route_to": "msp_support",
# "notification_method": ["email", "sms"],
# "sla_minutes": 15,
# "ehr_task_priority": null
# },
# "ai_comprehension_failure": {
# "priority": "normal",
# "route_to": "front_desk",
# "notification_method": ["ehr_task"],
# "sla_minutes": 120,
# "ehr_task_priority": "routine"
# }
# }
# --- WORKFLOW STEPS ---
# Step 1: Look up routing config for escalation_type
# Step 2: Create EHR task (if configured)
# POST to EHR task API with appropriate priority and assignee
# Include in task description:
# - Escalation reason
# - Patient name and ID
# - Medication requested
# - Link to call recording
# - Additional context from AI agent
# - Timestamp
# Step 3: Send SMS notification (if configured)
# To: Designated clinical staff phone number per route_to config
# Body: "[REFILL ESCALATION - {priority}] Patient: {patient_name}. Reason: {escalation_type}. Medication: {medication_name}. Call recording available in EHR task. Please respond within {sla_minutes} minutes."
# Step 4: Send email notification (if configured)
# To: Designated email per route_to config
# Subject: "[AI Refill Escalation] {patient_name} - {escalation_type}"
# Body: Full escalation details including call transcript excerpt
# Step 5: Send MSP alert for system errors
# If escalation_type == "system_error":
# Send to MSP monitoring system (email, PagerDuty, or ConnectWise ticket)
# Include: error details, affected patient count, suggested remediation
# Step 6: Patient notification
# If patient was NOT transferred in real-time:
# SMS to patient: "Your refill request requires additional review by our clinical staff. Someone will contact you within {sla_hours} hours. If this is urgent, please call {PRACTICE_MAIN_NUMBER}."
# Step 7: Audit log
# Log all escalation details, routing decisions, and notification timestamps
# --- SLA MONITORING (Keragon scheduled check) ---
# Create a separate scheduled workflow that runs every 30 minutes:
# Query open escalation tasks in EHR
# For any task exceeding its SLA:
# Send reminder notification to route_to staff
# After 2x SLA: escalate to practice manager
# After 3x SLA: alert MSP supportDaily Operations Dashboard and Reporting Agent
Type: agent An automated reporting agent that generates daily, weekly, and monthly operational reports on the AI refill system's performance. Provides KPIs including call volume, completion rate, escalation rate, average processing time, and cost savings. Delivered via email to practice management and MSP account manager.
Implementation
# Workflow Name: generate_operations_report
# --- DAILY REPORT STEPS ---Step 1: Query Retell AI Analytics API
GET https://api.retellai.com/v1/analytics
Parameters: date_from={yesterday}, date_to={yesterday}, agent_id={agent_id}- total_calls: integer
- completed_calls: integer (patient stayed on line through completion)
- transferred_calls: integer (transferred to human)
- abandoned_calls: integer (patient hung up)
- average_duration_seconds: float
- total_minutes_used: float
- total_cost: float (Retell billing)
Step 2: Query Keragon Workflow Logs
- Query internal Keragon API for workflow run statistics
- Filter: workflow_name in ['process_refill_request', 'handle_escalation']
- Date: yesterday
- refills_processed: integer (successfully created EHR tasks)
- escalations_total: integer
- escalation_breakdown: { controlled_substance: N, adverse_effects: N, ... }
- errors: integer
- average_processing_time_ms: float
Step 3: Calculate KPIs
completion_rate = completed_calls / total_calls * 100
escalation_rate = escalations_total / total_calls * 100
automation_rate = refills_processed / total_calls * 100
cost_per_refill = total_cost / refills_processed
estimated_savings = refills_processed * $11 - total_cost
daily_staff_hours_saved = refills_processed * 4.5 / 60Step 4: Generate Report HTML
# ========================================
# AI PRESCRIPTION REFILL DAILY REPORT
# {PRACTICE_NAME} | {DATE}
# ========================================
#
# CALL VOLUME
# Total Calls: {total_calls}
# Completed: {completed_calls} ({completion_rate}%)
# Transferred to Staff: {transferred_calls}
# Abandoned: {abandoned_calls}
#
# REFILL PROCESSING
# Successfully Routed to Provider: {refills_processed}
# Automation Rate: {automation_rate}%
# Average Processing Time: {avg_time}s
#
# ESCALATIONS
# Total Escalations: {escalations_total} ({escalation_rate}%)
# - Controlled Substances: {N}
# - Adverse Effects: {N}
# - No Active Rx: {N}
# - Unapproved Medication: {N}
# - Patient Requested Human: {N}
# - Other: {N}
#
# COST & SAVINGS
# AI Platform Cost: ${total_cost}
# Cost per Refill: ${cost_per_refill}
# Estimated Manual Cost (at $11/refill): ${manual_cost}
# Estimated Daily Savings: ${estimated_savings}
# Staff Hours Saved: {hours_saved} hours
#
# ERRORS / ISSUES
# System Errors: {errors}
# [List any error details]
#
# ========================================
# Generated by AI Operations Agent | MSP: {MSP_NAME}
# ========================================Step 5: Send Report
- Email to: practice_manager@practice.com, msp_account_manager@msp.com
- Subject: "[AI Refill Report] {PRACTICE_NAME} - {DATE}"
Step 6: Weekly/Monthly Aggregation
- For weekly (Monday): aggregate last 7 days, add trend analysis
- For monthly (1st): aggregate last 30 days, add month-over-month comparison
- Top 10 medications by refill volume
- Escalation trend analysis
- Total monthly savings
- ROI calculation: total_savings / total_system_cost * 100
Step 7: Alert on Anomalies
- If completion_rate < 70%: ALERT MSP — possible system issue
- If escalation_rate > 40%: ALERT — protocol may need adjustment
- If errors > 5 in a day: ALERT MSP — investigate system health
- If abandoned_calls > 30%: ALERT — AI agent quality issue, review call recordings
Testing & Validation
- IDENTITY VERIFICATION TEST: Call the AI refill line and provide the name and DOB of Test Patient Alpha (synthetic). Verify the agent confirms identity and proceeds. Then call again with a non-existent patient (Test Patient Epsilon) and verify the agent gracefully transfers to the front desk after failing verification.
- HAPPY PATH REFILL TEST: Call as Test Patient Alpha requesting a refill for Lisinopril 10mg at CVS Pharmacy on Main Street. Verify: (1) AI agent correctly collects all information, (2) call summary is accurate, (3) Keragon workflow executes successfully, (4) refill task appears in EHR test environment assigned to correct provider, (5) SMS confirmation is received on test phone, (6) audit log entry is created.
- CONTROLLED SUBSTANCE ESCALATION TEST: Call as Test Patient Gamma requesting a refill for Alprazolam 0.5mg. Verify: (1) AI agent detects controlled substance within 1-2 exchanges, (2) agent does NOT collect further refill details, (3) agent transfers call to clinical escalation number, (4) Keragon escalation workflow fires, (5) escalation notification is sent to clinical staff, (6) NO automated EHR refill task is created.
- NO REMAINING REFILLS TEST: Call as Test Patient Beta requesting Metformin 500mg refill. Verify: (1) Keragon workflow detects 0 remaining refills, (2) EHR task is created with NEW_PRESCRIPTION_NEEDED flag, (3) patient receives appropriate SMS explaining provider review is needed.
- OVERDUE VISIT TEST: Call as Test Patient Delta requesting Atorvastatin 20mg. Verify: (1) Keragon detects last visit was >12 months ago, (2) EHR task is created with OVERDUE_VISIT flag, (3) patient receives SMS noting potential visit requirement, (4) refill request is still submitted (flagged, not blocked).
- ADVERSE EFFECTS ESCALATION TEST: Call as Test Patient Alpha requesting Lisinopril refill, but when asked about side effects, report experiencing a persistent dry cough and dizziness. Verify: (1) AI agent detects adverse effect report, (2) agent immediately offers to transfer to clinical staff, (3) escalation workflow fires with adverse_effects type, (4) provider receives urgent notification.
- MEDICATION NAME RECOGNITION TEST: Call and request refills using common mispronunciations, abbreviations, and brand names: 'my water pill' (hydrochlorothiazide), 'Lipitor' (atorvastatin brand), 'sugar medicine' (metformin). Verify: (1) AI agent either correctly maps the name or asks clarifying questions, (2) if unable to determine after 3 attempts, agent transfers to human.
- PATIENT HANG-UP RECOVERY TEST: Call the AI line, provide identity verification, state a medication, then hang up mid-conversation. Verify: (1) Retell AI logs an incomplete call, (2) Keragon does NOT create an EHR task for incomplete data, (3) no false SMS confirmation is sent to patient.
- AFTER-HOURS OPERATION TEST: Call the AI refill line outside of practice business hours. Verify: (1) AI agent handles the call normally, (2) refill request is processed and EHR task is created, (3) patient SMS includes after-hours messaging ('will be processed during business hours'), (4) provider notification is deferred until morning batch.
- CONCURRENT CALL LOAD TEST: Have 3-5 people call the AI refill line simultaneously. Verify: (1) all calls are answered without busy signals, (2) each call is processed independently, (3) no data crossover between concurrent sessions, (4) Keragon workflows process all calls correctly.
- HIPAA COMPLIANCE VERIFICATION: Review 10 completed call recordings. Verify: (1) AI agent does not repeat back excessive PHI unnecessarily, (2) recordings are encrypted at rest, (3) SMS confirmations contain only minimum necessary PHI, (4) audit logs capture all PHI access events, (5) call recordings are accessible only to authorized users with proper authentication.
- NETWORK FAILOVER TEST: Temporarily block outbound traffic from the practice network to Retell AI endpoints at the FortiGate firewall. Call the refill line. Verify: (1) Twilio fallback TwiML bin activates, (2) patient hears the fallback message and is transferred to practice staff, (3) FortiGate logs the blocked connection, (4) MSP monitoring alert fires within 15 minutes.
- DAILY REPORT GENERATION TEST: Trigger the daily operations report workflow manually. Verify: (1) report email is received by practice manager and MSP account manager, (2) all KPIs are calculated correctly against known test data, (3) report formatting is clean and professional, (4) anomaly alerts fire correctly when test thresholds are exceeded.
- EHR API TOKEN REFRESH TEST: Allow the EHR OAuth2 access token to expire (typically 1 hour for athenahealth). Process a refill request and verify: (1) Keragon automatically refreshes the token using the refresh token, (2) refill processing is not interrupted, (3) no patient-visible error occurs.
- END-TO-END PROVIDER APPROVAL TEST: After AI creates a refill task in the EHR, have a provider log in, review the task, approve it, and send the prescription to the pharmacy via Surescripts. Verify: (1) the EHR task contains all necessary information for provider decision-making, (2) the approval workflow functions correctly, (3) the Surescripts transmission succeeds.
Client Handoff
The client handoff should be a structured 2-hour session with the practice owner/manager, clinical champion (physician), office manager, and lead nurse/MA. Cover the following:
Maintenance
Ongoing MSP Responsibilities:
Daily (Automated):
- Review AI operations daily report email (auto-generated by the reporting agent)
- Monitor FortiGate firewall alerts via FortiCloud dashboard
- Monitor Retell AI platform status page for outages
- Review any system error escalations from the escalation handler workflow
Weekly (15-30 min per practice):
- Review weekly operations report with focus on escalation rate trends and error patterns
- Check Keragon workflow error logs for any failed workflow runs
- Verify EHR API connectivity (check token refresh is working)
- Review 5-10 random call recordings for AI quality assurance
- Check Twilio account for any delivery failures or number issues
Monthly (1-2 hours per practice):
- Generate and review monthly operations report with ROI analysis
- Present monthly report to practice manager in brief (15 min) review call
- Update Retell AI Knowledge Base if practice adds new medications or changes pharmacy partnerships
- Review and rotate any API credentials approaching expiration
- Verify HIPAA compliance documentation is current (BAAs, access logs)
- Check for Retell AI, Keragon, and FortiGate firmware/platform updates — apply during maintenance window
- Review controlled substance reference data for any DEA schedule changes
Quarterly (2-4 hours per practice):
- Conduct HIPAA compliance review: audit access logs, verify minimum necessary compliance, review BAA status
- Meet with clinical champion to review and update refill protocols if needed
- Review AI agent conversation quality: analyze calls where patients requested human transfer, identify improvement opportunities
- Update the approved medication list based on practice formulary changes
- Retell AI voice model and prompt optimization based on call analytics
- FortiGate firmware update and security audit
- NordLayer access review: remove departed staff, add new users
- Test disaster recovery/rollback procedure
Annually:
- Full HIPAA Security Risk Assessment renewal
- BAA renewal and vendor compliance verification
- Comprehensive AI agent performance review and optimization
- Network security assessment and penetration test
- Review industry regulatory changes (state medical practice acts, HIPAA updates, DEA rules)
- Contract and pricing review with all SaaS vendors
- Recommend system upgrades or expanded automation opportunities
SLA Considerations:
- AI agent downtime target: <0.1% monthly (99.9% uptime) — depends on Retell AI SLA
- MSP response time for Tier 1 issues: 4 business hours
- MSP response time for Tier 2 issues (integration failures): 2 business hours
- MSP response time for Tier 3 issues (system down, no refills processing): 1 hour
- Monthly uptime report included in operations report
Model Retraining / Prompt Optimization Triggers:
- Completion rate drops below 70% for 3+ consecutive days
- Escalation rate exceeds 35% for 1+ week (investigate if protocol is too restrictive or AI is misclassifying)
- New medications added to practice formulary (update approved list and Knowledge Base)
- Patient satisfaction survey scores decline
- New controlled substance added to DEA schedules (update detection module immediately)
- Retell AI releases new voice models or features (evaluate and upgrade)
Escalation Path:
- Practice staff → MSP helpdesk (general issues)
- MSP L1 tech → MSP L2 integration engineer (EHR/API issues)
- MSP L2 → Retell AI support (AI platform issues)
- MSP L2 → Keragon support (workflow issues)
- MSP → Practice clinical champion (clinical protocol questions)
- Any HIPAA breach suspicion → MSP compliance lead + practice compliance officer (immediate)
Alternatives
Emitrr All-in-One Platform (Instead of Retell AI + Twilio)
Replace the Retell AI + Twilio combination with Emitrr, which provides an all-in-one patient communication platform including AI voice agent, SMS, chat, and VoIP in a single package. Emitrr offers 500+ native integrations and transparent pricing starting at $149/month base plus $30/user/month for VoIP and $99/month for the AI agent add-on.
Hyro AI Enterprise Platform (Instead of Retell AI + Keragon)
Replace both the AI agent layer (Retell AI) and workflow orchestration layer (Keragon) with Hyro AI, an enterprise healthcare AI platform that provides omnichannel voice and chat agents with deep Epic and major EHR integrations built in. Hyro handles the full stack from patient conversation through EHR integration in a single platform.
DrFirst Rcopia/iPrescribe Integration (Medication-Focused Approach)
Instead of building a general AI voice agent, integrate with DrFirst's existing prescription management platform which provides AI-powered renewal workflow automation natively integrated into 300+ EHRs. DrFirst focuses specifically on the prescription workflow rather than the patient communication layer.
Open-Source Build with n8n + Vocode/Pipecat (Maximum Control)
Build the entire solution using open-source components: n8n (self-hosted workflow automation) instead of Keragon, and an open-source voice AI framework like Vocode or Pipecat instead of Retell AI. Host on a HIPAA-compliant cloud (AWS GovCloud or Azure with BAA). This gives the MSP complete control over the stack.
Patient Portal + Chatbot Only (No Voice AI)
Skip the voice AI component entirely and implement a text-based chatbot on the practice website and patient portal that handles refill requests via typed conversation. Use a platform like Capacity or a custom chatbot built on Retell AI's chat capabilities. Patients request refills by typing rather than calling.
Want early access to the full toolkit?