CareCoreAI Use Policy

Responsible AI

The assistant handles the operations. People stay accountable for the care.

CareCore answers carer calls and family messages, and turns them into rota changes, tasks, incidents and escalations. This policy states plainly what that assistant is permitted to do, where it is blocked, how it is supervised, and what rights you have when it is involved in something about you.

Principles

Six rules the product is built around

Each is enforced in the system's behaviour, not only stated here.

Bounded autonomy

The assistant may act only inside a fixed set of operational actions. Everything else is a handover, never a guess.

Evidence by default

Every AI-handled call or message stores the detected intent, confidence, the policy extract relied upon and the records created.

A named human always owns risk

Clinical, safeguarding and out-of-scope matters are routed to a named coordinator through the on-call chain with time-to-human recorded.

Transparent to the caller

Callers and family members are told they are speaking to an assistant and can ask for a person at any point in the conversation.

Data minimisation at inference

Only the transcript snippet, message text and retrieved policy extract are sent to a model. Never a full client record. No training on your data.

Reversibility

Every AI-created task, incident or escalation can be reopened, amended or withdrawn by a coordinator, with the change audited.

Permitted actions

What the assistant is allowed to do

This is the complete operational scope. Anything not on this list is a human handover.

  • ✓Answer rota and visit questions from the agency's live schedule
  • ✓Answer policy questions grounded in the agency's own knowledge base
  • ✓Log a task or callback request for a coordinator
  • ✓Raise an incident when a fall, missed visit or safeguarding concern is reported
  • ✓Raise an escalation and page the on-call coordinator
  • ✓Broadcast a cover request to eligible carers and record their responses
  • ✓Transfer a live call to a human coordinator
  • ✓Confirm a request in writing with a reference number

Prohibited actions

What the assistant never does

These are hard boundaries. The assistant defers to a person rather than attempting them.

  • ✕Give medical, clinical or medication advice
  • ✕Change a care plan, risk assessment or medication instruction
  • ✕Close, downgrade or dismiss a safeguarding or clinical incident
  • ✕Decide whether a person receives or loses care
  • ✕Discipline, rank or performance-manage a carer
  • ✕Disclose one client's information to another caller
  • ✕Diagnose, triage clinically, or advise against calling 999
  • ✕Act on a request it cannot verify against the agency's own records

Confidence gating

When the assistant is blocked from acting

Certainty determines behaviour. Low certainty produces a handover, not a plausible-sounding answer.

SituationBehaviour
High confidence, in scopeThe assistant answers or acts, records the action and confirms with a reference number.
Medium confidenceThe assistant asks a clarifying question rather than acting. If still unclear, it logs a task for a coordinator instead of guessing.
Low confidence or out of scopeNo action is taken on the caller's behalf. The thread is marked as needing a human, a coordinator is paged, and the caller is told a person will come back to them.
Any urgency signalFalls, injuries, missed visits, safeguarding language and distress bypass confidence entirely — an incident and escalation are raised immediately and a human is paged.
Write failureIf a record cannot be written, the assistant does not claim success. It forces a human handover and pages the coordinator.

Human oversight

Who supervises the assistant

Meaningful oversight means a person can see the reasoning and reverse the outcome.

  1. 1

    Managers see every AI-created record in the same queues as human-created ones, tagged with source: ai.

  2. 2

    Call and thread detail pages show the reasoning trail: intent, confidence, retrieved policy and resulting records.

  3. 3

    Coordinators can override, reassign, amend or withdraw any AI decision; the override is written to the audit log.

  4. 4

    A registered manager is accountable for AI-assisted operations at each agency, as part of their CQC responsibilities.

  5. 5

    Persistent misrouting can be corrected by editing the knowledge base — the behaviour changes without a code release.

Models

What runs where

Models are used for inference only. Care content is not used to train or fine-tune any model, and material changes to triage behaviour are logged.

UseCapabilityNotes
Speech-to-text (calls)Deepgram Nova-3 with care-specific vocabulary boostingDiarised, UK English
Voice responsesManaged text-to-speech with tone steeringCached audio, private storage
Call & message triageManaged frontier LLM via the AI gatewayInference only, no training
Knowledge retrievalRetrieval over the agency's own knowledge baseExtracts stored with each answer

Accuracy & fairness

How we keep it honest

AI performance in care has to be measured, not assumed.

  • Transcription accuracy is boosted with agency-specific terms — client names, carer names and care vocabulary — and reviewed against recordings.
  • Every intent decision is stored with its confidence, so misclassification rates can be measured rather than estimated.
  • Escalation outcomes are reviewable: how many AI escalations a human agreed with, and how many were withdrawn.
  • Language and accent handling is monitored for uneven performance; where accuracy drops, the route defaults to a human rather than a guess.
  • Model or prompt changes that affect triage behaviour are recorded with a date and a reason.

Your rights

If the assistant handled something about you

Plain-language rights for service users, families and carers.

  • You can ask for a person at any time — say “coordinator” on a call or in a message and you will be handed over.
  • You can ask what the assistant recorded about your call or message and request a copy.
  • You can ask for a human review of any decision the assistant made about an operational request.
  • You can ask for a correction if the assistant recorded something inaccurately.
  • Data protection requests are handled by your care agency as controller, with CareCore supporting within the statutory month.

Questions about AI use: privacy@carecore.app. Your care agency remains the data controller and the decision maker for your care.