HR Inbox Agent
An LLM triage agent for an HR mailbox where the model proposes and plain code decides.
- Role
- Designer & engineer
- Team
- Solo, with HR as domain owner
- Timeline
- Sep 2026 – Ongoing
- Organisation
- Green Compare Ltd
- Code
- Proprietary (employer)
- Reading time
- 3 min
A Python agent that reads an HR mailbox through Microsoft Graph, asks a language model for a structured classification and draft reply, and lets deterministic code decide whether to send, save a draft or escalate to Slack.
Overview
An HR inbox receives job applications, sick-leave and urgent-leave notices — and phishing such as "please change my bank details". HR wanted fast, polite acknowledgements without an AI ever making a decision about a person.
The problem
- Replies must be quick, but some emails must never get an automatic answer.
- Email content is untrusted input — a perfect prompt-injection vector.
- Health information must not leak into logs, alerts or third-party systems.
Context & constraints
Mail lives in Microsoft 365, alerts go to Slack, and every model call goes through XEOO AI Control — our AI gateway — which enforces a model allowlist, rate limits and metadata-only capture. It starts on free OpenRouter models and can switch to Claude with a configuration change.
Approach
The pipeline runs check → triage → decide → act → label → alert → log:
- Code checks first — automated mail is detected from headers, Reply-To mismatches are treated as phishing, bank-change wording is flagged.
- LLM triage — the model returns strict JSON: a category (job application, sick leave, urgent leave, other, spam), risk flags and a draft acknowledgement. It has no tools.
- Policy decides — a reply is sent automatically only when every hold passes: an auto-reply category, no model or code risk flags, the model marked it safe with high confidence, no decision wording (offers, rejections, interview invites), a reply exists, no auto-reply to this sender in 24 hours, and the agent is in auto mode.
Anything doubtful becomes a draft or an escalation. Sick and urgent leave always send a Slack FYI — without health details.
Key capabilities
- Structured JSON classification with eight risk flags
- Eight independent safety holds before any auto-send
- Prompt-injection posture: email marked untrusted, model has no tools
- Least-privilege mailbox access scoped to one mailbox
- Crash-safe sending: a message marked "sending" is never retried
- Evaluation harness with an "unsafe" metric
Architecture
Inputs
Pipeline
Model access
Outputs
A deterministic pipeline with one probabilistic step in the middle — and code on both sides of it. Select a component to trace its connections.
Key decisions
ADR-01The model proposes; code disposes
- Context
- A model that is right 98% of the time will still, eventually, auto-reply to a resignation or a grievance.
- Decision
- The model only produces a structured proposal. Every rule that matters lives in deterministic, unit-tested code.
- Trade-offs
- Fewer emails are fully automated — the right trade for anything involving people.
ADR-02Measure "unsafe", not just accuracy
- Context
- Accuracy hides the one failure that matters: an email that should have escalated but would have been auto-sent.
- Decision
- The evaluation harness reports category accuracy and an unsafe count, and HR can label real mail for evaluation.
- Trade-offs
- Labelling takes HR time; the safety signal is worth it.
ADR-03Roll out in draft mode first
- Context
- Trust has to be earned on real mail, not fixtures.
- Decision
- Start with drafts only for one to two weeks, review, then enable supervised auto-replies.
- Trade-offs
- Slower time-to-value, much lower risk.
Challenges
Crashes mid-send
Problem. A crash between "send" and "record" could send the same reply twice.
Solution. Status `sending` is written before the send; a message found in that state is escalated, never retried.
Old mail on first start
Problem. Connecting to an existing mailbox could trigger replies to months-old email.
Solution. A first-start watermark ignores older mail, with a 10-minute overlap window to catch late timestamps.
Quality & delivery
Testing
58 offline tests run against a fake mailbox, a recording notifier and mocked HTTP — for example a crash while sending is never retried and code flags block sending even if the model says safe.
Deployment
Runs as a non-root Python 3.12 container with a data volume and a read-only knowledge mount. Before going live it needs the mailbox connection, an evaluation on real labelled mail and the gateway's data-processing agreement.
Outcome
- Unsafe auto-sends
- 0
- Across 3 evaluation runs on fixtures
- Fixture emails classified correctly
- 14–16 / 16
- Free-tier models
- Independent safety holds
- 8
- All must pass to auto-send
- Offline tests
- 58
- pytest
The agent is feature-complete and evaluated on synthetic fixtures with zero unsafe decisions. It is waiting on the mailbox connection and a real-mail evaluation before entering draft mode.
Lessons
- Guardrails are a product feature, not a prompt.
- An agent's most important output is often "I'm not sure — a human should look".
Building something similar?
Tell me what you’re working on — I reply to every serious message.
Related work
Green Compare CRM
A FastAPI modular monolith with a React 19 front end, where Postgres row-level security backs every permission check. I started the repository, then shipped features across finance, telesales, renewals, dashboards and access control.
- TypeScript
- Python
- React
- Ant Design
- TanStack Query
- FastAPI
- +9