The problem this solves
400,000 credit card transactions from the processor against 400,000 GL rows. Four people spend six days pairing them in spreadsheets, and the exceptions still arrive at period end as a wall. Transaction Matching is a high-volume matching engine at transaction level: configurable rules pair items between two or more data sources, auto-match clears 98%, and a human works the 2%. It is the most dramatic before-and-after in the product — one person, half a day — and it is also the module most often scoped by enthusiasm rather than by transaction volume.
It does not manage workflow. It produces matched sets and unmatched exceptions, and the unmatched balance feeds a Reconciliation Compliance reconciliation as its balance. That relationship is the reason to implement both rather than treating them as alternatives.
Audience
Treasury and cash-application teams reconciling bank, card-processor, and intercompany activity. Close managers deciding which three or four accounts genuinely need matching. Consultants who have to answer “why is our auto-match rate stuck at 71%?” with the composition of the exception population rather than another rule.
Two data sources, six rules, three data-quality scenarios
| Object | What it is | In this build |
|---|---|---|
| NLQ query: “Why is the auto-match rate stuck at 71%?” or “Fix the reference key on load” — DeepSeek resolves the scenario and, when the user asks for it, the normalise flag | ||
| Data Source A | The GL side | 1,200 rows: amount, day, reference (INV-10123), payee, description |
| Data Source B | The bank side | ~1,190 rows generated from A with perturbations: date shifts, reformatted or missing references, deposit splits, batch payments, fees, duplicates, items in transit, bank charges |
| Scenario | Source data quality | Clean shared reference · amount and date only · poor references with manual descriptions |
| Matching rules | The logic that pairs transactions, in sequence | r1 exact 1:1 → r2 date ±3 → r3 one-to-many → r4 many-to-one → r5 amount tolerance → r6 amount + date only |
| Load-time fixes | Data changes, not rules | Normalise references (transformation) · bank file carries the reference (the upstream file change) |
An auto-match rate you can explain, and an exception population you can act on
- KPI row: auto-match rate against the expected range for the scenario, matched/total rows, exceptions to work, and false matches — which must be zero
- Per-rule chart: rows matched by each rule with false matches overlaid, so loosening a rule shows its cost immediately
- Feeds RC: unmatched GL and bank totals, adjustments created for fees, and the net difference that becomes the reconciling item on the cash reconciliation
- Exception population: every unmatched row categorised by root cause with its real fix — and the honest total of how many are rule changes versus data changes
The second engine, deployed second
Most implementations deploy Reconciliation Compliance first across the whole portfolio, then add Transaction Matching for the three or four accounts that genuinely need it — bank, credit card, intercompany, sometimes AR cash application. Building TM first automates four accounts while the remaining 1,796 stay uncontrolled, which is not the finding the auditor raised. Its output lands on the cash reconciliation in UC10 and, through it, in the aging and evidence views of UC12.
Recommended integration points
- Discovery: the question that settles scope — “are they matching individual line items, or explaining a difference between two totals?”
- Daily close: TM loads daily; matching continuously through the month means period end presents a small exception queue rather than 400,000 rows at once
- Rate stuck low: sample 100 exceptions and categorise them — this page is that exercise, automated
The numbers
The honest checklist
- โA handful of accounts carry hundreds of thousands of transactions a month that people are pairing by hand — bank, card processor, intercompany, cash application
- โThere is, or can be, a shared reference between the two systems — or you accept that amount-and-date matching needs an ambiguity guard and a bigger exception queue
- โReconciliation Compliance is already in place to receive the unmatched balance with workflow and evidence
- โThe reconciliation pain is control and visibility, not volume — TM would be expensive over-engineering
- โThe two systems share no key and nobody will change the file — no volume of rule writing recovers a rate stuck at 71%
Try it now
The live demo runs the six-rule engine over 1,200 GL rows and their bank counterparts in real time.
Things to try
- Type “Why is the auto-match rate stuck at 71%?”, then tick Normalise references on load, then Bank file now carries the reference — watch ~75% become ~85% become ~96% without writing a single rule
- Choose Amount and date only and enable r6 — the rate jumps from ~12% to ~90%, and the false-match count tells you what the ambiguity guard is protecting
- Untick r1 and r2 on the clean scenario and read the exception table: the rows that should have matched are labelled as such, not lost
The scoping error, concretely
A client says they have a huge reconciliation problem in their intercompany and bank accounts. If that means volume of transactions, they need Transaction Matching and will be disappointed by Reconciliation Compliance alone. If it means no control or visibility over who did what, they need Reconciliation Compliance and Transaction Matching would be expensive over-engineering. Confusing the two engines is the most common scoping error on ARCS projects, and it is usually an expensive one. The discovery question that settles it: “When your team reconciles this account, are they matching individual line items against each other, or explaining a difference between two totals?”
| Reconciliation Compliance | Transaction Matching | |
|---|---|---|
| Unit of work | An account balance | An individual transaction |
| Volume | Hundreds to thousands of reconciliations | Millions of rows |
| Core question | Is this balance supported and approved? | Which of these items pair up? |
| Main config object | Profile + Format | Match Type + Matching Rules |
| Has workflow | Yes — preparer, reviewer, status | No |
| Licensing | Base | Often a separate entitlement |
Generate with known truth → run rules in sequence → categorise what is left
- Each bank row remembers its true GL counterpart (
truthGl) and each GL row its perturbation (truth): exact, date shift, reformatted ref, missing ref, split, batch, fee, duplicate, in transit - Scenario sets the reference-quality probabilities; the upstream fix zeroes the missing/garbled ones
- Indexes open bank rows by (normalised) reference; date and amount tolerances per rule
- Every match is checked against truth —
falseMatchesper rule is measured, not assumed
- r5 records the fee as an adjustment
- r6 only when exactly one candidate each way
- One deposit, several receipts
- Batch payment hitting the bank once
- Six findings; only date tolerance and one-to-many are rule changes
- Net unmatched balance computed as the RC reconciling item
- KPI row, per-rule chart with false matches, feed into RC, exception table, unmatched sample
- Two data-fix toggles kept visually separate from the six rules
App UI โ Component breakdown
| Component | Behaviour |
|---|---|
| Scenario & toggles | Data quality select; “normalise references on load” and “bank file carries the reference” — labelled as transformations and file changes, not rules |
| Rule checkboxes | r1–r6 in execution order; r6 defaults off because it is the loosening rule |
| KPI row | Rate vs the expected range for the scenario; matched/total; exceptions; false matches in red if non-zero |
| Per-rule chart | Matched rows per rule, disabled rules greyed, false matches overlaid |
| Feeds RC panel | Unmatched GL $, unmatched bank $, adjustments, net difference → the cash reconciliation’s reconciling item |
| Exception table | Category, count and share, the real fix, coloured green (rule) or red (data); totals of each |
Core objects and the four rule shapes
| Object | What it is |
|---|---|
| Match Type | The container. Defines the data sources, their attributes and the rules for one kind of matching |
| Data Source | One side of the match — GL, bank file, processor extract. A match type may have more than two |
| Attributes | The fields on each source: amount, date, reference, description. Mapped and transformed on load |
| Matching Rules | The logic that pairs transactions, run in sequence |
| Adjustments | System-created entries for known differences such as bank fees |
| Rule type | Pairs | Typical use | Here |
|---|---|---|---|
| One-to-one | 1 GL row ↔ 1 bank row | The bulk of matching. Amount + date + reference | r1, r2, r5, r6 |
| One-to-many | 1 GL row ↔ several bank rows | A single GL deposit covering multiple receipts | r3 |
| Many-to-one | Several GL rows ↔ 1 bank row | A batch payment hitting the bank as one debit | r4 |
| Many-to-many | Groups on both sides | Netting arrangements, sweep accounts | Out of scope for the demo |
Reference fields are rarely formatted identically in two systems (INV-00123 vs 123), so normalisation belongs on load. In this build tmNormalizeRef() strips non-digits and leading zeros, and it is a toggle separate from the rules because that separation is the point: a transformation on load fixes a whole category of exceptions that no matching rule can.
Start tight, measure, loosen deliberately
r1 amount ==, ref ==, date == // measure the rate first
r2 amount ==, ref ==, |date| <= 3 // settlement window
r3 sum(bank rows w/ same ref) == amount // one deposit, several receipts
r4 sum(GL rows w/ payee+date) == bank // batch payment
r5 |amount diff| <= max($5, 0.5%) // fees / FX โ adjustment
r6 amount ==, |date| <= 3, no ref // ONLY if exactly one candidate each way
Tolerances matter as much as the attributes: an amount tolerance for small FX or fee differences, a date tolerance for a settlement window of one to three days, and normalisation of references. The build pattern that works: start with a tight one-to-one rule, measure the match rate, then add progressively looser rules for what remains — checking each addition does not produce false matches. Loosening too fast produces confident wrong answers, which are worse than an exception queue because nobody finds them.
Because bank rows are generated from GL rows, this build can check every match against the true pairing and report falseMatches per rule. The amount-and-date-only rule (r6) is where that matters: without a reference, two $250.00 payments three days apart are indistinguishable, so r6 only fires when exactly one candidate exists on each side. That guard is why the amount-and-date scenario reaches ~90% with a false-match count you can read off the chart rather than discover at audit.
Where a rate is stuck low, the cause is upstream
| Source data quality | Expected auto-match | This build (default rules) |
|---|---|---|
| Clean shared reference between systems | 97–99% | ~97% → ~98% with normalisation |
| Amount and date only, no reference | 85–93% | ~12% until r6 is enabled, then ~90% |
| Poor references, manual descriptions | 70–85% | ~75% → ~85% normalised → ~96% once the file carries the key |
Do not write more rules against an undiagnosed exception population. Sample 100 unmatched items and categorise why each failed — the page does that for every row:
| What you find | Real fix | Rule? |
|---|---|---|
| No shared reference between the two systems | ERP or bank file change — a rule cannot invent a key | No |
| Reference formatted differently | Transformation on load, not a matching rule | No |
| Items consistently 1–3 days apart | Date tolerance | Yes |
| One GL row against several bank rows | One-to-many rule | Yes |
| Duplicates on one side | Upstream data problem; rules will match the wrong one | No |
| Different currencies or FX rounding | Amount tolerance, or match on entered currency | Partly |
Only two of those six are rule changes. A rate around 71% is characteristic of a missing or unusable reference key, and no volume of rule writing recovers it. The consultant move is to return with the composition of the exception population, what rules can fix, and the one file change that takes them from 71% to 95% — which is exactly the sequence the two toggles on the page reproduce. Unlike Reconciliation Compliance’s monthly rhythm, TM loads daily; pushing clients toward daily loads changes the close more than any amount of rule tuning.
Two fields, and one of them is a boolean
Matching’s NLQ schema is {intent, scenario, normalize, confidence}. There is no entity, no period, no amount — nothing about the transactions themselves reaches the model, which is the right boundary for bank data. The scenario is a 3-value enum (clean / amountdate / poorref) and normalize is a nullable boolean that only flips when the user says something like “fix the key” or “normalise references”. The few-shot examples include the question a client actually asks — “why is the auto-match rate stuck at 71%?” — and map it to the poor-reference scenario with normalisation off, so the page opens on the problem before the fix.
Cost model โ Cost per query
| Component | Detail | Cost |
|---|---|---|
| LLM input tokens | ~640 tokens (schema block + few-shot examples + query) | $0.000090 |
| LLM output tokens | ~45 tokens | $0.0000126 |
| Rules engine over 1,200 GL rows | Client-side, deterministic, once the query resolves | $0 |
| Total per query | ~$0.0001 | |
| With 2ร safety margin | ~$0.0002 |
The cheapest query in the ARCS module — a 3-value enum and a boolean. The matching engine runs six rules over ~2,400 rows client-side in well under a second; production ARCS runs the same idea over millions of rows in its own engine.
Key files
| File | Role |
|---|---|
epm-nlq-src/assets/epm-arcs-data.js | TM_SCENARIOS, TM_RULES, genTMData() (bank rows generated from GL rows with realistic perturbations so truth is known), runTransactionMatching() (rules in sequence, honest false-match counts), TM_EXCEPTION_FIXES |
epm-nlq-src/pages/matching.html | UI: NLQ bar, scenario select, the two data-fix toggles, six rule checkboxes, KPI row, per-rule chart, the feed into RC, exception population table, unmatched-row sample |
epm-nlq-src/assets/epm-fccs-data.js | genConsolidationBS() โ GL balances come from the same balance sheet FCCS consolidates, so ARCS proves the balances FCCS combines |
epm-nlq-src/assets/epm-data.js | Shared ENTITIES, PERIODS, ENTITY_SEEDS, SEASON |
functions/api/nlq-query.js | Shared NLQ endpoint; this use case uses useCase: "matching" |
Why ARCS is a separate data file from FCCS
They answer different questions and are separately licensed products. ARCS proves each balance is real; FCCS combines them into group results โ assurance, then aggregation. Keeping epm-arcs-data.js separate mirrors that: it consumes the FCCS balance sheet for its GL balances but adds its own objects (profiles, formats, reconciliations, match types, rules) and never writes back. The deterministic FNV-1a hash (arcsRand) means every reconciliation and every bank row is reproducible on any machine, which is what makes the trap scenarios teachable.
Tech stack โ Every tool in this build
| Layer | Tool | Why |
|---|---|---|
| Data layer | epm-arcs-data.js (vanilla JS) | A real rules engine with measurable false matches โ the LLM only resolves scenario and the normalise flag |
| LLM | DeepSeek V3 | Shared NLQ endpoint; the smallest schema in the ARCS module โ a 3-value enum and a nullable boolean |
| Charts | Chart.js 4.x | Rows matched by rule with false matches overlaid โ shows where each rule earns its place |
| Edge hosting | Cloudflare Pages | Static file, no server compute needed for this use case |
| Build | Eleventy v3.1.5 | Copies epm-nlq-src/pages/ and epm-nlq-src/assets/ to _site/ verbatim |
Known attack surfaces
| Threat | Mitigation in this build |
|---|---|
| Loosening rules produces confident wrong answers nobody finds | Bank rows are generated from GL rows, so every matched pair is checked against the truth pairing and falseMatches is reported per rule; the amount+date-only rule (r6) has an ambiguity guard that only matches when exactly one candidate exists on each side |
| Writing more rules against an undiagnosed exception population | Every unmatched GL row carries its root cause; the exception table labels each category as a rule fix or a data/file fix and totals both — the page makes the rule-vs-data split visible before anyone writes a rule |
| Duplicates on one side match the wrong row | Duplicates are generated deliberately (1%) and land in the exception population labelled as an upstream data problem, not silently matched |
| Treating TM as an alternative to RC | The “feeds Reconciliation Compliance” panel computes the net unmatched balance that becomes the reconciling item on the cash reconciliation — the two engines are shown as a pipeline, not a choice |
Guardrails โ What prevents bad control math
- Rules run most-restrictive-first, in a fixed order:
TM_RULESis an ordered array; a row consumed by r1 is never available to r6 - Every GL row is matched or in exactly one exception category — unit-tested for all three scenarios, so nothing falls through the accounting
- Reference normalisation is a load-time transformation, not a rule: it is a separate toggle from the six rules on purpose, because that distinction is the lesson
- Tolerances create adjustments, not silent matches: r5 records the fee difference as an adjustment amount so the GL can be corrected rather than left wrong
Who is asking, and what are they allowed to see?
The demo answers neither question — it has a cookie gate and no notion of a user. In production these are the two questions everything else rests on, and they have different answers: authentication is who you are, authorization is what you may see. Corporate SSO settles the first. Only Oracle EPM Cloud can settle the second, and the single most important rule in this section is that this application must never become the place where that decision is made.
9.1 · The identity chain, end to end
- MFA and Conditional Access are enforced here — device compliance, location, risk signals
- Returns an ID token (who the user is) and an access token (what they may call)
- Group membership arrives as a claim; the application never handles a password
- Verify signature, issuer, audience and expiry against the IdP’s published keys
- Read the group claims — there is no local user table and no local role table
- Short-lived access token with refresh-token rotation; session timeout set to the data classification
- The API is called as the user
- EPM enforces its own security natively
- Audit trail names the real user
- Preferred where the API supports it
- The application becomes the enforcement point
- Entitlements fetched separately, applied in one audited place
- Simpler and cacheable — and a filtering bug is a data breach
- Roles: Service Administrator, Power User, User, Viewer — assigned to groups, never to individuals
- Data level: Match-type level access in ARCS; bank and processor sources are usually restricted to treasury and cash application
- Group → role mapping lives in the platform, not in this application
- The user sees exactly what they would see logging into the source system directly — no more
- Every query logged against the real end user, never a shared account
9.2 · Federating the corporate identity provider
Oracle EPM Cloud does not replace your directory — it trusts it. The EPM Cloud identity domain is federated with the corporate IdP so authentication happens where it already happens, under policies security has already written.
| Identity provider | Protocol | Notes |
|---|---|---|
| Microsoft Entra ID (formerly Azure AD) | SAML 2.0 or OIDC | The common case. Conditional Access, MFA and device compliance are enforced at Entra and inherited automatically. On-premises Active Directory federates through Entra Connect rather than being integrated directly. |
| Okta | SAML 2.0 or OIDC | Same pattern; Okta groups drive EPM roles through SCIM provisioning. |
| OCI IAM (identity domains) | Native | Already present with Oracle EPM Cloud. Can be the primary IdP for a smaller estate, or a federated spoke of Entra/Okta for a larger one. |
| AD FS | SAML 2.0 | Still seen where the estate is not yet cloud-first. Works, but you inherit the on-premises availability of the token service — if AD FS is down, nobody logs in. |
For the browser application itself, use OIDC Authorization Code flow with PKCE. Not the implicit flow, which is deprecated and leaks tokens through the URL, and never a resource-owner password grant — a finance tool should not be capable of handling a password at all.
9.3 · From group membership to EPM roles
Roles are granted to groups, never to individuals, and the groups come from the directory. That one discipline is what makes joiner/mover/leaver work without anyone having to remember this application exists.
Entra ID group โ EPM role / entitlement
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
FIN-EPM-Analysts โ Planning User
FIN-EPM-Controllers-EMEA โ Power User + EMEA data scope
FIN-EPM-Admins โ Service Administrator
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Provisioned by SCIM. Remove the user from the group and the
entitlement disappears on the next sync โ including here.
For this use case the relevant native entitlement is: ARCS User with access to the relevant match types.
9.4 · The architectural decision: who enforces?
This is the choice that determines whether the deployment is defensible. Both patterns appear in the diagram above; the difference is where the security boundary actually sits.
| Pattern A — identity propagation | Pattern B — service account + filtering | |
|---|---|---|
| How | The user’s token is exchanged (OAuth 2.0 on-behalf-of) for one scoped to the EPM API; calls are made as the user | A single read-only integration account calls the API; the application filters the results |
| Enforcement point | Oracle EPM Cloud | This application |
| Audit trail shows | The real end user | The service account — you must log the real user separately |
| Failure mode | Token plumbing is more complex; per-user rate limits apply | A filtering bug is a data breach, and the entitlement copy drifts from reality |
| Verdict | Prefer this wherever the API supports user-token authentication | Acceptable with discipline: narrowest possible service account, filtering centralised in one tested place, real user in every log line |
The shortcut to refuse. Pattern B built with a Service Administrator account and no filtering at all is the most common way this gets delivered, because it works perfectly in UAT — testers are usually over-entitled, so nobody notices that everyone can see everything. It fails at the first access review, and by then it is in production with real users depending on it.
9.5 · Data-level security is the part that matters
Role membership decides whether a user can open the application. It does not decide which rows they get back, and confusing the two is the most expensive mistake available here.
- For this use case: Match-type level access in ARCS; bank and processor sources are usually restricted to treasury and cash application.
- Apply it before aggregation, not after. Filtering a total that has already been computed across entities the user cannot see still leaks the total.
- The NLQ layer needs its own check. Layer 4 already validates that the resolved point of view uses approved members; production adds a second test — that the resolved POV sits inside this user’s scope — and it runs before the data call, not after. A natural-language interface is very good at asking for things politely; the authorization check must not care how the question was phrased.
- Fail closed. If entitlements cannot be resolved, return nothing and say so. An empty result is a support ticket; a permissive default is an incident.
9.6 · Provisioning, sessions and the leaver problem
- SCIM provisioning from Entra or Okta into the EPM Cloud identity domain (OCI IAM, formerly IDCS), covering joiner, mover and leaver. The mover is the case people forget — somebody changing region should lose the old scope, not accumulate both.
- No local user store. If this application keeps its own copy of who may do what, a leaver keeps access until somebody remembers to update it. Nobody ever does.
- Short-lived access tokens with refresh-token rotation; align session timeout with the data classification rather than with convenience.
- MFA and Conditional Access at the IdP — not reimplemented here. Device compliance and location policy come free with federation.
- Quarterly recertification of both the groups that grant access and the service account’s own entitlements, evidenced and signed.
- Break-glass access is a named, monitored, time-boxed account — never a shared credential in a password manager.
9.7 · What this means for Transaction Matching
| Concern | Answer for this use case |
|---|---|
| Native entitlement required | ARCS User with access to the relevant match types |
| Data-level control | Match-type level access in ARCS; bank and processor sources are usually restricted to treasury and cash application |
| Use-case-specific sensitivity | Treasury-grade data. Bank and card-processor files carry account numbers and counterparty names — the schema in this build deliberately sends none of it to the model. Restrict match types by team, and treat an export of unmatched items as a payment-data export. |
9.8 · Security configuration checklist
- ✓Oracle EPM Cloud federated with the corporate IdP over SAML 2.0 or OIDC; the cookie gate removed entirely
- ✓Browser app uses OIDC Authorization Code + PKCE — no implicit flow, no password grant
- ✓MFA and Conditional Access enforced at the IdP, not reimplemented in the application
- ✓Roles granted to directory groups, never to individuals; SCIM covers joiner, mover and leaver
- ✓Enforcement pattern chosen deliberately — Pattern A where the API supports it, or Pattern B with filtering centralised and tested
- ✓Data-level security applied before aggregation, and the resolved POV checked against the user’s scope before the data call
- ✓No local user table and no local role table anywhere in the application
- ✓Every query logged against the real end user, even when a service account makes the call
- ✓Authorization failures fail closed and are logged as security events rather than swallowed
- ✓Quarterly recertification of access groups and of the service account’s own entitlements
From demo to a governed enterprise deployment
Everything above runs on synthetic data, a public LLM API key, a cookie gate, and no audit trail — deliberately, so the mechanics are inspectable. Taking Transaction Matching to production is not a rewrite; the 4-layer pipeline and the data-layer contract survive intact. It is a controlled-change program across six workstreams: architecture, LLM platform, security, SOX/audit, environment promotion, and operations. This section is the checklist we run with clients.
10.1 · Production reference architecture
- Terminates SSO, validates the session, attaches the user’s EPM groups to the request
- Rate limits per user, blocks anonymous access, scrubs PII patterns before anything reaches the orchestrator
- L2 grounding reads dimension metadata from EPM on a schedule — not a hardcoded schema
- Only the schema + user query go to the model; financial values never leave the data layer
- L4 rejects anything outside the approved member lists and falls back to the deterministic parser
- Data stays inside the OCI boundary
- Natural fit when EPM is already in OCI
- Use the hyperscaler the org already governs
- Enterprise DPA, no training on prompts
- For regulated or sovereign data
- Highest control, highest run cost
- Oracle ARCS Transaction Matching — match types, matched sets, adjustments, and the unmatched exception population via the ARCS REST API; rules execute in the ARCS engine on the daily load
- Least-privilege service account (read-only role, one app, one pod) with the token in a vault and rotated
- Results filtered to the requesting user’s EPM security before rendering
- Every query logged: user, timestamp, raw query, parsed intent JSON, model + prompt version, POV returned, latency, cost
- Exported to the SIEM; retained per the SOX evidence schedule
- Dashboards for fallback rate, eval pass rate, guardrail hits, p95 latency, spend
- The parsed JSON is shown to the user as the explanation (“AI: entity · year · scenario — 93% confident”) — the same line the demo prints today
- Every number on screen traces to an EPM cell intersection an auditor can reproduce
10.2 · Choosing the LLM platform
The demo’s DeepSeek call is a placeholder for a single adapter, callLLM(system, user), behind Layer 3. Swapping the provider changes one function and zero business logic. Pick the platform the organisation already governs — the security and procurement review is the long pole, not the integration.
| Option | Choose when | Data posture |
|---|---|---|
| Oracle OCI Generative AI (Cohere Command, Llama) | EPM Cloud already lives in OCI; you want one cloud boundary and one contract | Prompts stay in the OCI tenancy; no training on customer data; dedicated AI clusters available for isolation |
| Azure OpenAI Service | Microsoft-first finance estate (Entra ID, Purview, Sentinel already in place) | Private endpoint, regional deployment, zero-retention by default under the enterprise agreement |
| AWS Bedrock (Claude, Titan) / Google Vertex AI (Gemini) | The org’s landing zone is AWS or GCP; VPC endpoints and IAM already audited | VPC/PSC private access, no data used for training, CloudTrail/Cloud Audit Logs integration |
| Direct enterprise API (Anthropic, OpenAI) | Fastest model access; acceptable when a zero-data-retention agreement and DPA are signed | ZDR endpoint, SSO-managed keys, SOC 2 report on file |
| Self-hosted open weights (Llama, Mistral, Qwen via vLLM) | Sovereign or air-gapped requirements; regulated data classification forbids any external inference | Full control; you own patching, eval, and capacity — budget for an MLOps owner |
Put a model gateway in front of whichever you choose (Azure API Management, OCI API Gateway, Kong AI Gateway, LiteLLM, or Portkey): it owns key custody, per-team spend caps, routing and fallback between models, prompt/response logging, and lets you retire a deprecated model without touching the application.
10.3 · Security controls
| Control | Implementation |
|---|---|
| Identity & access | Covered in full in section 09 — corporate SSO, group-to-role mapping, and the decision about who enforces data-level security. Listed here because it is a production gate, not because it is optional. |
| Service account | One read-only EPM service account per application per pod, least-privilege role, no interactive login, credential in a vault (OCI Vault, Azure Key Vault, HashiCorp Vault), rotated on a schedule and on staff change. |
| Secrets & config | No secrets in code or build artifacts; environment-specific config injected at deploy; .dev.vars-style files never leave a developer machine. |
| Network | Private endpoints to the LLM provider and to EPM where the platform supports them; egress allow-list so the orchestrator can reach exactly two hosts; TLS 1.2+ everywhere. |
| Prompt-injection & input guardrails | Layer 1 (already in the demo) blocks instruction-override patterns, enforces length and scope; extend with a classifier on the gateway and log every rejection. |
| Output guardrails | Layer 4 (already in the demo) validates every returned member against the approved lists and strips unexpected keys; production adds a policy check that the resolved POV is inside the user’s security scope before the data call. |
| Data minimisation | Prompts contain metadata and the user’s query only. No cell values, no employee names, no free-text comments from EPM. Logged prompts are classified and retained accordingly. |
| Encryption | In transit (TLS) and at rest (provider-managed KMS); audit logs on immutable storage with customer-managed keys where policy requires. |
10.4 · SOX, audit, and model-risk controls
A read-only NLQ layer does not change a financial-reporting control, but it is an interface to a SOX-relevant system and lands squarely in ITGC scope. Treat prompts, schemas, and eval sets as code — that single decision satisfies most of what an auditor will ask for.
| Requirement | How it is satisfied |
|---|---|
| Complete, immutable audit trail | Append-only log of user, timestamp, raw query, parsed JSON, model and prompt version hash, POV returned, and row count — WORM storage, retained for the evidence period (typically 7 years), exported to the SIEM. |
| Change management | Prompt templates, few-shot examples, approved-member schema, and code are version-controlled; every change follows ticket → peer review → test evidence → CAB approval → deploy. A prompt edit is a code change. |
| Segregation of duties | Developers cannot deploy to production; the service-account owner is not a developer; production secrets are held by platform operations. |
| Access recertification | Quarterly review of who can use the tool and of the service account’s EPM roles, evidenced and signed. |
| Testing evidence | A golden-query regression suite (the few-shot examples plus a larger labelled set) runs in CI before every release; pass rate and diffs are archived as release evidence. |
| Model risk management | An inventory entry (intended use, limitations, owner, validation date) in the model-risk register — the SR 11-7 pattern for financial services; periodic re-validation when the model or prompt changes. |
| Reproducibility & lineage | Every displayed number traces to an EPM POV and a consolidation/calculation timestamp; an auditor can re-query the same intersection in EPM and match it. |
| Explainability | The parsed intent JSON is the explanation and is shown to the user on every response — no hidden reasoning between the query and the data call. |
10.5 · Dev → Test → Prod promotion
| Environment | EPM target | Data | Gate to leave |
|---|---|---|---|
| Dev | EPM Test pod (developer slice) | Synthetic or masked | Unit tests on the data layer; lint; eval suite ≥ threshold against the Test LLM deployment |
| Test / UAT | EPM Test pod (full refresh) | Masked copy of production | Business UAT sign-off on the golden queries; security scan; performance run (p95 latency, fallback rate) |
| Prod | EPM Production pod | Live | Change ticket approved; deploy in window; smoke test; hypercare with rollback ready |
- Promoted artifacts: application build, prompt templates (versioned), approved-member schema snapshot, eval set, infrastructure config (IaC) — all from the same Git tag.
- Pipeline: branch → PR review → CI (tests + evals) → deploy to Test → UAT sign-off → CAB → deploy to Prod → smoke test. Hosting can stay on Cloudflare Pages/Workers or move to OCI Functions + API Gateway or the org’s standard platform — the code does not care.
- Configuration: per-environment secrets and endpoints injected at deploy; the same build runs in every environment.
- Metadata sync: a scheduled job refreshes dimension metadata into Layer 2 grounding with change detection, so a new entity or account appears in the approved lists without a code release.
- Rollback: previous build and previous prompt version retained; rollback is a redeploy, and because prompts are versioned it also reverts a prompt regression.
10.6 · Operating it
- SLOs: p95 latency, availability of the read path (the deterministic fallback keeps it alive when the LLM is down — already built), fallback rate as a quality signal, eval pass rate per release.
- Cost governance: per-user and per-team token budgets at the gateway; alert on anomalies; the unit-cost model earlier in this kit is the baseline.
- Model lifecycle: providers retire models on a schedule — re-run the eval suite on the successor before switching, and record the switch as a change.
- Incident runbook: LLM outage → fallback parser; EPM API outage → cached metadata with a stale banner; guardrail spike → review logs for injection attempts.
10.7 · What changes for Transaction Matching
| Concern | Production answer |
|---|---|
| System of record | Oracle ARCS Transaction Matching — match types, matched sets, adjustments, and the unmatched exception population via the ARCS REST API; rules execute in the ARCS engine on the daily load |
| Read/write posture | Read-only. Manual matches, splits, and adjustments are made in ARCS; this layer categorises and explains the exception population. |
| Use-case-specific control | Bank and processor files carry account numbers and counterparties — treasury-grade data classification. Raw bank rows must never appear in an LLM prompt; only the scenario and rule selection do. |
10.8 · Production readiness checklist
- ✓LLM platform selected from the governed list, DPA / zero-retention terms on file, gateway in front of it
- ✓SSO integrated; authorisation derived from EPM security groups; cookie gate removed
- ✓Read-only EPM service account per pod, credential in a vault, rotation scheduled
- ✓Prompts, schema, and eval set version-controlled and under change management
- ✓Append-only audit log wired to the SIEM with the agreed retention
- ✓Golden-query eval suite passing in CI; results archived as release evidence
- ✓Dev / Test / Prod pipeline with gates, IaC, and a rehearsed rollback
- ✓Model-risk register entry and owner named; first re-validation date set
- ✓Metadata refresh job scheduled with change detection
- ✓SLOs, cost caps, and the incident runbook agreed with platform operations