Auditing across vendors who will not give you their database
An agent workflow touches a model provider, two SaaS platforms and your own services. Auditing it means asking four organisations for records they have commercial reasons to limit, in formats that do not join.
Why can nobody reconstruct what happened across four vendors?
Because each holds one segment of the trail in its own schema, with its own retention, its own redactions and no shared correlation identifier. The auditor's work becomes reconciliation rather than assurance, and the joins are guesses dressed as findings.
- Vendors limit log access for legitimate reasons: other customers' data, competitive detail, and the cost of bespoke extracts.
- What comes back is partial, differently timestamped, and keyed by identifiers that do not join across systems.
- Portable receipts move the evidence to the workflow rather than the platform, so no vendor cooperation is needed to verify it.
Part of AI oversight and regulation
Why vendors say no
It is tempting to read this as obstruction. Mostly it is not.
| Reason | Legitimacy |
|---|---|
| Logs contain other customers' data | Entirely legitimate — they cannot share it |
| Logs reveal internal architecture | Reasonable commercial position |
| Extraction is bespoke engineering work | Real cost, and they are not resourced for it |
| Retention is shorter than your audit period | Common, and usually documented in the contract |
| Contract does not require it | The gap was in procurement, not in their conduct |
The last row is where most of this originates. Audit access rights are rarely negotiated at purchase, and by the time they are needed the leverage is gone.
The join problem
Even when logs arrive, they do not reconcile. Four independent systems produce four independent record sets with no shared key.
# Vendor A
{"request_id":"req_8h2k","ts":"2025-11-19T14:22:01.442Z","user":"acct_44"}
# Vendor B
{"correlationId":"c-991f","timestamp":1763561521,"principal":"svc-integration"}
# Vendor C
{"eventId":"E88213","occurred":"11/19/2025 09:22:01 EST","actor":"system"}
# Your system
{"trace":"4471-88213","at":"2025-11-19T14:22:01Z","who":"u_4471"}
# No shared key. Three time formats. "actor": "system".An auditor reconstructs this by correlating timestamps and inferring causality. It is slow, it is approximate, and the conclusion is an inference rather than a record.
Moving the evidence to the workflow
The alternative is to stop treating each platform's logs as the evidence and to produce an artefact that travels with the transaction.
# Emitted at the point of the consequential action, by whoever
# performs it, and carried with the workflow.
{
"workflow": "wf-2025-88213",
"step": 3,
"action": "vendor_payment.release",
"payload_digest": "sha256:9c1f...a83e",
"actor": { "type": "human", "id": "[named]", "user_verified": true },
"under_delegation": "dlg-4471",
"at": "2025-11-19T14:22:01Z",
"signature": "...",
"prev": "sha256:2b7d...4056" # links to step 2's receipt
}The prev field chains the steps. An auditor receives an ordered sequence that verifies as a unit, rather than four log exports to correlate by hand.
What the auditor can do with it
- Verify every receipt against published keys, with no vendor involvement
- Confirm the chain is complete — a missing step breaks the linkage visibly
- Identify the human behind each consequential action, and the delegation they acted under
- Test the whole population rather than a sample, because verification is mechanical
The second point deserves emphasis. With log exports, a missing record is indistinguishable from an action that never happened. With a chain, a gap is detectable.
The obvious limit
This only covers actions instrumented to emit receipts. It says nothing about what a vendor did internally, and it does not replace the vendor's own assurance reporting.
That is a real boundary and it maps onto how responsibility is already allocated. You are responsible for the actions taken in your workflow; the vendor is responsible for their platform's controls, evidenced by their audit reports. The receipts cover the first, and always did.
What to negotiate at procurement
The time to fix this is before signing, and the asks are modest.
- The ability to attach an opaque identifier to requests and have it returned in responses and logs. Cheap for the vendor, solves the join problem.
- Log retention matching your audit period, stated in the contract.
- A defined export format and a committed turnaround for audit requests.
- Where the vendor performs consequential actions on your behalf, receipts or equivalent attestation for those actions.
The first is the highest value per unit of negotiating effort. Most platforms already support a correlation or metadata field; the ask is only that it round-trips reliably into their logs.
A worked example: the join
An action crosses an orchestration platform, a model provider, a retrieval service and an execution runtime. The auditor asks a simple question: which human authorised this, and what were they shown?
| Source | Contains | Missing |
|---|---|---|
| Orchestrator | Run ids, timestamps | What was rendered to a person |
| Model provider | Token counts, model version | Any customer-side context |
| Retrieval service | Document ids | Whether the human saw them |
| Execution runtime | The effect | Who authorised it |
| All four joined | A plausible narrative | The authorising human |
The missing item is the same in every column, because no vendor was ever in a position to record it — the human was on your side of the boundary.
Objections and honest limits
“We can mandate log formats at procurement.” Worth doing, and it improves the joins. It does not produce the authorising human, because that event happens in your interface, not theirs.
“This adds another evidence store.” It replaces four partial ones with a portable artefact you hold. The vendor logs remain useful for operations; they stop being the assurance basis.
Making the trail survive the vendors
- Identify where a human authorises, in your interface. That is the one event nobody else can log.
- Emit a signed receipt there. Over the rendered effect.
- Carry a chain identifier into every vendor call. So their logs can be joined to yours.
- Keep receipts under your own retention. Independent of vendor policy.
- Negotiate export format and retention anyway. It improves the supporting picture.
Terms used here
- Join problem
- Reconstructing one event from several systems with no shared identifier or clock.
- Correlation identifier
- A value carried across services so their records can be linked afterwards.
- Portable receipt
- A signed artefact verifiable without calling the system that produced it.
Frequently asked questions
Are vendors being obstructive? Mostly not. Logs contain other customers' data, reveal architecture, and extraction is unfunded bespoke work. The gap usually originates in procurement.
Why don't log exports reconcile? Independent systems use independent identifiers and time formats, and many record a service principal rather than a person. Correlation is inference, not record.
What does chaining receipts add? A missing step becomes visible. With separate log exports, a missing record is indistinguishable from an action that never occurred.
What should be negotiated at purchase? A correlation identifier that round-trips into vendor logs, retention matching your audit period, a defined export format and turnaround.
Why will vendors not give full logs? Their stores are multi-tenant and contain other customers' data, so exports are redacted, partial and schema-specific by necessity.
What can the auditor do with receipts? Verify them offline against a published key and read what a named human was shown before they approved.
Does this replace vendor logs? No. It replaces them as the assurance basis; they remain useful operationally.
Where this fits in Manav
Manav produces the one record no vendor can: a signature from the authorising human over the effect they were shown, verifiable offline and held by you.
Sources and further reading
- AICPA audit and attestation standards
- Published guidance on multi-cloud audit and evidence collection.
- Federal Acquisition Regulation
- RFC 8785 — JSON Canonicalization Scheme
- NIST AI Risk Management Framework