{ "slug": "okta-cross-tenant-impersonation-architectural-lessons-2023-support", "topic_id": "TOPIC-027", "cluster": "Privileged Identity & Help-Desk Takeover", "tier": "Tier B", "title": "Support access to customer tenants: the standing-authority problem", "summary": "Every multi-tenant platform gives support staff a way into customer environments. That path is usually invisible to the customer, always on, and governed by the vendor's own controls.", "lede": "A customer buys an identity platform to control who reaches their data. The support organisation running that platform holds a path into their tenant that the customer cannot see, cannot gate, and learns about only when something goes wrong.", "date": "2025-01-02", "category": "Developer", "author_id": "margot-reyes", "tags": [ "multi-tenancy", "support access", "cross-tenant", "SaaS security", "vendor risk", "impersonation" ], "image_title": "Support Access Standing Authority", "schema": "Article", "key_takeaways": [ "Support impersonation is a necessary capability that is usually built as standing authority rather than customer-gated access.", "Diagnostic artefacts uploaded to support systems routinely contain live session material, which converts a support-system breach into tenant access.", "Requiring a customer-signed grant before impersonation moves the control to the party whose data is at stake." ], "body": [ { "type": "h2", "text": "The capability nobody can remove" }, { "type": "diagram", "kind": "flow", "alt": "A customer uploads a HAR file to support; the file contains a live session token; the attacker in the support system replays it", "caption": "The customer's own troubleshooting step is what carries the credential into the vendor's systems.", "nodes": [ { "label": "Customer hits a bug", "note": "ordinary" }, { "label": "Support asks for a HAR capture", "note": "documented workflow" }, { "label": "HAR contains a live session cookie", "note": "rarely redacted", "bad": true }, { "label": "Support system is compromised", "note": "Okta, Sep–Oct 2023", "bad": true }, { "label": "Session replayed against the tenant", "note": "no password, no MFA", "bad": true } ] }, { "type": "p", "html": "Support engineers need to reproduce customer problems. Reproducing a problem in a configuration you cannot see is close to impossible, so every mature platform builds an impersonation path." }, { "type": "p", "html": "The question is not whether it exists. It is who authorises each use of it." }, { "type": "table", "head": [ "Design", "Who authorises access", "Customer visibility" ], "rows": [ [ "Standing support role", "The vendor, once, at role assignment", "Usually none" ], [ "Ticket-gated access", "The vendor's ticketing system", "After the fact, if audit logs are shared" ], [ "Customer toggle", "The customer, for a window", "Good — but a window, not an action" ], [ "Customer-signed grant", "The customer, per session, bound to scope", "Full — and independently verifiable" ] ] }, { "type": "p", "html": "Most platforms sit in the first two rows. The third is a meaningful improvement and is where the better vendors have landed." }, { "type": "h2", "text": "The diagnostic-artefact problem" }, { "type": "p", "html": "Support workflows routinely ask customers to upload a HAR capture, a debug bundle or a log archive. These artefacts are extraordinarily rich." }, { "type": "ul", "items": [ "Session cookies and bearer tokens, live at capture time", "Authorization headers on every recorded request", "Full request and response bodies, including customer data", "Internal hostnames, service topology and identifiers" ] }, { "type": "p", "html": "A support system holding thousands of these is a higher-value target than most of the tenants it serves, because a single compromise yields working credentials for many of them at once." }, { "type": "p", "html": "Redaction tooling helps and is rarely complete. Token formats change, custom headers are missed, and the customer uploading the file usually does not know what is in it." }, { "type": "h2", "text": "What a customer-gated design looks like" }, { "type": "code", "text": "# Support engineer opens a customer tenant\n\n1. Engineer requests access, stating scope and reason.\n\n2. The customer's designated administrator sees:\n \"Grant support access to your tenant\n Engineer: [named individual, vendor identity]\n Scope: read-only, billing configuration\n Duration: 4 hours from acceptance\n Reason: ticket #48210 — invoice discrepancy\"\n\n3. Administrator signs that statement with their own credential.\n\n4. The platform's impersonation path requires a valid, unexpired,\n unrevoked grant. No grant → access denied, including to staff\n with the support role.\n\n5. Actions taken under the grant are recorded against both\n identities and exported to the customer." }, { "type": "p", "html": "Step four is what distinguishes this from a policy. If the check is enforced in the access path rather than in a runbook, the vendor's own administrators cannot bypass it without a code change." }, { "type": "h2", "text": "Why vendors resist, and the answer" }, { "type": "p", "html": "The objection is real: an outage at 2am cannot wait for a customer administrator to wake up and sign something." }, { "type": "p", "html": "Two responses, both practical." }, { "type": "ol", "items": [ "Pre-authorised emergency scope. The customer signs, in advance, a narrow grant covering defined emergency scenarios — with a short duration, a restricted scope, and mandatory notification. The customer still authorised it; they simply did so ahead of time.", "Break-glass with two vendor signatures. Where no customer grant exists, access requires two named vendor individuals to sign, generates an immediate customer notification, and is treated as an incident requiring justification." ] }, { "type": "p", "html": "Neither prevents legitimate emergency work. Both make ungated access rare and visible rather than routine and invisible." }, { "type": "h2", "text": "Questions a customer should ask their platform vendor" }, { "type": "ol", "items": [ "Can your staff access my tenant data without an action by someone at my organisation? If yes, under what process?", "Do I receive a record of every support access, and is it delivered to me rather than available on request?", "What diagnostic artefacts do you ask me to upload, and what live credentials do they typically contain?", "If your support system were compromised, what would an attacker hold that grants access to my tenant?" ] }, { "type": "p", "html": "The fourth question is the one that gets an uncomfortable answer, and it is the one worth asking first." }, { "type": "h2", "text": "The broader point" }, { "type": "p", "html": "An identity platform is a concentration of authority by construction. That is its value and also its risk profile." }, { "type": "p", "html": "A customer who cannot see or gate the vendor's path into their tenant has not reduced their identity risk; they have moved it to an organisation whose controls they cannot inspect. The remedy is not distrust of vendors. It is designing the access path so that the customer holds the key to it." }, { "type": "h2", "text": "What happened at Okta, and why it generalises" }, { "type": "p", "html": "Okta disclosed that from 28 September to 17 October 2023 a threat actor accessed files in its customer support case management system belonging to 134 customers, under one per cent of its base. Session tokens embedded in uploaded HAR files were then used to hijack live sessions at five of those customers. Okta detected the breach on 13 October and disclosed publicly on 19 October." }, { "type": "p", "html": "Nothing in that sequence is specific to Okta. Every support organisation asks customers for diagnostic captures, and a HAR file is a faithful recording of authenticated traffic. The vendor ends up holding a pile of live credentials it never intended to collect, gathered through a workflow both sides consider routine." }, { "type": "table", "caption": "What a browser capture contains that the uploader did not intend to send", "head": [ "Artefact", "Present in a default capture?" ], "rows": [ [ "Session cookies", "Yes, unless explicitly stripped" ], [ "Authorization headers", "Yes, on every request" ], [ "Full request and response bodies", "Yes, including customer records" ], [ "Internal hostnames and service topology", "Yes" ], [ "Anything the redaction tool did not recognise", "Yes — custom headers are the usual miss" ] ] }, { "type": "h2", "text": "Objections and honest limits" }, { "type": "p", "html": "“We redact before upload.” Redaction helps and is incomplete by nature. Token formats change, custom headers are missed, and the person uploading usually cannot tell what is in the file. Treat redaction as reducing exposure, not removing it." }, { "type": "p", "html": "“Support access is contractual, so it is fine.” A contract allocates liability. It does not tell you when access happened, or let you gate it. Those are different properties and a customer usually wants both." }, { "type": "p", "html": "What a customer-signed grant does not fix: a vendor whose own infrastructure is compromised in a way that bypasses the impersonation path entirely, or data the vendor legitimately holds outside your tenant. It closes the gated path and leaves the rest to the vendor's controls." } ], "faq": [ { "q": "Isn't support access necessary?", "a": "Yes. The argument is about who authorises each use of it, not whether the capability should exist." }, { "q": "What about 2am emergencies?", "a": "Pre-authorised emergency scope signed in advance by the customer, or break-glass requiring two named vendor signatures with immediate notification. Both keep the path open and make it visible." }, { "q": "Why are diagnostic uploads a problem?", "a": "Browser session captures and debug bundles routinely contain live tokens and authorization headers. A support system holding many of them is a higher-value target than most individual tenants." }, { "q": "Can redaction solve it?", "a": "It helps and is rarely complete. Custom headers and changing token formats get missed, and the uploading customer usually cannot tell what is in the file." }, { "q": "How many customers were affected by the Okta support breach?", "a": "Okta reported that files belonging to 134 customers were accessed, and that session tokens from those files were used to hijack live sessions at five of them." }, { "q": "Why is a HAR file dangerous?", "a": "It records every header and cookie in an authenticated browser session. A capture taken while logged in contains a working session token, and redaction tooling routinely misses custom headers." }, { "q": "Can we simply refuse to send diagnostics?", "a": "Usually not, if you want the ticket resolved. Capture in an isolated session with a throwaway account, verify the redaction yourself, and revoke that session afterwards." } ], "sources": [ { "t": "FTC — business guidance on marketplaces and consumer protection", "u": "https://www.ftc.gov/business-guidance" }, { "t": "HAR 1.2 specification — captured headers, cookies and content", "u": "https://w3c.github.io/web-performance/specs/HAR/Overview.html" }, { "t": "Cloud Security Alliance — Cloud Controls Matrix", "u": "https://cloudsecurityalliance.org/research/cloud-controls-matrix" }, { "t": "Okta Security — root cause and remediation for the support case management system incident", "u": "https://sec.okta.com/articles/2023/11/unauthorized-access-oktas-support-case-management-system-root-cause/" }, { "t": "BleepingComputer — 134 customers exposed in the October support system hack", "u": "https://www.bleepingcomputer.com/news/security/okta-breach-134-customers-exposed-in-october-support-system-hack/" } ], "related": [ { "slug": "privileged-access-management-pam-under-attack-gating-just", "title": "Gating just-in-time escalation", "category": "Developer" }, { "slug": "session-hijacking-via-infostealers-mfa-login-protect-post", "title": "Why MFA at login does not protect post-login actions", "category": "Developer" }, { "slug": "illusion-zero-trust-network-microsegmentation-fails-against", "title": "The illusion of Zero Trust", "category": "Developer" } ], "image": "https://cdn.twc.sh/images/igcache/Support%20Access%20Standing%20Authority/1200_630/blog.jpg", "wordcount": 1205, "url": "/blog/okta-cross-tenant-impersonation-architectural-lessons-2023-support.html", "reading_time": "5 min read", "seo_title": "Support access to customer tenants", "meta_description": "Every multi-tenant platform gives support a path into customer environments. It is usually invisible to the customer and always on.", "hub": { "slug": "topics/privileged-identity", "title": "Privileged identity and account recovery" }, "answer": "That a support system holding customer diagnostic files can be more valuable than the tenants it serves. Between 28 September and 17 October 2023 an attacker read files belonging to 134 Okta customers; session tokens inside uploaded HAR files were used to hijack live sessions at five of them.", "answer_q": "What did the Okta 2023 support breach show about vendor access?", "entities": [ { "name": "Okta", "type": "Organization", "url": "https://www.okta.com/", "primary": true }, { "name": "HTTP Archive format", "type": "Thing", "url": "https://w3c.github.io/web-performance/specs/HAR/Overview.html" } ], "glossary": [ { "term": "HAR file", "def": "HTTP Archive: a JSON recording of a browser session including every request, response, header and cookie. Standard support diagnostic, and a faithful copy of live credentials." }, { "term": "Cross-tenant impersonation", "def": "Using access obtained in one tenant, or in the vendor's own systems, to act inside a different customer's environment." }, { "term": "Standing authority", "def": "Permission granted once, at role assignment, that persists until revoked — as opposed to authority granted per use and bounded in scope and time." } ], "checklist": { "title": "Reducing diagnostic-upload exposure", "id": "diagnostics", "desc": "What to change before the next support ticket asks for a browser capture.", "steps": [ { "name": "Capture in an isolated session.", "text": "Reproduce the issue in a private window with a throwaway account rather than an administrator's live session." }, { "name": "Strip and then verify.", "text": "Run the redaction tool, then grep the file for Cookie, Authorization and your own token prefixes before it leaves your network." }, { "name": "Revoke immediately after.", "text": "Treat any session used to produce a capture as burned, and invalidate it once the file is sent." }, { "name": "Ask what the vendor retains.", "text": "Get the retention period for support attachments in writing, and whether they are encrypted at rest separately from case metadata." } ] }, "cta": { "title": "Where this fits in Manav", "html": "Manav's model is that the party whose data is at stake holds the key to the access path. A support engineer's session is gated on a grant the customer signed, scoped and time-bounded, and every action under it verifies offline against the customer's own record.", "href": "../architecture.html", "label": "Read the architecture" } }