Identity-Aware Enterprise Search — Architecture

Unified NL search across 3 systems · Okta federation · query-time ACL enforcement · full audit · in-perimeter data

1 · System Overview
2 · Authentication Flow
3 · ACL & Ingestion
4 · Access Scenarios
5 · Connector Strategy
6 · Design Trade-offs
7 · Zero-Trust Layer
8 · Live Deployment (as-built)

End-to-End System Architecture

Three federated source systems (Wiki, HR, SecVault) behind a centralized identity-aware index. One Okta identity is federated into Google and reused to filter every query. Everything sits inside the company's VPC-SC perimeter with regional data residency.

↔ This is a wide diagram — scroll horizontally to see it all. Follow the arrows top-to-bottom: identity → zero-trust checks → segmented stores.

flowchart TB subgraph USER[" "] U["fa:fa-user Employee
(80,000 across 40 countries)"] end subgraph IDP["Identity & Federation Layer (normalized, NOT delegated)"] OKTA["Okta SSO
(existing single IdP)"] WIF["Workforce Identity Federation
Google STS · app sign-in only
short-lived creds · no stored keys"] MAP["Identity Mapping Store
source IDs ↔ Okta principals
holds high-cardinality claims
(mgr-of-X, need-to-know lists)"] BEYOND["BeyondCorp · Access Context Manager
device posture · network · IdP risk
RISC / Shared Signals → CAEP"] end subgraph SEARCH["Gemini Enterprise / Agentspace"] AGENT["NL Query / Agent Orchestration"] CAEP["ZERO-TRUST · CAEP session check
re-evaluated EVERY query
revoke = instant cut-off"] CAA["ZERO-TRUST · Context-Aware Access
trusted context? (device+network+risk)
gates restricted classifications"] FILTER["QUERY-TIME ACL FILTER
principals ∩ allow/deny tags
deny always wins"] CHECK["Check-back (sensitive stores only)
live ACL re-validation"] end subgraph INDEX["Segmented Data Stores (not one monolith · region-pinned)"] GENDS["General store (US)
Wiki"] HRDS["HR store (US)
HR — tighter blast radius"] SECDS["Security store (US)
SecVault — handling rules / audit logs"] EUDS["EU-only store (GDPR)
CMEK · Access Transparency"] end subgraph CONN["Custom Connector Layer (keyless via WIF)"] C1["Wiki connector"] C2["HR connector"] C3["SecVault connector
LDAP · ACL mirroring · deny-by-default"] end subgraph SRC["Source Systems"] S1["Wiki
spaces / pages"] S2["HR
HR records / org chart"] S3["SecVault (legacy)
handling rules / audit logs"] end subgraph GOV["Governance & Perimeter"] AUDIT["Cloud Audit Logs
→ BigQuery + WORM (immutable)"] PERIM["VPC-SC · Private Service Connect · CMEK"] end U -->|query + login| OKTA OKTA --> WIF WIF -->|app sign-in| AGENT AGENT --> CAEP CAEP -->|session valid| CAA CAA --> FILTER BEYOND -.context + risk signals.-> CAA BEYOND -.revocation events.-> CAEP FILTER -.resolve principals.-> MAP FILTER --> CHECK CHECK --> GENDS CHECK --> HRDS CHECK --> SECDS CHECK --> EUDS C1 -->|content + ACLs| GENDS C2 -->|content + ACLs| HRDS C3 -->|content + ACLs| SECDS C2 -.GDPR subset.-> EUDS C1 -.normalized IDs.-> MAP C2 -.normalized IDs.-> MAP C3 -.normalized IDs.-> MAP C1 --> S1 C2 --> S2 C3 --> S3 AGENT -.logs.-> AUDIT FILTER -.access decisions.-> AUDIT INDEX --- PERIM CONN --- PERIM classDef idp fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; classDef search fill:#2a1f4d,stroke:#9a6cff,color:#e6edf3; classDef index fill:#173a2f,stroke:#3ec98a,color:#e6edf3; classDef conn fill:#3a2f17,stroke:#d9a441,color:#e6edf3; classDef gov fill:#3a1f24,stroke:#e06c75,color:#e6edf3; classDef zt fill:#0f3a3a,stroke:#2fd6c3,color:#e6edf3; class OKTA,WIF,MAP idp; class BEYOND,CAEP,CAA zt; class AGENT,FILTER,CHECK search; class GENDS,HRDS,SECDS,EUDS index; class C1,C2,C3 conn; class AUDIT,PERIM gov;

Authentication Flow — One Search, End to End

No new credentials, and no fragile per-query token-delegation chain. Okta handles app sign-in; identities are normalized to canonical principals once at ingest and resolved at query time.

sequenceDiagram autonumber participant U as Employee participant O as Okta SSO participant W as Workforce Identity
Federation (Google STS) participant A as Agentspace participant M as Identity
Mapping Store participant I as Segmented
Data Stores participant L as Audit Log U->>O: 1. Authenticate (existing SSO + 2FA) O-->>U: 2. OIDC/SAML assertion (+ low-cardinality group claims) U->>W: 3. Present assertion W-->>U: 4. Short-lived federated credential
(app sign-in only · no long-lived keys) U->>A: 5. NL query + federated identity A->>M: 6. Resolve full principal set
(token groups + high-cardinality claims) M-->>A: 7. Canonical principals A->>I: 8. Retrieve, filtered by allow/deny principals I-->>A: 9. Only authorized documents A->>A: 10. (Sensitive stores) live ACL check-back A-->>U: 11. Grounded answer + citations A->>L: 12. Log who/query/docs/access-decision Note over O,M: Identity NORMALIZED, not delegated:
no per-query Okta → Azure AD → source token chain.
Source IDs are mapped to Okta principals ONCE at sync.

ACL Replication & Ingestion (incl. legacy SecVault)

Connectors ingest content + ACL metadata together, mapping each source's permission model to canonical Okta principals. The legacy LDAP+2FA system has no OAuth, so its connector mirrors LDAP group memberships into document ACLs at index time.

flowchart LR subgraph SOURCES["Source permission models"] P1["Wiki space perms
(+ need-to-know barrier)"] P2["HR org-chart
(manager → reports)"] P5["SecVault LDAP groups
(no OAuth)"] end subgraph MAP["Identity Mapping Service"] M["Map source principals
→ canonical Okta group/user IDs"] end subgraph SYNC["Freshness"] DELTA["Scheduled delta sync"] HOOK["Webhooks / change streams"] end subgraph DS["Vertex AI Search"] D["Documents tagged with
allow/deny principals + embeddings"] end P1 --> M P2 --> M P5 -->|service acct reads,
mirrors LDAP groups| M M --> D DELTA --> D HOOK --> D QT["Permission change after indexing?"] -->|fast path| HOOK QT -->|backstop| DELTA QT -->|sensitive| CB["Query-time check-back
against live source ACL"] classDef a fill:#3a2f17,stroke:#d9a441,color:#e6edf3; classDef b fill:#173a2f,stroke:#3ec98a,color:#e6edf3; class P1,P2,P5,M,DELTA,HOOK a; class D,CB b;

Access Control Scenarios (the proof)

Same query, different identities, different results — driven entirely by principal intersection and deny rules. These are the live-demo test cases.

flowchart TB Q1["Atlas engineer: 'data handling policy'"] --> R1["Strict Atlas version ONLY
(general version is for another division)"] Q2["HR manager: 'case file for Yamada'"] --> R2["HR docs for DIRECT REPORTS only
(HR org-chart principals)"] Q3["Security officer: 'security incidents'"] --> R3["Audit logs YES · HR files DENIED
(deny principal wins)"] Q4["General staff: 'account setup / MFA'"] --> R4["Wiki KB article
NOT HR admin procedures"] classDef q fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; classDef r fill:#173a2f,stroke:#3ec98a,color:#e6edf3; class Q1,Q2,Q3,Q4 q; class R1,R2,R3,R4 r;

Per-Source Connector Strategy

Buy-vs-build, ingest-vs-federate, and ACL translation are decided per source. This per-source rigor is the core guarantee: no over-sharing incident from day one.

SystemConnectorIngest / federateACL & identityOn permission change
Wiki (Confluence-style)build (native*)ingestspace / page / label ACL; need-to-know barrier → denywebhook
HR (Workday-style)buildingestOAuth; security group / role → canonical ID; org-chart scopingscheduler (HR events)
Legacy SecVaultbuild (required)federate / tight intervalLDAP DN → mapping store; deny-by-defaultin-VPC bridge, near-real-time

* Native connector may exist in-product; this design leads with a custom connector and revisits buy-vs-build per ACL fidelity in a real engagement. The legacy system is the only one federated live (volatile + most sensitive); everything else is indexed for latency, resilience, and a clean audit trail.

What I Chose, and What I Gave Up

Deliberate design decisions a technical stakeholder will probe. Each names the alternative that was rejected and why.

flowchart TB D1["Index + query-time enforcement
over live federation to every source"] --> W1["Chosen for latency, resilience, clean audit.
Federate ONLY the volatile + most-sensitive source (legacy)."] D2["Shared experience, segmented stores
not one monolithic index"] --> W2["HR / security / general / EU-only as separate stores.
CMEK + Access Transparency need multi-region — region is front-loaded."] D3["Identity normalized, not delegated
no per-query token chain"] --> W3["One-time mapping at sync; high-cardinality claims
(mgr-of, need-to-know) live in the mapping store, not the token."] D4["Per-agent crypto identity
not a shared service account"] --> W4["SPIFFE-style identity + user-delegated OAuth + prompt-injection
screening. Tool actions phased to Phase 2 (deny-by-default in VPC-SC)."] classDef d fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; classDef w fill:#173a2f,stroke:#3ec98a,color:#e6edf3; class D1,D2,D3,D4 d; class W1,W2,W3,W4 w;

Zero-Trust Enforcement — Every Single Query

Authentication happens once per session; authorization is re-decided on every query. Two Google zero-trust controls wrap the ACL filter: CAEP (continuous session evaluation — a revoked or risky session is cut off mid-session) and Context-Aware Access (device posture + network + IdP risk decide whether restricted classifications are even reachable). Identity alone is no longer enough — context and session validity are checked too.

flowchart TB Q["Employee query
(already signed in via Okta/WIF)"] --> PDP subgraph PDP["Policy Decision Point — runs per query"] S1["1 · CAEP session check
still valid? not revoked?
no new risk signal?"] S2["2 · Context-Aware Access
managed device? corp network?
low IdP risk?"] S3["3 · Identity ACL filter
principals ∩ allow/deny
deny always wins"] end subgraph SIG["Zero-Trust signals (BeyondCorp · Access Context Manager)"] DEV["Device posture
(managed / unmanaged)"] NET["Network
(corporate / external)"] RISK["IdP risk · RISC
Shared Signals events"] end S1 -->|revoked or high-risk| DENYALL["DENY ALL
session terminated"] S1 -->|valid| S2 S2 -->|untrusted context| RESTRICT["Drop RESTRICTED docs
general docs still allowed"] S2 -->|trusted context| S3 RESTRICT --> S3 S3 --> OUT["Authorized results only
+ grounded answer"] DEV -.-> S2 NET -.-> S2 RISK -.-> S1 RISK -.-> S2 S1 -.decision.-> LOG["Cloud Audit Log
who · context · session · docs"] S2 -.decision.-> LOG S3 -.decision.-> LOG classDef pdp fill:#0f3a3a,stroke:#2fd6c3,color:#e6edf3; classDef sig fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; classDef deny fill:#3a1f24,stroke:#e06c75,color:#e6edf3; classDef ok fill:#173a2f,stroke:#3ec98a,color:#e6edf3; class S1,S2,S3 pdp; class DEV,NET,RISK sig; class DENYALL,RESTRICT deny; class OUT,LOG ok;

POC mapping: CAEP = local session-revocation registry (zerotrust.check_session); Context-Aware Access = device/network toggles gating sensitivity (zerotrust.context_allows); ACL filter = Vertex AI Search server-side allow/deny principals. In production these map to BeyondCorp / Access Context Manager / RISC Shared Signals, and the classification gate moves server-side into Vertex AI Search so restricted content is never returned.

Live Deployment on Google Cloud (what actually runs)

The tabs above describe the production-grade design. This tab shows the as-built POC deployed in this project: two Cloud Run services behind Identity-Aware Proxy, each running as its own least-privilege service account, both calling the same enforcement pipeline (query.search_as) against a real Vertex AI Search engine. The mock IdP / Okta of the design is realized here by Google Workspace (the IdP that IAP redirects to).

flowchart TB U["fa:fa-user Browser"] subgraph EDGE["Edge auth (authentication)"] IAP["Identity-Aware Proxy
blocks unauthenticated traffic"] IDP["Google Workspace (IdP)
soonchang.me · OAuth login
IAM: iap.httpsResourceAccessor"] end subgraph RUN["Cloud Run · asia-northeast1"] UI["aegis-search-ui
FastAPI demo (app.py)
SA: aegis-search-ui@…
roles: discoveryengine.viewer"] AG["aegis-search-agent
ADK FastAPI + dev UI (main.py)
SA: aegis-search-agent@…
roles: aiplatform.user + discoveryengine.viewer"] end subgraph PIPE["In-app enforcement (authorization) · query.search_as"] ZT["CAEP session check + Context-Aware Access
(zerotrust.py)"] ACL["Identity ACL filter
principals ∩ allow / deny-wins"] end GEM["Vertex AI · Gemini 2.5-flash
(agent reasoning + tool calls)"] VAIS["Vertex AI Search (Discovery Engine)
engine aegis-knowledge-search
data store aegis-knowledge-ds · global"] LOG["Cloud Logging
immutable audit trail"] U -->|HTTPS| IAP IAP -.no session.-> IDP IDP -.authenticated.-> IAP IAP -->|verified identity| UI IAP -->|verified identity| AG AG -->|search_knowledge_base tool| GEM GEM -->|chosen query text| ZT UI -->|persona + context| ZT ZT --> ACL ACL -->|filtered query| VAIS VAIS -->|authorized docs only| ACL ZT -.access decisions.-> LOG ACL -.who, query, docs.-> LOG classDef edge fill:#0f3a3a,stroke:#2fd6c3,color:#e6edf3; classDef run fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; classDef pipe fill:#2a1f4d,stroke:#9a6cff,color:#e6edf3; classDef data fill:#173a2f,stroke:#3ec98a,color:#e6edf3; classDef gov fill:#3a1f24,stroke:#e06c75,color:#e6edf3; class IAP,IDP edge; class UI,AG run; class ZT,ACL pipe; class GEM,VAIS data; class LOG gov;

Build & deploy path

Images are built by Cloud Build, pushed to Artifact Registry, and deployed to Cloud Run. The agent image is built from Dockerfile.agent (via cloudbuild.agent.yaml); the UI is built from Dockerfile.

flowchart LR SRC["poc/ source
(.gcloudignore excludes .venv)"] --> CB["Cloud Build"] CB --> AR["Artifact Registry
cloud-run-source-deploy"] AR --> CR["Cloud Run revision"] CR --> IAPD["IAP-gated URL"] classDef a fill:#3a2f17,stroke:#d9a441,color:#e6edf3; classDef b fill:#1b3a5c,stroke:#4f8cff,color:#e6edf3; class SRC,CB,AR a; class CR,IAPD b;

As-built resources

ComponentWhat runsIdentity / access
Demo UICloud Run aegis-search-ui (FastAPI, app.py)SA aegis-search-ui@ · discoveryengine.viewer; behind IAP
ADK agentCloud Run aegis-search-agent (ADK FastAPI + dev UI, Gemini 2.5-flash)SA aegis-search-agent@ · aiplatform.user + discoveryengine.viewer; behind IAP
Search indexVertex AI Search engine aegis-knowledge-search / store aegis-knowledge-ds (global)Queried server-side with allow/deny principal filter
Edge authIdentity-Aware Proxy → Google Workspace loginUsers need roles/iap.httpsResourceAccessor
AuditCloud LoggingWho / query / returned docs / access decision

How this maps to the design tabs: IAP + Google Workspace stand in for Okta / Workforce Identity Federation (edge authentication); the in-app query.search_as pipeline (CAEP + Context-Aware Access + ACL filter) is the same authorization model from the Zero-Trust tab; Vertex AI Search is the real index. The single global data store here is segmented into HR / security / EU-only stores in the production design.