Permission-Aware RAG: Making Your Internal AI Assistant Respect Document Access

Permission-aware RAG means the retrieval step of your internal AI assistant only ever sees chunks the person asking is allowed to open. Here’s the short version. Store access metadata (user and group IDs, or an access label) on every chunk when you index it. Take the user’s identity from a verified token, never from the prompt. Filter inside the search query rather than after it. Then plan for the boring parts that usually cause the leaks: stale permissions, group overage, connectors with too much access, and missing audit logs.

I’ve been through this recently. I led the rollout of an internal assistant that answers staff questions with source references and reads from several business systems, Salesforce included, with sign-in secured through Entra ID and AWS Cognito. This article is the guide I wanted at the start. It covers the design decisions and the concrete settings for Azure AI Search and Amazon Bedrock Knowledge Bases, plus Salesforce, auditing and testing.

Why a normal RAG pipeline leaks

A basic RAG pipeline copies documents out of SharePoint, a file share or a CRM, chunks them, embeds them and puts them in one vector index. The source system’s permissions stay behind. Your assistant now runs every question against one big pool of company knowledge, and the model is happy to summarise the salary review spreadsheet for anyone who asks the right question.

OWASP lists this under LLM08:2025 Vector and Embedding Weaknesses. Its prevention advice starts with fine-grained access controls and “permission-aware vector and embedding stores”. A system prompt that says “don’t reveal confidential data” does not count. Once a restricted chunk is in the context window, you’re relying on the model to keep a secret it has already been handed. (I covered the wider version of this problem in the hidden security issues with AI agents.)

Filter at retrieval time, not after

There are three places you could enforce access:

Where	How it works	Verdict
Pre-filter (in the query)	The search engine only scores chunks whose ACL matches the user	Use this. Restricted content never leaves the index
Post-filter (in your app)	Retrieve top-k, then drop chunks the user can’t see	Leaks through bugs and returns empty or thin answers when most of the top-k gets dropped
Output guardrail	Ask the model, or a classifier, to spot restricted content in the answer	A detection layer at best. Never the control

Post-filtering looks fine in a demo. In production it has two problems. The restricted chunks have already reached your application tier, so a logging statement or a debugging session can expose them. And if you ask for five results and the user can see none of them, you get a bad answer even though the right document was sitting at position 40. Filtering inside the query avoids both. Microsoft’s ISE team lists “filtering results after retrieval” as a common pitfall in their write-up on propagating SharePoint permissions to RAG pipelines (April 2026).

The reference design

Whatever stack you pick, the moving parts are the same:

1. Ingestion connector reads content and its permissions from the source system, using the narrowest access it can get.
2. ACL resolver works out the effective permissions (inheritance, sharing links, groups) and writes them as metadata on every chunk.
3. Identity layer signs the user in (Entra ID, Cognito or both) and hands the backend a signed token.
4. Retrieval service validates the token, derives the user’s IDs and groups server-side, and builds the filter. The client never sends a filter.
5. Audit log records who asked what, which chunks came back, and which filter was applied.
6. Sync job keeps permissions in the index close to the source, on a schedule or triggered by change events.

Step 4 matters more than it looks. If your front end posts something like {"question": "...", "groups": ["finance"]} and the backend trusts it, anyone with browser dev tools can widen their own access. Build the filter from the verified token and from directory lookups you do server-side.

Getting identity right: Entra ID and Cognito

Use immutable IDs, not names or emails

Store Entra object IDs (GUIDs) in your ACL fields, not UPNs, emails or display names. Microsoft’s token claims reference says preferred_username and email are mutable and must not be used for authorization. It recommends oid (with tid) to identify a user across services. Names change, and a recycled email address can quietly give a new hire the previous owner’s documents.

Watch for group overage

It’s tempting to read the user’s groups straight from the groups claim in the token. That works until someone joins too many groups. Per the same reference, if a user is in more than 200 groups (JWT) or 150 (SAML), the groups claim is left out and replaced with an overage pointer (_claim_names / _claim_sources, or hasgroups in some flows). Your code then has to call Microsoft Graph (for example /users/{id}/getMemberObjects) to get the real list.

If you miss this, a user with lots of groups gets a filter with no groups in it. If you’re lucky that means they see too little. If your code treats an empty group list as “no restriction”, they see everything. Handle the overage case explicitly and fail closed.

Cognito groups

With Amazon Cognito user pools, group membership arrives in the cognito:groups claim of the ID token. According to the Cognito ID token docs, it’s always an array, even for a single group. The same page says to verify the token signature before trusting any claim, and to check token_use is id. Use sub as the stable user ID, not cognito:username.

If you run both (Entra for staff, Cognito for an AWS-hosted app or external users), pick one canonical identifier per person and map the other to it in your backend. Otherwise you end up with two ACL vocabularies in one index.

Option A: Azure AI Search

Azure AI Search’s document-level access control overview (updated August 2026 at the time of writing) describes four approaches:

Approach	Status	Best for
Security filters (string match on a field you populate)	Generally available, works with any API version	Custom identity, push-model indexes, non-Microsoft sources such as a CRM
POSIX-like ACL / RBAC scopes	Preview	Content in ADLS Gen2 or Blob Storage with existing ACL or RBAC assignments
SharePoint in Microsoft 365 ACLs	Preview	SharePoint libraries, lists and site pages
Microsoft Purview sensitivity labels	Preview	Content already governed by Purview labels

With the native (preview) options, your app passes the user’s Entra token in the x-ms-query-source-authorization request header. The service compares the token’s user and group claims with the permission metadata stored in the index and trims results. The app itself still needs the Search Index Data Reader role on the index. The docs reference the 2026-08-01-preview REST API for these features. Preview features carry no SLA and Microsoft doesn’t recommend them for production, so weigh that before relying on them.

The GA route: security filters

For production today, security filters are the safe option. Add a filterable string collection to the index and populate it at ingestion with the Entra object IDs of the users and groups allowed to read each chunk:

{
  "name": "allowed_principals",
  "type": "Collection(Edm.String)",
  "filterable": true,
  "retrievable": false,
  "searchable": false
}

Setting retrievable to false keeps the ACL out of search responses, so it never ends up in the prompt or in a citation panel. At query time, your backend builds the filter from the user’s oid plus their resolved group IDs:

POST https://<service>.search.windows.net/indexes/<index>/docs/search?api-version=2024-07-01
Content-Type: application/json
Authorization: Bearer <app token>

{
  "search": "travel policy for client visits",
  "filter": "allowed_principals/any(p: search.in(p, '<user-oid>,<group-oid-1>,<group-oid-2>', ','))",
  "top": 5
}

The filter runs inside the query, before results come back. The same pattern appears in an accepted Microsoft Q&A answer on document-level security, and it’s a question that keeps coming up there. One detail that’s easy to get wrong: if you chunk with a skillset (integrated vectorization), the ACL field has to be projected onto every chunk. Microsoft’s docs point this out for both the ACL fields and Purview labels. A chunk with no ACL either matches nothing or, worse, gets indexed without the field and slips past a badly written filter.

SharePoint connector permissions

If you write your own SharePoint ingestion with Microsoft Graph, the ISE post recommends the Sites.Selected application permission, so the connector can only read sites you’ve explicitly granted. It warns against Sites.Read.All, which opens up every site in the tenant to one app registration. Resolve the effective ACL (inheritance and sharing included) once at ingestion and store it on each chunk.

Option B: Amazon Bedrock Knowledge Bases

On AWS, the standard pattern is metadata filtering. For an S3 data source, each document gets a sidecar file with the same name plus .metadata.json, so policy.pdf pairs with policy.pdf.metadata.json. AWS’s metadata filtering walkthrough uses Cognito for sign-in and a metadata attribute to limit what each user retrieves:

{
  "metadataAttributes": {
    "access_label": "finance"
  }
}

Then pass a filter on every Retrieve or RetrieveAndGenerate call. Here the list of labels comes from the verified token’s cognito:groups, mapped server-side to the labels each group may read:

import boto3

client = boto3.client("bedrock-agent-runtime")

def retrieve_for_user(question, allowed_labels, kb_id):
    if not allowed_labels:
        # Fail closed: no labels means no retrieval
        return []
    resp = client.retrieve(
        knowledgeBaseId=kb_id,
        retrievalQuery={"text": question},
        retrievalConfiguration={
            "vectorSearchConfiguration": {
                "numberOfResults": 5,
                "filter": {
                    "in": {"key": "access_label", "value": allowed_labels}
                },
            }
        },
    )
    return resp["retrievalResults"]

Check the Bedrock filtering docs for your vector store before you design the metadata, because the operators aren’t equally supported everywhere:

• in, notIn and listContains are described as best supported on Amazon OpenSearch Serverless (and Neptune Analytics for in).
• andAll and orAll each take a list of up to 5 filter expressions. If you store a list of allowed groups on each chunk and try to orAll one listContains per user group, you hit that limit quickly. That’s why I’d rather use one access label per document with in than a per-chunk group list.
• On OpenSearch Serverless, filtering requires the vector index to use the faiss engine, not nmslib.
• Managed knowledge bases don’t support startsWith or stringContains.

Also be careful with implicit metadata filtering, where the model generates a filter from the user’s question. That’s a relevance feature. Never let it decide access.

AWS is upfront about what this pattern gives you. Its secure multi-tenant RAG architecture post (June 2026) calls it filter-level, logical isolation rather than IAM-enforced isolation. It recommends a separate knowledge base per tenant for hard boundaries between customers, with filters inside each one. The post also adds Amazon Verified Permissions (Cedar policies) to make the access decision outside application code and flags fail-open filter logic as a real risk. Inside one company, logical isolation is usually acceptable. For truly separate legal entities or clients, I’d split the knowledge bases.

Systems with complex sharing: Salesforce and other CRMs

Document stores have fairly static ACLs. CRMs don’t. Salesforce access depends on the org-wide defaults, role hierarchy, sharing rules, teams and manual shares, and these change constantly as accounts change owner. Copying that model into chunk metadata is fragile.

There are two approaches that hold up better:

• Query live as the user. Don’t index CRM records at all. Let the assistant call the Salesforce API with a per-user OAuth connection, so Salesforce enforces its own sharing model. This is slower and needs per-user consent, but there’s no permission copy to go stale.
• Index, then double-check. If you must index (for semantic search across notes, say), keep a coarse filter in the index and confirm access for the final candidates before they reach the prompt. Salesforce’s UserRecordAccess object lets you query HasReadAccess for a user across up to 200 record IDs at a time.

The second option is a post-retrieval check, which I argued against earlier. The difference is that it’s a second gate behind a pre-filter, not the only one. Where the source system’s model is too rich to mirror, a second check against the source is the honest compromise.

Permission drift: the problem nobody demos

Every index-based approach shares one weakness: the index holds a copy of permissions, and copies go stale. Microsoft’s docs say plainly that source permission changes only show up in results after the metadata is synchronised. For SharePoint, changes on items with unique permissions are picked up incrementally on each indexer run (from the 2026-05-01-preview API). Changes inherited from a parent site, library or folder need an explicit refresh. The ISE post makes the same point for custom pipelines: someone whose access was revoked may still get answers from that content until the next ingestion run.

What helps:

• Index groups, not individuals. When someone leaves a team, Entra removes them from the group and the next token reflects it. No re-index needed. Per-user ACL entries are what go stale.
• Resolve groups at query time. Short token lifetimes, or a fresh Graph lookup, mean membership changes take effect fast even when document ACLs lag.
• Sync by sensitivity. HR and finance libraries get a more frequent permission refresh than the company handbook. Use change notifications (webhooks) where the source supports them.
• Have a purge path. When a document is deleted or locked down urgently, you need a way to remove its chunks from the index immediately, not at the next scheduled run.
• Tell people about the lag. Data owners should know that a permission change can take until the next sync to reach the assistant.

Auditing and source references

OWASP’s LLM08 guidance recommends immutable, detailed logs of retrieval activity. For an internal assistant, I’d log at least these fields per question:

• User ID (oid or sub), timestamp, and the app or channel used
• The filter that was actually applied (or a hash of the principal list)
• The IDs of the chunks and source documents returned
• Any tool or connector calls (for example, a Salesforce query) and their outcome

Logging the question and the answer text is a separate decision, because those can contain personal or confidential data. Agree retention and who can read the log with whoever owns data protection in your company.

Source references in the answer are also a security feature, not just a trust feature. When every answer links to the documents it used, users notice and report odd results (“why can I see this?”) far sooner than any monitoring rule would. Make sure the citation links point at the source system, so opening one goes through that system’s own permission check.

How to verify it actually works

Don’t sign this off by asking the assistant a few questions as an admin. Build a small permission test set:

1. Create two or three test users in different groups (for example, one in Finance, one in Sales, one in neither).
2. Plant a “canary” document in each restricted area with a unique, made-up phrase that appears nowhere else.
3. As each user, ask questions that should hit each canary. Check the raw retrieval results, not just the model’s answer, because the model might paraphrase or refuse.
4. Remove a test user from a group, wait for your documented sync window, and confirm access is gone.
5. Try the obvious attacks: injecting a group name into the prompt, tampering with the request body, and sending a request with no token or an expired one. All should return nothing.
6. Test a user near the group overage limit, or mock the overage claim, and confirm your Graph fallback works.

Automate this and run it after every pipeline change. A refactor that renames the ACL field can silently turn your filter into one that matches everything, or nothing.

Common problems

• Users with zero results for everything: usually the group overage case, or ACLs written as emails while the filter uses GUIDs (or the other way round).
• Some chunks visible to everyone: the ACL field wasn’t projected to chunks, or a default value like "everyone" was applied when resolution failed. Failed resolution should mean “nobody”, and the item should go to a review queue.
• Filter ignored on Bedrock: check the vector store supports the operator you used, and that OpenSearch Serverless indexes use faiss.
• Revoked user still gets answers: permission drift. Check when the last sync ran and whether the change was inherited from a parent folder.
• Connector can read too much: replace tenant-wide Graph permissions with Sites.Selected and grant sites one by one.

Where to start

If you’re starting a company assistant now, I’d do it in this order. First, put only broadly shareable content in the index (policies, handbooks, public product docs) and launch with that. Second, add group-based ACL fields and the server-side filter before any restricted library goes in. Third, connect systems like a CRM through live, per-user queries rather than copying them. Then build the canary tests and the audit log before you widen access.

If you plan to expose these sources to agents through MCP servers, the same rules apply to every tool call. My analysis of MCP servers and the post on agentic RAG with planning, memory and tools cover how retrieval changes once an agent picks its own queries. If you’re still deciding which directory should be the source of truth for groups, FreeIPA vs Active Directory is a useful starting point.

Sources and further reading

• Microsoft Learn: Document-level access control in Azure AI Search
• Microsoft Learn: ID token claims reference (groups overage)
• AWS: Configure and customize Bedrock Knowledge Base queries (metadata filtering)
• AWS: Understanding the Cognito ID token
• Salesforce: UserRecordAccess object reference
• OWASP: LLM08:2025 Vector and Embedding Weaknesses
• AWS Architecture Blog: Secure multi-tenant RAG with Amazon Bedrock and Verified Permissions
• AWS ML Blog: Access control for vector stores using metadata filtering
• Microsoft ISE Blog: Propagating SharePoint document permissions to AI Search and RAG pipelines