Enterprise AI (RAG): compliance, reliability, costs, etc. The complete guide.

Enterprises have a vast documentary heritage: procedures, contracts, meeting minutes, knowledge bases, support tickets, CRM content, product sheets, HR documents, internal standards, and security policies. Yet this knowledge often remains difficult to find, interpret, and reuse.

The RAG, for Retrieval-Augmented Generation, connects generative AI to the company’s internal data to produce more contextualized, verifiable, and business-appropriate responses.

This guide explains what a RAG architecture is, when to use it, how to design it, its potential costs, which risks to anticipate, and how to address security, confidentiality, and compliance challenges.

Generative AI has introduced a new way of interacting with information: asking a question in natural language and receiving a structured answer. For businesses, the promise is significant: saving time, reducing reliance on a few internal experts, streamlining access to knowledge, and improving the quality of operational decisions.

However, a general-purpose model used alone does not know the company’s internal documents. It can help with writing, rephrasing, or structuring, but it cannot spontaneously answer based on your contracts, procedures, quality database, or technical documentation. This is precisely the role of RAG: to provide the model with the right document excerpts at the right time, so it can respond based on a controlled business context.

RAG is therefore not simply about “plugging ChatGPT into files.” It is a complete architecture that combines data, document retrieval, security, governance, user experience, evaluation, and maintenance. Recent sources confirm that enterprise RAG implementations must address complex issues: query understanding, multi-source access, token constraints, response times, access rights, and content governance.

In this white paper, you will learn:

• what RAG is, simply put;
• when it is relevant;
• when it should not be used;
• which technical components make up a RAG architecture;
• which structural decisions to make;
• which costs to anticipate;
• how to address security, GDPR, AI Act, and confidentiality;
• how to launch a RAG project in a gradual and realistic way.

Key takeaways
RAG is a pragmatic answer to a very common problem: businesses have a wealth of knowledge, but it is scattered, hard to access, and rarely usable in natural language.

The principle is simple:

1. the user asks a question;
2. the system searches internal documents for the most relevant passages;
3. these passages are added to the context sent to the language model;
4. the model generates a response based on these excerpts;
5. the response can cite the sources used.

AWS describes RAG as a technique that augments a large language model with external data, such as internal documents, to provide the model with the context needed to produce a useful response in a specific use case.

A simple analogy

Imagine an expert who needs to answer a complex question. Without documents, they rely on memory. With RAG, the right files are automatically opened to the right pages before they respond.

The language model acts as the writer who formulates the answer. The search engine acts as the archivist who retrieves the sources. The RAG architecture organizes the collaboration between the two.

RAG vs. Traditional Chatbot

A traditional chatbot responds based on what it learned during training or its initial prompt. It can be fluent, but it risks producing answers that are too generic or misaligned with your internal rules.

A RAG chatbot, however, first searches your internal sources: HR policies, product documentation, quality procedures, contracts, FAQs, support tickets, sales documents. The response is thus more grounded in your operational reality.

RAG vs. Fine-Tuning

The fine-tuning consists of retraining or specializing a model using examples. It can be useful for learning a style, response structure, classification, or specific behavior.

But for leveraging internal documents that change frequently, RAG is often more suitable. It allows knowledge to be updated by reindexing or syncing sources, without retraining the model. It also facilitates source citation, traceability, and rights management.

ApproachUseful forLimitations

Simple prompt

One-off tasks, writing, rephrasing

Not connected to internal data

Fine-tuning

Style, classification, recurring behaviors

Less suited to evolving knowledge

RAG

Document search, sourced answers, internal knowledge

Heavily dependent on data quality and retrieval

Key takeaway
RAG is generally more relevant than fine-tuning when the main challenge is to leverage an evolving internal knowledge base with sourced answers.

Problem: employees frequently ask the same questions about leave, remote work, expense reports, benefits, and onboarding processes.

RAG solution: connect an assistant to HR policies, internal agreements, FAQs, and procedures.

Business value: reduced repetitive requests, greater employee autonomy, consistent and traceable answers.

Augmented customer support

Problem: support teams must search through past tickets, product sheets, escalation procedures, and knowledge bases.

RAG solution: provide suggested answers from internal sources, with citations of the documents used.

Business value: reduced processing time, improved response quality, leveraging past incidents.

Intelligent document search engine

Problem: documents are scattered across SharePoint, Google Drive, Notion, Confluence, ERP, CRM, or PDF files.

RAG solution: index priority sources and enable users to ask questions in natural language.

Business value: faster access to information, reduced dependency on experts, better knowledge reuse.

Legal or compliance assistant

Problem: teams need to quickly retrieve clauses, obligations, procedures, or internal decisions.

RAG solution: query contracts, policies, frameworks, internal notes, and validated archives.

Business value: accelerated search, more effective analysis preparation, better traceability.

Key consideration: this type of assistant must be carefully framed. It can help retrieve and synthesize information but should not be presented as automated legal advice.

Sales assistant

Problem: sales teams waste time finding the right pitch decks, case studies, offer sheets, objection responses, or proposal elements.

RAG solution: create an assistant connected to sales documentation and validated content.

Business value: faster responses to prospects, message consistency, faster onboarding of new hires.

Technical knowledge base

Issue: technical documentation is extensive, partially outdated, and difficult to navigate.

RAG solution: query manuals, tickets, changelogs, schemas, maintenance procedures, and internal guides.

Business value: faster diagnostics, reduced errors, support for field teams.

Employee onboarding

Issue: a new employee must quickly absorb a large number of documents.

RAG solution: onboarding assistant capable of answering questions based on validated internal documents.

Business value: smoother integration, reduced time spent by managers answering repetitive questions.

Key takeaway
The best RAG use cases are those where answers already exist in documents but are hard to find, synthesize, or contextualize.

5. When RAG is not the right solution

RAG is not a one-size-fits-all solution.

Poorly structured or insufficient data

If documents are incomplete, contradictory, outdated, or scattered without logic, RAG may produce poor responses. A RAG assistant does not magically fix a weak knowledge base.

Need for complex business reasoning

RAG excels at retrieving and synthesizing information. It is less suited for applying complex business rules, calculating scenarios, arbitrating between multiple constraints, or executing transactional processes.

In these cases, a dedicated business application, a rules engine, or a structured workflow may be more appropriate.

Unreliable data

If no one knows which document is authoritative, the AI won’t either. The project must start by clarifying authorized sources, outdated documents, and update owners.

Automation over document search

If the issue mainly involves sending reminders, filling out forms, routing tickets, or updating a CRM, a standard automation may suffice.

Need for absolute guarantees

RAG can reduce hallucinations, improve traceability, and provide sources. However, it does not guarantee that every answer will be perfect. For critical decisions, human validation, deterministic rules, or control processes must be implemented.

Key takeaway
Good scoping also means knowing when to say no to RAG. If the need falls under a workflow, rule engine, or traditional business application, it should be acknowledged.

Poor or poorly structured data

If documents are incomplete, contradictory, outdated, or scattered without logic, RAG may produce subpar answers. A RAG assistant does not magically fix a weak knowledge base.

Need for complex business reasoning

RAG excels at retrieving and synthesizing information. It is less suited for applying complex business rules, calculating scenarios, arbitrating between multiple constraints, or executing transactional processes.

In these cases, a business application, rule engine, or structured workflow may be more appropriate.

Unreliable data

If no one knows which document is authoritative, the AI won’t either. The project must start by clarifying authorized sources, outdated documents, and those responsible for updates.

Automation rather than document search

If the issue mainly involves sending reminders, filling out forms, routing tickets, or updating a CRM, a standard automation may suffice.

Need for absolute guarantees

RAG can reduce hallucinations, improve traceability, and provide sources. However, it does not guarantee that every answer will be perfect. For critical decisions, human validation, deterministic rules, or control processes must be implemented.

Key takeaway
Good scoping also means knowing when to say no to RAG. If the need falls under a workflow, rule engine, or traditional business application, it should be acknowledged.

Internal sources
├─ PDF, Word, PowerPoint
├─ SharePoint / Google Drive / Notion / Confluence
├─ CRM / ERP / support tickets
└─ business databases

↓

Document ingestion
├─ text extraction
├─ OCR if needed
├─ cleaning
├─ structuring
└─ metadata detection

↓

Chunking
├─ by sections
├─ by headings
├─ by paragraphs
└─ with controlled overlap

↓

Embeddings + indexing
├─ embedding model
├─ vector database
├─ optional lexical index
└─ metadata and permissions

↓

User query
↓

Retrieval
├─ vector search
├─ keyword search / BM25
├─ hybrid search
├─ access control filters
└─ reranking

↓

Enriched prompt
├─ question
├─ relevant excerpts
├─ response instructions
└─ citation rules

↓

LLM
↓

User response
├─ summary
├─ citations
├─ limitations
└─ potential actions

↓

Logs, feedback, evaluation, monitoring

Data sources

Sources can be simple, such as a folder of PDFs, or complex, like a set of internal systems. The priority is not to connect everything, but to connect the right sources.

Document ingestion

Ingestion involves extracting, cleaning, and transforming documents to make them usable. Scanned PDFs, tables, images, diagrams, and poorly formatted documents may require OCR, advanced parsing, or manual reprocessing.

AWS Bedrock Knowledge Bases, for example, describes managed workflows spanning ingestion to retrieval, including text chunking, embeddings, and storage in compatible vector databases.

Chunking

Chunking involves splitting documents into segments. Overly long chunks dilute information. Overly short chunks lose context. The strategy depends on the document type: procedures, contracts, FAQs, technical documentation, or support tickets.

LlamaIndex notes that changing chunk size and overlap alters the computed embeddings, and that a smaller chunk may be more precise while a larger chunk may be more general.

Embeddings and vector database

Embeddings transform text into numerical vectors, enabling semantic proximity measurement between a query and documentary passages.

The vector database stores these vectors and retrieves the passages most relevant to a query.

Hybrid search and reranking

Vector search is useful, but not always sufficient. Modern architectures often combine:

• vector search;
• keyword search;
• metadata filters;
• reranking;
• citations;
• access control.

Microsoft outlines common RAG pipeline techniques: full-text search, vector search, chunking, hybrid search, query rewriting, and reranking. Pinecone also explains that two-step reranking can improve quality by combining large-scale search with more precise re-ranking.

Access rights management

A critical point: a user must not receive an answer based on a document they do not have access to. The architecture must enforce permissions at the search stage, not just in the interface.

Microsoft highlights the need for granular access control when private content is exposed to LLM, with mechanisms such as security trimming, query-time filters, and network isolation.

Key takeaway
A robust RAG architecture is a complete chain: ingestion, document quality, search, permissions, prompt, model, citations, logs, and continuous improvement.

The model choice depends on several criteria:

• response quality;
• cost per token;
• latency;
• maximum context;
• regional availability;
• contractual commitments;
• multilingual capability;
• compatibility with security requirements.

A highly powerful model isn’t always necessary. For many RAG use cases, the best trade-off is to use a fast, cost-effective model for simple queries and a more advanced one for complex cases.

External API or self-hosted model?

Option | Advantages | Limitations

External API

Quick to deploy, high-performing models, reduced maintenance

Vendor dependency, contractual clauses to review, variable costs

Self-hosted model

Enhanced control, environment mastery, potential sovereignty

Infrastructure, MLOps, monitoring, higher costs and required expertise

Hybrid

Optimisation by use case

More complex architecture

Which vector database?

The choice depends on volume, latency, filters, security model, cloud environment, and in-house expertise. An SME with a few thousand documents doesn’t have the same needs as an international group with millions of chunks.

Which chunking strategy?

Testing is required. The right segmentation depends on the documents. A FAQ can be split question by question. A contract must retain clauses and sub-clauses. A procedure must keep steps together.

How to manage metadata?

Metadata is essential:

• document type;
• date;
• version;
• author;
• department;
• confidentiality;
• language;
• client or project;
• access rights;
• status: draft, approved, obsolete.

They help filter, prioritise, and explain responses.

How to limit hallucinations?

No method eliminates them entirely. However, they can be reduced:

• high-quality retrieval;
• mandatory citations;
• instruction not to answer without a source;
• confidence threshold;
• displaying limitations;
• regular evaluation;
• user feedback;
• adversarial testing;
• human validation for sensitive cases.

How to assess quality?

A RAG system should be evaluated with real questions. Criteria may include:

• accuracy;
• completeness;
• faithfulness to sources;
• ability to say “I don’t know”;
• relevance of citations;
• latency;
• satisfaction rate;
• reduced search time;
• no data leakage.

Key takeaway
Technical decisions are not just technical. They impact cost, security, answer quality, maintenance, and user adoption.

Main cost drivers

ItemDescriptionImpact

Scoping

business workshops, objectives, success criteria

essential to avoid a useless POC

Document audit

sources, quality, rights, obsolescence

often underestimated

Data preparation

cleaning, structuring, OCR, metadata

can become significant

Development

ingestion, retrieval, interface, orchestration

core of the project

Integrations

SSO, SharePoint, CRM, ERP, ticketing

depends on the IS

LLM

cost of API calls or model infrastructure

depends on volume

Embeddings

initial indexing and updates

depends on document volume

Vector database

storage, queries, availability

depends on scale

Security

IAM, logs, encryption, testing

non-negotiable

Monitoring

cost tracking, errors, quality

essential in production

Maintenance

fixes, reindexing, improvements

recurring cost

Indicative costs of major LLM providers

Prices change frequently. The figures below should be read as orders of magnitude observed as of May 27, 2026, and verified before any contractual commitment.

ProviderExample modelsPublic indicative prices

OpenAI

GPT-5.5, GPT-5.4, GPT-5.4 mini

GPT-5.5: $5 / 1M tokens input and $30 / 1M tokens output; GPT-5.4: $2.50 / 1M input and $15 / 1M output; GPT-5.4 mini: $0.75 / 1M input and $4.50 / 1M output.

Anthropic

Claude Opus 4.7, Sonnet 4.6, Haiku 4.5

Opus 4.7: $5 / MTok input and $25 / MTok output; Sonnet 4.6: $3 / MTok input and $15 / MTok output; Haiku 4.5: $1 / MTok input and $5 / MTok output.

Google

Gemini 3.5 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Flash-Lite

Gemini 3.5 Flash standard: $1.50 / 1M input and $9 / 1M output; Gemini 2.5 Pro: $1.25 / 1M input up to 200k tokens and $10 / 1M output; Gemini 2.5 Flash: $0.30 / 1M input text/image/video and $2.50 / 1M output; Flash-Lite: $0.10 / 1M input and $0.40 / 1M output.

Mistral AI

Mistral Small 4, Large 3, Medium 3.5

Small 4: $0.15 / M tokens input and $0.60 / M tokens output; Large 3: $0.50 / M input and $1.50 / M output; Medium 3.5: $1.50 / M input and $7.50 / M output.

Simplified cost calculation example

Assume:

• 200 users;
• 20 questions per user per business day;
• 22 days per month;
• 1,500 input tokens per question, including context;
• 500 output tokens per answer.

This amounts to approximately:

• 132 million input tokens per month;
• 44 million output tokens per month.

Monthly costs will vary significantly depending on the chosen model. With an economy model, they can remain manageable. With a premium model, they can become substantial. The key is therefore to control costs based on actual usage, not just the listed price.

What drives costs up

• sending too much context to the model;
• use a premium model for all queries;
• unnecessarily reindex the entire document base;
• multiply LLM calls in an agentic workflow;
• omit caching;
• fail to limit retrieved documents;
• neglect usage monitoring;
• connect non-priority sources;
• process large volumes of scanned PDFs without a strategy.

How to control costs

• select the cheapest model that meets the expected quality level;
• limit the context sent;
• use reranking to send fewer but more relevant chunks;
• cache recurring answers or contexts;
• separate simple and complex queries;
• track costs by team, usage, source, and query type;
• set quotas;
• start with a limited document scope.

Key takeaways
The cost of RAG depends less on the “model price” than on the overall architecture: document volume, number of users, context length, usage frequency, retrieval quality, and integration level.

As soon as a RAG system processes personal data, GDPR must be considered: purpose, legal basis, data minimisation, retention period, individual rights, security, subcontracting, and documentation.

The CNIL notes that GDPR does not prevent AI innovation but requires addressing risks to individuals when personal data is used. It emphasises the need for a legal basis, data minimisation, and documentation of processing.

In a RAG project, the questions to ask are practical:

• do the documents contain personal data?
• sensitive data?
• HR data?
• customer data?
• health data?
• confidential contractual information?
• information covered by professional secrecy?
• do all users have the right to access this information?

DPIA

A Data Protection Impact Assessment may be necessary or strongly recommended when processing poses high risks. The CNIL notably recommends conducting a DPIA for AI system development in several cases, such as when sensitive data is collected, personal data is processed on a large scale, data from vulnerable individuals is used, or datasets are cross-referenced.

AI Act

The AI Act introduces a risk-based approach. Obligations vary depending on whether the system is prohibited, high-risk, subject to transparency requirements, or minimal risk. The European Commission states that prohibited practices and AI literacy obligations have applied since 2 February 2025, rules for general-purpose AI models since 2 August 2025, and the timeline for high-risk systems was adjusted under the Digital Omnibus, with some rules set for 2 December 2027 and others for 2 August 2028, depending on the category.

For a company deploying an internal RAG assistant, the main challenge is to classify the use case: simple document assistant, HR tool, decision support, or a tool used in recruitment, training, compliance, credit, healthcare, safety, or other sensitive areas. European guidelines on high-risk system classification were still under consultation and adjustment as of May 2026.

RAG system security

The main risks are:

• sensitive data leaks;
• incorrect access rights enforcement;
• prompt injection;
• indirect prompt injection via documents;
• system prompt exposure;
• document poisoning;
• unsourced responses;
• logs containing sensitive data;
• over-permissioned agents;
• vendor lock-in;
• lack of auditability.

The OWASP Top 10 for LLM Applications 2025 ranks prompt injection, disclosure of sensitive information, supply chain risks, data/model poisoning, excessive agency, system prompt leakage, vector and embedding weaknesses, disinformation, and unbounded resource consumption among the major risks of LLM applications.

The UK’s NCSC notes that prompt injection is not comparable to a simple SQL injection: LLM systems do not inherently separate data from instructions. It recommends treating LLMs as “inherently confusable” components and mitigating risks through design, privilege limitation, monitoring, and deterministic safeguards.

Recommended measures

• SSO and strong authentication;
• user-based access control;
• document filtering at query time;
• encryption in transit and at rest;
• environment segmentation;
• separation of dev, test, and prod;
• controlled logging;
• log retention policy;
• masking or pseudonymisation where relevant;
• security testing;
• AI red teaming;
• internal usage policy;
• contractual review of suppliers;
• subprocessing clauses;
• monitoring of transfers outside the EU;
• documentation of decisions.

The NIST AI RMF Generative AI Profile provides a voluntary framework for integrating trust, governance, measurement, and risk management considerations into generative AI systems. The Cloud Security Alliance also recommends applying Zero Trust principles to LLM environments: least privilege, micro-segmentation, continuous monitoring, identity management for both human and non-human entities, and governance.

Key takeaway
RAG security does not rely on a magic prompt. It relies on architecture: access rights, isolation, logging, encryption, monitoring, testing, and governance.

Choosing a vector database or model before clarifying the business need often leads to an impressive but useless proof of concept.

Connect all data

The more sources there are, the higher the risks: document noise, obsolescence, unclear rights, costs, latency. It’s better to start with a limited, reliable corpus.

Overlooking document quality

A high-performing RAG relies on reliable, up-to-date, and well-structured documents. Corpus quality is often more important than the model choice.

Forgetting access rights

This is one of the most dangerous mistakes. An assistant must never become a backdoor to access confidential documents.

Not planning for citations

Without sources, users cannot verify. Citations improve trust and make corrections easier.

Not testing responses

You need to create a set of real questions, with expected answers and evaluation criteria. Otherwise, quality will remain subjective.

Underestimating maintenance

Documents change, sources move, models evolve, pricing varies, users discover new use cases: a RAG must be maintained.

Confusing prototype with product

A prototype can overlook certain topics: fine-grained security, monitoring, UX, error handling, costs. A product cannot.

Launching a POC without success criteria

A POC must answer clear questions: does it save time? Are the answers reliable? Do users adopt it? Is the cost acceptable?

Key takeaways
Most RAG failures stem not from the model, but from poor scoping, a weak corpus, overlooked security, or lack of evaluation.

1. Identify the business need

Frame the problem without mentioning AI: “support teams waste 20 minutes retrieving a procedure”, “new hires keep asking the same questions”, “sales teams don’t know which document to use”.

2. Prioritize use cases

Not all use cases are equal. Focus on those that combine business value, document feasibility, and manageable risk.

3. Audit available data

List sources, their quality, freshness, format, owner, and confidentiality level.

4. Check security/GDPR constraints

Identify personal, sensitive, or confidential data. Define rights, roles, and documentation obligations.

5. Design a target architecture

Select components: ingestion, embeddings, vector database, model, interface, logs, monitoring, authentication.

6. Develop a limited prototype

Narrow the scope: one team, one corpus, one use case, a simple interface.

7. Test with real cases

Use actual user questions, not just ideal demos.

8. Measure quality

Track accuracy, citations, justified non-response rate, satisfaction, time saved, errors, costs.

9. Deploy gradually

Expand in waves: more users, more sources, more features.

10. Maintain and improve

Reindex, correct, monitor, train, adjust prompts, review costs, enhance sources.

Key takeaways
A successful RAG project resembles a business product more than an isolated technical experiment.

Business problem

• Who is losing time today?
• What problem do we want to solve?
• Which decision or task will be improved?
• What would be a useful outcome in 3 months?

Users

• Who will use the tool?
• How many users?
• What user profiles?
• What different access rights?
• What level of training?

Data

• Which sources will be used?
• Are the documents reliable?
• Are they up to date?
• Who maintains them?
• Are there duplicates?
• Are there outdated documents?
• Are the formats usable?

Security and compliance

• Do the documents contain personal data?
• Sensitive data?
• Confidential information?
• Are access rights clearly defined?
• Is a DPIA required?
• Which vendors will have access to the data?
• Where will the data be hosted?
• What is the retention period for logs?

Response and quality

• Should the model cite its sources?
• Should it refuse to answer without a source?
• What level of accuracy is expected?
• Who validates the answers?
• How do you correct a wrong answer?
• How is quality measured?

Costs

• What initial budget?
• What recurring budget?
• How many queries are expected?
• Which model to use?
• What usage thresholds?
• What monitoring?

Deployment

• Prototype or product?
• Which pilot team?
• What timeline?
• What user support?
• What maintenance plan?

Key takeaways
If you can’t answer this checklist, the project is probably not ready for development.

A B2B services company with 600 employees has:

• 1,200 internal procedures;
• 300 contract templates;
• 450 support FAQs;
• 20,000 historical tickets;
• project committee minutes;
• a poorly structured SharePoint database;
• several outdated but still accessible documents.

Support, sales, and operations teams spend significant time searching for information. Answers vary depending on the person asked. New hires rely heavily on internal experts.

Initial problem

The company wants to enable its teams to ask questions in natural language:

• “What is the escalation procedure for a critical incident?”
• “Which clause template should be used for a public sector client?”
• “How should an out-of-time refund request be handled?”
• “What are the prerequisites before going live?”

It requires sourced answers, limited to validated documents, while respecting access rights.

Chosen approach

The project starts with a limited scope: validated support procedures and FAQs. Contracts and historical tickets are excluded from the first batch as they contain more sensitive data and document noise.

Architecture

• SSO via internal identity;
• ingestion of validated documents from SharePoint;
• text extraction and cleaning;
• metadata: department, date, version, owner, status;
• chunking by title and section;
• embeddings;
• vector database with access-right filters;
• hybrid search;
• reranking;
• answer with citations;
• anonymised logs where possible;
• user feedback;
• quality/cost dashboard.

Identified risks

• outdated documents still present;
• inconsistent SharePoint permissions;
• contradictory procedures;
• out-of-scope questions;
• risk of copying and pasting client data into prompts;
• unrealistic user expectations.

Measures taken

• manually validated corpus;
• exclusion of unapproved documents;
• refusal to answer without a source;
• display of documents used;
• usage policy;
• short user training;
• cost thresholds;
• weekly feedback review at launch.

Expected benefits

• reduced search time;
• greater consistency in responses;
• faster onboarding;
• reduced requests to experts;
• identification of documentation gaps.

Limitations

• the assistant does not replace experts;
• it does not yet handle contracts;
• complex answers require validation;
• quality depends on document updates.

Next steps

After 8 weeks of pilot:

1. measure quality;
2. identify unanswered questions;
3. correct the corpus;
4. gradually integrate non-sensitive contract templates;
5. add simple workflows, e.g., opening a ticket or suggesting a procedure.

Key takeaways
A realistic RAG project starts small, proves its value, corrects its data, then scales gradually.

RAG is one of the most useful architectures for applying generative AI to a company’s internal data. It transforms a hard-to-exploit document base into a conversational interface capable of searching, synthesising, and citing its sources.

But a successful RAG project is not just about connecting a chatbot to documents. Its success depends on several factors:

• a clear business need;
• a reliable corpus;
• a robust retrieval architecture;
• strict access control;
• security by design;
• compliance with GDPR and the AI Act;
• cost management;
• a simple user experience;
• continuous evaluation;
• real maintenance.

RAG is not magic. But when well designed, it can become a very practical tool to reduce search time, capitalise on internal knowledge, streamline operations and improve the quality of business responses.

Key takeaway
The value of RAG does not come from the model alone. It comes from the whole: data, architecture, security, usage, governance and adoption.

Scroll helps you frame the need, assess feasibility, design the architecture and develop a solution tailored to your business, technical and regulatory constraints.

Our approach is pragmatic:

• start with the business need;
• audit the available data;
• identify the risks;
• choose a suitable architecture;
• develop a useful prototype;
• measure quality;
• prepare for a secure deployment.

Let’s discuss your RAG project

Before developing, have your AI project framed: use cases, data, architecture, costs, security, compliance and deployment roadmap.

Sources used

Specifications

Brief provided by the user for the white paper’s positioning, structure, target audience, tone and constraints.

RAG and modern architectures

Microsoft Learn, “Retrieval-augmented generation in Azure AI Search”, accessed 27 May 2026. Source used for enterprise RAG challenges: query understanding, multi-source access, token constraints, governance, security, classic RAG and agentic retrieval.

Microsoft Cloud Blog, “Common retrieval augmented generation techniques explained”, 4 February 2025. Source used for RAG techniques: full-text search, vector search, chunking, hybrid search, query rewriting and reranking.

AWS Prescriptive Guidance, “Understanding Retrieval Augmented Generation”. Source used for the definition of RAG, basic steps and the role of embeddings, vector databases and the orchestrator.

AWS, “Amazon Bedrock Knowledge Bases”. Source used for managed RAG workflows, ingestion, vector stores, advanced chunking, GraphRAG, reranking and source attribution.

LlamaIndex, “Basic Strategies”. Source used for considerations on chunk size, chunk overlap, embeddings, and hybrid search.

Pinecone, “Rerankers and Two-Stage Retrieval”. Source used to explain the value of reranking and two-stage retrieval.

GDPR, CNIL, and personal data

CNIL, “Developing AI systems: CNIL recommendations for GDPR compliance”, 22 July 2025. Source used for GDPR principles applicable to AI: legal basis, data minimisation, security, sensitive data, DPIA, and documentation.

CNIL, sections on legal basis, minimisation, and the responsibilities of data controllers/processors.

CNIL, sections on DPIA and risk criteria.

European Data Protection Board, thematic page “Artificial intelligence”, including Opinion 28/2024 on certain data protection aspects related to AI models.

AI Act

European Commission, “AI Act | Shaping Europe’s digital future”. Source used for the risk-based approach, GPAI obligations, transparency requirements, implementation timeline, and Digital Omnibus adjustments.

AI Act Service Desk, “Timeline for the Implementation of the EU AI Act”. Source used for the phased implementation timeline and 2025–2027 milestones, with reference to the Digital Omnibus context.

European Commission, consultation of 19 May 2026 on guidelines for classifying high-risk AI systems. Source used to note that high-risk system classification remains a case-by-case assessment.

European Commission, “Standardisation of the AI Act”. Source used for the link between high-risk rules, harmonised standards, and the adjusted timeline.

AI security, LLMs, and RAG

OWASP GenAI Security Project, “2025 Top 10 Risk & Mitigations for LLMs and Gen AI Apps”. Source used for key LLM risks: prompt injection, sensitive information disclosure, supply chain, poisoning, excessive agency, system prompt leakage, vector/embedding weaknesses, misinformation, and unbounded consumption.

OWASP, “LLM01:2025 Prompt Injection”. Source used to clarify that RAG and fine-tuning do not fully eliminate prompt injection risks.

NIST, “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile”, published 26 July 2024. Source used for the generative AI risk management framework.

NCSC, “Prompt injection is not SQL injection (it may be worse)”, 8 December 2025. Source used to explain why prompt injection is a structural risk of LLMs and why it must be managed through architecture, impact limitation, and monitoring.

Cloud Security Alliance, “Using Zero Trust to Secure Enterprise Information in LLM Environments”, 3 February 2026. Source used for Zero Trust recommendations applied to LLM environments: least privilege, micro-segmentation, continuous monitoring, IAM, and governance.

CISA, “New Best Practices Guide for Securing AI Data Released”, 22 May 2025. Source used for best practices in securing data used to train and operate AI systems.

LLM pricing and API costs

OpenAI, “API Pricing”, accessed 27 May 2026. Source used for GPT-5.5, GPT-5.4, GPT-5.4 mini, batch API pricing, and cost-related options.

Anthropic, “Claude API Pricing”, accessed 27 May 2026. Source used for Claude Opus 4.7, Sonnet 4.6, and Haiku 4.5 pricing.

Google AI for Developers, “Gemini Developer API pricing”, accessed 27 May 2026. Source used for Gemini 3.5 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 2.5 Flash-Lite, and Gemini embeddings pricing.

Mistral AI Docs, model sheets for Mistral Small 4, Mistral Large 3, and Mistral Medium 3.5, accessed 27 May 2026. Sources used for indicative pricing per million tokens and context characteristics.