What is retrieval augmented generation?

The UK Government Digital Service describes RAG as a mechanism used to make additional data available to a large language model so that data can be incorporated into queries. It works by slicing documents into chunks, converting those chunks into a numerical format known as embeddings, and storing them in a vector database. At query time the system finds the chunks that best match the question, puts them into the prompt, and asks the model to answer using only that material. The model supplies language skill and your documents supply the facts, which matters because the model was never trained on your enterprise agreement or last Thursday's policy update.

Does Australian government guidance actually recommend RAG?

Yes. The DTA technical standard for government's use of artificial intelligence includes Criterion 70, which recommends that for generative AI, agencies build reference or contextual datasets to improve the quality of AI outputs. The standard says such a dataset can be in the form of a retrieval-augmented generation dataset or a prompt dataset, with the objective of more accurate and context aware AI outputs and reduced hallucinations. RAG is not a vendor fashion here. It is the pattern the national technical standard describes for grounding a generative system in your own material.

How should a RAG system handle sensitive or classified content?

Authorisation is evaluated at retrieval time against the requesting user, not applied as a filter after generation. Every chunk carries its source classification and permissions, and if the user cannot read the document the chunk never enters the prompt. The National Archives states that where AI technologies access source data containing sensitive or privileged information, the original sensitivity requirements continue to apply unless the sensitivity is downgraded. The DTA agentic addendum requires that classifications and sovereignty controls are incorporated into agent-to-agent data exchange, and ASD's guidance sets the surrounding expectations on access controls, input sanitisation and logging.

Do we need an impact assessment before deploying a RAG assistant?

If you are a non-corporate Commonwealth entity in scope of the Policy for the responsible use of AI in government v2.0, yes. The assessment must commence at the design stage and be finalised, with agreed risk treatments applied, before deployment. Existing use cases that have not been assessed must be dealt with by 30 April 2027. Every in-scope use case also needs a named accountable owner, and the internal register is shared with the DTA every six months. Victorian buyers work to a different stack, including the VPS AI Assurance Framework, which they are told to complete and attach to the procurement plan.

RAG for Australian Government Agencies

Q: Why do government RAG projects fail?

Four failure modes, and none of them is chunk size. Stale source data, because the demo indexed a folder once while production has to keep answering correctly after a guideline is superseded, and a RAG system fails confidently. Permission leakage across classifications, where retrieval does not enforce the same access controls as the source repository and the leak arrives as fluent prose with no access log fired. Answers you cannot cite, which are unusable because the officer who acts on one is accountable for it. And records you did not realise you were creating, because every question and answer may be a Commonwealth record.

Q: Are AI generated answers Commonwealth records?

Yes. The National Archives of Australia advises that AI generated content created or received by Australian Government agencies constitutes a Commonwealth record under the Archives Act 1983, and that the Act is technology and format neutral. The trap follows immediately: NAA notes that AI systems are not designed as recordkeeping systems and generally lack adequate records management functionality, so outputs that must be kept have to be captured into a system where they can be properly managed, with adequate metadata. Where outputs are relied on for decisions, the prompts, inputs and source content may need retaining too. Victorian agencies carry an equivalent obligation under the Public Records Act 1973.

Most agencies asking about retrieval augmented generation have already sat through the demo. Someone loaded a folder of PDFs into a vector database, asked a question, and got a fluent answer with a citation underneath it. It looked finished.

It was not. The technology in that demo is commoditised. What is not commoditised is everything an Australian public sector deployment must satisfy before it can answer a real staff member's question about a real citizen: classification handling, records obligations, impact assessment, transparency statements, and a defensible answer to "where did that sentence come from?"

This guide is written for Australian agencies, because almost every guide to government RAG is written for another jurisdiction.

What RAG actually does

The clearest short description in public sector writing comes from the UK Government Digital Service. Its AI Insights: Agentic RAG note describes RAG as "a mechanism used to make additional data available to a large language model (LLM) so that data can be incorporated into queries," working by "slicing documents into chunks, converting those chunks into a numerical format (known as embeddings), and storing them in a special kind of database called a vector database."

At query time the system finds the chunks that best match the question, puts them into the prompt, and asks the model to answer using only that material. The model supplies language skill. Your documents supply the facts. That division of labour is the point: the model was never trained on your enterprise agreement, your clinical guidelines, or last Thursday's policy update, and you do not want it inventing them.

Why agencies reach for RAG specifically

Because the Australian Government's own technical guidance points there. The Digital Transformation Agency's Technical standard for government's use of artificial intelligence includes Criterion 70, which recommends that for generative AI, agencies "build reference or contextual datasets to improve the quality of AI outputs." Such a dataset, the standard says, "can be in the form of (and not limited to) a retrieval-augmented generation (RAG) dataset or a prompt dataset," with the objective of "more accurate and context aware AI outputs, and reduce hallucinations."

RAG is not a vendor fashion here. It is the pattern the national technical standard describes for grounding a generative system in your own material.

Where RAG sits in Australian AI obligations

A RAG assistant is an AI use case, and it lands inside an existing obligation stack.

Obligation	Source	What it means for a RAG build
Use case impact assessment	Policy for the responsible use of AI in government v2.0 (effective 15 December 2025)	Must commence at the design stage and be finalised, with agreed risk treatments applied, before deployment. Existing unassessed use cases must be dealt with by 30 April 2027.
Accountable use case owner and internal register	Policy v2.0, Strategy and oversight	Every in-scope use case needs a named owner. The register is shared with the DTA every 6 months.
Public AI transparency statement	Policy v2.0, Strategy and oversight	Your assistant forms part of what the agency publicly discloses about its AI use, reviewed at least annually.
End-to-end auditability and disclosure of AI interaction	AI technical standard, Criteria 9 and 24	Both required, not recommended. Users must be able to tell they are talking to an AI system, and the system must be auditable end to end.
Recordkeeping	National Archives of Australia advice (January 2025)	AI generated content created or received by Australian Government agencies constitutes a Commonwealth record under the Archives Act 1983.
Privacy	OAIC guidance on commercially available AI products	Privacy obligations apply to personal information input into an AI system, and to the output it generates where that output contains personal information.
Victorian protective markings	Administrative Guideline for Generative AI in the VPS	Information put into an agency-approved tool "must not exceed the protective marking determined by the in-scope organisation as appropriate for that tool".
Victorian procurement	Navigating AI in procurement	Complete the VPS AI Assurance Framework and include it with the procurement plan and approvals.

All of this sits under the National framework for the assurance of artificial intelligence in government, agreed by the Data and Digital Ministers Meeting on 21 June 2024 as a joint approach by the Australian, state and territory governments. For the Victorian layer, see the VPS AI Assurance Framework supplier guide.

Four failure modes specific to government

Generic RAG advice tells you to tune your chunk size. That is not what breaks these builds.

1. Stale source data

The demo indexed a folder once. Production has to keep answering correctly after the guideline is superseded, the form is replaced and the delegation changes. This failure has the sharpest consequences, because a RAG system fails confidently: it will happily cite a document that was withdrawn in March. The DTA's Agentic AI addendum, last updated 4 June 2026, is blunt about the prerequisite. Agencies "must not progress beyond early design or experimental stages unless data quality, governance, and security are confirmed and have been assessed as fit for the level of autonomy."

In Victoria the same exposure runs through information privacy. OVIC's guidance on enterprise Generative AI tools flags IPP 3 risks, covering poor data quality and accuracy, when these tools are used in decision-making, including earlier steps like research and summarisation that materially assist a decision.

The index needs an owner, a refresh schedule tied to the source system rather than a calendar reminder, and a retirement path so superseded documents leave the index when they leave the intranet. Freshness is a content operations problem wearing an engineering costume, which is why we treat it as part of AI knowledge management rather than a pipeline detail.

2. Permission leakage across classifications

A RAG index flattens a document library into a pile of text chunks. If retrieval does not enforce the same access controls as the source repository, a staff member can ask a question and receive an answer synthesised from material they were never cleared to read. No document was opened, no access log fired, and the leak arrives as fluent prose.

The National Archives states the principle plainly: where AI technologies access source data containing sensitive or privileged information, or generate outputs from that content, "the original sensitivity requirements will continue to apply unless the sensitivity is downgraded." OVIC similarly expects organisations to consider "applying appropriate protective markings for newly generated information assets," and the DTA's agentic addendum requires that "classifications and sovereignty controls are incorporated into agent-to-agent data exchange."

The engineering answer is that authorisation is evaluated at retrieval time against the requesting user, not applied as a filter after generation. Every chunk carries its source classification and permissions. If the user cannot read the document, the chunk never enters the prompt.

Filtering after generation means restricted content already entered the model context. Permission-aware retrieval excludes it before the prompt is assembled, and logs every retrieval against the requesting identity.

The ACSC-co-authored guidance on deploying AI systems securely sets the surrounding expectations: role-based or attribute-based access controls, input validation and sanitisation to reduce prompt injection risk, and logging covering "inputs, outputs, intermediate states, and errors". More in securing AI assistants in government.

3. Answers you cannot cite

An answer without a traceable source is not usable in a public sector context, because the officer who acts on it is accountable for it. The Victorian guideline is explicit that personnel "remain responsible and accountable for the accuracy and quality of their work," and warns that relying on generative AI content to support decision-making without verifying its accuracy could itself breach the VPS Code of Conduct. OVIC puts it more sharply for incidents: VPS organisations "will not be able to simply say that the incident or interference was caused by AI."

Citation is therefore not an interface flourish. It is how a human discharges their accountability. A citation has to resolve to a specific version of a specific document, not a domain name, and checking the source has to be trivial. The system also needs a refusal path: the technical standard requires agencies to "identify and address situations when AI outputs should not be provided" (Criterion 80). An assistant that answers everything is worse than one that admits it lacks the grounding to answer.

4. Records you did not realise you were creating

Every question and answer may be a record. The National Archives advice is direct: the Archives Act 1983 is technology and format neutral, and records created using AI "will be subject to the same requirements and obligations as other Commonwealth records created for similar business purposes with similar content and value."

The trap follows immediately. NAA notes that AI systems creating records "are not designed as recordkeeping systems and generally lack adequate records management functionality," so outputs that must be kept have to be captured into a system where they can be properly managed, with adequate metadata alongside them. Where outputs are relied on for decisions, the prompts, inputs and source content may need retaining too. Victorian agencies carry an equivalent obligation under the Public Records Act 1973, with Public Record Office Victoria's AI Technologies and Recordkeeping Policy as the operative guidance.

Design for this at the start; retrofitting after go-live is expensive and rarely done well.

A build checklist

Stage	What good looks like
Scoping	Impact assessment started at design, not after the pilot. Accountable use case owner named.
Corpus	An owner per document set. Superseded content has a removal path. Classification travels with every chunk.
Retrieval	Access control evaluated per user at query time against source permissions. No post-generation filtering.
Generation	Grounded answers only. Explicit refusal when retrieval is weak. Citations resolve to a document version.
Interface	Clear indication the user is interacting with an AI system. One click to the source. A visible way to report a bad answer.
Records	Retention decided per interaction type, with a capture path into the approved records system.
Operations	Logging of inputs, outputs and errors. Scheduled re-evaluation of answer quality. Contractual visibility of vendor model changes.
Residency	Documented data flows, including where inference happens and where embeddings are stored.

Buyers underestimate that last row. Where the model runs and where the vectors live are separate questions, and both have to survive a security assessment. See AI data sovereignty for Australian government.

Where we sit

Noice runs its AI workloads on AWS Bedrock in ap-southeast-2, Sydney, so inference stays onshore by default rather than by exception. We are a Magnolia Platinum Partner and a Progress reseller through BlueChip Infotech, so the retrieval layer and the content platform underneath it can be procured and supported locally. We are currently building the Knowledge Sharing Platform for the Victorian Collaborative Centre for Mental Health and Wellbeing, which is this exact problem in a sector where accuracy matters a great deal.

For the wider picture, including where iterative and agentic retrieval changes the risk profile, start with agentic RAG for Australian government. To talk about a specific corpus, that is what our AI for government practice does.