Basic AI Concepts Related to Cybersecurity · Topic 3 of 4
Data Security for AI, RAG and Embeddings
Why data security matters for AI: cleansing, verification, lineage, provenance, integrity, balancing, data types, watermarking, RAG and vector storage.
- 6 min
- Includes a diagram
- Provenance is the origin and trust of data; lineage is its path and transformations.
- Cleansing removes errors, verification confirms data is genuine, and integrity controls detect changes.
- Augmentation adds samples; balancing fixes the class mix.
What it is
An artificial intelligence (AI) model learns its behaviour from data, so whoever controls the data controls the model. Bad, biased or tampered data produces a bad, biased or compromised model, even when the code is perfect.
Data security for AI therefore covers more than confidentiality. It also covers quality (is the data correct?), origin (where did it come from?) and integrity (has it changed?), for training data and for the documents a model reads at run time.
How it works
Data processing steps
| Step | What it means | Why it matters for security |
|---|---|---|
| Data cleansing | Removing duplicates, errors, corrupt records and irrelevant fields | Removes junk that skews results and can strip planted records |
| Data verification | Checking data is accurate, complete and from the expected source | Catches tampered or fake data before training |
| Data lineage | The record of how data moved and was transformed through pipelines | Lets you trace a bad model output back to the step that introduced it |
| Data provenance | The origin and ownership history of the data: who created it, when and under what terms | Proves the data is genuine and licensed for use |
| Data integrity | Assurance that data has not been altered without authorisation | Hashes, signatures and access controls detect poisoning |
| Data augmentation | Creating extra training samples by transforming existing ones or generating synthetic ones | Improves robustness, but synthetic data must also be checked |
| Data balancing | Adjusting the mix of classes, for example by oversampling rare cases or undersampling common ones | Stops a model ignoring rare but important events such as fraud |
Lineage and provenance are easy to confuse. Provenance answers "where did this come from and can I trust it?", while lineage answers "what happened to it on the way here?".
Why balancing matters
Security data is usually unbalanced. In a log set, malicious events may be a tiny fraction of the total.
A model trained on that set can score high accuracy by labelling everything benign, which is useless. Balancing, or weighting the rare class, forces the model to learn what an attack looks like.
Data types
| Type | Description | Examples |
|---|---|---|
| Structured | Fixed fields in rows and columns | Database tables, firewall logs in a fixed schema |
| Semi-structured | Tagged or keyed data without a rigid table schema | JavaScript Object Notation (JSON), Extensible Markup Language (XML), syslog with key value pairs |
| Unstructured | No predefined model | Emails, Portable Document Format (PDF) reports, images, audio, chat transcripts |
Unstructured data is the hardest to secure. Sensitive details such as names or account numbers hide inside free text, so they must be found by scanning before the data is used for training or retrieval.
Watermarking
Watermarking embeds a hidden, detectable signal in content or a model. For AI it has two main uses: marking AI-generated output so it can later be recognised as synthetic, and marking data sets or models so theft or unlicensed use can be proven.
Watermarks help with provenance and deepfake detection. They are not perfect, because editing, cropping or re-encoding content can weaken some of them.
Retrieval-augmented generation (RAG)
RAG lets a large language model (LLM) answer from an organisation's own documents without retraining. Documents are split into chunks, each chunk is turned into an embedding, and the embeddings are kept in a vector store.
An embedding is a list of numbers that represents the meaning of a piece of text, so similar meanings sit close together. At question time the query is also embedded, the closest chunks are retrieved, and they are added to the prompt.
Securing RAG and vector storage
- Permission-aware retrieval: filter chunks by the user's own access rights, or the model will happily summarise documents the user could never open.
- Ingestion checks: scan and approve documents before indexing, because a planted document can carry hidden instructions or false facts.
- Protect the vector store: encrypt it and restrict access. Embeddings are derived from the source text and can leak information about it, so treat them as being as sensitive as the documents.
- Keep provenance metadata: store the source and owner with each chunk so answers can cite and be traced.
A worked example
A help desk assistant answers staff questions from an internal wiki using RAG. A user asks about salary bands and receives figures from a human resources (HR) page they have no right to see.
The cause is that retrieval ignored document permissions. The fix is to store each chunk with its access labels, filter results by the asking user's identity before they reach the prompt, and remove HR pages from the general index.
How the exam asks about this
- "A team must show auditors where each training record originated and under which licence. Which of the following should they maintain?" Data provenance records.
- "A fraud model labels nearly every transaction as legitimate because fraud is rare in the training set. Which of the following would best address this?" Data balancing, such as oversampling the fraud class.
- "A RAG chatbot reveals contents of restricted documents to unauthorised users. Which of the following is the best control?" Permission-aware retrieval that filters chunks by the user's access rights.
What to remember for the exam
- Provenance is the origin and trust of data; lineage is its path and transformations.
- Cleansing removes errors, verification confirms data is genuine, and integrity controls detect changes.
- Augmentation adds samples; balancing fixes the class mix.
- Structured data has fixed fields, semi-structured data has tags or keys, and unstructured data has neither.
- RAG retrieves chunks by embedding similarity from a vector store and adds them to the prompt.
A RAG system inherits the access level of its index, not of the user. If retrieval does not check the user's permissions, everyone can read everything the index holds.
Common mistakes
Treating embeddings as harmless numbers. Embeddings encode the meaning of the source text and can leak it. Protect the vector store like the original documents.
Using "lineage" when the question is about origin. A question about where data came from, who made it or whether it is authentic points to provenance.
Assuming RAG removes the need for data controls. RAG avoids retraining, but the retrieved documents still need classification, ingestion checks and access filtering.
Check yourself
What is an embedding?
A list of numbers that represents the meaning of text or other content, so that items with similar meaning are close together.
Why is data balancing important for security models?
Attacks are usually rare in training data, and without balancing a model can ignore them while still looking accurate.
What does watermarking add for AI-generated content?
A hidden, detectable signal that helps show the content is synthetic and supports provenance and deepfake detection.
Topic 3 of 4 in Basic AI Concepts Related to Cybersecurity
Test yourself on Basic AI Concepts Related to Cybersecurity
The full CompTIA SecAI+ CY0-001 question bank has 654 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by CompTIA. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.