Security, Compliance and Governance · Topic 2 of 5
Secure Data Engineering and Lineage
Apply secure data engineering for AI: data quality, privacy enhancing techniques, access control, integrity, data lineage and source citation.
- 4 min
- Data lineage tracks the path and changes of data from source to use.
- A data catalog is an inventory of datasets, and AWS Glue Data Catalog holds dataset metadata.
- Source citation shows which sources support an AI answer.
What it is
AI quality and safety depend on the data it uses. Secure data engineering means preparing, storing and moving data in ways that keep it accurate, private and traceable.
Two ideas are central: knowing where your data came from, and protecting it while you use it.
How it works
Documenting data origins
You should be able to say where each piece of data came from, what happened to it, and who may use it.
| Concept | Meaning |
|---|---|
| Data origin (provenance) | The source of the data and the rights attached to it |
| Data lineage | The path of data from source through each change to its final use |
| Data catalog | An organised inventory of datasets with descriptions and owners |
| Source citation | Showing which sources an AI answer is based on |
| Model documentation | Records of the data and settings behind a model, such as SageMaker Model Cards |
AWS Glue Data Catalog is a place to hold metadata about datasets. Lineage helps when something goes wrong: if bad data was loaded, you can see which models and reports used it. It also helps when a source owner withdraws permission.
Source citation in GenAI
Source citation means the application shows where an answer came from.
With retrieval augmented generation (RAG), the system can point to the documents it retrieved. Citations let users verify answers, and they support trust and audits. Amazon Bedrock Knowledge Bases can return source references with its answers.
Assessing data quality
Poor data makes poor and sometimes unsafe models. Check these points.
- Accuracy: values and labels are correct.
- Completeness: important fields are not missing.
- Consistency: formats and meanings match across sources.
- Timeliness: data is up to date for the use.
- Representativeness: it covers the real cases and groups.
- Uniqueness: duplicates are removed.
Privacy enhancing technologies
Privacy enhancing technologies reduce the exposure of personal data while keeping it useful.
| Technique | Idea |
|---|---|
| Masking | Hide parts of a value, such as showing only the last digits |
| Anonymisation | Remove or alter details so a person cannot be identified |
| Pseudonymisation | Replace identifiers with substitute values kept separately |
| Encryption | Make data unreadable without a key |
| Data minimisation | Collect and keep only what is needed |
| Differential privacy | Add controlled noise so individual records cannot be singled out |
Amazon Macie can find sensitive data in Amazon S3, and Amazon Bedrock Guardrails can mask or block personal information in prompts and answers.
Data access control
Limit who can read, change or share data.
- Use IAM policies and roles with least privilege.
- Separate sensitive data into its own buckets and accounts.
- Control access by role or purpose, so only approved users and applications reach training and retrieval data.
- For RAG, respect user permissions so people only see content they may read.
- Review access regularly and remove what is no longer needed.
Data integrity
Integrity means data is not altered by accident or by an attacker. Keep versions, use checksums or validation, restrict write access, and log changes. This also defends against data poisoning, where someone slips bad records into a training or retrieval source to change a model's behaviour.
Secure pipelines
Treat the data pipeline as part of the attack surface. Encrypt data at rest and in transit, validate incoming data, keep an audit trail, and test the pipeline for failures.
What to remember for the exam
- Data lineage tracks the path and changes of data from source to use.
- A data catalog is an inventory of datasets, and AWS Glue Data Catalog holds dataset metadata.
- Source citation shows which sources support an AI answer.
- Data quality includes accuracy, completeness, consistency, timeliness and representativeness.
- Privacy enhancing techniques include masking, anonymisation, pseudonymisation, encryption and differential privacy.
- Access control uses least privilege and respects user permissions in RAG.
- Data integrity protects against accidental change and poisoning.
- Model Cards record the data and purpose behind a model.
A question asking how to prove where training data came from points to lineage and cataloguing. A question asking how to show users the basis for an answer points to source citation.
Common mistakes
Treating lineage and cataloguing as the same. A catalog lists datasets. Lineage traces how data moved and changed.
Assuming anonymised data can never be traced back. Weak anonymisation can be reversed by combining data.
Collecting extra data just in case. Data minimisation reduces risk.
Ignoring permissions in RAG. A shared index may expose content to the wrong user.
Forgetting poisoning. Unchecked data sources can change model behaviour.
Check yourself
What does data lineage show?
The path of data from its source through each change to its final use.
What is source citation in a GenAI application?
Showing which source documents support an answer, so users can verify it.
Name two privacy enhancing techniques.
Examples are masking, anonymisation, pseudonymisation, encryption and differential privacy.
Topic 2 of 5 in Security, Compliance and Governance
Test yourself on Security, Compliance and Governance
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.