Data Masking and Anonymization

What it is

Encryption protects data from people who should not see it at all. Masking and anonymization protect fields that some people may use but should not read in full, such as a card number or a national ID.

The goal is to let analysts, developers and partners work with useful data while the sensitive values stay hidden or unlinkable to a person.

How it works

Core terms

Hashing without a salt can be attacked by guessing common values, such as email addresses. A salt is extra data added before hashing so lookup tables of known hashes do not work.

Static and dynamic masking

Static masking changes the stored data, for instance when you create a masked copy for a test environment. Dynamic masking leaves the stored data intact and changes what a query returns depending on who asks.

Amazon Redshift dynamic data masking

Redshift dynamic data masking lets you define a masking policy in SQL and attach it to a column for a user or role. Users and roles that the policy is attached to see the masked value, and a priority setting decides which policy wins when several apply.

CREATE MASKING POLICY mask_card
WITH (card_number VARCHAR(19))
USING ('XXXX-XXXX-XXXX-0000'::VARCHAR(19));

ATTACH MASKING POLICY mask_card
ON payments(card_number)
TO ROLE analyst;

The policy uses SQL expressions, so it can show the last four characters instead of a constant. Redshift also has row level security for hiding entire rows.

Lake Formation filters and views

Lake Formation data filters exclude sensitive columns or rows for a principal, as the earlier page explains. This is the right tool for a data lake. A SQL view that omits or transforms columns offers a simple alternative in Athena, but remember that users need access to the view, not the base table, for the protection to work.

SELECT customer_id,
       to_hex(sha256(to_utf8(email))) AS email_hash,
       substr(postcode, 1, 3) AS postcode_area
FROM customers;

This Athena query hashes the email and generalizes the postcode. Hashing keeps the field usable for joins and counting distinct people, since the same email always gives the same hash.

Detecting sensitive data

You cannot protect what you cannot find. Amazon Macie scans S3 buckets for sensitive data such as names, financial details and credentials.

AWS Glue has a sensitive data detection transform that finds and can mask or hash personal data inside an ETL job. Glue DataBrew offers PII transformations that substitute, hash, shuffle, encrypt or null out values.

Anonymization methods

Combining several quasi identifiers, such as birth date, postcode and gender, can still identify a person, so generalize them together. Test the result for re identification risk before sharing.

Where to apply protection

Apply masking at the layer closest to the consumer. Keep the raw zone encrypted and restricted, create masked or hashed copies in the curated zone for broad use, and use dynamic policies where one table serves users with different rights.

Combining controls

Masking does not replace encryption or access control. Use all three: encrypt data at rest and in transit, restrict who can reach raw data, and mask what each audience sees.