Lake Formation Permissions

What it is

AWS Lake Formation adds central, fine grained access control to a data lake. Instead of writing S3 and IAM policies for every team, you grant permissions on catalog resources: databases, tables, columns and rows.

Engines that integrate with Lake Formation, such as Athena, Redshift Spectrum, EMR and Glue, check those grants when they read the data.

How it works

Registering locations

You register an S3 location with Lake Formation, and it then manages access to the data under that path using a service role. After registration, engines receive temporary, narrowly scoped credentials from Lake Formation, so users do not need direct S3 permissions on the data.

Grants on catalog resources

Lake Formation uses a GRANT and REVOKE model similar to a relational database. You grant permissions to IAM users, roles, groups and, through IAM Identity Center, to directory users.

A data filter defines which columns and which rows a principal may see. This is the Lake Formation answer to "analysts in each region should see only their region's rows, without the national ID column".

aws lakeformation grant-permissions \
  --principal DataLakePrincipalIdentifier=arn:aws:iam::123456789012:role/analyst \
  --permissions SELECT \
  --resource '{"Table":{"DatabaseName":"sales","Name":"orders"}}'

LF tags and tag based access control

Lake Formation tags (LF tags) are key value pairs attached to databases, tables and columns, such as sensitivity=high. You grant a principal access to all resources that carry certain tag values.

This scales better than naming tables one by one. A new table with the right tag is covered by existing grants, which cuts administration for large catalogs.

How IAM and Lake Formation combine

Both layers apply. The caller needs IAM permission to call the service API, for example Athena and Glue actions, and Lake Formation must grant access to the data resource.

Lake Formation also has a hybrid access mode, so you can adopt it for selected databases and tables while existing IAM and S3 based access continues for others. A common pitfall is leftover IAM permissions that let a user bypass Lake Formation, because S3 access granted by IAM still works for tools that read S3 directly.

Cross account sharing

You can share databases and tables with another account or an organization.

The consumer account receives a resource link in its own catalog and can grant its own users access. Sharing works by granting to the account, with AWS Resource Access Manager (RAM) behind the scenes. The consumer's data lake administrator can then grant its own users access.

Lake Formation and other services

Lake Formation can also ingest data through blueprints and governs the Glue Data Catalog in the same account. It records permission changes and data access in CloudTrail, which supports audits.

Redshift data sharing and governance

Redshift data sharing controls which schemas and tables a consumer cluster can read. Redshift also has its own row level security and dynamic data masking policies. Pick the mechanism that matches where the data lives: Lake Formation for lake tables, Redshift native features for warehouse tables.

Operational tips

Start with a small number of admin roles, use LF tags for scale, and test with a restricted user before rollout. Remove broad grants to the special group named IAMAllowedPrincipals when you want Lake Formation to be the only gatekeeper.