IAM Authorization and Access Control

What it is

Authorization decides what an authenticated identity may do. After the role is proven, policies determine which actions on which resources are allowed.

For data engineers the goal is least privilege: every job, user and service gets only the permissions it needs, on only the resources it needs.

How it works

Policy types

An explicit deny always wins. Without a deny, a request is allowed if some policy allows it and no boundary or SCP blocks it.

Within one account, an identity policy or a resource policy that allows is enough. Across accounts, both sides must allow.

Reading a policy

A policy is JSON with statements that list an effect, actions, resources and optional conditions.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:ListBucket"],
    "Resource": [
      "arn:aws:s3:::my-lake",
      "arn:aws:s3:::my-lake/curated/*"
    ]
  }]
}

Bucket level actions such as s3:ListBucket apply to the bucket ARN (Amazon Resource Name), while object actions such as s3:GetObject apply to the object ARN with /*. Mixing them up is a classic cause of access denied errors.

Least privilege in practice

Give a Glue job role read access to its source prefix and write access to its target prefix, not to every bucket. Prefer AWS managed policies for quick starts and customer managed policies for tight control. Use IAM Access Analyzer to find unused permissions and resources shared outside the account.

Conditions narrow access further, for example by source VPC endpoint, by requiring TLS with aws:SecureTransport, or by tag.

Role based and attribute based access

Role based access control (RBAC) gives permissions through roles for job functions, such as analyst or engineer. Attribute based access control (ABAC) uses tags on principals and resources, so a policy says "access resources whose tag matches the user's tag".

ABAC scales well because new resources only need the right tag, not a new policy. Lake Formation tags apply the same idea to catalog resources.

S3 access controls

Modern designs avoid ACLs and rely on policies.

Database and service level authorization

Redshift uses SQL GRANT and REVOKE on schemas and tables, roles, and optionally row level security policies. Athena access depends on IAM permissions for Athena, the Glue Data Catalog and S3, and workgroups can restrict who runs queries and where results go. DynamoDB supports fine grained access using condition keys such as the leading key, which restricts a user to items with their own key.

GRANT SELECT ON TABLE sales.orders TO ROLE analyst;

Cross account access

To share a bucket with another account, add a bucket policy that names the principal, or use an access point. For catalog and table sharing at scale, Lake Formation provides cross account grants. Use ExternalId when a third party assumes a role in your account, which guards against the confused deputy problem.

Auditing permissions

IAM Access Analyzer reports external access and policy problems. IAM policy simulator tests what a policy allows. CloudTrail records which calls were made, and "last accessed" information shows which permissions are unused, so you can remove them.