AWS Learning Journey

Who Can See the Salary Column? Data Governance With Lake Formation and Macie

Lake Formation permissions down to databases, tables, columns, and rows, LF-Tags, data sharing, IAM integration, KMS encryption, Amazon Macie and PII detection, masking, retention, audit logging, lineage, and quality controls.

9 min read · 1699 words
Data GovernanceAWSData EngineeringData PrivacyCloud Security
Who Can See the Salary Column? Data Governance With Lake Formation and Macie cover image

AWS from Zero to Production — Phase 9, Part 10 of 11 · See the full series roadmap


Your data lake is a success. Ten teams use it. Then three questions arrive in the same week:

If your answers are “probably,” “we don’t know,” and “we’d have to search every bucket,” your lake has a governance problem.

Governance isn’t about locking data away. Done well, it lets more people use data safely — because access is precise, sensitive data is known and protected, and everything is auditable. This article shows how to do it on AWS.

What you’ll walk away with


The problem with S3 permissions alone

With IAM and bucket policies only, access is defined on S3 paths:

Allow s3:GetObject on arn:aws:s3:::acme-lake-curated/hr/employees/*

That’s all or nothing per file. You can’t say “everything except the salary column,” or “only rows for your own region.” And every engine — Athena, Redshift Spectrum, EMR, Glue — needs separately managed permissions.

AWS Lake Formation moves permissions up to the level people think in: databases, tables, columns, and rows — enforced consistently across integrated engines.


AWS Lake Formation

Lake Formation builds governance on top of the Glue Data Catalog and S3:

  1. Register S3 locations with Lake Formation.
  2. Grant permissions on catalog resources — databases, tables, columns — to IAM principals, IAM Identity Center users and groups, or other accounts.
  3. Integrated engines — Athena, Redshift Spectrum, EMR, Glue, and others — ask Lake Formation what the caller can see, and Lake Formation vends temporary credentials scoped to exactly that data.
aws lakeformation grant-permissions \
  --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:role/analyst \
  --permissions SELECT \
  --resource '{
    "TableWithColumns": {
      "DatabaseName": "hr_curated",
      "Name": "employees",
      "ColumnNames": ["employee_id", "department", "hire_date"]
    }
  }'

The analyst can query employees — but only those three columns. salary doesn’t exist for them.

Permission levels

Row-level and cell-level security

A data filter restricts rows — and optionally columns:

aws lakeformation create-data-cells-filter --table-data '{
  "TableCatalogId": "111122223333",
  "DatabaseName": "sales_curated",
  "TableName": "orders",
  "Name": "emea_only",
  "RowFilter": { "FilterExpression": "region = '\''EMEA'\''" },
  "ColumnWildcard": { "ExcludedColumnNames": ["customer_email"] }
}'

Grant SELECT on the filter, and the EMEA team sees only EMEA orders, without customer emails.

Hybrid access mode

Migrating an existing lake? Hybrid access mode lets you turn on Lake Formation permissions gradually, table by table and principal by principal, while existing IAM-based access keeps working.


LF-Tags: permissions at scale

Granting per table doesn’t scale to thousands of tables. LF-Tags let you grant by attributes:

Tags on resources:
  table hr_curated.employees            domain=hr,    sensitivity=confidential
  column employees.salary               sensitivity=restricted
  table sales_curated.orders            domain=sales, sensitivity=internal

Grants on tags:
  analysts      -> SELECT where sensitivity IN (public, internal)
  hr_analysts   -> SELECT where domain=hr AND sensitivity IN (internal, confidential)
  payroll       -> SELECT where domain=hr (all sensitivities)

New tables automatically become visible to the right people as soon as they’re tagged — this is tag-based access control (TBAC), the Lake Formation version of ABAC from Phase 2.


Data sharing

Lake Formation shares data across accounts without copying it:

This is the foundation of a data mesh: domain teams own and publish data products from their own accounts; consumers get governed, discoverable access.

For browsing, requesting, and approving access across an organization, Amazon DataZone adds a business data catalog and subscription workflow on top.


IAM integration

Lake Formation doesn’t replace IAM — they work together:

Watch out: If principals also have broad direct S3 permissions to registered locations, they may be able to bypass Lake Formation by reading files directly. Remove direct S3 access for users who should go through Lake Formation.


KMS encryption

From Phase 2 and Part 2:


Amazon Macie: finding sensitive data

You can’t protect what you don’t know you have. Amazon Macie discovers sensitive data in S3:

Typical response: when Macie finds credit card numbers in a raw bucket that shouldn’t contain them, an EventBridge rule notifies the data owner — or triggers quarantine automation.


PII detection inside pipelines

Catch sensitive data before it spreads:


Data masking

Not everyone needs raw values:


Data retention

Keep data as long as it’s needed and allowed — then delete it:


Audit logging

Answer “who accessed what, and when?”:


Data lineage

Lineage tracks how data moves and transforms:

orders (RDS) -> DMS -> raw.orders -> Glue job -> curated.orders -> Redshift.fact_orders -> revenue dashboard

On AWS:


Data quality controls

Governance includes trust:


A practical governance setup

Catalog:      Glue Data Catalog, one database per domain and zone
Permissions:  Lake Formation with LF-Tags (domain, sensitivity)
Sharing:      cross-account grants to consumer accounts
Encryption:   SSE-KMS, separate keys for restricted domains
Discovery:    Macie automated sensitive data discovery on all lake buckets
Pipelines:    PII detection and masking between raw and curated
Retention:    lifecycle rules per dataset; Iceberg for erasure requests
Audit:        CloudTrail org trail + Athena/Redshift logs in a log archive account
Quality:      Glue Data Quality gates; owners alerted on failures

Common mistakes


Key takeaways


Quick self-check

Test yourself before moving on:

  1. How do you let analysts query a table without seeing the salary column?
  2. What’s the scalable alternative to granting permissions table by table?
  3. Which service scans S3 for credit card numbers and other PII?
  4. Why can broad S3 permissions undermine Lake Formation?

Answers

  1. Grant column-level SELECT in Lake Formation that excludes salary — or a data filter.
  2. LF-Tags — tag-based access control.
  3. Amazon Macie.
  4. Principals could read files directly from S3, bypassing Lake Formation’s checks.

What’s next

In Part 11, the Phase 9 capstone lab: build the end-to-end data platform — the orders database replicated by DMS into S3, transformed by Glue into curated Parquet, queried by Athena and Redshift, orchestrated by Step Functions, and monitored with CloudWatch and SNS.

How does your organization decide who gets access to sensitive data? Tell me in the comments.


Share this article

LinkedIn post draft