Implement S3 Lifecycle Policies for Log Data
Specification & Description
Configure S3 lifecycle policies on the central logging buckets in the Log Archive account to automatically transition log data through cheaper storage classes over time and enforce retention-based expiration.
This control applies to all log data centralised in the Log Archive account. The sources, retention requirements, and bucket structure for each log type are defined in the following controls:
- [AWS-SEC-XXX] Enable AWS CloudTrail at the Organization Level - defines CloudTrail log delivery configuration, bucket naming, and minimum retention requirements.
- [AWS-NET-XXX] Enable VPC Flow Logs - defines Flow Log delivery targets, format, and account-level activation scope.
- [AWS-SEC-XXX] Enable AWS Config Recording - defines Config snapshot and history delivery to the Log Archive bucket.
Lifecycle policies defined in this control must be applied to the buckets referenced in those controls and must not reduce retention below the minimums they specify.
Technical Specifications
Log data delivered to the Log Archive account accumulates continuously from all Landing Zone accounts. Without lifecycle management, standard S3 storage costs compound linearly with organisational growth. This control defines storage class transition schedules and expiration rules per log type, balancing access frequency patterns against cost and compliance obligations.
Recommended transition schedule by log type
| Log Type | Standard → Intelligent-Tiering | → Glacier IR | → Glacier Deep Archive | Expiration |
|---|---|---|---|---|
| CloudTrail | 30 days | 90 days | 365 days | 3650 days (10 years) |
| VPC Flow Logs | 30 days | 60 days | 180 days | 365 days |
| ALB / WAF Access Logs | 14 days | 60 days | 180 days | 365 days |
Retention minimums for CloudTrail and Config are driven by compliance standards. Adjust expiration only after confirming with your compliance team.
S3 Intelligent-Tiering is preferred over a direct Standard → Glacier transition for the first 30–90 day window because access patterns for recent logs are non-uniform - incident response queries frequently target logs from the past 30 days.
Security Rationale
Log storage is a significant and predictable cost driver in Landing Zone architectures. Lifecycle policies are the lowest-effort, highest-return cost optimisation available for the Log Archive account. Establishing them at Landing Zone build time prevents cost accumulation that becomes difficult to remediate retroactively once bucket objects number in the billions.