28 of 50 in All Controls
aws

Selective CloudWatch Logs Aggregation with Cost Guardrails

IDAWS-LOG-006
Severity
medium
Automation
Auto
Category
logging
Enforcementrecommended
Complexity / Impact
medium(low impact)
Risk Profile
medium likelihood(low residual)
Cloud Scopeaws:account
Environments
productionnon-production
Target Resources
AWS::Logs::DestinationAWS::Logs::SubscriptionFilterAWS::Logs::LogGroupAWS::KinesisFirehose::DeliveryStream
Services
cloudwatch-logskinesis-firehoses3kmsiam
Tags
loggingcloudwatchcentralizationcross-accountapplication-logscontainer-logscost-optimizationlanding-zone
Frameworks
NIST 800-53ISO 27001SOC 2PCI DSS
Governance
Cloud Security & FinOps (quarterly)
Created
2026-05-03
Last Updated
2026-08-22

Specification & Description

Selectively aggregate security-relevant, compliance-mandated, and audit-grade CloudWatch Log Groups from workload accounts into the central Log Archive account using CloudWatch Logs cross-account destinations, Amazon Data Firehose, and durable S3 cold storage. Explicitly filter out high-volume ephemeral debug, trace, and synthetic health-check streams to prevent severe CloudWatch ingestion and data transfer cost escalation.

Technical Specifications

While AWS CloudTrail captures management and data API events, workload environments (EC2, ECS, EKS, Lambda) generate critical application audit trails, authentication events, and operating system security logs within CloudWatch Logs. However, streaming all application logs to a central account is a costly anti-pattern that creates signal fatigue and runaway bills.

1. Selective Log Group Scope & Security Classification

  • In-Scope for Centralization: Authentication/authorization events, administrative audit logs, financial transaction records, operating system audit streams (/var/log/secure, /var/log/audit/audit.log), and production container error streams.
  • Out-of-Scope (Local Only): High-frequency debug/trace logs, ephemeral stdout container noise, synthetic ALB/NLB health-check probes, and non-production sandbox streams. These should remain in local accounts with short retention windows.

2. Cost Dimensions & Streaming Economics

  • Cross-account log aggregation incurs multiple distinct cost layers:
    • CloudWatch Logs Ingestion: ~$0.50 per GB ingested in member accounts.
    • Cross-Account Data Transfer: Inter-account and cross-region egress network fees.
    • Amazon Data Firehose Processing: ~$0.029 per GB ingested and delivered.
    • S3 Storage & PUT Requests: Per-request and ongoing storage fees in the central Log Archive bucket.
  • Subscription Filter Patterns: Enforce strict filter_pattern criteria (e.g., [time, level = ERROR || level = WARN || level = AUDIT, ...]) on subscription filters to drop benign debug lines at the source before transmission.

3. Retention Hygiene & Storage Optimization

  • Member account CloudWatch Log Groups must enforce strict local retention periods (e.g., 7 to 14 days) to eliminate local storage fees ($0.03/GB/month).
  • Long-term compliance retention is offloaded entirely to S3 Lifecycle policies (transitioning from S3 Standard to S3 Glacier Instant Retrieval / Deep Archive) in the central Log Archive account.

Security Rationale

Decentralized logging scattered across individual AWS accounts creates security blind spots during incident response. However, indiscriminately streaming all workload and debug logs across an enterprise leads to runaway CloudWatch and Data Firehose ingestion costs. Applying selective subscription filters balances forensic visibility and SIEM correlation with strict cost governance.

Business & Security Impact

Failing to centralize security logs delays incident response and risks permanent evidence loss if accounts are decommissioned. Conversely, failing to filter out high-volume debug and health-check logs results in massive, unbudgeted CloudWatch ingestion, data transfer, and Firehose processing charges.

Compliance Framework Mappings (4)

Regulatory baselines and industry standards mapped to this control specification.

Beta
Framework / StandardMapped Requirements & ControlsControls
NIST 800-53 Rev 5NIST 800-53
AU-2AU-4AU-12
3
ISO 27001:2022ISO 27001
A.8.15
1
SOC 2SOC 2
CC7.2
1
PCI DSS v4.0PCI DSS
10.2.1
1

* Compliance framework mappings are provided for informational architectural reference. Always verify control alignment against your organization's formal audit requirements.

Automated Detection & Rule Logic

Detection Engine: aws-api • Automated: Yes

Audits member account CloudWatch Log Groups matching security and production audit patterns (e.g., /aws/containerinsights/production/, /var/log/secure, /app/audit/) to verify they have an active cross-account subscription filter and short local retention (<= 30 days) configured.

AWS CLI: Check Subscription Filters on Log Groupbash
aws logs describe-subscription-filters \
  --log-group-name "/aws/containerinsights/production/application" \
  --output table
Python (Boto3): Audit Selective Subscriptions and Retention Hygienepython
import json
import boto3

logs = boto3.client('logs')

# Target security and production audit prefixes (avoiding noisy debug/sandbox streams)
SECURITY_AUDIT_PREFIXES = (
    '/aws/containerinsights/production',
    '/var/log/secure',
    '/var/log/audit',
    '/aws/eks/prod',
    '/app/audit'
)

def audit_log_groups(destination_arn='arn:aws:logs:us-east-1:LOG_ARCHIVE_ACCOUNT_ID:destination:CentralAppLogDestination'):
    paginator = logs.get_paginator('describe_log_groups')
    findings = []

    for page in paginator.paginate():
        for group in page.get('logGroups', []):
            group_name = group['logGroupName']
            retention = group.get('retentionInDays')
            
            # Only audit security-critical log groups for centralization
            is_security_group = any(group_name.startswith(p) for p in SECURITY_AUDIT_PREFIXES)
            
            if is_security_group:
                subs = logs.describe_subscription_filters(logGroupName=group_name).get('subscriptionFilters', [])
                has_central_dest = any(s.get('destinationArn') == destination_arn for s in subs)
                
                issues = []
                if not has_central_dest:
                    issues.append("MISSING_CENTRAL_SUBSCRIPTION")
                if retention is None or retention > 30:
                    issues.append(f"UNOPTIMIZED_RETENTION ({retention or 'Never Expire'} days)")
                    
                if issues:
                    findings.append({"log_group": group_name, "issues": issues})

    if findings:
        return {"status": "NON_COMPLIANT", "unforwarded_or_unretentioned_groups": findings}
    return {"status": "COMPLIANT", "message": "All security log groups forwarded with cost-optimized retention"}

if __name__ == "__main__":
    result = audit_log_groups()
    print(json.dumps(result, indent=2))

Preventative Enforcement Policies

CloudWatch Logs Destination Access Policy in Log Archive Accountjson
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowOrgCrossAccountLogSubscriptions",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "logs:PutSubscriptionFilter",
      "Resource": "arn:aws:logs:us-east-1:LOG_ARCHIVE_ACCOUNT_ID:destination:CentralAppLogDestination",
      "Condition": {
        "StringEquals": {
          "aws:PrincipalOrgID": "o-xxxxxxxxxx"
        }
      }
    }
  ]
}
Terraform: Central Destination & Filtered Member Subscriptionhcl
# 1. Central Destination in Log Archive Account
resource "aws_cloudwatch_log_destination" "central_destination" {
  name       = "CentralAppLogDestination"
  role_arn   = aws_iam_role.cloudwatch_to_firehose.arn
  target_arn = aws_kinesis_firehose_delivery_stream.log_archive_stream.arn
}

resource "aws_cloudwatch_log_destination_policy" "central_destination_policy" {
  destination_name = aws_cloudwatch_log_destination.central_destination.name
  access_policy    = data.aws_iam_policy_document.destination_access.json
}

# 2. Member Account Selective Subscription Filter with Pattern Guardrails
resource "aws_cloudwatch_log_subscription_filter" "security_audit_filter" {
  name            = "stream-security-audit-logs"
  log_group_name  = "/aws/eks/prod-cluster/application"
  # Filter pattern only forwards security, audit, warning, and error events to control costs
  filter_pattern  = "[timestamp, request_id, level = *ERROR* || level = *WARN* || level = *AUDIT*, message]"
  destination_arn = "arn:aws:logs:us-east-1:${var.log_archive_account_id}:destination:CentralAppLogDestination"
}

# 3. Enforce 14-day local retention to prevent local storage cost bloat
resource "aws_cloudwatch_log_group" "app_logs" {
  name              = "/aws/eks/prod-cluster/application"
  retention_in_days = 14
}

Remediation Guide & Automation

Provision a centralized CloudWatch Logs Destination in the Log Archive account with Amazon Data Firehose, and attach selective cross-account subscription filters with filter patterns to security-critical workload log groups.

Step-by-Step Instructions

  1. Classify application and system log groups to identify security-critical audit and error streams.
  2. Provision the central S3 log bucket (with Glacier lifecycle transitions) and Amazon Data Firehose stream in the Log Archive account.
  3. Create the CloudWatch Logs Destination with a resource policy allowing `aws:PrincipalOrgID` access.
  4. Deploy subscription filters with specific `filter_pattern` rules targeting only audit and error events to avoid streaming debug noise.
  5. Set explicit retention periods (7 to 14 days) on member account log groups to eliminate local storage cost accumulation.
AWS CLI: Create Central Destination and Filtered Subscriptionbash
# 1. Create central destination in Log Archive account
aws logs put-destination \
  --destination-name "CentralAppLogDestination" \
  --target-arn "arn:aws:firehose:us-east-1:LOG_ARCHIVE_ACCOUNT_ID:deliverystream/central-logs-stream" \
  --role-arn "arn:aws:iam::LOG_ARCHIVE_ACCOUNT_ID:role/CWLtoFirehoseRole"

# 2. Put selective subscription filter with pattern in member account
aws logs put-subscription-filter \
  --log-group-name "/aws/containerinsights/production/application" \
  --filter-name "CentralSecurityArchiveSubscription" \
  --filter-pattern "[timestamp, level = *ERROR* || level = *AUDIT*, ...]" \
  --destination-arn "arn:aws:logs:us-east-1:LOG_ARCHIVE_ACCOUNT_ID:destination:CentralAppLogDestination"

# 3. Set short retention on local member account log group
aws logs put-retention-policy \
  --log-group-name "/aws/containerinsights/production/application" \
  --retention-in-days 14

We value your privacy

We use analytics cookies to understand how visitors interact with our site and to improve the user experience. You can choose to accept or decline these cookies.