Most cloud projects look clean in a diagram and messy in production.

That was the point of AuditTrail SDK, a DSB capstone project I built to automate a basic AWS compliance task the right way: use the official SDK, authenticate with temporary credentials only, handle throttling, log every API call in structured form, provision the stack with Terraform, and validate the API with Postman. GitHub Actions OIDC lets AWS access happen without long-lived GitHub secrets, and Postman supports AWS Signature auth for signed requests.

The project in one sentence

AuditTrail SDK is a Terraform-provisioned AWS compliance auditor that inventories compute and storage resources with boto3, stores findings, exposes audit endpoints, and creates enough logging and cloud-side visibility to let you audit the auditor. Boto3 supports configurable retry behavior, and CloudTrail provides control-plane visibility for AWS API activity.

What it does

The platform checks resources such as EC2, EBS, S3, and Lambda, then runs basic compliance checks, including required-tag validation, EBS encryption checks, and S3 public access checks. The results are stored and exposed via an API, allowing the system to be validated operationally rather than just by reading logs after the fact. For S3 public access checks, AWS documents that the bucket-level public access block configuration is retrieved via GetPublicAccessBlock, which requires the s3:GetBucketPublicAccessBlock permission.

Why this project is stronger than a script demo

A throwaway script can list resources. That proves almost nothing.

A real cloud security project has to answer harder questions:

• Can it authenticate without static keys?

• Can it survive throttling?

• Can it be reproduced with infrastructure as code?

• Can it be tested like a service?

• Can it explain what happened when something fails?

This project was built to answer yes to all five. GitHub’s AWS OIDC flow exists specifically so that workflows can access AWS without storing long-lived credentials, making it the right default for CI/CD.

Project Architecture

Here’s the shape of the system:

• Terraform provisions the infrastructure

• Lambda + boto3 runs the inventory and compliance checks

• DynamoDB stores findings

• S3 stores raw reports

• API Gateway exposes /audit/run and /audit/results

• CloudWatch Logs stores structured JSON logs

• CloudTrail records AWS-side activity

• Postman validates the API behavior

That architecture matters because it moves the project from “script” to “small internal platform.” CloudTrail’s DescribeTrails and related API coverage make it useful for verifying control-plane setup and visibility.

GitHub Actions CI/CD
  |                         |
  | deploy.yml              | terraform.yml
  | (src/** changes)        | (terraform/** changes)
  v                         v
OIDC → GitHubActionsDeployRole (Terraform-managed)
  |                         |
  | lambda:UpdateFunctionCode  | terraform plan/apply
  v                         v
Lambda code deploy    HCP Terraform (remote state)
                            |
                            v
              Terraform-managed AWS environment
                - API Gateway (HTTP API)
                - Lambda audit function
                - DynamoDB findings table (with GSI for query)
                - S3 reports bucket
                - CloudWatch Logs
                - CloudTrail
                - EventBridge schedule (daily full audit)
                - EventBridge rule (real-time CloudTrail S3 events)
                - SNS topic for public bucket alerts
                - IAM roles (least-privilege)
                            |
                            v
              boto3 multi-region inventory + compliance checks
                            |
                            +--> findings saved to DynamoDB
                            +--> raw reports saved to S3
                            +--> findings imported to Security Hub
                            +--> SNS notifications for public buckets
                            +--> structured JSON logs to CloudWatch
                            |
                            v
              CloudTrail S3 event --> EventBridge --> Lambda micro-audit
                            |
                            +--> single-bucket compliance check
                            +--> findings + SNS notification with actor context
                            |
                            v
              Postman collection / GET /audit/results
                - trigger audit
                - query findings by check_id
                - validate responses

What I wanted to prove

Secure SDK usage is not just “call an API.”
It is authentication design, IAM precision, resilience under throttling, and evidence you can actually inspect afterward.

Temporary credentials only

One hard rule shaped the whole project: no static AWS access keys.

For CI/CD, I used the GitHub Actions OIDC model. GitHub documents that OIDC allows workflows to access AWS without storing long-lived AWS credentials as GitHub secrets. That is the security baseline I wanted the project to enforce from day one.

For local execution and certain bootstrap tasks, the pattern is short-lived role assumption instead of hardcoded keys. That kept the authentication story consistent: trust relationships and temporary credentials, not secret sprawl.

Throttling is not an edge case

When a script starts walking multiple AWS services, throttling is not hypothetical.

So the project uses two layers of protection:

• SDK-level retry behavior in boto3/botocore.

• Lightweight pacing and exponential backoff with jitter in the application logic

That combination matters because retries alone are not always enough if your script is needlessly aggressive. It is better to reduce pressure up front than to slam into rate limits and “recover” from self-inflicted errors. AWS’s SDK guidance supports configurable retry behavior for exactly these kinds of transient failures and throttling scenarios.

Audit the audit

A big part of the assignment was not just running checks, but creating the means to audit the audit.

So I built two layers of visibility:

• Structured JSON logs from the application for every API call

• CloudTrail for AWS-side history of account activity

That gives you two useful perspectives. App logs tell you what the code thought it was doing. CloudTrail tells you what AWS actually recorded. For a security-oriented capstone, that is a much stronger story than “the script printed some output.”

Postman made the project operational

Adding Postman turned the project into something demoable and repeatable.

Instead of only proving that the Lambda function exists, I could validate:

• health checks

• audit triggers

• findings retrieval

• response structure

• signed request behavior where needed

Postman officially supports AWS Signature authentication, which makes it a clean fit for testing AWS-backed endpoints.

The most valuable part: what broke

The best lessons in this project did not come from the happy path. They came from the implementation failures.

Below are the biggest issues I hit and what they taught me.

1) Security Hub AccessDeniedException

I ran into a nasty issue where setting AWS_ACCOUNT_ID the SNS caused Security Hub logic to attempt BatchImportFindings instead of skipping that path. The result was an AccessDeniedException.

The fix was not to force permissions through. The fix was to wrap the call in a try/except function and return a graceful skipped status with the error message. That was the right move because AWS documents that BatchImportFindings have caller/account requirements, including account alignment in certain import paths. Optional integrations should fail safely rather than crash the entire audit flow.

Lesson

Optional security integrations should degrade gracefully.
“Skip with evidence” beats “hard fail for everything.”

2) The S3 IAM action name was wrong

The original IAM policy used:

"s3:GetPublicAccessBlock"

Well, that looked right. It was wrong.

For bucket-level retrieval, the correct permission is:

"s3:GetBucketPublicAccessBlock"

AWS’s S3 API reference is explicit: the GetPublicAccessBlock bucket operation requires s3:GetBucketPublicAccessBlock. Fixing that permission unblocked the S3 public-access checks immediately.

3) DynamoDB GSI permissions were more specific than expected

A query against a DynamoDB global secondary index failed even though the table permissions looked fine.

The issue was that querying a GSI requires permissions on the index ARN, not just the base table ARN. AWS documents IAM patterns that grant access to a table and all of its indexes, which is the pattern I needed. Once the policy included the index ARN path, the query worked.

That is one of those AWS details that punishes surface-level understanding. “Table access” does not automatically mean “index access” as people often assume.

4) Some describe actions that cannot be tightly ARN-scoped

This one is easy to miss if you are trying to be extremely strict with IAM.

During Terraform state refresh, these actions failed under tight scoping:

cloudtrail:DescribeTrails
logs:DescribeLogGroups

The practical fix was to give them their own IAM statement with Resource: "*". That is not laziness; it is matching the way these read/list operations are authorized. AWS’s service authorization references and CloudWatch Logs documentation make clear that some describe-style actions do not behave like normal resource-scoped operations. CloudWatch Logs also notes limits on IAM control patterns for DescribeLogGroups.

Lesson

Least privilege does not mean “force every action onto an ARN.” Sometimes the tightest correct policy is a narrow action set with Resource: "*".

5) OIDC IAM bootstrapping was real

This was one of the most practical issues in the whole build.

I was using Terraform to manage the CI/CD role’s own permissions, but terraform plan needed permission to inspect the very OIDC and IAM resources the role depended on. Initially, the role lacked read permissions like:

• iam:GetOpenIDConnectProvider

• iam:GetRolePolicy

• iam:ListAttachedRolePolicies

That caused plan/state-refresh failures.

The fix was to bootstrap once locally: apply the GitHubActionsDeployPolicy first, then push to CI. After that, the role had enough visibility to read and manage its own policy path, and later CI runs became self-managing. GitHub’s AWS OIDC documentation supports this overall trust model; the bootstrap nuance is the real-world layer you only feel when you actually implement it.

Lesson

Self-managing IAM in Terraform is possible, but the first step often cannot come from a role that does not yet have permission to see itself.

What this project actually proves

This project proves more than “I can write boto3.”

It shows:

• secure use of the official AWS SDK

• temporary-credential authentication design

• Terraform-based reproducibility

• throttling-aware engineering

• structured operational logging

• API validation through Postman

• real IAM debugging under CI/CD pressure

That is why the implementation issues matter so much. They make the project more credible, not less.

Final takeaway

AuditTrail SDK began as a compliance automation capstone project. It became a much better lesson than that.

It exposed how AWS services behave when IAM is tight, how Terraform refresh interacts with identity resources, how subtle permission names can break security checks, and how optional integrations like Security Hub need graceful failure paths to avoid turning useful tooling into brittle tooling. AWS’s own docs back up the key mechanics here: OIDC for GitHub Actions avoids long-lived secrets; S3 public access checks depend on the right bucket-level permissions; DynamoDB index access requires index-aware policies; and describe-style APIs sometimes need wildcard resources.

That is the difference between building a project for screenshots and building one that survives contact with reality.