Encryption at Rest and in Transit: How Envelope Encryption and Managed KMS Actually Work

During compliance audits, I often hear teams say: "Our database and S3 buckets have encryption at rest turned on, so our data cannot be stolen." Then I ask a follow-up question: "Which key encrypts that bucket—a default provider key or a Customer-Managed Key (CMK) with a strict key policy?" The silence that follows reveals one of the most misunderstood concepts in cloud security.

---

In Part 4 of our Modern Security Architecture series (Secrets Management Architecture), we eliminated hardcoded credentials from source code and runtime configs.

Now, in Part 5, we turn to the mathematical foundation protecting the data itself: Encryption in Transit and Encryption at Rest using Managed Key Management Services (KMS).

To architect data protection that actually stops modern cloud breaches—rather than merely checking a compliance box—you must understand three core cryptographic mechanics:

1. How TLS 1.3 and Mutual TLS (mTLS) protect data moving across networks.
2. Why cloud-managed default encryption only protects against physical hard-drive theft, whereas Customer-Managed KMS Keys (CMKs) create a cryptographic access boundary.
3. How Envelope Encryption solves the performance and payload-size limits of Hardware Security Modules (HSMs).

---

1. Encryption in Transit: TLS 1.3, Ephemeral Diffie-Hellman, and mTLS

Whenever data leaves a process's memory—whether traveling from a user's browser to your CDN, from an API Gateway to a microservice, or from an application pod to a PostgreSQL database—it is data in transit.

Without cryptographic protection on the wire, any network hop along the path (a compromised router, a misconfigured load balancer, or an ARP-spoofed neighbor) can passively eavesdrop on plaintext payloads or actively tamper with bytes in flight.

Why TLS 1.3 Changed the Standard

Defined in IETF RFC 8446, TLS 1.3 overhauled the legacy TLS 1.2 protocol in three decisive ways:

1. Mandatory Perfect Forward Secrecy (PFS): TLS 1.3 completely removed static RSA key exchange. Every single TLS 1.3 connection uses Ephemeral Diffie-Hellman (ECDHE / X25519 / P-256) to negotiate a unique session key. Even if an attacker records encrypted network traffic for years and later steals the server's long-term private key, they still cannot decrypt a single past session.
2. Authenticated Encryption with Associated Data (AEAD) Only: Legacy weak ciphers (RC4, 3DES, CBC-mode ciphers vulnerable to padding-oracle attacks like POODLE and Lucky13) were stripped out. TLS 1.3 mandates modern AEAD cipher suites such as TLS_AES_256_GCM_SHA384 and TLS_CHACHA20_POLY1305_SHA256, which simultaneously guarantee confidentiality (encryption) and integrity (authentication tags, building on the hashing foundations you can test in our Hash Generator).
3. 1-RTT Handshake: By sending key shares in the very first ClientHello, TLS 1.3 completes its cryptographic handshake in a single network round-trip—proving that stronger security can actually reduce network latency.

Enforcing Encryption in Transit End-to-End

A common architectural anti-pattern is terminating TLS at the public Load Balancer and forwarding plaintext HTTP (port 80) across internal VPC subnets to backend containers. In a Zero Trust / Defense-in-Depth architecture:

• Public edges enforce HTTPS only with HSTS (Strict-Transport-Security).
• Internal service-to-service communication uses Mutual TLS (mTLS) so both client and server workloads verify X.509 certificates.
• Cloud storage and databases enforce transport encryption via policy—such as attaching an S3 bucket policy that explicitly denies any request where "aws:SecureTransport": "false" or setting rds.force_ssl = 1 on PostgreSQL/MySQL instances.

---

2. The "Encrypt At Rest Checkbox" Illusion: Provider Keys vs. Customer-Managed KMS Keys

When you enable default encryption on a cloud storage volume (such as Amazon EBS, S3 SSE-S3, or default cloud database encryption), what does that actually protect you against?

Default provider-managed encryption protects against someone physically walking into an AWS, Azure, or Google data center and pulling a hard drive out of a server rack.

It does not stop an attacker who compromises an over-privileged IAM role inside your cloud account! Why? Because with default provider-managed keys (SSE-S3), any IAM principal in your account with s3:GetObject permission gets the data transparently decrypted by the storage service automatically.

Why Customer-Managed KMS Keys (CMKs) Are a Game Changer

When you instead encrypt sensitive S3 buckets, RDS databases, EBS volumes, and Secrets Manager entries using a Customer-Managed Key (CMK) in AWS KMS (or a customer-managed key in Azure Key Vault / Google Cloud KMS), you introduce a second, independent cryptographic authorization gate:

1. To read an encrypted S3 object, the calling workload must be allowed by the IAM/S3 Policy (s3:GetObject) AND must also be explicitly permitted by the KMS Key Policy (kms:Decrypt)!
2. Every single encrypt or decrypt operation against the KMS CMK is immutably recorded in AWS CloudTrail (or Azure Monitor / Cloud Audit Logs).
3. You control the Key Rotation schedule, Key Deletion windows, and Cryptographic Encryption Context (additional authenticated data bound to the ciphertext).

---

3. How Envelope Encryption Works Under the Hood

Here is a puzzle every systems architect eventually encounters:

In AWS KMS, the root Customer-Managed Key (CMK) lives inside a FIPS 140-3 validated Hardware Security Module (HSM). That root key can never leave the HSM in plaintext, and the direct kms:Encrypt API only accepts payloads up to 4 Kilobytes (4 KB)!

If KMS can only encrypt 4 KB directly—and making a network RPC to KMS for every 4 KB block of a 50 GB database file would saturate the network—how does cloud storage encrypt terabytes of data using KMS?

The answer is Envelope Encryption:

flowchart TB
    subgraph EncryptFlow["1. Envelope Encryption Workflow (Encrypting a 10 GB File)"]
        direction LR
        KMS1["Cloud KMS HSM<br/>(Root CMK Never Leaves HSM)"] -->|"GenerateDataKey API"| DEKPair["Returns Two Keys:<br/>1. Plaintext DEK (In-Memory Only)<br/>2. Encrypted DEK (Ciphertext)"]
        DEKPair -->|"Use Plaintext DEK Locally<br/>(AES-256-GCM at RAM Speed)"| EncData["Encrypt 10 GB File Locally<br/>-> Wipe Plaintext DEK from RAM!"]
        EncData --> Bundle["Store Together on Disk:<br/>[Encrypted DEK] + [Encrypted File Ciphertext]"]
    end

    subgraph DecryptFlow["2. Envelope Decryption Workflow (Reading the File)"]
        direction LR
        BundleRead["Read from Disk:<br/>[Encrypted DEK] + [Ciphertext]"] -->|"Send Only 64-Byte<br/>Encrypted DEK to KMS"| KMS2["Cloud KMS HSM<br/>Verifies IAM + Key Policy<br/>& Logs to CloudTrail"]
        KMS2 -->|"Returns Plaintext DEK"| DecLocal["Decrypt 10 GB Ciphertext Locally<br/>with Plaintext DEK -> Wipe from RAM"]
    end

Step-by-Step Mechanics of Envelope Encryption

1. Generate a Data Encryption Key (GenerateDataKey): When your service needs to encrypt a large payload, it makes a single API call to kms:GenerateDataKey specifying your Customer-Managed Key ID.
2. Dual-Key Response: Inside the HSM, KMS generates a cryptographically random 256-bit Data Encryption Key (DEK) and returns two things over TLS:
   • The Plaintext DEK
   • A Ciphertext Blob (Encrypted DEK)—which is that exact DEK encrypted under your HSM-resident root CMK.
3. Local High-Speed AES-256-GCM Encryption: Your application (or the storage service like S3/EBS) uses the Plaintext DEK locally in CPU memory (leveraging hardware AES-NI instructions) to encrypt the arbitrary-sized data payload using AES-256-GCM, and immediately zeroes/wipes the Plaintext DEK from memory.
4. Store the Envelope: The small Encrypted DEK is stored right alongside the encrypted data payload (as object metadata or a file header).
5. Decryption with Context Verification: To decrypt the file later, the application reads the Encrypted DEK from the file header and calls kms:Decrypt with the required Encryption Context (e.g., {"tenantId": "acme-corp", "purpose": "payroll"}). If the caller's IAM role and KMS Key Policy permit decryption and the Encryption Context matches byte-for-byte, KMS returns the Plaintext DEK to decrypt the payload in memory.

Optimizing KMS Calls: Data Key Caching & S3 Bucket Keys

What if a high-throughput service reads 50,000 objects per second from an S3 bucket encrypted with a KMS CMK? Calling KMS 50,000 times per second would hit KMS API rate limits and rack up massive API charges.

• Amazon S3 Bucket Keys solve this by generating a short-lived bucket-level intermediate key from KMS that S3 uses to mint unique object DEKs locally—reducing AWS KMS request traffic and KMS costs by up to 99% while preserving CMK control and CloudTrail visibility.

---

4. Under the Hood: Testable Envelope Encryption with Encryption Context (AAD) in TypeScript

Below is a complete, runnable TypeScript implementation demonstrating Envelope Encryption and Decryption with Additional Authenticated Data (EncryptionContext) and strict KMS Key Policy enforcement.

Notice how even if an attacker obtains the full encrypted envelope from disk, tampering with a single byte of ciphertext—or attempting to decrypt with the wrong tenantId Encryption Context—causes cryptographic verification to fail immediately.

Copy and run this code in our JS/TS Playground:

/**
 * Cloud KMS Envelope Encryption & EncryptionContext (AAD) Reference Engine
 * Demonstrates:
 *  1. Root CMK isolated inside simulated HSM (never exported)
 *  2. GenerateDataKey producing Plaintext DEK + Encrypted DEK
 *  3. Binding EncryptionContext (AAD) to prevent cross-tenant ciphertext substitution
 *  4. Dual authorization check (KMS Key Policy + Caller Principal)
 */

interface EncryptionContext {
  [key: string]: string;
}

interface EncryptedEnvelope {
  cmkKeyId: string;
  encryptedDekHex: string;
  ivHex: string;
  ciphertextHex: string;
  authTagHex: string;
  contextSnapshot: EncryptionContext;
}

class SimulatedCloudKmsHsm {
  // Root 256-bit CMK material sealed inside HSM memory (never exposed to callers!)
  private readonly hsmRootKeys = new Map<string, number[]>();
  private readonly keyAllowedPrincipals = new Map<string, Set<string>>();
  private dekCounter = 1;

  public createCustomerManagedKey(keyId: string, allowedPrincipals: string[]): void {
    const rootSecret = Array.from({ length: 32 }, (_, i) => (i * 37 + 0x5a) & 0xff);
    this.hsmRootKeys.set(keyId, rootSecret);
    this.keyAllowedPrincipals.set(keyId, new Set(allowedPrincipals));
  }

  /**
   * Simulates kms:GenerateDataKey(KeyId, EncryptionContext)
   */
  public generateDataKey(
    callerPrincipal: string,
    keyId: string,
    context: EncryptionContext
  ): { plaintextDek: number[]; encryptedDekHex: string } {
    this.verifyKeyPolicy(callerPrincipal, keyId);
    const rootKey = this.hsmRootKeys.get(keyId)!;

    // Generate unique 32-byte (256-bit) Data Encryption Key (DEK)
    const seed = this.dekCounter++;
    const plaintextDek = Array.from({ length: 32 }, (_, i) => (seed * 73 + i * 19 + 0xa3) & 0xff);

    // Bind EncryptionContext (AAD) into the wrapped DEK inside HSM
    const contextTag = this.hashContext(context);
    const wrappedBytes = plaintextDek.map((b, i) => b ^ rootKey[i] ^ contextTag[i % contextTag.length]);
    const authDigest = this.computeMac(wrappedBytes, rootKey, contextTag);

    const encryptedDekHex = this.toHex([...wrappedBytes, ...authDigest]);
    return { plaintextDek, encryptedDekHex };
  }

  /**
   * Simulates kms:Decrypt(CiphertextBlob, EncryptionContext)
   */
  public decryptDataKey(
    callerPrincipal: string,
    keyId: string,
    encryptedDekHex: string,
    context: EncryptionContext
  ): number[] {
    this.verifyKeyPolicy(callerPrincipal, keyId);
    const rootKey = this.hsmRootKeys.get(keyId)!;
    const raw = this.fromHex(encryptedDekHex);

    const wrappedBytes = raw.slice(0, 32);
    const providedMac = raw.slice(32);
    const contextTag = this.hashContext(context);
    const expectedMac = this.computeMac(wrappedBytes, rootKey, contextTag);

    if (!this.constantTimeEquals(providedMac, expectedMac)) {
      throw new Error(
        "KMSInvalidCiphertextException: EncryptionContext mismatch or corrupted Encrypted DEK!"
      );
    }

    return wrappedBytes.map((b, i) => b ^ rootKey[i] ^ contextTag[i % contextTag.length]);
  }

  private verifyKeyPolicy(callerPrincipal: string, keyId: string): void {
    const allowed = this.keyAllowedPrincipals.get(keyId);
    if (!allowed || !allowed.has(callerPrincipal)) {
      throw new Error(
        `KMSAccessDeniedException: Principal '${callerPrincipal}' is not authorized by Key Policy on CMK '${keyId}'.`
      );
    }
  }

  private hashContext(ctx: EncryptionContext): number[] {
    const canonical = Object.keys(ctx)
      .sort()
      .map((k) => `${k}=${ctx[k]}`)
      .join("&");
    let h1 = 0x811c9dc5;
    let h2 = 0x01000193;
    for (let i = 0; i < canonical.length; i++) {
      const c = canonical.charCodeAt(i);
      h1 = Math.imul(h1 ^ c, 0x01000193) >>> 0;
      h2 = Math.imul(h2 + c, 0x85ebca6b) >>> 0;
    }
    return [
      h1 & 0xff, (h1 >>> 8) & 0xff, (h1 >>> 16) & 0xff, (h1 >>> 24) & 0xff,
      h2 & 0xff, (h2 >>> 8) & 0xff, (h2 >>> 16) & 0xff, (h2 >>> 24) & 0xff,
    ];
  }

  private computeMac(data: number[], key: number[], aad: number[]): number[] {
    let acc = 0x9e3779b9;
    for (let i = 0; i < data.length; i++) {
      acc = Math.imul(acc ^ data[i] ^ key[i % key.length] ^ aad[i % aad.length], 0x01000193) >>> 0;
    }
    return [acc & 0xff, (acc >>> 8) & 0xff, (acc >>> 16) & 0xff, (acc >>> 24) & 0xff];
  }

  public constantTimeEquals(a: number[], b: number[]): boolean {
    if (a.length !== b.length) return false;
    let diff = 0;
    for (let i = 0; i < a.length; i++) diff |= a[i] ^ b[i];
    return diff === 0;
  }

  public toHex(bytes: number[]): string {
    return bytes.map((b) => b.toString(16).padStart(2, "0")).join("");
  }

  public fromHex(hex: string): number[] {
    const out: number[] = [];
    for (let i = 0; i < hex.length; i += 2) {
      out.push(parseInt(hex.slice(i, i + 2), 16));
    }
    return out;
  }
}

class EnvelopeEncryptionService {
  constructor(private readonly kms: SimulatedCloudKmsHsm) {}

  public encryptPayload(
    callerPrincipal: string,
    cmkKeyId: string,
    plaintext: string,
    context: EncryptionContext
  ): EncryptedEnvelope {
    // 1. Request fresh DEK from KMS HSM
    const { plaintextDek, encryptedDekHex } = this.kms.generateDataKey(callerPrincipal, cmkKeyId, context);

    // 2. Encrypt payload locally in memory using Plaintext DEK
    const iv = [0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc, 0xde, 0xf0];
    const plaintextBytes = Array.from(plaintext).map((ch) => ch.charCodeAt(0));
    const ciphertextBytes = plaintextBytes.map(
      (b, idx) => b ^ plaintextDek[idx % plaintextDek.length] ^ iv[idx % iv.length]
    );

    // Compute AEAD Authentication Tag over Ciphertext + IV + DEK
    let tagAcc = 0xcbf29ce4;
    for (let i = 0; i < ciphertextBytes.length; i++) {
      tagAcc = Math.imul(tagAcc ^ ciphertextBytes[i] ^ plaintextDek[i % 32], 0x01000193) >>> 0;
    }
    const authTag = [tagAcc & 0xff, (tagAcc >>> 8) & 0xff, (tagAcc >>> 16) & 0xff, (tagAcc >>> 24) & 0xff];

    // 3. Zero out Plaintext DEK in memory immediately after local encryption!
    plaintextDek.fill(0);

    return {
      cmkKeyId,
      encryptedDekHex,
      ivHex: this.kms.toHex(iv),
      ciphertextHex: this.kms.toHex(ciphertextBytes),
      authTagHex: this.kms.toHex(authTag),
      contextSnapshot: context,
    };
  }

  public decryptPayload(
    callerPrincipal: string,
    envelope: EncryptedEnvelope,
    providedContext: EncryptionContext
  ): string {
    // 1. Ask KMS HSM to unwrap the Encrypted DEK (verifies Key Policy + EncryptionContext)
    const plaintextDek = this.kms.decryptDataKey(
      callerPrincipal,
      envelope.cmkKeyId,
      envelope.encryptedDekHex,
      providedContext
    );

    // 2. Verify Ciphertext Integrity Tag (AEAD) before outputting any plaintext
    const ciphertextBytes = this.kms.fromHex(envelope.ciphertextHex);
    let tagAcc = 0xcbf29ce4;
    for (let i = 0; i < ciphertextBytes.length; i++) {
      tagAcc = Math.imul(tagAcc ^ ciphertextBytes[i] ^ plaintextDek[i % 32], 0x01000193) >>> 0;
    }
    const computedTag = [tagAcc & 0xff, (tagAcc >>> 8) & 0xff, (tagAcc >>> 16) & 0xff, (tagAcc >>> 24) & 0xff];
    const expectedTag = this.kms.fromHex(envelope.authTagHex);

    if (!this.kms.constantTimeEquals(computedTag, expectedTag)) {
      plaintextDek.fill(0);
      throw new Error("AEADIntegrityError: Ciphertext or AuthTag was tampered with on disk!");
    }

    // 3. Decrypt locally and wipe Plaintext DEK from memory
    const iv = this.kms.fromHex(envelope.ivHex);
    const decryptedChars = ciphertextBytes.map((b, idx) =>
      String.fromCharCode(b ^ plaintextDek[idx % plaintextDek.length] ^ iv[idx % iv.length])
    );
    plaintextDek.fill(0);
    return decryptedChars.join("");
  }
}

// ============================================================================
// Verification Scenarios
// ============================================================================
const kmsHsm = new SimulatedCloudKmsHsm();
kmsHsm.createCustomerManagedKey("alias/prod-payments-cmk", ["arn:aws:iam::123456789012:role/PaymentsServiceRole"]);

const cryptoService = new EnvelopeEncryptionService(kmsHsm);
const context = { tenantId: "tenant-9942", resourceType: "wire-transfer" };

// 1. Authorized Service Encrypts Sensitive Record via Envelope Encryption
const envelope = cryptoService.encryptPayload(
  "arn:aws:iam::123456789012:role/PaymentsServiceRole",
  "alias/prod-payments-cmk",
  JSON.stringify({ account: "IBAN-DE89370400440532013000", amountUsd: 250000 }),
  context
);
console.log("1. Stored Encrypted Envelope on Disk:", envelope);

// 2. Authorized Service Decrypts with Matching Context
console.log(
  "2. Authorized Decryption:",
  cryptoService.decryptPayload("arn:aws:iam::123456789012:role/PaymentsServiceRole", envelope, context)
);

// 3. Attacker with Storage Read Access (Different IAM Role) Attempts Decryption -> Blocked by KMS Key Policy!
try {
  cryptoService.decryptPayload("arn:aws:iam::123456789012:role/AnalyticsReadOnlyRole", envelope, context);
} catch (err) {
  console.log("3. Unauthorized Role Blocked by CMK Key Policy:", (err as Error).message);
}

// 4. Cross-Tenant Confused Deputy Attack (Wrong EncryptionContext) -> Blocked by AAD Verification!
try {
  cryptoService.decryptPayload("arn:aws:iam::123456789012:role/PaymentsServiceRole", envelope, {
    tenantId: "tenant-attacker-0001",
    resourceType: "wire-transfer",
  });
} catch (err) {
  console.log("4. Cross-Tenant Context Tampering Blocked:", (err as Error).message);
}

---

5. Production Architecture Checklist for Encryption & KMS

Control Domain	Baseline Mistake ❌	Production Security Architecture ✅
TLS Configuration	Allowing legacy TLS 1.0/1.1 or terminating TLS at edge with plaintext internal hops	Enforce TLS 1.3 (or TLS 1.2 with ECDHE+AEAD only) at edge and mTLS for East-West traffic
At-Rest Key Ownership	Relying solely on default storage-managed keys (SSE-S3) for sensitive PII/financial data	Use Customer-Managed KMS Keys (CMKs) with least-privilege Key Policies and automatic annual rotation
Envelope Context	Calling KMS without EncryptionContext (vulnerable to ciphertext cut-and-paste across records)	Pass structured EncryptionContext (tenantId, tableName) on every GenerateDataKey and Decrypt call
High-Volume Storage	Making raw KMS RPC calls for every individual S3 object read	Enable S3 Bucket Keys or local KMS Data Key Caching with strict TTLs

In our sixth and final installment of the Modern Security Architecture series, we examine the network fabric underneath all of these workloads: Private Cloud Networking: VPC/VNet Isolation, Private Endpoints, and Stateful Security Groups.