Secrets Management

Volume 5 — Platform Engineering

The Azure Key Vault chapter covered one specific secret store in depth. This chapter is broader: which store is the right one for a given context, how secrets move (or shouldn't move) between them, and the handful of absolute rules worth never breaking regardless of how much time pressure is involved.

1. A Secret's Lifecycle Has Multiple Contexts

The same conceptual secret — say, the database connection string — legitimately needs to exist in several different places, each with a different appropriate storage mechanism:

Context	Right store	Wrong store
A developer's personal reference (initial setup, one-off manual testing)	Bitwarden	A Slack DM, a text file on the desktop, a sticky note
Local development environment	.env file (gitignored), or appsettings.Development.json (gitignored)	Hardcoded directly in checked-in source
CI/CD pipeline execution	Azure DevOps secret variable / variable group	Plain pipeline variable, visible in logs
Running production application	Key Vault, resolved via managed identity	A pipeline variable baked into a deployed config file

The same secret value, three or four legitimate homes, each serving a genuinely different need — the goal isn't "one store for everything," it's the right store for each context, with the fewest possible additional copies of the actual value floating around.

2. Bitwarden — the Team's Secret Store

Bitwarden is used across Traxs for the secrets that don't belong in any automated system at all — credentials a human needs to reference or hand-enter: third-party service logins, initial account setup credentials, personal API keys for services not yet wired into Key Vault.

Practices worth holding to:

• One entry per credential, named specifically enough that "which one is this" is never a guessing game six months later.
• Shared organization vault for anything team-relevant (a shared service account), never a personal vault, so access doesn't disappear if one person is unavailable.
• Never paste a Bitwarden-stored secret into Slack, email, or a ticket to "share" it — share the Bitwarden entry (or grant vault access), not the value in plaintext somewhere that itself becomes a new location the secret now lives, unencrypted, indefinitely.

3. What Should Never Happen, Regardless of Time Pressure

These aren't stylistic preferences — each has a direct, concrete failure mode:

Never	Because
Commit a secret to Git, even briefly, even in a private repo	Git history retains it indefinitely — removing it in a later commit does not remove it from history (see the Git Troubleshooting chapter's section on this exact scenario)
Share a secret over Slack/email/chat "just this once"	Chat history is itself a persistent, searchable, often broadly-accessible store the secret now lives in, outside any secret management system's visibility
Reuse the same secret across dev and production	A leaked or overly-broadly-shared development credential becomes a production compromise
Hardcode a secret "temporarily" during debugging, intending to remove it before commit	This is exactly how secrets end up committed — the intention to remove it is not a safeguard, only actually not committing it is
Log a secret value, even at debug level	Log aggregators (Seq, App Service Log Stream) retain history and are often more broadly accessible than the secret store itself

If any of these happen anyway — treat the exposed credential as compromised immediately and rotate it. Don't wait to assess whether it was "actually seen" by anyone; the cost of an unnecessary rotation is far lower than the cost of a credential that was genuinely compromised and left in place.

4. Local Development: .env Files

# .env — gitignored, never committed
DB_PASSWORD=LocalDevOnly!2026
SENDGRID_API_KEY=SG.local-dev-key-here

# .gitignore
.env
appsettings.Development.json

Covered mechanically in the Docker Compose chapter — the principle worth restating here is that a local development secret should be genuinely different from the production one, not a shortcut copy of it, specifically so a leaked local .env file never translates into meaningful production exposure.

5. Secret Rotation

Trigger	Response
Scheduled/routine rotation	Rotate on a defined interval regardless of any known issue — limits the window a leaked-but-undetected credential remains useful
Known/suspected exposure	Rotate immediately, don't wait to confirm actual misuse first
Team member offboarding	Rotate any shared credential they had access to, not just revoke their individual account access
Third-party breach notification	Rotate the specific credential(s) affected, even if Traxs's own systems weren't directly compromised

Rotation is only as good as the update propagating everywhere the old value was actually used — the Azure Key Vault chapter's stop/start caching gotcha is a direct, concrete example of a rotation that looks complete (the new value is in Key Vault) but silently isn't actually in effect yet until the specific follow-up step is taken.

6. Least Exposure, Not Just Least Privilege

Beyond Azure RBAC's least-privilege principle (who can access a secret store), apply the same discipline to how many places a secret's actual value exists at all:

• Prefer a system resolving a secret at the point of use (a Key Vault reference resolved by App Service) over that value being copied into multiple configuration files or pipeline variables.
• When a secret genuinely needs to exist in more than one place, know and be able to enumerate exactly where — an unaccounted-for third copy is exactly the kind of thing that gets missed during a rotation.

7. Troubleshooting Playbook

Symptom	Likely cause	What to check
Secret was committed to Git	Bypassed .gitignore, or added before a .gitignore entry existed	Rotate the credential immediately (Git Troubleshooting chapter covers cleaning history, but rotation comes first, always)
Rotated a secret, an old value is still somehow in use	A copy exists somewhere beyond the store that was updated — a cached value, a second config file, an unmanaged pipeline variable	Enumerate every known location the secret's value exists (Section 6) and confirm each was actually updated
Local .env value ended up committed	.gitignore entry missing, added too late, or the file was force-added at some point (git add -f)	Confirm .env is genuinely gitignored; check git log for whether it was ever actually tracked despite the current .gitignore
Team member's access revoked, but a shared service credential they knew is unrotated	Offboarding only removed their individual account access, not shared secrets they had visibility into	Rotate any shared credential a departing team member had access to, as a standard part of offboarding, not an afterthought
Unsure whether a specific value is still "live" anywhere	No enumeration of the secret's known locations exists	Treat this as a gap to close going forward — document where each sensitive value legitimately lives as part of introducing it, not after the fact

8. Quick Reference

Context	Store	Notes
Human reference / manual credentials	Bitwarden	Shared org vault, never a personal one, for team-relevant credentials
Local development	.env / gitignored config	Genuinely different values from production
CI/CD pipeline execution	Azure DevOps secret variable/group	Masked in logs automatically
Running application, runtime	Azure Key Vault, via managed identity	See the Azure Key Vault chapter for the stop/start rotation gotcha
Absolute rule	Never commit, never chat-share, never reuse dev/prod	Treat any violation as immediate compromise, rotate regardless of certainty

---

Part of the Traxs Engineering Handbook — Volume 5: Platform Engineering. Companion chapters in this volume: Linux Fundamentals, Linux Administration, Kubernetes Fundamentals, CI/CD, Observability.

Volume 5 — Platform Engineering is now complete: Linux Fundamentals, Linux Administration, Kubernetes Fundamentals, CI/CD, Observability, and Secrets Management.