Least privilege is a claim until the database refuses you: building RBAC and custom roles on a financial-grade MongoDB cluster

The previous build ended in a defensible but incomplete state: a three-region replica set, every connection authenticated by an x.509 certificate, and exactly one principal — an operator certificate mapped to the root role. That was correct for bootstrapping. It is indefensible as a steady state. A root identity used for routine work is a root identity exposed to routine risk, and the entire premise of a financial-grade deployment is that risk is bounded, attributable, and proportional to need. One omnipotent certificate is the opposite of all three. This build replaces that single identity with a deliberate set of them — each one a distinct certificate mapped to a distinct, narrowly-scoped role — and then does the part that most access-control write-ups skip: it proves the scoping holds. Creating a user with a limited role is a claim. Demonstrating that the user can do its job and is structurally refused everything beyond it is the proof. The difference between those two is the difference between an access model you describe in a runbook and one an auditor will sign off on. The governing principle throughout: no human operates as root day to day, no application ever holds administrative rights, and every identity is granted exactly what its function requires and nothing more. The work is to make that principle concrete, certificate by certificate, role by role, and then to make the cluster enforce it in front of us.

Before any command runs, the access model has to be designed, because the certificates and roles encode decisions that are expensive to change once issued. The question is not "what logins does the system need" but "what are the distinct jobs that touch this cluster, and what is the minimum privilege each job requires."

For a system that moves money, six jobs are distinct enough to warrant separate identities:

Cluster administrator. Operates the cluster and its databases — reconfiguration, index management, diagnostics. Built on clusterAdmin and dbAdminAnyDatabase. Deliberately not root: it has no business reading the raw internals or holding anyResource grants. And — this is the consequential decision — it cannot manage users.

User administrator. Creates and manages users and roles, and nothing else. userAdminAnyDatabase. This identity exists as a separate principal from the cluster administrator on purpose, and the purpose is separation of duties: the entity that grants access should not also be the entity that operates the system and reads its data. An identity that could both create privileges and exercise them is precisely the concentration of power an auditor flags. Splitting them means provisioning a new privileged identity requires one certificate, and operating the cluster requires another, and no single certificate does both.

Application. The identity the application authenticates as in production. readWrite on the application database — and only that database. No other namespace, no administrative action, no cluster visibility. This is the blast-radius control: if the application is ever compromised, the privileges the attacker inherits are exactly the privileges the application legitimately held, and not one grant more.

Backup. Reads everything required for a consistent snapshot and writes nothing. The built-in backup role. It is broad in read scope by necessity — a backup must see all data — and that breadth is itself a reason to keep it isolated to a single, tightly-controlled identity rather than folding it into the application or admin certificates.

Monitoring. The narrowest and most interesting of the set. It must observe cluster health — server status, replication state, operational metrics — and it must be incapable of reading a single document of application data. No built-in role draws that line precisely, which is why this identity gets a custom role, defined action by action.

Read-only analytics. Read access to the application database, no write. For dashboards, reporting, and ad-hoc queries, ideally served from a secondary so analytical load never touches the primary's write path.

Each identity is a separate x.509 certificate with a distinct subject distinguished name. They share the organizational tail the certificate authority enforces — OU=cluster-clients, O=Meetadev, DC=mongo, DC=internal — and differ in their common name: meetadev-clusteradmin, meetadev-useradmin, meetadev-app, meetadev-backup, meetadev-monitor, meetadev-analytics. The common name is what makes every connection attributable to a named principal in the audit log. There is no shared service account, no anonymous access, no certificate that stands for more than one job.

One design decision governs how the six keys are generated, and it is worth stating because the wrong default here is a common production failure.

Two of these identities are operated by a human — the cluster administrator and the user administrator. A human is present to type a passphrase, so their private keys are encrypted with one. If the key file is ever exposed, it is useless without the passphrase, and the human who holds it supplies the passphrase interactively at the moment of use.

The other four are operated by services — the application, the backup job, the monitoring agent, the analytics consumer. A service restarts unattended, often at three in the morning with no human watching, and cannot pause to be prompted for a passphrase. Encrypting a service key with a passphrase that then has to be stored in plaintext next to the key, or baked into an environment variable, or fed in by a script, provides the inconvenience of encryption with none of the protection. The honest posture for a service identity is an unencrypted private key whose protection comes from filesystem permissions and a secrets-management system — not from a passphrase that has to be available to an automated process anyway.

So the split is deliberate: human identities get passphrase-encrypted keys; service identities get unencrypted keys locked down by permissions. This is not a shortcut. It is the correct model, and pretending a service key is protected by a passphrase it must hand to a script at startup is the actual shortcut.

The certificate authority built earlier in this series — a root in Azure Key Vault, an intermediate on the West Europe node — exists precisely so that new identities can be minted on demand. This is where that investment is collected: issuing these six is not new PKI work but the CA doing the single job it was built for, six times. Each certificate follows the same three-step sequence the operator certificate followed: generate a key, create a signing request carrying the identity's distinguished name, sign it with the intermediate under the client-certificate profile. The mechanics are routine. What makes them worth attention is what the profile and the signing policy quietly enforce.

That profile matters. The intermediate's configuration defines a v3_client_cert extension block that stamps each issued certificate with basicConstraints: CA:FALSE (this is a leaf, it can sign nothing), keyUsage: digitalSignature (it authenticates, it does not encrypt), and — the critical one — extendedKeyUsage: clientAuth (it is a client credential, not a server identity). MongoDB will only accept a certificate as a client identity if it carries clientAuth, so the profile is what makes these certificates usable for authentication at all.

The CA's signing policy also enforces the namespace discipline. Its policy_strict block requires that every issued certificate match the CA's organization and domain components exactly, supplying only the organizational unit and common name per leaf. This is why all six identities necessarily carry O=Meetadev, DC=mongo, DC=internal — the CA refuses to sign anything that does not — and differ only in the common name. The structure of the access model is enforced by the certificate authority itself, not merely by convention.

The human certificates are issued first, each with a passphrase set on its key at generation. The four service certificates follow with unencrypted keys. Each signing operation requires the intermediate CA's own passphrase and an explicit confirmation, so the four service certificates are generated as keys and requests in a batch but signed individually — the signing step is the one place a human deliberately authorizes each new identity into existence, and that should not be automated away.

When the six are issued, the CA's own ledger records them. The index.txt database — the authoritative, append-only record of every certificate this authority has ever signed — now shows ten entries: the three node certificates and the operator certificate from earlier parts, followed by the six new role identities, each marked V for valid, each with its serial number and full subject.

A detail worth understanding, because it surfaces in that ledger and trips up the unprepared: the filename column in the OpenSSL database reads unknown for every entry. This is not an error or a missing record. The openssl ca command simply does not populate that optional field; it tracks certificates by serial number and subject, which is what matters. Every CA database in the world looks like this. Recognizing it as normal — rather than chasing a phantom problem — is the kind of small fluency that distinguishes someone who has actually operated a CA from someone following a tutorial.

Five of the six identities map cleanly onto built-in roles. The monitoring identity does not, and the reason is the most instructive design point in the build.

A monitoring system needs to read the cluster's operational state — how the server is performing, whether replication is healthy, what the connection pool looks like, how large each collection is. It does not need, and for a financial system must not have, the ability to read the contents of those collections. The built-in clusterMonitor role grants the cluster-level read access but is a blunt instrument; what the situation calls for is a role that draws the line exactly at the boundary between metrics about data and the data itself.

That role has to be defined explicitly, action by action:

The first privilege block grants cluster-scoped diagnostic actions — server status, replica set status, connection pool statistics, command-line options, parameters, host information, operation visibility. These are the vital signs of a running cluster. The second block grants statistical actions across all databases and collections: the sizes, the document counts, the index usage. The empty database and collection strings mean "every namespace," and that breadth is intentional — monitoring should see the shape of all data.

What the role conspicuously does not grant is find. The monitoring identity can determine that a collection holds ten million documents and occupies four gigabytes; it cannot read what any one of those documents contains. That single omission is the entire point. Metrics, yes. Data, never. For a system holding financial transactions, a monitoring credential that could also read transactions would be a backdoor wearing an observability badge, and defining the role by explicit action is what closes that door precisely rather than approximately.

This is also where the cost of the two-tier PKI built earlier pays for itself. Having a real certificate authority means each of these roles can be bound to its own attributable certificate; having the discipline to define a custom role means the binding is to exactly the privilege the job needs. Neither alone is sufficient. Together they are what "least privilege" actually means in practice.

With the custom role defined, the six users are created — each one binding a certificate's distinguished name to a role. Under x.509 authentication there is no password to set; the "user" is the certificate subject, and creating the user is the act of telling the cluster which role that subject carries. The username must be the subject distinguished name exactly as MongoDB derives it from the certificate, in the canonical RFC 2253 order — leaf component first, root component last — or authentication will succeed at the transport layer and then fail at the lookup, because the name on the certificate and the name in the database will not match byte for byte. Each user is therefore created with its DN in that exact canonical form.

The two human identities encode the separation of duties directly in their role grants. The cluster administrator receives cluster and database administration:

The user administrator receives user management alone:

The four service identities follow the same pattern, each scoped to its function: the application receives readWrite on meetadev_ledger; the backup identity receives the built-in backup role on admin; the monitoring identity receives the custom meetadevMonitor role; the analytics identity receives read on meetadev_ledger. Note the deliberate gradient between the application and analytics identities — both touch the same single database, but one may write and the other may only read. Same namespace, different verbs, different certificates.

Listing the resulting users shows the complete model in one view: seven $external identities, each authenticating by certificate (the mechanism reads external for every one, confirming there is not a password anywhere in the set), each bound to precisely the roles its job requires. The original operator identity remains, still holding root — for now.

Everything to this point is a set of claims. The cluster says the application can only readWrite one database; it says the cluster administrator cannot create users. A claim is not a control. The control is the cluster refusing the action when the identity reaches beyond its grant — and the only way to know the control exists is to reach beyond the grant and watch the refusal.

What follows is four pairs of demonstrations. Each pair shows an identity doing the job it was scoped for, and then being structurally refused something outside that scope. The refusals are not configuration we added; they are the natural consequence of the roles, surfacing as authorization errors. This is the section that converts an access model from described to proven.

Authenticated as the cluster administrator, reading the replica set configuration succeeds — rs.conf().members.length returns 3, exactly the kind of operational visibility the role exists to provide. Then, attempting to create a user:

The cluster administrator — an identity with real, substantial power over the cluster — is refused the creation of a user. Not asked not to. Refused. To provision an identity, you must hold the user administrator's certificate, which this is not.

The inverse identity demonstrates the other half of the separation. Authenticated as the user administrator, creating a user succeeds — the exact action the cluster administrator was just refused — and dropping it succeeds too, confirming full lifecycle control over identities. Then, attempting a cluster operation:

Read together with the previous pair, the separation is complete and symmetric: the cluster administrator operates the cluster and is refused user creation; the user administrator creates users and is refused cluster operations. Neither human identity can do the other's job. There is no single certificate in the system that can both grant access and exercise administrative control — which is exactly the property an audit of a money-moving system is looking for.

This is the blast-radius proof, and it is the most important demonstration for the financial framing. Authenticated as the application identity, writing a transaction to the ledger succeeds:

The insert is acknowledged, the document reads back in full, and the meetadev_ledger database now exists with its first transaction. The application can do precisely its job. Then, three probes outward, each testing a different direction of escape, each refused:

Reading the cluster's user list — an attempt to enumerate other identities — fails: not authorized on admin. Reading a different database entirely — config, the cluster's own metadata — fails: not authorized on config. Running a server-status command — an attempt at administrative visibility — fails: not authorized on meetadev_ledger. Three directions, three walls, three Unauthorized. The application identity can read and write the ledger and do nothing else. It cannot enumerate users, cannot reach another namespace, cannot inspect server internals. This is the containment guarantee that makes the financial framing real rather than aspirational: a full compromise of the application — the most exposed component in any system, the one reachable from the internet — yields the attacker read and write access to the ledger and a wall in every other direction. The damage is bounded by design, and the boundary is the role.

The custom role's payoff. Authenticated as the monitoring identity, reading server status succeeds — serverStatus returns ok: 1, the exact health visibility the role was built to provide, and notably the same command the application was refused moments earlier. Reading collection statistics succeeds too: the transaction count comes back as 1. The monitor can see that the ledger holds one transaction and how large the collection is. Then, attempting to read an actual document:

Health: visible. Metrics: visible. Document contents: refused. This is the line the custom role was defined to draw, enforced exactly where it was drawn. The monitoring identity has everything an observability stack needs and no path to a single transaction's contents. A built-in role could not have produced this boundary; defining the role action by action did. For a system whose data is financial transactions, a monitoring credential that stops precisely at the data is not a nicety — it is the difference between an observability integration and an exfiltration path.

The backup and analytics identities round out the set, each authenticating cleanly and carrying exactly its intended role — backup for the one, read on the ledger for the other. With all six demonstrated, the model is no longer a description. It is a set of boundaries the cluster has refused, in front of us, to let us cross.

The last act of building a least-privilege model is to apply it to the identity that built the model.

The operator certificate that bootstrapped this cluster holds root, and it had to — it created the replica set, the first identity, the custom role, all six users. But with the scoped identities now covering every real task, root has no remaining day-to-day function. An unrestricted certificate left in routine circulation is the precise standing risk the entire model exists to eliminate. The discipline the build enforced on the application and the monitor has to be enforced on the operator too, or it was never really a principle.

The financial-grade posture for root is break-glass: it is not destroyed, it is sealed. The certificate moves out of the everyday working directory into separated, locked-down storage, and it is never used for routine work again. Its user record stays in place — removing it would also remove the recovery path it exists to provide — but the certificate itself becomes a fire-axe behind glass, used only for genuine emergencies, and any use of it is a deliberate, audited event rather than a casual reach. From here forward, every routine operation uses a scoped identity. The certificate that can do anything is the one certificate you resolve never to use.

Here honesty requires naming the gap, in the same spirit as this series' standing practice of inventorying what a layer is not. Moving the operator certificate into a locked directory on a workstation is the gesture of vaulting, not the substance of it. True break-glass storage is offline and physically separated — a hardware token, a sealed credential in a safe, an air-gapped device — such that using root requires a deliberate physical act, not merely a change of directory. The relocation enforces the operational discipline (do not reach for root casually) but not the cryptographic guarantee (root is unreachable without breaking a physical seal). And because this PKI deliberately does not publish certificate revocation lists — a gap named when the CA was built — retiring root here is break-glass by discipline, not break-glass by cryptographic revocation. A production financial system would want the latter: a real offline vault and a working revocation path. What is implemented here is the correct operational pattern with its boundaries stated plainly, which is the honest distance between a demonstration and a fully hardened system.

The cluster now has a real access model. Six identities, each a distinct certificate, each bound to the minimum privilege its job requires, each attributable by name in the audit log. Separation of duties between the two human roles, enforced by the cluster and demonstrated in both directions. A custom role drawing the metrics-without-data line exactly where a financial system needs it. The application contained to a single database, its compromise bounded by design. And root retired from routine use, least privilege applied even to the identity that built everything. Every claim in that paragraph was demonstrated against the running cluster, not asserted.

What it is not, yet, is a complete operational security posture. The roles are static; a mature system rotates the certificates behind them on a schedule and has a tested path to revoke one urgently — which this PKI, by its own stated design, cannot yet do cryptographically. The audit log now carries attributable identities, but nothing in this build consumes that log, alerts on anomalous use, or retains it to a compliance standard. The break-glass storage is a gesture toward vaulting rather than the offline reality. And the access model governs who may do what, but says nothing yet about the guarantees the data itself carries under concurrent, multi-region writes — whether a posted transaction is durable across regions, whether a read reflects the latest write, what happens to an in-flight write when a region fails.

That last gap is the next piece of the build: read concerns, write concerns, and the durability guarantees a financial transaction actually requires when the cluster spans three regions and the network between them is neither instant nor reliable. The access model decides who may write to the ledger. The next question is what a write to the ledger is allowed to mean.

The certificate authority that issued all six of these identities was built earlier in this series — the two-tier PKI with the root in Azure Key Vault and the intermediate on the wire. The single root identity these six replaced was created when the cluster first came alive — the cluster learns to say yes. That build taught the database to say yes to one identity. This one taught it to say yes to six, each only to exactly what it needs — and to say no, in front of us, to everything else.

Building something similar, or want to argue with the choices here? I run a community for MongoDB builders on Meetadev — MongoDB AI Builders — where this series and the decisions behind it get discussed in the open. Come break things with us. 🔧