Learn

Cosmos DB Design for Integrations

The main Cosmos DB design mistakes in Entra-adjacent systems are usually made before the first production incident: the wrong partition key, documents that blur several access patterns together, and a change-feed plan that assumes downstream processing will stay simple forever. This page focuses on those design pressures for integration workloads, not on Cosmos DB as a general-purpose schema design topic.

Read this together with Cosmos DB for Identity State for the baseline mental model and State and Artifacts for the concrete split between workflow state and larger payloads.

Start From The Workflow Boundary

Cosmos DB is usually carrying state that Entra and Microsoft Graph do not own for you. In identity-oriented integrations, that often means:

  • onboarding or remediation progress,
  • reconciliation snapshots and mismatch records,
  • per-tenant connector state,
  • checkpoint positions for polling or export flows,
  • ownership or lease information for workers.

The important question is not just what the document looks like. The important question is which unit of work reads and writes that document repeatedly.

For most Entra-adjacent integrations, the useful boundaries are things like tenant, connector, workflow instance, or target system. Those are better starting points than generic IDs chosen without an access pattern behind them.

Partition Key Consequences Show Up Late

Partition-key mistakes often stay hidden until the system becomes multi-tenant, noisy, or expensive.

Good partition keys in this topic usually do one of two things:

  • keep the records for one operational owner together, or
  • keep the records for one active workflow together.

That often points to keys such as:

  • tenantId for tenant-scoped state,
  • connectorId for integration-specific progress,
  • workflowId when a case or onboarding journey is the real ownership boundary.

Weak keys usually fail one of two ways:

  • Hot partitions because a single tenant or connector receives disproportionate traffic.
  • Cross-partition query pressure because the data you need together is scattered everywhere.

The trade-off matters in identity systems because “multi-tenant” and “bursty” are common together. A partition key that looks balanced in a test environment can become noisy when one enterprise tenant starts driving most of the writes.

Multi-Tenant Design Needs An Honest Default

For many Entra-adjacent integration systems, tenantId is the most natural partition key because operators, reporting, and failure ownership are usually tenant-scoped.

But it is not automatically correct.

Use tenantId when:

  • most reads and writes stay within one tenant,
  • operators investigate failures by tenant,
  • per-tenant data volume is unlikely to create a persistent hot partition.

Use something narrower like workflowId or connectorId when:

  • one tenant can produce very large bursts,
  • several independent workflows exist inside the same tenant,
  • write amplification within a tenant is the dominant scaling problem.

The important part is to decide based on who owns the data operationally and where the write pressure really lands.

RU Cost Pressure Comes From Chattiness

Cosmos DB becomes expensive in identity workloads when the system quietly turns every small step into a document rewrite or a fan-out query.

Common RU pressure points include:

  • workers repeatedly upserting large state documents,
  • dashboards querying across partitions for recent failures,
  • checkpoint records being updated too frequently,
  • change-feed handlers writing several secondary records for each input change,
  • wide indexing on properties that are rarely queried.

The cheapest design is usually not the most normalized one. It is the one that keeps the common reads and writes targeted, keeps documents compact, and avoids cross-partition scans for routine operations.

Practical ways to reduce pressure:

  • keep workflow state documents small and purpose-built,
  • store large artifacts in Azure Storage Basics instead of embedding them,
  • index for the queries operators and workers actually run,
  • separate high-churn records from slower-moving summary records when one document shape is doing too many jobs.

Document Shape Should Match Operational Questions

Identity integrations often need to answer a narrow set of questions quickly:

  • what step is this tenant onboarding case in,
  • which connector owns this failure,
  • what was the last successful Graph checkpoint,
  • has this downstream entitlement already been applied.

If those questions require reconstructing several unrelated documents or running broad queries on every request, the design is already drifting away from the operational model.

That does not mean everything belongs in one document. It means the dominant operational read paths should be cheap and obvious.

Consistency Is A Workflow Contract Decision

Consistency choice should follow the failure cost of stale reads.

For many Entra-adjacent workflows, session consistency is the practical default because:

  • a writer can observe its own updates,
  • the workflow keeps reasonable freshness,
  • the system avoids the strongest global constraints.

Stronger consistency is worth considering when a narrowly scoped coordination rule really depends on it, but teams should treat that as a deliberate cost and flexibility trade-off, not a safe default.

Weaker consistency can be acceptable when the data is supporting analytics, reconciliation views, or eventually corrected summaries. It is much less acceptable when two workers could make contradictory decisions because one read stale state.

The useful question is: what breaks if this reader is briefly behind?

Change Feed Is Powerful, But It Changes Downstream Design

The change feed is attractive because it lets downstream processors react to item changes without polling the container for everything. In identity-oriented integrations, that can be a strong fit for patterns like:

  • a Graph export job writes checkpointed results,
  • a container change triggers enrichment,
  • another processor creates an artifact or analytics projection.

But the change feed also pushes certain consequences downstream:

  • consumers need their own checkpointing and recovery model,
  • duplicate processing still needs to be tolerated,
  • the order is meaningful within the feed’s partitioned model, not as one global sequence for the whole container,
  • downstream services inherit the shape and frequency of state changes whether or not that is the best event contract for them.

That last point matters most. If the downstream system really wants a workflow command with retries, operator inspection, or dead-lettering, the clearer fit is often Service Bus for Workflows. If it wants to react to durable state transitions, the change feed can be a clean model.

Do Not Turn Change Feed Into A Hidden Broker

The design starts to go wrong when teams use one container as a catch-all event source for every downstream need.

Warning signs:

  • application state updates exist mainly to trigger consumers,
  • several unrelated processors depend on the same document mutation pattern,
  • operators need broker-like remediation but the system only has state changes,
  • downstream teams are forced to infer business events from generic document updates.

The change feed works best when the state transition itself is the event. It works worse when the real need is explicit workflow messaging.

Integration-Specific Patterns That Age Well

These patterns usually hold up better in this topic’s scope:

  • Keep workflow state in Cosmos DB and artifacts in Storage.
  • Use one container shape for active operational state and separate projections only when the query pressure proves it is necessary.
  • Choose a partition key that matches either the tenant or the workflow boundary, not a generic document identity with no operational meaning.
  • Let change feed trigger downstream work only when the state change is the natural source of truth.

These patterns usually age badly:

  • one giant per-tenant document rewritten by every step,
  • wide cross-partition queries for routine operational views,
  • embedding export payloads or audit files in state documents,
  • using change feed where a real command broker was needed.

Practical Recommendation

For Entra-adjacent integrations, design Cosmos DB around the operational owner of the state first, then validate the RU and downstream consequences of that choice. Prefer session consistency as the normal default, keep high-churn state compact, and use change feed only when downstream consumers genuinely want state transitions rather than brokered commands.

If the design starts to look like a queue, a file store, and an analytics sink all inside one container, stop and split the responsibilities before cost and recovery pressure force the issue.