Learn

Diagnostics with Data Explorer

Azure Data Explorer becomes valuable when an Entra-adjacent system has too much event and diagnostic data for service-by-service inspection, but the team still needs fast operational answers. The point is not to build a full observability platform. The point is to give builders a place to ask investigative questions across large event sets using Kusto Query Language (KQL).

Read this after Azure Data Explorer and KQL for the baseline framing and Event Stream to Data Explorer for the ingestion pattern.

[!IMPORTANT] Scope guard: this page does not try to cover AKS, Azure SQL, API Management, or full observability-platform design for the initial release. It stays focused on Azure Data Explorer and KQL as an investigative surface for Entra-adjacent systems.

What Builders Actually Use It For

Builders normally reach for Data Explorer after the question stops being “did this one function invocation fail” and becomes “what is happening across tenants, connectors, or time windows?”

Typical uses include:

  • investigating repeated provisioning failures across many runs,
  • correlating Graph-driven automation problems by tenant or connector,
  • exploring retry storms from worker fleets,
  • checking whether a deployment changed event shape or timing,
  • comparing current behavior with earlier event windows.

Those are operational analytics questions. They require fast filtering, grouping, summarizing, and correlation over a lot of data.

Data Explorer Is Downstream, Not The Control Plane

In this topic, Data Explorer usually sits after the core workflow services have already done their job:

  • Graph or another source emits events or logs,
  • workers enrich and forward them,
  • Event Hubs or another ingestion path lands them,
  • Data Explorer stores the event history for KQL queries.

That means Data Explorer is not where the workflow is coordinated, retried, or made authoritative. It is where operators and builders inspect the evidence after the system runs.

KQL Patterns That Matter In Practice

The useful KQL patterns for this topic are rarely complicated. They are the everyday building blocks of investigation.

Narrow the time window first

Start by reducing the search space aggressively:

IdentityEvents
| where Timestamp > ago(2h)
| where TenantId == "contoso"
| limit 50

This keeps investigations responsive and makes later filters easier to trust.

Summarize before reading raw rows

When the event volume is high, summarize first to see where the problem clusters.

IdentityEvents
| where Timestamp > ago(24h)
| summarize Failures = countif(Status == "Failed") by Connector, TenantId
| top 20 by Failures desc

This helps builders find whether the issue is one connector, one tenant, or the whole estate.

Correlate by workflow or request identity

If the telemetry carries a workflow, request, or correlation identifier, use it early.

IdentityEvents
| where Timestamp > ago(6h)
| where WorkflowId == "wf-12345"
| project Timestamp, Stage, Status, TenantId, ErrorCode, CorrelationId
| order by Timestamp asc

This is often the fastest way to turn noisy logs into a readable operational story.

Compare failure shapes over time

Trend views matter when the question is whether something regressed after a deployment or configuration change.

IdentityEvents
| where Timestamp > ago(7d)
| summarize Failures = countif(Status == "Failed") by bin(Timestamp, 1h)
| render timechart

The value here is operational pattern detection, not just raw query output.

Investigative Questions Data Explorer Answers Well

In Entra-adjacent systems, Data Explorer is especially good for questions like:

  • which tenants are seeing the highest failure rate,
  • which connector version introduced a burst of retries,
  • whether one Graph operation or one downstream target is dominating incidents,
  • how long a delay or backlog persisted,
  • whether duplicate processing is isolated or systemic.

Those questions are larger than a single application log and smaller than a whole enterprise observability strategy. That middle ground is where Data Explorer fits best in this topic.

Operational Analytics vs Broader Observability

This distinction needs to stay visible.

Operational analytics in this topic means exploring landed event and telemetry data to understand system behavior, failures, and trends.

Broader observability usually means a wider platform concern: dashboards, alerts, traces, SLOs, service maps, and estate-wide instrumentation strategy.

Data Explorer can overlap with observability work, but this page is about the first category. It helps builders inspect and reason about event-heavy identity systems. It does not try to define a complete monitoring architecture.

Where Application Insights Fits Nearby

Application Insights often remains useful for service-local telemetry, request traces, and near-source debugging. That is adjacent to this page, but it is not the main surface here.

The practical relationship is:

  • use Application Insights when you need service-level request or dependency detail close to the application runtime,
  • use Data Explorer when the investigation spans large event sets, many workers, many tenants, or landed telemetry over time.

That keeps the tools complementary instead of pretending one should replace the other.

Failure Patterns To Watch For

Data Explorer helps investigate failures, but it also has its own design traps.

Common issues include:

  • event payloads that are too inconsistent to query cleanly,
  • missing correlation fields that make cross-service analysis guesswork,
  • retaining more data than the investigation model really needs,
  • landing diagnostic data that still needs heavy cleanup before it is usable,
  • treating ad hoc KQL as a substitute for durable workflow state.

Mitigation patterns:

  • standardize the core fields builders investigate most often,
  • carry tenant, connector, workflow, and correlation identifiers consistently,
  • separate operational diagnostics from authoritative state,
  • keep Data Explorer focused on analysis rather than workflow coordination.

Practical Recommendation

Use Azure Data Explorer when Entra-adjacent events, logs, and telemetry have grown beyond one-service troubleshooting and the team needs fast KQL-based investigation across the dataset. Keep it downstream from the workflow, keep the event shape queryable, and keep the scope bounded to operational analytics rather than expanding into a full observability guide.