Troubleshooting
Cloud Sync troubleshooting differs fundamentally from Connect Sync. There is no on-premises sync engine to debug, no Synchronization Service Manager, no metaverse search. Instead, troubleshooting centers on two things: the agent’s connectivity and the cloud provisioning service’s logs.
This page covers the most common failure patterns and how to resolve them.
Troubleshooting Model
flowchart TD
A[Sync problem detected] --> B{Agent visible in portal?}
B -- No --> C[Check agent installation and services]
B -- Yes --> D{Agent status healthy?}
D -- No --> E[Check network, ports, proxy config]
D -- Yes --> F{Objects syncing?}
F -- No --> G{Job in quarantine?}
G -- Yes --> H[Check quarantine error code]
G -- No --> I[Check scoping filters and provisioning logs]
F -- Yes but wrong --> J[Check attribute mappings and expressions]
C --> K[Reinstall or re-register agent]
E --> L[Fix connectivity, restart agent service]
H --> M[Resolve quarantine cause, clear or restart]
I --> N[Fix scoping or filter rules]
J --> O[Fix mappings, run on-demand provisioning]
Agent Problems
The provisioning agent is the on-premises component. When it fails, nothing syncs. Start here when troubleshooting any Cloud Sync issue.
Verify Agent Health
Check the agent from two sides:
In the Entra admin center:
- Navigate to Identity > Hybrid management > Microsoft Entra Connect > Cloud sync.
- Look for your agent(s). A healthy agent shows Active (green) status.
- If the agent is missing or shows inactive status, the problem is on-premises.
On the local server:
- Open Windows Services (
services.msc). - Verify both services are running:
- Microsoft Entra Provisioning Agent - the sync agent itself
- Microsoft Entra Connect Agent Updater - handles automatic updates
If either service is stopped, start it. If it fails to start, check the next section.
Agent Fails to Start
The most common cause: group policy prevents the service account NT SERVICE\AADConnectProvisioningAgent from logging on as a service.
Resolution:
- Open the service properties, go to the Log On tab.
- Change the account to a domain admin temporarily.
- Restart the service.
- Fix the underlying group policy so the NT SERVICE account has log-on-as-a-service rights.
Agent Registration Fails
Two common patterns:
Certificate or timeout error during registration. The agent cannot reach the hybrid identity service. This is almost always a network or proxy issue. Configure an outbound proxy by editing C:\Program Files\Microsoft Azure AD Connect Provisioning Agent\AADConnectProvisioningAgent.exe.config:
<system.net>
<defaultProxy enabled="true" useDefaultCredentials="true">
<proxy
usesystemdefault="true"
proxyaddress="http://proxy-server:port"
bypassonlocal="true"
/>
</defaultProxy>
</system.net>
Add this before the closing </configuration> tag and restart the agent service.
Security error during registration. PowerShell execution policy is blocking the registration scripts. Set the execution policy to RemoteSigned (not Unrestricted, which also triggers this error):
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope LocalMachine
Network Requirements
The agent needs outbound access to these endpoints:
| Port | Destination | Purpose |
|---|---|---|
| 443 | *.msappproxy.net, *.servicebus.windows.net | Agent-to-cloud communication |
| 443 | login.windows.net, *.microsoftonline.com, *.msauth.net | Authentication and registration |
| 80 | crl3.digicert.com, crl4.digicert.com, crl.microsoft.com | Certificate revocation checks |
If your firewall does domain-based filtering, allow these wildcard domains. If it does IP-based filtering, use the Azure IP ranges and service tags (updated weekly).
Do not use TLS inspection or termination on traffic between the agent and Azure. This breaks the Service Bus connection.
Agent Log Collection
Agent trace logs are stored in C:\ProgramData\Microsoft\Azure AD Connect Provisioning Agent\Trace. For detailed log collection, use the AADCloudSyncTools PowerShell module:
Import-Module AADCloudSyncTools
Export-AADCloudSyncToolsLogs -TracingDurationMins 5
This captures verbose logs for the specified duration and saves them to your Documents folder. Use -OutputPath to specify a different location.
Sync Error Patterns
Once the agent is healthy, sync errors manifest in the provisioning logs (Entra admin center > Identity > Monitoring & health > Provisioning logs).
Common Error Codes
| Error Code | What It Means | Resolution |
|---|---|---|
HybridIdentityServiceNoAgentsAssigned | No agent registered for the domain | Re-install and register an agent |
HybridIdentityServiceNoActiveAgents | Agent installed but not running or unreachable | Check services, firewall, Service Bus connectivity |
HybridIdentityServiceAgentSignalingError | Service Bus cannot reach the agent | Check network; if persistent, contact support |
AzureActiveDirectoryInvalidCredential | Sync service account ADToAADSyncServiceAccount deleted or invalid | Run Repair-AADCloudSyncToolsAccount |
AzureActiveDirectoryExpiredCredentials | Sync service account credentials expired | Run Repair-AADCloudSyncToolsAccount |
AzureDirectoryServiceServerBusy | Entra ID throttling requests | Transient; wait and retry. Contact support if it persists >24 hours |
HybridIdentityServiceInvalidResource | Agent registration is corrupted | Re-register the agent and restart configuration |
TimeOut | Request to the identity service timed out (10 min) | Check agent and network performance |
Repairing the Sync Service Account
Several error codes point to the cloud sync service account being deleted or having expired credentials. Fix it with the AADCloudSyncTools module:
Import-Module AADCloudSyncTools
Connect-AADCloudSyncTools # Authenticate with Global Admin
Repair-AADCloudSyncToolsAccount
Skipped Objects
If objects are not syncing but no error appears, they may be filtered out. Check the provisioning logs for events with status Skipped. Common reasons:
IsCriticalSystemObjectisTRUE. Built-in AD accounts and groups are excluded by default.- Replication victim objects. Objects flagged as replication conflicts are skipped.
- Scoping filter mismatch. The object does not match the configured OU or attribute scoping filters.
- Synchronization schema restrictions. Additional exclusions may be defined in the sync schema.
Search provisioning logs by the object’s AD ObjectGuid to find the specific skip reason.
Quarantine
When the provisioning service detects consistent failures, it places the sync job in quarantine. This is a safety mechanism: rather than retrying the same failing operation thousands of times, the service pauses and alerts you.
How Quarantine Triggers
The provisioning service tracks the ratio of failed calls. If most or all calls to the target system fail consistently (for example, invalid credentials, expired tokens, authorization failures), the job enters quarantine.
You will see the quarantine status on the Cloud Sync configuration page in the Entra admin center. The status shows the error code and message.
Common Quarantine Causes
| Cause | Error Code | Fix |
|---|---|---|
| Sync service account deleted | AzureActiveDirectoryInvalidCredential | Repair-AADCloudSyncToolsAccount |
| Credentials expired | AzureActiveDirectoryExpiredCredentials | Repair-AADCloudSyncToolsAccount |
| Directory Sync service principal missing | AzureDirectoryServiceAuthorizationFailed | See authorization fix below |
| Mass deletion threshold hit | Delete threshold exceeded | See accidental deletion prevention below |
Resolving the Authorization Failure
The AzureDirectoryServiceAuthorizationFailed error means the sync service principal is missing from your tenant. Check first:
GET /servicePrincipals?$filter=displayName eq 'Microsoft Entra AD Synchronization Service'
If no result, trigger recreation:
PATCH /organization/<tenant-id>
Body: { "onPremisesSyncEnabled": true }
Allow up to 24 hours for the authorization errors to clear after restoring the service principal.
Clearing Quarantine
Two options:
-
Clear quarantine. Right-click the configuration status and select Clear quarantine. This resets the watermark and runs a delta sync.
-
Restart the provisioning job. On the agent configuration page, select Restart sync. Alternatively via MS Graph API:
POST /servicePrincipals/{id}/synchronization/jobs/{jobId}/restartYou can selectively clear escrows (the failure counter), quarantine status, or watermarks.
Fix the underlying cause before clearing quarantine, or the job will re-enter quarantine on the next cycle.
Accidental Deletion Prevention
Cloud Sync has a built-in safeguard against mass deletions. If the number of objects to be deleted in a single cycle exceeds a configured threshold, the sync job pauses and sends a notification email.
Configuration
In the Entra admin center, on your Cloud Sync configuration properties:
- Notification email - where alerts go
- Prevent accidental deletions - enable the feature
- Accidental deletion threshold - object count that triggers the pause
When It Triggers
The configuration status shows Delete threshold exceeded. The provisioning logs contain StagedDelete entries for each object that would have been deleted.
Resolution
If the deletions are expected (OU restructure, decommissioning), right-click the status and select Allow deletes. The next cycle processes the deletions.
If the deletions are unexpected, investigate the source:
- Check if an OU was accidentally removed from sync scope.
- Check if objects were moved in AD.
- Fix the root cause.
- Select Restart sync to resume.
Connect Sync Deletion Threshold Interaction
If you are running Connect Sync alongside Cloud Sync (coexistence), or recently migrated from Connect Sync, the Connect Sync accidental deletion threshold may also block Cloud Sync exports. This is the tenant-level threshold set by Connect Sync, separate from Cloud Sync’s own threshold.
If you no longer have a Connect Sync server to toggle this setting, use the Cloud Sync agent’s PowerShell module:
Import-Module AADCloudSyncTools
Disable-AADCloudSyncToolsDirSyncAccidentalDeletionPrevention -tenantId "your-tenant-id"
Only disable this after confirming the deletions are expected.
Duplicate Object Errors
The HybridSynchronizationActiveDirectoryUnexpectedDuplicateEntriesFound error occurs when a query returns multiple AD objects where one was expected. This commonly affects group provisioning to AD.
Diagnosing Duplicates
Find AD groups with duplicate msDS-ExternalDirectoryObjectId values:
$attr = "msDS-ExternalDirectoryObjectId"
$groups = Get-ADGroup -LDAPFilter "($attr=Group_*)" -Properties $attr
$duplicates = $groups |
Group-Object -Property $attr |
Where-Object { $_.Count -gt 1 }
foreach ($dup in $duplicates) {
Write-Host "Value: $($dup.Name) (Count: $($dup.Count))"
$dup.Group | Select-Object Name, DistinguishedName |
Format-Table -AutoSize
}
Remove the duplicate groups from AD to resolve the quarantine.
Password Writeback Issues
Cloud Sync supports password writeback (SSPR writes password changes back to on-premises AD). When it does not work, check these areas:
GMSA Permissions
The agent’s gMSA account needs write permissions on user objects, specifically the Unexpire Password extended right. After setting permissions with Set-AADCloudSyncPermissions, allow up to an hour for AD replication to propagate the ACL changes.
Permissions must be applied to This object and all descendant objects on the target OU.
Inheritance Disabled on User Objects
If specific accounts fail password writeback while others work, check whether permission inheritance is disabled on those AD user objects. The password write permission must flow down from the parent OU.
Minimum Password Age Policy
Group policy’s Minimum password age setting can block password writeback. If you are testing and need to reset passwords more than once per day, temporarily set this to 0:
Computer Configuration > Policies > Windows Settings > Security Settings > Account Policies
After changing, run gpupdate /force on the domain controllers and wait for replication.
For production, if the minimum password age is greater than 0, password resets will fail during that window, which is by design from AD’s perspective.
Comparing Troubleshooting: Cloud Sync vs Connect Sync
| Aspect | Cloud Sync | Connect Sync |
|---|---|---|
| Primary diagnostic tool | Provisioning logs in Entra admin center | Synchronization Service Manager on-prem |
| Agent health check | Portal status + local services check | Server health + SQL database health |
| Error visibility | Cloud provisioning logs, filterable by date and object | Event Viewer + sync run profiles |
| Quarantine concept | Yes, automatic with threshold | No (has accidental delete prevention but no job quarantine) |
| On-demand testing | On-demand provisioning (single object) | Preview in Sync Rules Editor |
| Log collection | Export-AADCloudSyncToolsLogs PowerShell | Event Viewer export + SQL queries |
| Service account repair | Repair-AADCloudSyncToolsAccount | Reconfigure AAD Connector credentials |
| Rule debugging | Check attribute mappings in portal | Synchronization Rules Editor with precedence |
The operational model is simpler with Cloud Sync. Most troubleshooting happens in the portal, not on a server. But the trade-off is less granular visibility into individual object join and projection states that the Synchronization Service Manager provides.
Quick Reference: AADCloudSyncTools Commands
| Command | Purpose |
|---|---|
Connect-AADCloudSyncTools | Authenticate to the service |
Export-AADCloudSyncToolsLogs | Capture detailed agent logs |
Repair-AADCloudSyncToolsAccount | Fix deleted or expired service account |
Disable-AADCloudSyncToolsDirSyncAccidentalDeletionPrevention | Disable tenant-level deletion threshold (from Connect Sync) |
Install the module from the agent installation directory, or follow the AADCloudSyncTools reference.
See Also
- Architecture - how the agent and provisioning service interact
- Configuration - setting up Cloud Sync correctly from the start
- Connect Sync operations and troubleshooting - server-side troubleshooting for the older engine
- Error codes reference - full list of Cloud Sync error codes