Password hash synchronization agent is continuously getting RPC error from domain ""

When changed user password in on-prem AD, it does not reflect in Office365.

Run AADC troubleshoot and got these errors

------------------------------------------start---------------------------------------------------------

AD Connector - abc.ad

Password Hash Synchronization is enabled

Latest Password Hash Synchronization heartbeat is detected at: 05/21/2020 09:19:18 UTC

        Directory Partitions:

        =====================

        Directory Partition - abc.ad

        Password Hash Synchronization agent is continuously getting failures for domain "abc.ad"

        Please check 611 error events in the application event logs for details

        The latest 611 error event for the domain "abc.ad" is generated at: 05/21/2020 09:37:26 UTC

        Password Hash Synchronization agent is continuously getting RPC errors from domain "abc.ad"

        Please setup reliable preferred domain controllers. Please see "Connectivity problems" section at https://clicktime.symantec.com/3RkdZT5JN8p6wng3WtLAJGz7Vc?u=https%3A%2F%2Fgo.microsoft.com%2Ffwlink%2F%3Flinkid%3D847231

        Please check 611 error events in the application event logs for details

        The latest RPC error event for the domain "abc.ad" is generated at: 05/21/2020 09:37:26 UTC

        Last successful attempt to synchronize passwords from this directory partition started at: 5/21/2020 9:03:09 AM UTC and ended at: 5/21/2020 9:03:10 AM UTC

        Only Use Preferred Domain Controllers: False

        Checking connectivity to the domain...

        Domain "abc.ad" is reachable

        Directory Partition - child.abc.ad

        Last successful attempt to synchronize passwords from this directory partition started at: 5/21/2020 9:37:26 AM UTC and ended at: 5/21/2020 9:37:26 AM UTC

        Only Use Preferred Domain Controllers: False

        Checking connectivity to the domain...

        Domain "child.abc.ad" is reachable

-------------------------------------end------------------------------------------------------------------------------------------

Below are error from Event Viewer. Event ID 611

------------------------------------start--------------------------------------------------------------------------

System 

  - Provider 

   [ Name]  : Directory Synchronization 
 
  - EventID : 611 

   [ Qualifiers]  : 0 
 
   Level : 2 
 
   Task : 0 
 
   Keywords : 0x80000000000000 
 
  - TimeCreated 

   [ SystemTime]  : 2020-05-21T04:33:31.498363500Z 
 
   EventRecordID : 364538 
 
   Channel : Application 
 
   Computer : aadc-server 
 
   Security 
 

- EventData 

   Password hash synchronization failed for domain: abc.ad, domain controller hostname: dc01.child.abc.ad, domain controller IP address: xxx.xxx.xxx.xxx. Details: Microsoft.Online.PasswordSynchronization.SynchronizationManagerException: Recovery task failed. ---> Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsException: RPC Error 8465 : The replication synchronization attempt failed because a master replica attempted to sync from a partial replica. There was an error calling _IDL_DRSGetNCChanges. at Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsRpcConnection.OnReplicateSingleObject(DsName directoryName) at Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsConnection.ReplicateSingleObject(Guid objectGuid, String distinguishedName) at Microsoft.Online.PasswordSynchronization.RecoveryTask.<>c__DisplayClass9_0.<RetrieveObjectChangesFromAD>b__1(IDrsConnection c) at Microsoft.Online.PasswordSynchronization.RetryUtility.ExecuteWithRetry[T](Func`1 operation, Func`1 shouldAbort, RetryPolicyHandler retryPolicy) at Microsoft.Online.PasswordSynchronization.RecoveryTask.RetrieveObjectChangesFromAD(List`1 retryObjects) at Microsoft.Online.PasswordSynchronization.RecoveryTask.SynchronizeCredentialsToCloud() --- End of inner exception stack trace --- at Microsoft.Online.PasswordSynchronization.RecoveryTask.SynchronizeCredentialsToCloud() at Microsoft.Online.PasswordSynchronization.PasswordSynchronizationTask.SynchronizeSecrets() at Microsoft.Online.PasswordSynchronization.SynchronizationExecutionContext.SynchronizeDomain() at Microsoft.Online.PasswordSynchronization.SynchronizationManager.SynchronizeDomain(SynchronizationExecutionContext syncExecutionContext) Microsoft.Online.PasswordSynchronization.SynchronizationManagerException: Recovery task failed. ---> Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsException: RPC Error 8465 : The replication synchronization attempt failed because a master replica attempted to sync from a partial replica. There was an error calling _IDL_DRSGetNCChanges. at Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsRpcConnection.OnReplicateSingleObject(DsName directoryName) at Microsoft.Online.PasswordSynchronization.DirectoryReplicationServices.DrsConnection.ReplicateSingleObject(Guid objectGuid, String distinguishedName) at Microsoft.Online.PasswordSynchronization.RecoveryTask.<>c__DisplayClass9_0.<RetrieveObjectChangesFromAD>b__1(IDrsConnection c) at Microsoft.Online.PasswordSynchronization.RetryUtility.ExecuteWithRetry[T](Func`1 operation, Func`1 shouldAbort, RetryPolicyHandler retryPolicy) at Microsoft.Online.PasswordSynchronization.RecoveryTask.RetrieveObjectChangesFromAD(List`1 retryObjects) at Microsoft.Online.PasswordSynchronization.RecoveryTask.SynchronizeCredentialsToCloud() --- End of inner exception stack trace --- at Microsoft.Online.PasswordSynchronization.RecoveryTask.SynchronizeCredentialsToCloud() at Microsoft.Online.PasswordSynchronization.PasswordSynchronizationTask.SynchronizeSecrets() at Microsoft.Online.PasswordSynchronization.SynchronizationExecutionContext.SynchronizeDomain() at Microsoft.Online.PasswordSynchronization.SynchronizationManager.SynchronizeDomain(SynchronizationExecutionContext syncExecutionContext) . <forest-info><partition-name>abc.ad</partition-name> <connector-id>6d0e17a4-f299-47a7-af40-4f536ccbfda2</connector-id> </forest-info>