vRA 7.3 PhysicalHostPingService: ignoring exception. IAAS Service error MSDTC Issue

Pretty much EVERY time I install vRA Version 7.* I always get an error relating to the vRA IAAS Service.

It stops Endpoint Compute Services from being displayed, stops data collection of all added endpoints and generally makes vRA a paperweight.

The error can be found under Infrastructure>Monioring, you’ll know you have issues if you have hundreds\thousands of the same error!

5.JPG

The full error looks like this…

6.JPG

The error is related to the MS DTC service, if your Windows SQL Server and Windows IAAS Manager Servers have been rolled out from the same vSphere template then you will likely get the error, I read a large number of blogs that say this only happens if Sysprep hasn’t been run, however I see this issue whether SYSPREP is run or not. The GUID of MS DTC doesn’t seem to change and the duplicate GUID entries of the SQL & IAAS Manager Server results in the issue.

Uninstalling\Re-Installing\Re-Configuring MS DTC seems to be the most appropriate resolution to this issue, I run the following commands on all IAAS Manager Servers and SQL Servers. I then restart The SQL, IAAS MAnager & vRA Appliance for good measure.


Uninstall-Dtc -confirm:$false


Install-Dtc


Set-DtcNetworkSetting -DtcName "Local" -RemoteAdministrationAccessEnabled:$False -RemoteClientAccessEnabled:$True -InboundTransactionsEnabled:$True -OutboundTransactionsEnabled:$True -LUTransactionsEnabled:$True -XATransactionsEnabled:$False -AuthenticationLevel Mutual -Confirm:$False

vCenter 6u2a Upgrade failure with error 3010

I had an issue upgrading a VC to 6.0u2a recently, when running the upgrade I received the error “failed with error code 3010”

1

There’s a VMware KB here which explains the issue, however the resolution description is light on detail.

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2149266

Check the pkgmgr-comp-msi.log to find out what DLL is locked; the log files will be in the ZIP file downloaded when the installation fails.

I found it useful to search for “is being held in use” within this log file to identify the dll being held.

2

In my case it was vmeventmsg.dll.

I stopped the vCenter Service (which in turn stops all other services that the VC is a decency of)

Using process explorer

Download – https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx

Open process explorer and select “show details for all processes”  under file.

3

Under view ensure “Show processes from all users” is selected.

4

Select  Find > find handle or ddl (ctrl+f) and seach for the dll identified in the pkgmgr-comp-msi.log, the PID will be listed (in my case 912)

5

In process explorer kill the process that’s holding the dll hostage, by right clicking and selecting “Kill Process”

6

NOTE – I first tried using taskkill to end the process however it didn’t work, even though it would be listed when running tasklist.

Double check in processes explorer that the DLL is no longer in use.

Re-run the VC upgrade, go to the Winchester, have a nice cold pint, and wait for all of this to blow over.

7