ESXi Host Sync Issue with vCenter – A general System error occurred: Failed to login with vim administrator password

Few weeks back, have experienced an ESXi host not responding state issue. The symptoms are –

  • Host reported as not responding in vCenter
  • Unable to connect the host directly through vSphere client
  • Not responding to any of the commands in DCUI
  • But VMs are running without any issues and accessible in network

ESXi host found enabled with Lockdown mode, SSH also disabled and no remote logging configured. So, there is no option available to connect it remotely during that scenario to troubleshoot.

Few days later, ESXi host starts responding to commands in DCUI. So, we have enabled the SSH and disabled the Lockdown mode which would help us to troubleshoot further in this state.

Fortunately, we could connect the ESXi host directly through vSphere client post disabling lockdown mode. But the host found still reported as “Not responding” in vCenter.

We have tried reconnecting the host back to vCenter since it is finely accessible through vSphere client in direct. But the reconnect operation fails with below error:

Error – A general System error occurred: Failed to login with vim administrator password

We could see – vim.fault.InvalidLogin event traces in vpxa log reporting “Cannot complete login due to incorrect user name and password” 

Considering the above error, we suspect issue with vpxa agent. So we were in plan to completely remove the host from vCenter inventory and add it back.

But at the same time, While processing the vCenter connect request from hypervisor end, below mentioned event found logged in ESXi events:

Error – The ramdisk ‘var’ is full. As a result, the file /var/run/vmware/tickets/vmtck  could not be written

The above error clearly indicates that the var partition is full and we have confirmed the same with following command –>  vdf -h in putty session of ESXi host.

Ramdisk                Size    Used  Available Use% Mounted on
root                         32M     2M       29M         9%        —
etc                            28M  396K      27M         1%        —
opt                            32M      0B      32M         0%        —
var                            48M   48M        0M     100%        —
tmp                         256M  180K    255M         0%        —
iofilters                    32M       0B       32M        0%        —
hostdstats            2053M      7M   2045M       0%         —
stagebootbank      250M      4K      249M       0%         —

The above output confirms that the var partition is full.

So, we have moved the old log files which blocks the space in var partition to local Datastore. Post which we could see the sufficient free space available in var partition. If you are not familiar with command to move the files, you can use WinSCP tool for the same.

Ramdisk                Size    Used  Available Use% Mounted on
root                         32M     2M       29M         9%        —
etc                            28M  396K      27M         1%        —
opt                            32M      0B      32M         0%        —
var                            48M      1M     47M        2%        —
tmp                         256M  180K    255M         0%        —
iofilters                    32M       0B       32M        0%        —
hostdstats            2053M      7M   2045M       0%         —
stagebootbank      250M      4K      249M       0%         —

Going back to the main problem, now we have tried connecting back the ESXi host in vCenter. The task has gone successful and host also came in sync with vCenter.

So whenever you face host sync issue with vCenter, please have a check on ramdisk usage as well.


Never Stop Learning !

Advertisements

One thought on “ESXi Host Sync Issue with vCenter – A general System error occurred: Failed to login with vim administrator password

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s