VM-Aware

I recently came across a problem whereby a Rescan SAN was taking an absolute age, like 15 minutes from the VI Client. Now I love the VI Client, but sometimes it just doesn’t give you enough information when it is doing things. To be fair most of the time the verbose information is simply not needed, but when things go wrong it is a god send.

In comes the Service Console command line, more specifically the esxcfg-rescan and esxcfg-mpath commands.

Firstly I ran esxcfg-mpath -l, which will list all the LUNs and the paths associated with each LUN. After running this I noticed that some of the LUNs had dead paths listed for LUNs that had been deleted from both the Storage Processor (DataCore) and the VI Client.

I had previously read somewhere (I can’t remember where) that Brocade switches can sometimes cause ESX to remember LUN locations even if they have been removed from the Storage Processor. So this meant that I had to routes to take 1 - Add the LUN back with the original paths and try and remove again gracefully or 2 -reboot the ESX host.

The second option was not really desirable as far as the customer was concerned so I had to go and add the LUNs back. This wasn’t such a big deal, but here’s what I did:

• Create the LUN in DataCore
• Set up mirroring to create the original 4 paths
• Present the storage to both the affected ESX hosts
• Run esxcfg-rescan vmhba?? on both HBAs
• Use esxcfg-mpath -l to check that the original paths are now showing as either Active or On and that there are no Dead paths
• Open the VI Client and add the LUNs using the Add Storage wizard
• Now remove the LUNs from the VI Client
• Run esxcfg-mpath -l to make sure that all the paths are still happy
• Go back to the Storage Processor and remove the LUNs presented to the ESX hosts
• finally run esxcfg-rescan vmhba?? on both HBAs again and check that the LUNs have been removed and that there are no Dead paths to any of the LUNs

I know all this seems a little long winded, but it was at the time not desirable to reboot the ESX hosts.

Here is a good Tech Note from VMware to help with the use of esxcfg-mpath.

Bookmarks

Tags: Brocade, ESX, Service Console, Storage

This entry was posted on Tuesday, July 15th, 2008 at 8:56 am and is filed under ESX, Storage, VMware, Virtualisation. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.