Monday, March 16, 2015

"no network adapters" in ESXi 6.0 GA (workstation 10, possibly any case)

Summary: If you don't meet the minimum 4GB of RAM you may get this error.

I used to do a few custom ESXi VMs for my nested hosts, keep them there and modify them as needed. 

Reason for that is the default settings are not of my particular liking. I don't let them have a 40GB disk - 1GB is more than fine for my lab. Likewise, I started my templates with the least amount of ram; in 5.0 and 5.1, that meant 2GB. 5.5 raised the minimum limit to 4GB, and told you very clearly about it! (felt like RTFM after that)

This will become important later. 

I naively installed my first ESXi 6.0 GA host with 2GB of RAM, and ran into this

"Well WTH?" i thought. I googled around but there wasn't anything too clear for "no network adapters" in ESXi 6.0 GA or VMware Workstation. I assumed there had to be a mistake so I checked the settings. Workstation doesn't let you choose the NIC adapter, although you can edit the VMX file.

Checking the VMX files, one for 5.5 and 6.0 I could see the e1000 adapter clearly listed.

Then I took a step back, (RTFM echoed in my head), and I went to check what the requirements were for ESXi 6.0 and saw 4GB ram - I thought, well let's get that fixed although there wasn't an error like in 5.5. Lo and behold, the installer now worked and I was able to install without any other problems.

So, there's a "gotcha" there, and that's what this blog is about :)

Monday, March 2, 2015

P420m/P320h Micron PCIe SSDs - firmware update is different for retail and Dell branded SSDs

This one came up when I was just starting my deployment with PernixData FVP. Peter Chang, @virtualbacon, was instrumental in getting me out of the confusion, as well as Dan Florence from Micron.

I got my new R620 servers with Micron P420m PCIe drives (Dell part 468-7978). I always make sure I upgrade the firmware on any server i'm deploying, and I noticed the Dell OME bootable CD I created on that day didn't include any items for the SSD. I checked the VMware HCL and there were several recommendations depending on ESXi version, so it seemed I did want to make sure I was on the latest firmware to avoid some known issues. I figured, no biggie, and decided it could be too new, and would need to update it separately.

When I browsed the Dell downloads for this server, I only found a software package in the Dell website for Windows, and quite older than the VMware HCL; so I went to the Micron website, browsed in products for SSDs, PCIe interface, and there was the P420m, in my case a Half Height Half Length (HHHL) card. 

I found both a bootable and non-bootable version, saw both used the same firmware, and proceeded to download the latest file. This included firmware, drivers and manuals (at that time, the latest was 145.03.00, released 09/2014).

Fast forward a lot of details and reading - I got RSSDM installed and working, and was able to poll information using the cli

/opt/micron/bin # rssdm -L -d

Drive Id             : 0
Total Size           : 700.15GB
Drive Status         : Drive is in good health
SMARTSupport         : Yes
SMARTEnabled         : Yes
Interrupt Coalescing : D200F
WriteBufferEnabled   : Yes
Power Limit Status   : Not Supported
Est. Life Remaining  : 100%

Listing the detailed drive information is retrieved successfully
CMD_STATUS   : Success

Copyright (C) 2014 Micron Technology, Inc.

/opt/micron/bin # rssdm -L

Drive Id             : 0
Device Name          : mtip_rssd0
Model No             : Micron P420m-MTFDGAR700MAX
Serial No            : 000000001404whatever
FW-Rev               : B2085108
Total Size           : 700.15GB
Drive Status         : Drive is in good health
PCI Path (B:D.F)     : 41:00.0
Vendor               : Micron
Temp(C)              : 57

Drive information is retrieved successfully
CMD_STATUS   : Success

Copyright (C) 2014 Micron Technology, Inc.

Note the 08 in bold, it becomes important later. I then tried to do a firmware update and got an error:

/opt/micron/bin # rssdm -T B145.03.00.ubi -n 0

Trying to update for drive 0, from current firmware B208.51.08 to B212.05.00.
Are you sure you want to continue(Y|N):Y

Unified Image update for drive 0 will take a few seconds to complete.
Please wait

Drive Id     : 0
Unified image update operation failed
CMD_STATUS   : Order of Unified Update : Firmware, UEFI main driver, Option ROM.                                                Firmware Update failed in Unified image download

Copyright (C) 2014 Micron Technology, Inc.

After back and forth over the "why", and reaching out to PernixData and Micron through e-mail and twitter, finally the answer was provided to me. Remember the numbers in bold? Peter Chang, working with a Micron engineer, was able to confirm this was the source of my problems:

"It appears they are trying to upgrade disti FW on an OEM driver. Xx.xx.08 to xx.xx.00. The .08 is a Dell release.

RSSDM will prevent this operation.  This is likely the cause for the error. "

Aha! Turns out the correct page to get the Dell latest firmwares is:

Dan Florence from Micron was later able to confirm:

"Yes, B208.51.08 is the latest firmware for the Dell-branded version of P420M. Dell has their own internal testing and validation program for these drives, with their own unique cadence for releases. They also have their own warranty/service program. For these reasons a Dell drive is disallowed from loading our standard distribution firmware."

This was back in December 2014. Since then, I've seen version 145.03.08 show up in January, which is firmware "B212.05.08" - the Dell tested release of the Micron I was trying to upgrade to in the command excerpts above (see the 08 at the end?). 

It is important to note the Dell releases are somewhat delayed from Micron's, so when you see a new firmware out for Micron, just wait a bit and you will probably see Dell's. This Dell section of the Micron webiste does provide a nice little PDF explaining the upgrade process in good detail, including how a reboot and secure erase are needed.

I'm sharing this because "there's a gotcha there" and guess what this blog is about :)

Bonus - Late February 2015 I received an advance support bulletin from PernixData that new firmware is being made available for the P420m/P320h to fix a new issue, so keep an eye out on the respective locations if you are using these SSDs. 

If you are using retail SSDs in VMware and are a PernixData customer, a new firmware (B145.04.00) is available for your testing now. If you have not received such bulletin, open a case with Pernix to get the related instructions, as they are going out of their way (the issue is not related to FVP) to make sure all their affected customers are notified.

I'd like to finish saying that so far, these SSDs have behaved very very well for me and I am pleased with them. Whenever you are using new technology, you have to be ready to accept the uncharted territory - but you do expect support. The fact that both Micron and PernixData are jointly reaching out with a solution speaks to their engagement and respect for the customer, so I couldn't be happier :)