Monday, March 2, 2015

P420m/P320h Micron PCIe SSDs - firmware update is different for retail and Dell branded SSDs

This one came up when I was just starting my deployment with PernixData FVP. Peter Chang, @virtualbacon, was instrumental in getting me out of the confusion, as well as Dan Florence from Micron.

I got my new R620 servers with Micron P420m PCIe drives (Dell part 468-7978). I always make sure I upgrade the firmware on any server i'm deploying, and I noticed the Dell OME bootable CD I created on that day didn't include any items for the SSD. I checked the VMware HCL and there were several recommendations depending on ESXi version, so it seemed I did want to make sure I was on the latest firmware to avoid some known issues. I figured, no biggie, and decided it could be too new, and would need to update it separately.

When I browsed the Dell downloads for this server, I only found a software package in the Dell website for Windows, and quite older than the VMware HCL; so I went to the Micron website, browsed in products for SSDs, PCIe interface, and there was the P420m, in my case a Half Height Half Length (HHHL) card. 

I found both a bootable and non-bootable version, saw both used the same firmware, and proceeded to download the latest file. This included firmware, drivers and manuals (at that time, the latest was 145.03.00, released 09/2014).

Fast forward a lot of details and reading - I got RSSDM installed and working, and was able to poll information using the cli

/opt/micron/bin # rssdm -L -d

Drive Id             : 0
Total Size           : 700.15GB
Drive Status         : Drive is in good health
SMARTSupport         : Yes
SMARTEnabled         : Yes
Interrupt Coalescing : D200F
WriteBufferEnabled   : Yes
Power Limit Status   : Not Supported
Est. Life Remaining  : 100%

Listing the detailed drive information is retrieved successfully
CMD_STATUS   : Success

Copyright (C) 2014 Micron Technology, Inc.

/opt/micron/bin # rssdm -L

Drive Id             : 0
Device Name          : mtip_rssd0
Model No             : Micron P420m-MTFDGAR700MAX
Serial No            : 000000001404whatever
FW-Rev               : B2085108
Total Size           : 700.15GB
Drive Status         : Drive is in good health
PCI Path (B:D.F)     : 41:00.0
Vendor               : Micron
Temp(C)              : 57

Drive information is retrieved successfully
CMD_STATUS   : Success

Copyright (C) 2014 Micron Technology, Inc.

Note the 08 in bold, it becomes important later. I then tried to do a firmware update and got an error:

/opt/micron/bin # rssdm -T B145.03.00.ubi -n 0

Trying to update for drive 0, from current firmware B208.51.08 to B212.05.00.
Are you sure you want to continue(Y|N):Y

Unified Image update for drive 0 will take a few seconds to complete.
Please wait

Drive Id     : 0
Unified image update operation failed
CMD_STATUS   : Order of Unified Update : Firmware, UEFI main driver, Option ROM.                                                Firmware Update failed in Unified image download

Copyright (C) 2014 Micron Technology, Inc.

After back and forth over the "why", and reaching out to PernixData and Micron through e-mail and twitter, finally the answer was provided to me. Remember the numbers in bold? Peter Chang, working with a Micron engineer, was able to confirm this was the source of my problems:

"It appears they are trying to upgrade disti FW on an OEM driver. Xx.xx.08 to xx.xx.00. The .08 is a Dell release.

RSSDM will prevent this operation.  This is likely the cause for the error. "

Aha! Turns out the correct page to get the Dell latest firmwares is:

Dan Florence from Micron was later able to confirm:

"Yes, B208.51.08 is the latest firmware for the Dell-branded version of P420M. Dell has their own internal testing and validation program for these drives, with their own unique cadence for releases. They also have their own warranty/service program. For these reasons a Dell drive is disallowed from loading our standard distribution firmware."

This was back in December 2014. Since then, I've seen version 145.03.08 show up in January, which is firmware "B212.05.08" - the Dell tested release of the Micron I was trying to upgrade to in the command excerpts above (see the 08 at the end?). 

It is important to note the Dell releases are somewhat delayed from Micron's, so when you see a new firmware out for Micron, just wait a bit and you will probably see Dell's. This Dell section of the Micron webiste does provide a nice little PDF explaining the upgrade process in good detail, including how a reboot and secure erase are needed.

I'm sharing this because "there's a gotcha there" and guess what this blog is about :)

Bonus - Late February 2015 I received an advance support bulletin from PernixData that new firmware is being made available for the P420m/P320h to fix a new issue, so keep an eye out on the respective locations if you are using these SSDs. 

If you are using retail SSDs in VMware and are a PernixData customer, a new firmware (B145.04.00) is available for your testing now. If you have not received such bulletin, open a case with Pernix to get the related instructions, as they are going out of their way (the issue is not related to FVP) to make sure all their affected customers are notified.

I'd like to finish saying that so far, these SSDs have behaved very very well for me and I am pleased with them. Whenever you are using new technology, you have to be ready to accept the uncharted territory - but you do expect support. The fact that both Micron and PernixData are jointly reaching out with a solution speaks to their engagement and respect for the customer, so I couldn't be happier :)


  1. Hi Ariel, I am wondering if you can tell me what are the drivers you installed on your ESXi.

    I have two vib installed on my ESXi 6:
    mtip32xx-native 3.9.4-1OEM.550.0.0.1331820 MICRON VMwareCertified 2015-12-03
    micron-rssdm 2.10.5818.00-01 Micron PartnerSupported 2015-12-03

    But on my Dell R820, rssdm can't find any SSD device:

    /opt/micron/bin/rssdm -L

    CMD_STATUS : Unable to find any P320 drives connected
    STATUS_CODE : 10

    I see yours has
    Device Name : mtip_rssd0
    wondering if you are using different driver.

    my two SSD disks are showing as:

    esxcfg-mpath -m | grep ATA
    vmhba2:C0:T0:L0 vmhba2 pscsi.vmhba2 pscsi.0:0 t10.ATA_____Micron_P420m2DMTFDGAR700MAX______________01111111111111
    vmhba1:C0:T0:L0 vmhba1 pscsi.vmhba1 pscsi.0:0 t10.ATA_____Micron_P420m2DMTFDGAR700MAX______________02222222222222

    Thanks for your help!

  2. Hi Jack!

    I'm running 5.5. Here is my output

    mtip32xx-native 3.9.4-1OEM.550.0.0.1331820 MICRON VMwareCertified 2014-12-05
    micron-rssdm 2.20.11180.00-01 Micron PartnerSupported 2015-10-05

    ~ # esxcfg-mpath -m | grep ATA
    vmhba1:C0:T0:L0 vmhba1 pscsi.vmhba1 pscsi.0:0 t10.ATA_____Micron_P420m2DMTFDGAR700MAX______________0000000013490D...

    The thing I see is a difference in the rssdm version. Check this tweeter conversation I had with Micron Storage

    Ariel Sanchez @arielsanchezmor
    Oct 02
    @MicronStorage q for you. I'm installing RSSDM on VMware. I have two versions of VIBs, which is the newer one? 2.10.5818.00 or 2.20.11180.00

    Micron Storage
    Oct 05

    @arielsanchezmor 2.20.11180.00 Have a great Monday!

    But, this version is NOT in the firmware bundle - that download is only good for the firmware. The latest rssdm version is in the retail firmware version pack. Link here:,-d-,07,-d-,00_linux_vmware.gz

    Finally, since the time I made this post, Micron/Dell have released a new firmware version (B145.07.08.ubi) which I highly encourage you to use. This is my current output:

    ~ # cd /opt/micron/bin/
    /opt/micron/bin # ./rssdm -L

    Drive Id : 0
    Device Name : mtip_rssd0
    Model No : Micron P420m-MTFDGAR700MAX
    Serial No : 0000000013490.....
    FW-Rev : B2180108
    Total Size : 700.15GB
    Drive Status : Drive is in good health
    PCI Path (B:D.F) : 41:00.0
    Vendor : Micron
    Temp(C) : 42

    Drive information is retrieved successfully
    CMD_STATUS : Success

    Copyright (C) 2014 Micron Technology, Inc.

    /opt/micron/bin #

    contact me in twitter if you have more questions, @arielsanchezmor - and you could also ask Micron Storage directly :)

  3. Hey there,

    Do you have B145.03.00.ubi file with you? I purchased an old 420m for my homelab and it is currently on FW version B138.00.05.ubi , and this has some issues which causes my R620 to freeze to death.
    I crawled through Micron's website, just getting 404s there. Internet archive was no help as well. Hopefully you still have it saved somewhere.

  4. Hi Ariel,

    I have a couple of the Dell branded Micron drives and I'm running into the issue. I'm trying to use the Micro rssdm utility to upgrade the firmware and I'm receiving STATUS_CODE: 51. I'm unable to locate the Dell firmware and utilities to update these drives. Does anyone know where I can locate the Dell firmware and utilities?