CIMC and the fun of upgrading the bare metal drive firmware

Today I had to upgrade the CIMC firmware and oh what fun.  Basically, the RAID controller we replaced did nothing to solve the problem despite it having issues.  TAC came back and said it is now time to upgrade the firmware of the CIMC.  Normally, this might sound like an easy task, download file, upload, and done right?  Negative....

First, you need to download the file from Cisco's support site, if you aren't familiar with this and you are an engineer, shame on you.   What they don't tell you is that this is a .ISO file.  You either reboot the whole server and do everything from the KVM or you use the .bin files to do it manually.  At first, I thought I was going to be slick and just upgrade the CIMC firmware and be done since it doesn't require a server reboot.  You only need to restart the CIMC interface which will not interrupt any VM data traffic or processes.

Second, you need to update the BIOS, this will require a server reboot to finish.  Again, you can use the BIOS file and just restart the server after complete while watching the KVM.  Don't do this unless you have a KVM!!!!  The web KVM should suffice but if you are in my situation and are 2 hours away, it is better to be on-site in the event something blows up and you need to manually power cycle the server.  Please, for the love of god, make sure you properly shutdown the VMs first!

Once all of this is done, all you need to do is log back into the CIMC and verify that the upgrade was good and there aren't any issues present.  Below is an image of what the CIMC looks like before the switch to the new CIMC version.  Note that I had used the browser to upload the firmware and let it do its thing that way since TFTP is just more complexity (Yes I know TFTP is easy, but why use the TFTP option when you can just slap the file into a web box and be done?).

Above you can see that the running version is 1.4(7g) and my backup version is 1.5(2).  Once I click Activate CIMC firmware, the uploaded firmware will take effect.  Remember, you must update the BIOS afterwards or Cisco docs say your server will not boot!  Of course, you can disregard that statement if TAC says otherwise.


Summary of what to do:

You have two options
    1. Use the .ISO and reboot the whole server and let it do everything for you or
    2. Extract just the CIMC and BIOS firmware from the .ISO
    3. Install the CIMC firmware first, then you can activate the firmware as it will not interrupt traffic on the VMs
    4. Install the BIOS firmware after the CIMC firmware has been activated, then schedule time for a reboot so the BIOS can finish the update process
    5. Log back into the CIMC and verify that everything is updated to the version you downloaded and that there are no errors.


Here is the link to the UCS C Series Server firmware guide:

http://www.cisco.com/en/US/docs/unified_computing/ucs/c/sw/gui/config/guide/1.5/b_Cisco_UCS_C-series_GUI_Configuration_Guide.151_chapter_01101.html#task_94832943DCAA4854A4AA135D54FB2B50