CISCO UCS Firmware Upgrade – AgileOps https://agileops.co.uk Virtualization made simple for Everyone. Fri, 31 Oct 2014 22:30:11 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.8 https://agileops.co.uk/wp-content/uploads/2017/10/favicon-01-50x50.png CISCO UCS Firmware Upgrade – AgileOps https://agileops.co.uk 32 32 194680508 UCS Blade unable to ping or connect to vCenter https://agileops.co.uk/ucs-blade-unable-to-ping-or-connect-to-vcenter/ https://agileops.co.uk/ucs-blade-unable-to-ping-or-connect-to-vcenter/#comments Fri, 31 Oct 2014 22:30:11 +0000 https://quraishi.wordpress.com/?p=913 ]]> Hi All, I came across a strange issue where 2 blades was unable to ping I could get 2 ping and then Request time outs Problem : 2 Servers( in the same chassis server 3,4 ) unable to ping their gateway. Ping drops are after 1-2 packets. We cannot ping/ssh to them from outside and they cannot be added to vCenter. Other blades in the same chassis are working ( 1,2) Steps Taken :
  1. Confirmed that we do not have any IP conflicts
  2. Made sure that we do not use the same subnet anywhere else in the network
  3. Also checked and confirmed that the Mgmt(CIMC) subnet is different
ucs1
  1. Started a ping from inside a Host ( Server-3 ) to check if you are able to get to the gateway
  2. Checked vobd logs  to find if you can find any  entrie for duplicate ip/mac
  3. Checked the Mgmt Interface : vmk0
  4. It is connected to Port Group : Management Network  on VLAN ID, Check if we are  using the right vlan
  5. Found that the MAC Address of the vmk0 is same as vnic (vmnic0) : This is a known issue VMware KB article here. This is the vnic mac address of the service profile which is attached to the esx blade.
  6. Deleted and re-created the vmk0 interface
  7. Confirmed that the MAC Address has changed
  8. Started a continuous ping which work fine now
  9. Host can now be added to the vCenter
    So let me go through of the process of how we found the duplicate mac address. The vnic mac address was taken by vmknic  as you might know vmknic mad starts with 00:50:56 but in my case it was same as the mac address of vnic from the service profile. login to the host and give the command #esxcfg-nics -l

this gives the list of vnics assigned to the service profile, note the management nics in my case vnic 0 and 1 then give the below command to get the vmkernal nics #esxcfg-vmknic -l

as you can see above the mac address is not starting from 00:50:56 and it is been duplicated from the vnic template from the service profile vnic0 which is the root cause for the connectivity issue. Now to fix this we have to delete the virtual switch and re-create it. #esxcfg-vswitch -l This lists the virtual switches

#esxcfg-vmknic -d -p ‘Management Network’ you have to specify exact name of the port Now we will need to add the vmknic with the same Management network #esxcfg-vmknic -a -i <IPaddress> -n <Netmask> ‘Management Network’  and also give the command #esxcfg-vswitch -l  to check the Management network created as shown below. note you can not give a gateway at this point you can specify frm DCA

p3

 you should now be able to ping to the host as it has got a new mac address to check it you can give the command #esxcfg-vmknic -l

p34

  for a continus ping you can give #ping <gateway ip> -c 100 this will give 100 continuous pings   Reference: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111]]>

]]> https://agileops.co.uk/ucs-blade-unable-to-ping-or-connect-to-vcenter/feed/ 1 913 Cisco UCS Firmware Overview https://agileops.co.uk/cisco-ucs-firmware-overview/ https://agileops.co.uk/cisco-ucs-firmware-overview/#respond Wed, 03 Sep 2014 20:50:01 +0000 https://quraishi.wordpress.com/?p=819 ]]> Cisco UCS FlexFlash Configuration of Secure Digital (SD) Card So, for the newbee  to Cisco UCS,  its confusing about the firmware. In this post I am trying to clarify few aspects of this, feel free to post comment below if you have any questions and I will get back to you. The CISCO UCS firmware is bundled into two packages as given below: ucs_firmware   1. Infrastructure Firmware In Infrastructure Software firmware we update the firmware of the following
  • Cisco UCS Manager software
  •  Kernel and system firmware for the fabric interconnects
  •  I/O module firmware
2. Server Firmware Cisco UCS B-Series Blade Server Software Bundle
  • CIMC firmware
  • BIOS firmware
  • Adapter firmware
  • Board controller firmware
  • Third-party firmware images required by the new server
This also includes the firmware of Adapter, BIOS, FC Adapters,  HBA Option ROM, Storage, Controller, Local Disk Cisco UCS C-Series Rack-Mount UCS-Managed Server Software
  • CIMC firmware
  • BIOS firmware
  • Adapter firmware
  • Board controller firmware
  • Third-party firmware images required by the new server
As most of you might be aware the UCS manager is the GUI Java based which allows you to fully manage the CISCO Unified Computing System. Note I have noticed that the Java version  7 update 40 is only compatible with the UCS Manager.  So, where does the UCS Manager run ? This was one of my first questions, Well the Software of the UCS Manager runs on the FI ( Fabric Interconnect 62xx) as we have a pair of Fabric Interconnect it has got resilience. The UCS System is designed to be highly scalable and available, so upgrading the firmware should not bring any outage to the environment. However Cisco still suggests to arrange a Change windows and get this done out of hours or the quite period of your business. This UCS Manager is bundled in the Infrastructure Firmware Software along with the Kernal update of the FI which is the NXOS, Finally we have the I/O Module firmware which is a part of the Infrastructure firmware software too. The Other Bundle is the Server Firmware which has the firmware for BIOS, the Blades, Adapter, Board controller etc. If you want to learn more about the process of doing a Firmware update please checkout my link CISCO UCS Firmware Update Process Hope this information is useful, feel free to comment.]]>

]]> https://agileops.co.uk/cisco-ucs-firmware-overview/feed/ 0 819 CISCO UCS Firmware Update Process https://agileops.co.uk/cisco-ucs-firmware-update-process/ https://agileops.co.uk/cisco-ucs-firmware-update-process/#comments Tue, 02 Sep 2014 15:56:25 +0000 https://quraishi.wordpress.com/?p=807 ]]> updatestarted This is released on 24-JUL-2014, you can find more details on the release notes for this firmware version: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/ucs_3_0_rn.html#pgfId-225587 Just to give everyone an idea there are two from of Firmware that needs to be updated on UCS system 1. Infrastructure Firmware In Infrastructure Software firmware we update the firmware of the following
  • Cisco UCS Manager software
  •  Kernel and system firmware for the fabric interconnects
  •  I/O module firmware
2. Server Firmware Cisco UCS B-Series Blade Server Software Bundle
  • CIMC firmware
  • BIOS firmware
  • Adapter firmware
  • Board controller firmware
  • Third-party firmware images required by the new server
This also includes the firmware of Adapter, BIOS, FC Adapters,  HBA Option ROM, Storage, Controller, Local Disk Cisco UCS C-Series Rack-Mount UCS-Managed Server Software BundleCIMC firmware
  • BIOS firmware
  • Adapter firmware
  • Board controller firmware
  • Third-party firmware images required by the new server
So,  without wasting any time these are the steps to follow: Step One
  1. Create an all Configuration Backup
  2. Verifying HA status of Fabric Interconnects ( as we do not have any compute it should not affect us in anyway)
  3. Verifying the status of I/O Modules
  4. Verifying the Status of the Servers and Adaptors
  5. Download Firmware Images to the Fabric Interconnect.
  Step Two
  1. Disable the Call Home ( Email alerts for reboot)
  2. Updating the Firmware on the Adapters, CIMCs and IOMs
  If the Service profiles are configured using host/management firmware policy this will create an error and we might need to think of Plan B (If we do not get any error message we can progress with upgrade)  
  1. Activating the Firmware on the Adapters
  2. Activating the Firmware on the CIMCs
  3. Activating the Board Controller Firmware
  4. Activating the I/O Modules
  5. Activating the Subordinate Fabric Interconnect Firmware
  6. Making the primary FI the Subordinate by login to FI ssh to the primary and to make it subordinate first check
# connect local-mgmt   FI-01-A(local-mgmt)# show cluster extended-state Cluster Id: xxxxxxxxxxxxxxxxxxxxxxxx   Start time: Mon Jun 23 14:41:56 2014 Last election time: Tue Oct  7 11:54:01 2014   A: UP, SUBORDINATE B: UP, PRIMARY   A: memb state UP, lead state SUBORDINATE, mgmt services state: UP B: memb state UP, lead state PRIMARY, mgmt services state: UP    heartbeat state PRIMARY_OK   INTERNAL NETWORK INTERFACES: eth1, UP eth2, UP   HA READY Detailed state of the device selected for HA storage: Chassis 1, serial: xxxxx, state: active Chassis 2, serial: xxxxx, state: active Chassis 3, serial: xxxxx, state: active   In my case as shown above FI B is the Primary so I can connect t0 FI B with hte below command .   FI-01-A(local-mgmt)# connect local-mgmt B # cluster lead A
  1. Activating the Primary Fabric Interconnect Firmware
  Step Three
  1. Listing the BIOS Upgrade Versions
  2. Create a Host Firmware Package
  3. Applying the Host Firmware Package
  4. Enabling Call Home.
  Additional Notes   Prerequisite for Firmware update  https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/upgrading/from2-0/to2-0MR/b_UpgradingCiscoUCSFrom2-0To2-0MR/b_UpgradingCiscoUCSFrom1-4To2-0_chapter_010.html#concept_DD68485034FB4E258D5827880D37225C
Process of Upgrading and Activating the Firmware https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/upgrading/from2-0/to2-0MR/b_UpgradingCiscoUCSFrom2-0To2-0MR/b_UpgradingCiscoUCSFrom1-4To2-0_chapter_0100.html#task_F1119EB4EDBA42578ADE662462B5C4CE UCS Firmware deep dive https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/firmware-mgmt/gui/2-1/b_GUI_Firmware_Management_21/b_GUI_Firmware_Management_21_chapter_0100.html#concept_2474DE9D2048450CA1A37A8A3D3EE597  
You can also find the process of upgrading on this youtube video  httpss://www.youtube.com/watch?v=b6LQt2DYVWA its broken down into 3 steps.
]]>

]]> https://agileops.co.uk/cisco-ucs-firmware-update-process/feed/ 7 807