Correct way for an ISSU / Reflection of my mistakes

[Linkk]

Hello Community,

I did my first ISSU last Friday and ran into some minor problems. I would like to hear your opinion about how I did it and how I can prevent my mistakes the next time. Feel free to criticise my, as long as it is constructive.

About the Update:

I updated an IRF of two JG296A HPE 5920AF 24XG connected via one Stack Link. I updated from R2418P06 to R2432P01, which is a compatible ISSU.

I generally followed the instructions of this document: http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=mmr_sf-EN_US000005131

So, I copied the ipe file to the current master switch (Switch 1) and verified the compatibility. I set the rollback timer to 0, because I didn't want any rollback. Then I wanted to load the new firmware to switch 2 but Switch 1 didn't have enough space to extract the firmware. There were 83836 KB free space on switch 1 and 173944 KB on switch 2.

Here comes my biggest mistake, I believe. I deleted the at the moment running firmware from switch 1 and started the "issu load file ipe" to switch 2. The switch extracted the ipe file to both switches and started the upgrade an the slave switch (Switch 2). 

After the reboot I issued the issu switchover and switch 1 failed to boot. I connected via console and saw that switch 1 was stuck at the BootRom. My guess is that he tried to load the old firmware before the update, which I deleted. I upgraded the firmware manually to the new version and rebooted the switch. He joined the IRF without problems, but the ISSU state was still in switchover mode. I entered "issu commit slot 1" and after another reboot the ISSU was complet. The ISSU state was "Init" again.

There was no downtime since slot 2 handled all the traffic during the upgrade of slot 1.

After my reflection I guess I should have done the following:

After loading the new firmware to slot 2, I should have deleted the ipe file from slot 1 and then copy the old firmware from slot 2 back to slot 1. 

Do you have any other advises for me?

Thanks!

[parnassus]

Not properly an expert about ISSU (which, as I understood, is a sort of procedural upgrade): looking at general ISSU/non-ISSU software upgrade recommendations - not only related to IRF deployments - a primary condition I think must be met...a necessary flash space should be available and that free flash space is required to perform an upgrade correctly.

In other terms, speaking about an IRF deployment, it seems that each IRF Member switch must have free storage space (on its Slot) that is at least two times the size of the upgrade image file used (the IPE compressed file containing the system software - Boot and System - images)...that because IPE file will be decompressed automatically so, during its deflating, space used on Flash literally doubles.

If there is no enough free space on the IRF Master's flash and/or on any other IRF Member's flash the only viable option is to delete some [*] unused files in each Slot (as example using the delete /unreserved flash:/backup.bin command on the IRF Master and/or the delete /unreserved slotn#flash:/backup.bin command on any other IRF Member, so referring to Slot n)...that as a mandatory and preparatory step before any upgrade.

I don't know if a workaround to overcome free space shortages realy exists...but, looking at your reported free space values (83836 KB free space on Switch 1 and 173944 KB on Switch 2), your troubles started exactly there.

I'm quite sure also that the IRF Master will reboot - if instructed to do so - using the new IPE file as per this command (see R2432P01 Release Notes): boot-loader file flash:/newest.ipe slot 1 main where newest.ipe is the filename of the newest IPE...so I don't understand why, after the switchover, the Switch 1 should have booted with the old system software (which didn't found because you deleted...)...I mean, I'm not sure it should have done the boot that way.

What files there were (so how much free space there was) on Switch 1 Flash before you started the IPE file download on it (the same free space you had on IRF Member 2)?

[*] which files did you deleted from your Switch 1 Flash (you wrote: "I deleted the at the moment running firmware from switch ")? boot and/or system bin files?

[Linkk]

so I don't understand why, after the switchover, the Switch 1 should have booted with the old system software (which didn't found because you deleted...)...I mean, I'm not sure it should have done the boot that way.

That were my thoughts. In my mind it should upgrade the boot file to the new boot file and then start the new bin. 

Because it was an ISSU, I did not enter "boot-loader file flash:/newest.ipe slot 1 main". After switch 2 rejoined the IRF with the new firmware, I entered "issu run switchover", which gave the following output:

<IRF-5920>issu run switchover 
Upgrade summary according to following table: 

flash:/5900_5920-cmw710-boot-r2432p01.bin

  Running Version             New Version        
  Release 2418P06             Release 2432P01    

flash:/5900_5920-cmw710-system-r2432p01.bin

  Running Version             New Version        
  Release 2418P06             Release 2432P01    

  Slot                        Switchover Way                         
  1                           Master subordinate switchover          

Upgrading software images to compatible versions. Continue? [Y/N]:y
This operation maybe take several minutes, please wait...%Feb 10 18:03:40:436 2017 IRF-5920 PKG/5/PKG: This operation starts to reboot the specified slot to complete upgrade.

which files did you deleted from your Switch 1 Flash (you wrote: "I deleted the at the moment running firmware from switch ")? boot and/or system bin files?

I deleted the old bin file and left the boot file, because the bin file freed up enough space. 

[MichaelM55]

If you run the ISSU process, do not allow the switch to delete the new *.ipe firmware, although it suggests you to do so.

[Linkk]

Thanks for the reply.

Like you said, I kept the ipe file in case something would went wrong during the update, so I don't have to copy the file again (The copy process is soo slow!) .

 But I deleted it after the update was complete.