Cannot upgrade SSMC to 3.8.2

[BBARBAROS]

Hi all,

I installed new SSMC 3.8.0 as appliance.

Then upgraded to 3.8.1 without any issue

But I cannot upgrade to 3.8.2 or 3.8.2.1  no matter how much I tried.

I start upgrade, reboot ssmc and it still is 3.8.1 

[support_s]

System recommended content:

1. HPE SSMC 3.8 User Guide | Upgrading SSMC

2. HPE SSMC 3.8.2 and 3.8.2.1 Release Notes | Description

If the above information is helpful, then please click on "Thumbs Up/Kudo" icon.

Thank you for being a HPE community member.

[sbhat09]

Hello @BBARBAROS,

Your query has been discussed already in the other discussion chain - https://community.hpe.com/t5/HPE-3PAR-StoreServ-Storage/SSMC-and-log4j-vulnerability/td-p/7156368/page/5#.YcBE7GhByUk

Please let me know if you need further details.

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.

[Bertram Stoeckler]

If you can´t workaround the problem by doing a new-install, then you should open a support-case.

If you want to do some troubleshooting on your own, then you can

1. logon via the TUI to the SSMC-appliance

2. check

a) 

/var/opt/hpe/ssmc/data/logs/ssmc.log

It should have something like this:

2021-12-20 09:33:01.808+0100 INFO c.h.t.s.u.l.t.DeviceApplianceUpgradeLogicV1 - Applicance upgrade initiated
2021-12-20 09:33:04.810+0100 INFO c.h.t.s.u.l.t.DeviceApplianceUpgradeLogicV1 - Execution of Upgrade script will start
2021-12-20 09:33:04.971+0100 INFO c.h.t.s.u.l.t.DeviceApplianceUpgradeLogicV1 - Response Status:0Response Message:[Mon 20 Dec 2021 09:33:04 AM CET][16239][INFO] /var/opt/hpe/upgrade/work/HPEssmcappliance_3.8.2.1.9.iso mounted
[Mon 20 Dec 2021 09:33:04 AM CET][16248][INFO] Scheduling the upgrade job...
[Mon 20 Dec 2021 09:33:04 AM CET][16248][INFO] Successfully scheduled the upgrade job.

b)

/var/log/appliance.log 

That file shows the upgrade process in more detail.

Let me know if you have further questions.

[BBARBAROS]

@Bertram Stoeckler 

These are the screenshot of the appliance.log from the last upgrade attempt. I`d be grateful if you look at them to see if anything is obvious. My Linux knowledge is almost nothing

https://www.dropbox.com/s/1sp4zfrcrdx4ey7/Screenshot%202021-12-20%20130904.jpg?dl=0

https://www.dropbox.com/s/wztuntuucq6ruz7/Screenshot%202021-12-20%20131007.jpg?dl=0

https://www.dropbox.com/s/vhr2h1hpcgfnc4l/Screenshot%202021-12-20%20131053.jpg?dl=0

https://www.dropbox.com/s/y2ax3060cvb00sy/Screenshot%202021-12-20%20131129.jpg?dl=0

[Bertram Stoeckler]

Looks like something is inconsistent on your appliance as the upgrade is trying to reinstall hpessmc, but fails.

I have not seen this before.

How many arrays do you have connected to this SSMC ?

I am asking because if you only have a handful, a reinstallation is probably the fastest way to fix the issue.

Would this be an option for you? Can you try this? Note that you can keep the current SSMC and simply install a second appliance-VM.

[BBARBAROS]

I have 22 arrays and they all are compatible with SSMC 3.8/3.8.2/3.8.2.1

I did try a new installation. Brand new 3.8 from OVA. I didn`t add any arrays. Brand new SSMC install.

I ran the upgrade.  And same issue. 

[sbhat09]

Hello @BBARBAROS,

The issue is something strange and we never case across such issue before. Can't guess anymore reason causing this issue.

We may have to analyse the logs to find the cause and assist you further. I recommend you to open a support case for this.

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.

[BBARBAROS]

@sbhat09  Yeah, I usually find weird things

There is an open case (5360878829) but nobody contacted me yet 

[BBARBAROS]

@Bertram Stoeckler 

Bertram, I might have a solution but they all require to run some commands that need elevated or sudo permisison. How can I do that? All I have is ssmcadmin user which can`t do anything much.

[Bertram Stoeckler]

> Bertram, I might have a solution but they all require to run some commands that need elevated or sudo permisison. 

Can you share more details? What command do you want to run ?

What was/is the outcome of the support-case that you had opened ?

[BBARBAROS]

@Bertram Stoeckler 

This is what I saw in the logs

[Mon 03 Jan 2022 10:52:12 PM EST][4899][INFO] Stopping appliance watch process ...

[Mon 03 Jan 2022 10:52:13 PM EST][4899][INFO] Stopped appliance watch process

[Mon 03 Jan 2022 10:52:13 PM EST][4899][INFO] Stopping ssmc ...

[Mon 03 Jan 2022 10:52:21 PM EST][4899][INFO] Stopped ssmc

[Mon 03 Jan 2022 10:52:21 PM EST][4899][INFO] Stopping ultron ...

[Mon 03 Jan 2022 10:52:27 PM EST][4899][INFO] Stopped ultron

[Mon 03 Jan 2022 10:52:27 PM EST][4899][INFO] Stopping elasticsearch ...

[Mon 03 Jan 2022 10:52:28 PM EST][4899][INFO] Stopped elasticsearch

[Mon 03 Jan 2022 10:52:28 PM EST][4899][INFO] No /var/lib/dpkg/lock was found to be acquired. Proceeding with subsequent installation steps.

[Mon 03 Jan 2022 10:52:28 PM EST][4899][INFO] Clean incomplete dpkg processes, if there are any : dpkg --configure -a  >> /var/log/appliance.log

[Mon 03 Jan 2022 10:52:28 PM EST][4899][INFO] Installing packages... The the following command is being run : apt-get -y install HPESSMCAppCore-3.8.2.29.amd64.deb >> /var/log/appliance.log

Reading package lists...

Building dependency tree...

Reading state information...

E: The package hpessmc needs to be reinstalled, but I can't find an archive for it.

[Mon 03 Jan 2022 10:52:32 PM EST][4899][ERROR] Installation of one or more items failed...

[Mon 03 Jan 2022 10:52:32 PM EST][4899][INFO] Starting appliance watch process ...

[Mon 03 Jan 2022 10:52:32 PM EST][4899][INFO] Started appliance watch process

[Mon 03 Jan 2022 10:53:41 PM EST][7541][INFO] Starting the appliance services...

[Mon 03 Jan 2022 10:53:41 PM EST][7541][INFO] appliance_services 'start' applied lock

[Mon 03 Jan 2022 10:53:47 PM EST][7541][INFO] appliance_services 'start' released lock

And I tried these solutions

https://www.linuxuprising.com/2018/07/how-to-fix-could-not-get-lock.html

https://askubuntu.com/questions/15433/unable-to-lock-the-administration-directory-var-lib-dpkg-is-another-process

https://phoenixnap.com/kb/fix-could-not-get-lock-error-ubuntu

https://itsfoss.com/could-not-get-lock-error/

https://www.linuxtechi.com/apt-dpkg-lock-error-in-ubuntu/

[Bertram Stoeckler · Accepted solution]

@BBARBAROS 

Since the root-access is completly disabled on the SSMC-appliance, manual changes to the dpkg status information is NOT possible. My guess is that something went wrong during the migration from 3.3.1 to 3.6, or that something went wrong during the first upgrade attempt from 3.6 to 3.8.

You can check /var/lib/dpkg/status and search for the "hpessmc" package. Most likly it shows a status of "reinstreq half-installed".

As a solution i suggest that you re-do the migration followed by an upgrade to 3.8.2.1

I recently worked on a simliar issue and that got resolved after a second migration attempt.

Hope that helps.

[BBARBAROS]

@Bertram Stoeckler 

Yes, "hpessmc" package in /var/lib/dpkg/status shows as "reinstreq half-installed"

I`m bringing up a new 3.7 appliance and will try to upgrade to 3.8.2.1 from on top of that

I`ll let you know how it goes.

Thank you

[BBARBAROS]

@Bertram Stoeckler 

Thank you so much for your help

@sbhat09 

Thank you too for your time