HPE 3PAR StoreServ Storage
1748204 Members
3346 Online
108759 Solutions
New Discussion

Re: SSMC 3.4.1 Appliance Filled Itself Up

 
HughletClapbrd
Occasional Advisor

SSMC 3.4.1 Appliance Filled Itself Up

A few days ago I tried to logon to my 3Par management console and noticed it was down. Using the the text user interface (TUI), I rebooted the appliance which did not help. I dropped to the shell on the appliance and noticed that the file system /dev/sda1 was full:

Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 17M 783M 3% /run
/dev/sda1 28G 26G 0 100% /
tmpfs 4.0G 0 4.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 4.0G 0 4.0G 0% /sys/fs/cgroup
/dev/sda2 430G 692M 408G 1% /var/opt/hpe
tmpfs 799M 0 799M 0% /run/user/1002

All of the logs listed on page 89 of the SSMC Administrator guide are stored under the physical location of the directory /var/opt/hpe/ssmc/data. This directory only contains 442 MB of files and is actually located on the /dev/sda2 filesystem, however it is the /dev/sda1 file system that is full. So, removing logs will not fix the space issue on /dev/sda1 and will not fix the issue here.
 
The culprit files seem to be java .hprof files that aren't getting removed:
 
$ ls -alh /opt/hpe/ssmc/ssmcbase java*
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Nov 6 21:20 java_pid19316.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5951217084 May 1 2019 java_pid21541.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Aug 21 22:29 java_pid27199.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 1154846720 Jun 26 22:07 java_pid31097.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 6107854431 Apr 3 2019 java_pid32662.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5968203231 May 29 21:57 java_pid6705.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Jul 24 21:15 java_pid8632.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5489545102 Mar 6 2019 java_pid966.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 41848832 Oct 9 22:11 java_pid980.hprof
 
My user doesn't have sudo permissions or the permissions to re/move any of these files.
 
Anyone else's SMCC appliance filling up? Anyone know how to get the appliance to get rid of these files?
 
Thanks

 

3 REPLIES 3

Re: SSMC 3.4.1 Appliance Filled Itself Up

This is a known problem.

The root cause of the filesystem-fillup is fixed in SSMC 3.5. (Current version is SSMC 3.6)

Updating to that release won´t fix the full filesystem ! and i don´t think that you will be able to upgrade while the filesystem is full.

Unfortunately , to clean up the filesystem you will need to open a support case, or simply re-install SSMC. 

If your setup does not have multiple arrays with complex policy setup, the reinstall is the easiest and fastest solution.

 

 

I am an HPE Employee

Accept or Kudo

HughletClapbrd
Occasional Advisor

Re: SSMC 3.4.1 Appliance Filled Itself Up

Thanks for your reply, as this is a lot more information than I've yet to receive on my support ticket with HPE! We do have multiple arrays and also would prefer not losing our history, re-do our LDAP setup, etc. Hopefully we can get a few of these files removed so that I can upgrade the appliance instead of starting over. 

Maybe I missed it, but this issue seems like this should be a customer communication, or KB article, or documented in the 3.5 release notes as fixed?

Re: SSMC 3.4.1 Appliance Filled Itself Up

The removal of the files can only be done by HPE support. The reason for this is that the root-user on an SSMC appliance is disabled and need first to be enabled which is a procedure that requires remote-access to the SSMC-ESX/HyperV console.

 

I am an HPE Employee

Accept or Kudo