HPE 3PAR StoreServ Storage

3PAR SMCC 3.4.1 Appliance Filled Itself Up

 
HughletClapbrd
Occasional Advisor

3PAR SMCC 3.4.1 Appliance Filled Itself Up

Found our SMCC GUI was no longer online today, then logged into the text user interface and gave the appliance a restart. After boot, the situation was no better and so I dropped to the shell and found our SMCC appliance had filled its /dev/sda1 partition:

Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 17M 783M 3% /run
/dev/sda1 28G 26G 0 100% /
tmpfs 4.0G 0 4.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 4.0G 0 4.0G 0% /sys/fs/cgroup
/dev/sda2 430G 692M 408G 1% /var/opt/hpe
tmpfs 799M 0 799M 0% /run/user/1002

All of the logs listed on page 89 of the SSMC Administrator guide are stored under the physical location of the directory /var/opt/hpe/ssmc/data which only contains 442 MB of files and is actually located on the /dev/sda2 filesystem, however it is the /dev/sda1 file system that is full. So, removing logs will not fix the space issue on /dev/sda1 and will not fix the issue here.
 
The culprit seems to be java hprof files that aren't getting removed:
/opt/hpe/ssmc/ssmcbase$ ls -alh java*
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Nov 6 21:20 java_pid19316.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5.6G May 1 2019 java_pid21541.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Aug 21 22:29 java_pid27199.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 1.1G Jun 26 22:07 java_pid31097.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5.7G Apr 3 2019 java_pid32662.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5.6G May 29 21:57 java_pid6705.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 0 Jul 24 21:15 java_pid8632.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 5.2G Mar 6 2019 java_pid966.hprof
-rw------- 1 hpe3parssmcuser hpe3parssmcuser 40M Oct 9 22:11 java_pid980.hprof

 

I'd like to remove one or two of these but my user doesn't have sudo rights or the ability to re/move any of these files.

Anyone run into this before? Anyone else's appliance starting to fill up?

Thanks