Urgent: log file on Simplivity 3.7.10 overloads the disk

[jfrobs]

Hello, we have once again a big problem on our simplivity infrastructure, one of our servers to its log file that has saturated the disk. 

I thought this bug had been fixed on the new versions, but obviously not.
My second machine is 70% disk-based...It's critical.

@robkennedy you were the one who managed to find and fix the problem last time, I think you remember that.

Ticker Open on HP : 5342251019 

Help  

[robkennedy]

@jfrobs  Good morning, sorry to hear that you are having issues here again.  When you mention "one of our servers to its log file that has saturated the disk"  is this in reference to the Omnistack Virtual Controller running out of disk space or is it actually one of your running VMs ? I see that you have a support case open so the correct process is being followed here. If the same issue as before is diagnoised by support then it will most like come through to the Engineering Resolution Team for further analysis. Thank you

[jfrobs]

Hello Rob,

Yes its the Omnistack Virtual Controller same problem as last time.

I think that the compression of logs, in any case the process must have a problem.

[BenH10]

Thanks for posting this, I just checked my cluster that I am testing 3.7.10 on, the OVC's have doubled in size since the upgrade, they are now around ~20GB each. Where as my production cluster is still around ~8-9GB per OVC. 

I haven't put a support ticket in yet, but I will delay my production update for at least another week to see if the OVC's keep growing. Do you know where this space is being used up? A specific log or something? 

[jfrobs]

Hello @BenH10 , for Hyper-v System, you can see at /var/svtfs/0/log/remoteps.log

In our system, this log increase to 8 go and system crash.

[BenH10]

I found out what caused the storage increase. It looks like it will be normal for 3.7.10 to double in size. 3.7.10 brought a new version of Ubuntu to the OVC. 3.7.9 was Ubuntu 14.0.5 LTS and 3.7.10 is 18.04.2 LTS. The old version of Linux does not seem to get deleted during the upgrade. It would be intersting to see if factory reseting the node to 3.7.10 would bring the storage back under 10GB.