HPE SimpliVity

Is it ok to edit replica_distribution_file while auto rebalance is happening?

 
adidasnmotion
Advisor

Is it ok to edit replica_distribution_file while auto rebalance is happening?

We recently had to rebuild one of the ovc's with the help of HP support because it was in a faulty state.  Now that its working properly the vm's are auto rebalancing between the hosts which I would expect after the problem we had had.  However, I'm watching as the primary and secondary replicas are getting created using the dsv-balance-show and dsv-balance-manual commands and one of our hosts is dangerously close to running out of space. 

I thought I could edit the replica_distribution_file before it fills up to try and move some of those vm's to the host with much more free space but when I try to edit it I get a warning that I'm attempting to edit a read-only which I haven't seen before.  I assume its because its currently in the middle of its rebalancing.

Is its read-only at the moment because of the auto-rebalance?  Would editing this file while its in read-only cause any issues?  What happens if the host runs out of space and how would I fix that if it does?

 

3 REPLIES 3
kylerj
HPE Pro

Re: Is it ok to edit replica_distribution_file while auto rebalance is happening?

Hi @adidasnmotion 

The read-only message might be a permission issue. However, solely editing the replica distribution file does not change the actual distribution of replicas unless its coupled with the dsv-balance-manual command. I personally don't recommend that you take the manual balancing responsibility upon yourself and you should call and create a support ticket with the SimpliVity team to alleviate your capacity problems.

I work for HPE

Accept or Kudo

adidasnmotion
Advisor

Re: Is it ok to edit replica_distribution_file while auto rebalance is happening?

I can't really contact support because we're on legacy Lenovo equipment and don't currently have a support contract.  As it is I had to contact HP paid hourly support yesterday for help recreating the faulty ovc thats led to this.  So last night I realized the reason I was seeing a "readonly" message when trying to edit the replica_distribution_file was because I was forgetting to add "sudo" to the command (it was a long day).  I was using the dsv-balance-manual command and have adjusted the location of primary and secondary replicas in the past.

Regardless, during its auto rebalancing one of the hosts eventually hit zero disk space which is what I see when using the dsv-balance-show command:

Screen Shot 2021-04-16 at 11.02.44 AM.png

So far I haven't seen any issues because of this but obviously I don't want to leave it like that.  I tried editing the replica distribution file but it won't let me because it looks like there are still 2 vm's that still don't have a secondary replica:

Screen Shot 2021-04-16 at 11.07.51 AM.png

I'm sure part of the reason for this is because these are our 2 largest vm's (4.5TB and 11TB respectively).  My question now is will Simplivity eventually, on its own, create replicas for those 2 vm's or do I need to do something to try and resolve this?

db13
HPE Pro

Re: Is it ok to edit replica_distribution_file while auto rebalance is happening?

@adidasnmotion, you would need to first run the dsv-hive-job-show command from node 1 (having the p copy) and see if the guids for those 2 VMs is listed. If so, the service is actively rebuilding the replicas. If these are in an error state, its likely that the node selected for the secondary does not have enough space to hold what the service has estimated is needed. I that case, you would need to open a support case to investigate further.

I am an HPE Employee
A quick resolution to technical issues for your HP Enterprise products is just a click away HPE Support Center Knowledge-base

Accept or Kudo