Can't remove node probably because I can't stop fileserver

[Tim11]

When I execute this

maprcli node remove -nodes ip-10-123-207-186.us-west-2.compute.internal

I get this

ERROR (22) -  node remove failed for node ip-10-123-207-186.us-west-2.compute.internal, Error: Invalid argument; For hostname: ip-10-123-207-186.us-west-2.compute.internal, there are fileservers with ids: 4668085203722062592 and nfsserver with id: 9132956263961481312 . Specify server id for removal

I assume it's tell me in some abstruse way that I still have a file server running.  Indeed it is running

And no matter how many times, I click the box and then Stop Service, it will say 'Service stopped successfully' And likewise when I do this at the command line

 maprcli node services -fileserver stop -nodes ip-10-123-207-186.us-west-2.compute.internal

It pauses for a second or two and then returns the command prompt implying it succeeded without error. Yet the service won't stop.

Any help would be appreciated.

thanks,

Tim

[shiva_jr]

Hi Tim11,
    I assume the information provided as as node IP or Node ID is not proper. 
1) Run the command to view the list of the nodes. Identify the node to be removed and note down their ID and IP address.  Refer these documents (Doc1 and doc2)
    maprcli node list -columns hostname,id

2) Run the command to remove the node.   I believe, IP address & Node Ids can be used as parameter here. Refer the document.
    maprcli node remove -nodes 10.20.30.40

3) You can use the same node parameter for stopiing the service.

Regards,6
Shiva_JR
Please mark as 'Accepted solution' if my post worked and award me the 'Kudos'.

[Tim11]

Shiva_JR,

Thank you for the detailed reply. My two issues are these:

1) Why didn't it work in the MCS? All there is to do there is click a box and then 'Remove Node'

2) I used those commands you gave and saw this

[root@ip-10-123-207-186 ~]# maprcli node list -columns hostname,id
hostname                                      ip              id                   
ip-10-123-207-159.us-west-2.compute.internal  10.123.207.159  8703801987189684448  
ip-10-123-207-160.us-west-2.compute.internal  10.123.207.160  2474245913408905440  
ip-10-123-207-186.us-west-2.compute.internal  10.123.207.186  4668085203722062592  
ip-10-123-207-188.us-west-2.compute.internal  10.123.207.188  5766698336990939200  
ip-10-123-207-192.us-west-2.compute.internal  10.123.207.192  4140954407743139008  
ip-10-123-207-203.us-west-2.compute.internal  10.123.207.203  390243887232304160   
ip-10-123-207-207.us-west-2.compute.internal  10.123.207.207  1737110396736002912  
ip-10-123-207-212.us-west-2.compute.internal  10.123.207.212  6416815679606025664  
ip-10-123-207-213.us-west-2.compute.internal  10.123.207.213  4117215511011946880  
ip-10-123-207-23.us-west-2.compute.internal   10.123.207.23   6010795263368165344  
ip-10-123-207-28.us-west-2.compute.internal   10.123.207.28   3819959219516663328  
ip-10-123-207-8.us-west-2.compute.internal    10.123.207.8    7480259858129756800  
ip-10-123-207-91.us-west-2.compute.internal   10.123.207.91   5419590221285718304  

[root@ip-10-123-207-186 ~]# maprcli node remove -nodes ip-10-123-207-186.us-west-2.compute.internal
ERROR (22) -  node remove failed for node ip-10-123-207-186.us-west-2.compute.internal, Error: Invalid argument; For hostname: ip-10-123-207-186.us-west-2.compute.internal, there are fileservers with ids: 4668085203722062592 and nfsserver with id: 9132956263961481312 . Specify server id for removal

[root@ip-10-123-207-186 ~]# maprcli node remove -nodes 4668085203722062592
ERROR (2) -  node remove failed for node 4668085203722062592, Error: No such hostname, need valid hostname; No node with name 4668085203722062592

Even though I used the proper hostname, it still didn't recognize it. Also I don't see how to use the id

thanks,

Tim

[Tim11 · Accepted solution]

Shiva_JR,

UPDATE: 

I just ran and received the below

[root@ip-10-123-207-186 ~]# maprcli node remove -hostids 4668085203722062592 -service fileserver
ERROR (16) -  node remove failed for hostid: 4668085203722062592, Error: Device or resource busy

1. Stop services on the node with ID: 4668085203722062592
2. Wait for 5 minutes until NODE_ALARM_NO_HEARTBEAT is raised for node: 4668085203722062592
3. Run this operation again

So I think this is telling me explicitly that the fileserver service is still running and needs to be stopped. As I said in my original post, running this 

maprcli node services -nodes ip-10-123-207-186.us-west-2.compute.internal -name fileserver -action stop

returns no error but seems to execute just fine. But in fact the service is still running. 

thanks,

Tim

[VineetKumar]

Hi TIm,

It appears, command used to stop fileserver is not correct. Kindly try to run below command from "cluster admin" user, after taking user ticket(in case, cluster is secure):

$ maprcli node services -name fileserver -nodes <FQDN_hostname> -action stop

Then verify using below command fileserver service stops on the server:

$ ps -ef | grep -i mfs

Thanks,

Vineet

[Tim11]

UPDATE:

I looked away for a minute and the fileserver service went to 'not configured'. I tried using the MCS to 'Remove Node' but that threw an error, but this worked

maprcli node remove -hostids 4668085203722062592

so I'll accept this as the solution.

Thanks again for your help

-Tim

[Sunitha_Mod]

Hello @Tim11,

That's excellent! 

We are extremely glad to know the problem has been resolved and we appreciate you for keeping us updated.