Error when provisioning new hosts - Ceph Installation Gets Stuck in multiple places.

[AballahSonDi]

Is there some special magic to get the ceph setup to run correctly? 

The script always stops at the point right after creating the partition (created by script) and says that it failed because the partition already exists.  Even though it didn't until the script created it. 

I can usually restart it running from here but that it will hang later in the script doing the following:
Run Script: Add Ceph Pools to libvirt

[cbunge]

I believe you can run a wipefs -a /dev/blah and retry the task (the circular arrow icon on the specific task) from the history on the cluster. I have not seen ceph fail outside of a partition table being present on the disk. 

[DiegoDelgado]

Just to add to cbunge's answer; I've also had issues when you enter the host's IP as the name in the cluster creation dialogue, it fails on a different step, but please make sure you enter a name starting with a letter  (it doesn't need to be the the hostname and it's not needed in your DNS)

[cbunge]

Oh! It may be due to using NVMe drives.  There is a fix to how the partitions are named in v8.0.4 releasing this month.

[AballahSonDi]

I do have NVMe on the primary drives.  The others are SSDs. 

2 of the 4 identical machines I eventually got working.  Still "retrying" on the other two.  

[AballahSonDi]

So any update here.   I blew away the whole cluster and tried to re-provision and now I can't get any of them to work?  

I would really like to get some testing done but if I can't get ceph to come up I can't do much testing. 

Is there a log I can provide or anything?

[AballahSonDi · Accepted solution]

So for anyone coming to this page here is ultimately what I did to resolve all of my issues:

This was an install into my test cluster which I wipe and rebuild fairly often wtih different types of technology.  So there is no local DNS, no local domain, etc.  When I tear it down there is nothing but a gateway in front of it.  

1. DNS was my issue across the board. 
2. After getting morpheus vm up I had put a DNS name in the appliance url that is resolvable by the gateway but not by the local hosts, meaning I could reach it from the outside of this cluster but not inside.  
3. I temporarily added an entry in the hosts file and reprovisioned.  Ceph still failed a few times but just continually retrying at the point it failed eventually lead it to finish.  

This is my bad in general, however, I wish someone could have pointed me to the logs that I should have been looking at.  Also I dont' understand why it was failing at different spots all the time if the DNS was really the issue.