HPE VME POC

[chypsa]

Hi all,

I'm working on a POC for HPE VME, so this is truly a hodge-podge of tech stapled together just to see if it will work. To be precise:

• 1x PC running Windows Server with HyperV
• 1x VM running HPE VME Ubuntu server (slimmed down, official HPE version, updated) with nested virtualization enabled and inside is the HPE VME Console installed
• 1x VM with domain controller, iSCSI, NFS and SMB shares faking as storage
• 1x VME cluster with the VME node created
• 1x Added Storage, NFS target to Cluster
• 1x Added SMB storage to Infrastructure - Storage - File Shares as ISO storage

When provisioning a new instance, which is a Windows Server 2025 built from an ISO, I eventually get the following error:

Error:Failed to create server volumes: Failed to create volume: qemu-img: /mnt/06930b3d-e543-47c6-adec-2906548d3fd2/Windows Server 2025 003/hvm_4-disk-0: Could not write qcow2 header: Resource temporarily unavailable

All permissions have been set "lab-like", meaning everyone has full control on every NFS and SMB share. I just want to get a VM up and running, so I don't care about permissions.

Are there any other logs that might indicate where the process gets stuck? Does anyone have any thoughts on what might be the cause of getting stuck?

Bonus question: Is there a way to retry the entire provisioning process, so I don't have to type in all the info from scratch on every test?

[Ted Laurent]

I have a similar issue.   I set up an NFS server and verified that I could write to the export.  When I PUTTY into my Ubuntu server I can create new file directly and I can see the file created under /mnt but get pretty much the same error you mentioned when trying to start the VM.

I am running 8.0.10.  What version are you running?

You could look under Administration | Health for morepheus logs but I didn't find any answers to my issue there.

I haven't looked under /var/morepheus ... I think there may be some logs there.

[CalvinZito]

@chypsa I suggest you contact hpe-sw-trial-vmessentials@hpe.com to see if they can look at what you have. It is our HPE Support for customers running the trial software. 

[chypsa]

Thanks for the suggestion. I have another test environment and I'll try standing up the environment from scratch, then engage other channels if need be.

[chypsa]

So, basically...I upped a new lab, created a shared storage iSCSI and it's provisioning VM's fine. From other news, I don't understand the need for virtual images to be stored on a shared folder. Why not designate a folder on the existing iSCSI targets or create a new iSCSI location and provision a share on it...is beyond me.

Other than that, it works, so I guess this can be closed. However, be tuned, as I'm just about to raise some other questions regarding backups.

[Ted Laurent]

echoing @CalvinZito - the trailware team definitely rocks.

I do appreciate you posting in the forums though ... that helped me.

Ted

[jamgwert81]

I agree with your comments on not understanding why Virtual Images can only be hosted on S3 Buckets and Files Shares.

I ran into a similar issue while using Ceph storage on my HVM cluster.

I had to setup an external smb share to host the images so I could check the box for Default Virutal Image Store.  I also noticed the SMB setup doesn't appear to support SMBv3 with encryption which is pretty much the default now from most OSes using SMB storage.  I had to disable the enforcement of encryption on SMBv3 to allow the VME manager VM to connect to the share.  Seems like step backwards to me.

I'm also concerned about overall network speed when using SMB paths to host the Virutal Image store, seems like it takes forever trying to push the first Instance to the HVM cluster because it has to seed those files to the clusters own Virutal Image storage location.  Once it does that its kind of a non issue, but if you make a lot of images and constantly update them, then this make the process take longer.  I get it you're trying to be cloud agnostic, but if I'm trying to run this product on-prem in place of my on-prem VMware cluster, why can't the VME manager VM be able to talk directly to my Ceph storage as well as iSCSI data storage for the Virtual Images piece?

I also had issue with all the stuff I uploaded or captured to Virtual Images before I updated it to making the SMB share be the Default Virtual Image Store.  All the Virtual Images I'd setup before that broke when I setup the SMB share and checked that box.  New stuff worked fine but all the old stuff got errors until I manually figured out a way to copy the data in the old appliance location out to the new SMB path.  It would be nice if you change from one location to another, the data just syncs to the new location.  Someone on another thread indicated this is like a Content Library in VMware, but the syncing concept of Content Library when you make changes after the fact appears to be missing.  I'm also concerned about when and if we have to change the location or storage path of Virtual Images store at a later date.  The documentation just says make sure it's "mirrored" but doesn't really explain how to go about mirroring the data, especially if it was initially configured to store those images locally on the appliance.

[Arnout_Verbeken]

Virtual Image location can indeed be an SMB, NFS,S3,... type of share but can also be a "local fileshare". I'll come back on that later.

You should read https://community.hpe.com/t5/hpe-morpheus-vm-essentials/disk-space-filling-up-on-host-where-vmessentails-console-is/m-p/7252607#M513 to get some more info about the different types of "storage" in VM Essentials.

As you know, VM Essentials it the little brother of HPE Morpheus, which is specifically designed to "manage multiple clouds".  In that concept, having a Virtual Image location on prem that can be used by your multiple clouds is the best way.  You manage your images locally, and they only need to sync once (to the Image Target) of a specific cloud during deployement of VM's.  Subsequent deployments of a specific Virtual Image will use the image that is "cached" in the Image Target.
This makes deployments very fast since there is no need to copy the Virtual Image with every new deployment over the wire to your 'far-away-cloud".

VM Essentials uses the same concept.  I understand that it seems overkill since everything is on-prem, but also with VM Essentials you can run the VM Essentials Manager on a different location as your cluster.  This is certainly true when running multiple clusters with only 1 manager.

As said, you can also use a "Local Fileshare" as Virtual Image Location.  It is explained in the link above and you can find the official documentation on https://support.hpe.com/hpesc/public/docDisplay?docId=sd00006775en_us&page=GUID-4885ECA7-94F5-481E-A2FE-A433FAA899C8.html

This share will use "a mountpoint" inside your VM Essentials Manager as location for the Virtual Images.  It removes the need for an external SMB/NFS/S3/... and the data actually lives on your real Datastores (albeit in a virtual disk attached to your VM Essentials Manager).
The best practice is of course to add a 2nd disk to your manager, format and mount it (automount with /etc/fstab) and use that mountpoint for the Virtual Images location.
An extra advantage is that your images will be backed up together with the Manager VM (if you would want that).

All references/metadata of Virtual Images are stored in the internal DB of the VME Essentials Manager, including the location.  This is all done during the creation/edit of a new Virtual Image.
There is indeed no clean way to copy/move Virtual Images to another location because the DB does not get updated.  All data (ISO's...) that existed before adding the location as Virtual Image repository will also not be added to the DB.

There is a way to clone/copy the Virtual Image Library to another location by following the procedure on https://support.hpe.com/hpesc/public/docDisplay?docId=sd00006775en_us&page=GUID-AE35F0B9-BDF0-4A25-9E84-763F156878B7.html
This should help you.
It boils down to the following procedure:

1. Follow the process to add an image in the UI -- don't add any files
2. After the name shows up in the list, hover over it to find the folder number  ("<name> (ID: XX)")
3. Create that folder on the file share, I use CIFS but I imagine NFS is identical:  \morpheus-virtual-images\XX
4. Copy the image ISO or QCOW into the folder
5. Back in the HVM UI the size will state "N/A" if you want to update this edit then save the image