HPE Morpheus VM Essentials Software
1859963 Members
15579 Online
110409 Solutions
New Discussion

Shut down a virtual machine from the WebUI

 
dya
Valued Contributor

Shut down a virtual machine from the WebUI

I am testing this on v8.1.1.

When I stop a virtual machine from the WebUI, does it crash during the shutdown process?

●Host journal when stopped via the WebUI
Apr 21 14:20:30 vme-iscsi1 sudo[421230]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown vme-iscsi-manager
Apr 21 14:20:40 vme-iscsi1 sudo[421302]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy vme-iscsi-manager
⇒“virsh destroy” is executed approximately 10 seconds after “virsh shutdown”.

● Virtual Machine Journal
root@vme-iscsi-manager:~# journalctl -b -1 -n 10
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: fwupd.service: Consumed 6.188s CPU time.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: upower.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager blkdeactivate[16733]: Deactivating block devices:
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped upower.service - Daemon for power management.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: ssh.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: systemd-random-seed.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped systemd-random-seed.service - Load/Save OS Random Seed.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: blk-availability.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped blk-availability.service - Availability of block devices.
root@vme-iscsi-manager:~#
⇒There is no “Journal stopped” in the last line

 

18 REPLIES 18
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

When I shut down a virtual machine from the WebUI, it appears that “virsh shutdown” is executed on the HVM host, followed by “virsh destroy” approximately 10 seconds later, causing the virtual machine to be forcefully terminated.
Does this happen to anyone else?

mixdrop
Advisor

Re: Shut down a virtual machine from the WebUI

Hi.
 
The following is a Japanese manual, but if the qemu-agent or Morpheus Agent is not installed on the virtual machine, it will be forcibly shut down after 10 seconds (virsh destroy).
 

VM Essentials managerを使用したSVCのシャットダウン | HPE SimpliVity 6.2.0 for HPE Morpheus VM Essentials Softwareドキュメント

We have also confirmed the issue where the system shuts down after 10 seconds in our verification environment (v8.0.13 / v8.1.1). While we cannot say this with complete certainty, we believe it is likely related to the agent. However, in the case of Linux, the qemu-agent is usually installed by default, so it is possible that it was not installed correctly or that the agent is not running. I hope this information is helpful.
 
Thanks.
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

Thank you.

I checked both the “HVM Manager” and other Linux virtual machines.
The QEMU guest agent seems to be working properly, and the Morpheus agent is also installed.

I have some concerns about the shutdown process for HVM virtual machines.
At the very least, when shutting down the “HVM Manager,” I do not use the Web UI and instead manually execute “virsh shutdown” on the host.

I’m concerned that “virsh destroy” is being executed only about 10 seconds after “virsh shutdown” — such a very short amount of time.

 

I am only including partial logs.

----------------------------------------------------
Guest 1: Ubuntu Server 24.04
■Host
・journalctl
May 02 03:09:08 vme-iscsi3 sudo[340011]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown esxi-ubuntu
May 02 03:09:13 vme-iscsi3 systemd-machined[903]: Machine qemu-2-esxi-ubuntu terminated.
May 02 03:09:19 vme-iscsi3 sudo[340114]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy esxi-ubuntu

・QEMU Guest Agent check
# virsh domifaddr esxi-ubuntu --source agent
Name MAC address Protocol Address
-------------------------------------------------------------------------------
lo 00:00:00:00:00:00 ipv4 127.0.0.1/8
- - ipv6 ::1/128
enp3s0 52:54:00:4b:f5:16 ipv4 172.16.1.222/24
- - ipv6 fe80::5054:ff:fe4b:f516/64
#
# virsh qemu-agent-command esxi-ubuntu '{"execute":"guest-exec", "arguments":{"path":"/bin/ls"}}' --pretty
{
"return": {
"pid": 1582
}
}
#

■Guest
・journalctl
May 02 03:09:08 esxi-ubuntu systemd-logind[742]: System is powering down (hypervisor initiated shutdown).
May 02 03:09:11 esxi-ubuntu systemd-journald[346]: Journal stopped

■Observations
・The QEMU Guest Agent in the guest appears to be working normally.
・virsh shutdown is executed on the host, and the shutdown process starts in the guest.
・The guest’s journal ends with "Journal stopped".
・About 11 seconds after running virsh shutdown, virsh destroy is executed on the host (after the systemd-machined VM termination log).

----------------------------------------------------

----------------------------------------------------
Guest 3: Ubuntu Server 24.04 (HVM manager)
■Host
・journalctl
May 02 05:34:25 vme-iscsi1 sudo[8056]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown vme-iscsi-manager
May 02 05:34:36 vme-iscsi1 sudo[8076]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy vme-iscsi-manager
May 02 05:34:38 vme-iscsi1 systemd-machined[911]: Machine qemu-1-vme-iscsi-manager terminated.

・QEMU Guest Agent check
# virsh domifaddr vme-iscsi-manager --source agent
Name MAC address Protocol Address
-------------------------------------------------------------------------------
lo 00:00:00:00:00:00 ipv4 127.0.0.1/8
- - ipv6 ::1/128
eth0 52:54:00:61:51:37 ipv4 172.16.2.100/24
- - ipv6 fe80::5054:ff:fe61:5137/64

#
# virsh qemu-agent-command vme-iscsi-manager '{"execute":"guest-exec", "arguments":{"path":"/bin/ls"}}' --pretty
{
"return": {
"pid": 58454
}
}
#

■Guest
・journalctl
May 02 05:34:25 vme-iscsi-manager systemd-logind[683]: System is powering down (hypervisor initiated shutdown).
May 02 05:34:28 vme-iscsi-manager systemd[1]: Stopped finalrd.service - Create final runtime dir for shutdown pivot root.

■Observations
・The QEMU Guest Agent appears to be working normally.
・virsh shutdown is executed on the host, and shutdown starts in the guest.
・Unlike Guest 1, there is no "Journal stopped" message at the end of the guest logs (last log is at 05:34:28).
・About 11 seconds after virsh shutdown, virsh destroy is executed (before the systemd-machined termination log).
・It appears that the guest was terminated by virsh destroy.

----------------------------------------------------

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

This issue does not occur only when shutting down the “HVM Manager”; it also appears to affect other virtual machines (Linux and Windows).
In the case of Linux, the execution of virsh destroy after approximately 10 seconds seems too early, even considering systemd’s DefaultTimeoutStopSec=90s.
I believe that using the “Stop” operation for virtual machines from the Web UI is risky. This seems to be a problem that should be improved.

■ What I tested
・Windows Server 2025
・Configured a shutdown delay batch file using the Local Group Policy “Shutdown Script”
timeout /t 180 /nobreak
・Ubuntu Server 24.04
・Configured a systemd unit file to delay shutdown
# cat /etc/systemd/system/delay-shutdown.service
[Unit]
Description=Delay Shutdown Service
After=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecStop=/bin/sleep 180

[Install]
WantedBy=multi-user.target
#

■ Results
・Virtual Machine (Windows Server 2025)
→ virsh destroy was executed approximately 1 minute 10 seconds after virsh shutdown
・Virtual Machine (Ubuntu Server 24.04)
→ virsh destroy was executed approximately 10 seconds after virsh shutdown

■ Additional Information
This applies only to Ubuntu Server 24.04, but I also tested virtual machines with the same shutdown delay unit file on ESXi 8.0u3 and Proxmox VE 9.0.10. In both cases, the systems shut down normally.

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

When I initiate a shutdown operation from the WebUI, the guest journal also shows the message “guest-shutdown called, mode: powerdown,” which indicates that the shutdown is definitely being started via virsh shutdown in agent mode.

What I consider to be the issue is why virsh destroy is executed about 10 seconds later. As I mentioned previously, I believe this behavior is problematic.

I would like to hear HPE’s opinion or explanation regarding this behavior. What do you think?

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

We consider this issue to be a serious problem, but so far we have not received any response from HPE.
Is anyone else experiencing this issue, or are we the only ones affected?
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

Could you please provide an update on the status of this issue? Are there any plans to improve or fix this in the future?

Additionally, I would like to know if this issue is specific to a limited environment or if it is a known general issue.

RFK
HPE Pro

Re: Shut down a virtual machine from the WebUI

Hi

I see you are trialing VME, is this the start of looking for another hypervisor, what's the plan?

What O/S is the VM you are seeing this with?

thanks



I work at HPE
HPE Support Center offers support for your HPE services and products when and how you need it. Get started with HPE Support Center today.
[Any personal opinions expressed are mine, and not official statements on behalf of Hewlett Packard Enterprise]
Accept or Kudo
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

Hello,

Thank you for your message.

Regarding your question, I am currently evaluating VME as we are looking for a migration path from VMware. As for our roadmap, my team has already started proposing VME solutions to our customers.

The issue has been confirmed on virtual machines running Windows Server 2025 and Ubuntu Server 24.04. It also occurs in the VME Manager. I suspect this behavior persists across other guest OSs as well, but I have not been able to test them all yet.

I consider this to be a critical issue, and since it could hinder our future proposals, I would like to get to the bottom of this.

RFK
HPE Pro

Re: Shut down a virtual machine from the WebUI

Thanks for all the info.

As I understand, the issue is:
You issue a 'Stop' command (from the VM, not the Instance) from the VME Manager, which runs a 'virsh shutdown', but then 10 seconds later a 'virsh destroy' gets run regardless of the shutdown state?

Having an Agent installed or not, makes no difference?

 

 



I work at HPE
HPE Support Center offers support for your HPE services and products when and how you need it. Get started with HPE Support Center today.
[Any personal opinions expressed are mine, and not official statements on behalf of Hewlett Packard Enterprise]
Accept or Kudo
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

Thank you for your quick confirmation.

Your understanding is generally correct. This issue occurs with both [Instances] and [Computes]. Additionally, while "virsh shutdown" is executed followed by "virsh destroy" about 10 seconds later in most cases, there have been instances where the delay was approximately 1 minute and 10 seconds. I am not certain what causes this difference, but regardless of the duration, I view the fact that "virsh destroy" is executed within such a short time frame as a significant problem.

Both the QEMU guest agent and the Morpheus agent are installed. I believe this is the intended configuration, so I have not tested any cases where they are not installed.

I tested this again in v9.0.0, and the details are provided below.

■Tested guests
root@cloud-ubuntu:~# cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.4 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
root@cloud-ubuntu:~#
root@cloud-ubuntu:~# systemctl status qemu-guest-agent
● qemu-guest-agent.service - QEMU Guest Agent
Loaded: loaded (/usr/lib/systemd/system/qemu-guest-agent.service; static)
Active: active (running) since Mon 2026-06-15 23:37:15 JST; 2min 19s ago
Main PID: 652 (qemu-ga)
Tasks: 2 (limit: 2202)
Memory: 1.4M (peak: 1.6M)
CPU: 33ms
CGroup: /system.slice/qemu-guest-agent.service
└─652 /usr/sbin/qemu-ga

Jun 15 23:37:15 cloud-ubuntu systemd[1]: Started qemu-guest-agent.service - QEM>
root@cloud-ubuntu:~#
root@cloud-ubuntu:~# morpheus-node-ctl status
● morpheus-morphd.service - Morpheus Agent (morphd)
Loaded: loaded (/etc/systemd/system/morpheus-morphd.service; enabled; pres>
Active: active (running) since Mon 2026-06-15 23:37:16 JST; 2min 28s ago
Main PID: 690 (java)
Tasks: 41 (limit: 2202)
Memory: 134.1M (peak: 135.4M)
CPU: 9.264s
CGroup: /system.slice/morpheus-morphd.service
└─690 /opt/morpheus-node/embedded/java/bin/java -XX:+UseG1GC -Xms3>

Jun 15 23:37:16 cloud-ubuntu systemd[1]: Started morpheus-morphd.service - Morp>
root@cloud-ubuntu:~#

■Stop from [Compute]
・Part of the host's journal
Jun 15 23:41:34 vme-iscsi2 sudo[21395]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown cloud-ubuntu
Jun 15 23:41:45 vme-iscsi2 sudo[21502]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy cloud-ubuntu

・Part of the guest's journal
Jun 15 23:41:34 cloud-ubuntu qemu-ga[652]: info: guest-shutdown called, mode: powerdown

■Stop from [Instance]
・Part of the host's journal
Jun 15 23:51:45 vme-iscsi2 sudo[27287]: root : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown cloud-ubuntu
Jun 15 23:51:55 vme-iscsi2 sudo[27372]: root : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy cloud-ubuntu

・Part of the guest's journal
Jun 15 23:51:45 cloud-ubuntu qemu-ga[649]: info: guest-shutdown called, mode: powerdown

⇒While the guest has initiated the shutdown in agent mode via virsh shutdown, it does not seem normal that "Journal stopped" is missing from the last line.

 

mixdrop
Advisor

Re: Shut down a virtual machine from the WebUI

Hello.

We are also working daily to test solutions for migrating from VMware to VME for our customers.

Our investigation suggests that the issue occurs regardless of whether the Guest Agent is installed or not, and regardless of the Guest OS. While we can trace it on Linux, it appears to be occurring on Windows as well.

Dya,> I apologize, I provided incorrect information. Thank you for testing it on 9.0. Has the UI changed? I'm hesitant to test 9.0 here.

Actually, our investigation has revealed that this issue occurs not only in the Standalone version but also in the dHCI/PCBE versions.

We have submitted a request, but we still don't know the trigger for the issue.

We will let you know if there are any further developments.

 

Thanks.

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

Thank you for the information. So, this also occurs in the dHCI/PCBE version as well.
Regarding v9.0, I think the following video will be helpful. The UI hasn't changed much, but the underlying mechanism has changed.
https://www.youtube.com/watch?v=stq-38_POdE

I performed a test migration of a Windows VM from VMware using the standard migration tool in v9.0, and this issue also occurred during the migration. Looking at the journal, it showed that "destroy" was executed by morpheus-node.

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

This is the host's morpheus log when stopping the VM from Compute on the WebUI.

It seems it's configured to run destroy after sleep 10 right from the beginning. The delimiter is a semicolon (;), but is this expected?
Even when virsh shutdown ends successfully, virsh destroy is executed about 10 seconds afterwards.

shutdown.png

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

I don't think there's anything the user side can do about this...

I unpacked the following file to look into it: /opt/morpheus/lib/tomcat/webapps/ROOT/WEB-INF/lib/mvm-9.0.0.jar

The screenshot is attached below.

kakunin.png
dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

@RFK 


Is there any progress?
I think we've reached the limit of what we can check on our end.
It would be great if you could keep us all updated on how this progresses.

dya
Valued Contributor

Re: Shut down a virtual machine from the WebUI

"I have shared all available information regarding this matter and have answered all questions raised by HPE representatives. However, HPE has yet to provide any clear indication of their stance or intentions.
It is my understanding that HPE personnel also monitor the community site, yet there has been complete silence from their end. One cannot help but wonder if this platform exists solely for HPE's own purposes.
I shared this information in the hope of helping everyone, as the issue at hand seems quite fundamental. It is truly disappointing that HPE has chosen to remain silent."
mixdrop
Advisor

Re: Shut down a virtual machine from the WebUI

Hello.
I agree with you on this matter.

The fact that performing GUI operations—despite instructions not to use `visrh`—results in a forced disconnection after 10 seconds is a critical issue, particularly regarding DFS2 fencing.
This phenomenon effectively means that the system should not be deployed for services such as database servers (other than Oracle), file servers, or authentication services.

The findings from dya's verification work are extremely valuable.

I have also looked through various threads, but they mostly just say "please contact us individually," and there seems to be no feedback on the eventual outcome.

Even though users are told to submit requests, there is often no acknowledgment that the request was received;
in the end, it seems requests only get processed via Slack.

Since this is a forum, I believe the results of individual requests (excluding those involving private configurations) should be made public.

dya, thank you for raising this point.

thanks.