Shut down a virtual machine from the WebUI

[dya]

I am testing this on v8.1.1.

When I stop a virtual machine from the WebUI, does it crash during the shutdown process?

●Host journal when stopped via the WebUI
Apr 21 14:20:30 vme-iscsi1 sudo[421230]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown vme-iscsi-manager
Apr 21 14:20:40 vme-iscsi1 sudo[421302]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy vme-iscsi-manager
⇒“virsh destroy” is executed approximately 10 seconds after “virsh shutdown”.

● Virtual Machine Journal
root@vme-iscsi-manager:~# journalctl -b -1 -n 10
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: fwupd.service: Consumed 6.188s CPU time.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: upower.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager blkdeactivate[16733]: Deactivating block devices:
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped upower.service - Daemon for power management.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: ssh.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped ssh.service - OpenBSD Secure Shell server.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: systemd-random-seed.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped systemd-random-seed.service - Load/Save OS Random Seed.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: blk-availability.service: Deactivated successfully.
Apr 21 14:20:30 vme-iscsi-manager systemd[1]: Stopped blk-availability.service - Availability of block devices.
root@vme-iscsi-manager:~#
⇒There is no “Journal stopped” in the last line

[dya]

When I shut down a virtual machine from the WebUI, it appears that “virsh shutdown” is executed on the HVM host, followed by “virsh destroy” approximately 10 seconds later, causing the virtual machine to be forcefully terminated.
Does this happen to anyone else?

[mixdrop]

Hi.

The following is a Japanese manual, but if the qemu-agent or Morpheus Agent is not installed on the virtual machine, it will be forcibly shut down after 10 seconds (virsh destroy).

VM Essentials managerを使用したSVCのシャットダウン | HPE SimpliVity 6.2.0 for HPE Morpheus VM Essentials Softwareドキュメント

We have also confirmed the issue where the system shuts down after 10 seconds in our verification environment (v8.0.13 / v8.1.1). While we cannot say this with complete certainty, we believe it is likely related to the agent. However, in the case of Linux, the qemu-agent is usually installed by default, so it is possible that it was not installed correctly or that the agent is not running. I hope this information is helpful.

Thanks.

[dya]

Thank you.

I checked both the “HVM Manager” and other Linux virtual machines.
The QEMU guest agent seems to be working properly, and the Morpheus agent is also installed.

I have some concerns about the shutdown process for HVM virtual machines.
At the very least, when shutting down the “HVM Manager,” I do not use the Web UI and instead manually execute “virsh shutdown” on the host.

I’m concerned that “virsh destroy” is being executed only about 10 seconds after “virsh shutdown” — such a very short amount of time.

I am only including partial logs.

----------------------------------------------------
Guest 1: Ubuntu Server 24.04
■Host
・journalctl
May 02 03:09:08 vme-iscsi3 sudo[340011]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown esxi-ubuntu
May 02 03:09:13 vme-iscsi3 systemd-machined[903]: Machine qemu-2-esxi-ubuntu terminated.
May 02 03:09:19 vme-iscsi3 sudo[340114]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy esxi-ubuntu

・QEMU Guest Agent check
# virsh domifaddr esxi-ubuntu --source agent
Name MAC address Protocol Address
-------------------------------------------------------------------------------
lo 00:00:00:00:00:00 ipv4 127.0.0.1/8
- - ipv6 ::1/128
enp3s0 52:54:00:4b:f5:16 ipv4 172.16.1.222/24
- - ipv6 fe80::5054:ff:fe4b:f516/64
#
# virsh qemu-agent-command esxi-ubuntu '{"execute":"guest-exec", "arguments":{"path":"/bin/ls"}}' --pretty
{
"return": {
"pid": 1582
}
}
#

■Guest
・journalctl
May 02 03:09:08 esxi-ubuntu systemd-logind[742]: System is powering down (hypervisor initiated shutdown).
May 02 03:09:11 esxi-ubuntu systemd-journald[346]: Journal stopped

■Observations
・The QEMU Guest Agent in the guest appears to be working normally.
・virsh shutdown is executed on the host, and the shutdown process starts in the guest.
・The guest’s journal ends with "Journal stopped".
・About 11 seconds after running virsh shutdown, virsh destroy is executed on the host (after the systemd-machined VM termination log).

----------------------------------------------------

----------------------------------------------------
Guest 3: Ubuntu Server 24.04 (HVM manager)
■Host
・journalctl
May 02 05:34:25 vme-iscsi1 sudo[8056]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh shutdown vme-iscsi-manager
May 02 05:34:36 vme-iscsi1 sudo[8076]: morpheus-node : PWD=/tmp ; USER=root ; COMMAND=/usr/bin/virsh destroy vme-iscsi-manager
May 02 05:34:38 vme-iscsi1 systemd-machined[911]: Machine qemu-1-vme-iscsi-manager terminated.

・QEMU Guest Agent check
# virsh domifaddr vme-iscsi-manager --source agent
Name MAC address Protocol Address
-------------------------------------------------------------------------------
lo 00:00:00:00:00:00 ipv4 127.0.0.1/8
- - ipv6 ::1/128
eth0 52:54:00:61:51:37 ipv4 172.16.2.100/24
- - ipv6 fe80::5054:ff:fe61:5137/64

#
# virsh qemu-agent-command vme-iscsi-manager '{"execute":"guest-exec", "arguments":{"path":"/bin/ls"}}' --pretty
{
"return": {
"pid": 58454
}
}
#

■Guest
・journalctl
May 02 05:34:25 vme-iscsi-manager systemd-logind[683]: System is powering down (hypervisor initiated shutdown).
May 02 05:34:28 vme-iscsi-manager systemd[1]: Stopped finalrd.service - Create final runtime dir for shutdown pivot root.

■Observations
・The QEMU Guest Agent appears to be working normally.
・virsh shutdown is executed on the host, and shutdown starts in the guest.
・Unlike Guest 1, there is no "Journal stopped" message at the end of the guest logs (last log is at 05:34:28).
・About 11 seconds after virsh shutdown, virsh destroy is executed (before the systemd-machined termination log).
・It appears that the guest was terminated by virsh destroy.

----------------------------------------------------

[dya]

This issue does not occur only when shutting down the “HVM Manager”; it also appears to affect other virtual machines (Linux and Windows).
In the case of Linux, the execution of virsh destroy after approximately 10 seconds seems too early, even considering systemd’s DefaultTimeoutStopSec=90s.
I believe that using the “Stop” operation for virtual machines from the Web UI is risky. This seems to be a problem that should be improved.

■ What I tested
・Windows Server 2025
・Configured a shutdown delay batch file using the Local Group Policy “Shutdown Script”
timeout /t 180 /nobreak
・Ubuntu Server 24.04
・Configured a systemd unit file to delay shutdown
# cat /etc/systemd/system/delay-shutdown.service
[Unit]
Description=Delay Shutdown Service
After=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecStop=/bin/sleep 180

[Install]
WantedBy=multi-user.target
#

■ Results
・Virtual Machine (Windows Server 2025)
→ virsh destroy was executed approximately 1 minute 10 seconds after virsh shutdown
・Virtual Machine (Ubuntu Server 24.04)
→ virsh destroy was executed approximately 10 seconds after virsh shutdown

■ Additional Information
This applies only to Ubuntu Server 24.04, but I also tested virtual machines with the same shutdown delay unit file on ESXi 8.0u3 and Proxmox VE 9.0.10. In both cases, the systems shut down normally.