Cloning + grub2 + mtab issue

CalcUA — Thu, 04 May 2017 09:09:53 GMT

Hi all,

I have the following issue. I have a CentOS 7 node which I can succesfully backup using CMU 7.2. But when I clone the image, it fails to properly restore /etc/mtab which causes several things in the boot process to fail. In the backed up image, the file /etc/mtab is a softlink to /proc/self/mounts, but in the cloning process I see this file gets removed:

 [CerbereClient] Set the boot: Configuring Bootloader
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 rm -f /etc/mtab  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 rm -f /etc/mtab   &> /opt/cmu/tmp/CMU_TEMPO_FILE_wN1S3r<--  cmu_system_helper
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 mv /etc/fstab /etc/fstab.cmu.labels  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mv /etc/fstab /etc/fstab.cmu.labels   &> /opt/cmu/tmp/CMU_TEMPO_FILE_Wy06hk<--  cmu_system_helper
 [CerbereClient] touching fstab: chroot /opt/cmu/mnt/sda3 touch /etc/fstab  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 touch /etc/fstab   &> /opt/cmu/tmp/CMU_TEMPO_FILE_Svu9wc<--  cmu_system_helper
 [CerbereClient] chroot mount: chroot /opt/cmu/mnt/sda3 mount -f /dev/sda3 /  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mount -f /dev/sda3 /   &> /opt/cmu/tmp/CMU_TEMPO_FILE_kJwZM4<--  cmu_system_helper
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 mv /etc/fstab.cmu.labels /etc/fstab  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mv /etc/fstab.cmu.labels /etc/fstab   &> /opt/cmu/tmp/CMU_TEMPO_FILE_os8G3W<--  cmu_system_helper
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 mount /proc  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mount /proc   &> /opt/cmu/tmp/CMU_TEMPO_FILE_GlkblP<--  cmu_system_helper
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 mount /boot  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mount /boot   &> /opt/cmu/tmp/CMU_TEMPO_FILE_Y3SzDH<--  cmu_system_helper
 [CMUTools          ] log:mount: special device /dev/sda1 does not exist

 [CerbereClient] Non zero returned status: 8192 (chroot /opt/cmu/mnt/sda3 mount /boot ), may not be fatal if you don't have a separate /boot CMUCrbrInstallBootLoader
 [CerbereClient] Executing: chroot /opt/cmu/mnt/sda3 mount /usr  CMUCrbrInstallBootLoader
 [CMUTools          ]  about to run: -->chroot /opt/cmu/mnt/sda3 mount /usr   &> /opt/cmu/tmp/CMU_TEMPO_FILE_XCoQWz<--  cmu_system_helper
 [CMUTools          ] log:mount: can't find /usr in /etc/fstab

 [CerbereClient] Non zero returned status: 256 (chroot /opt/cmu/mnt/sda3 mount /usr ), not fatal if /usr is not a separate partition CMUCrbrInstallBootLoader
 [CerbereClient]  (info) -->/opt/cmu/mnt/sda3//boot/grub/menu.lst<-- file does not exist CMUCheckFileExists
 [CerbereClient]  (info) -->/opt/cmu/mnt/sda3//boot/boot.scr<-- file does not exist CMUCheckFileExists
 [CerbereClient] Sends status CERBST_CLONED for node r6c3cn07
 [CerbereServer] Sends status CERBST_READY_TO_SERVE for node r6c3cn07
 [CerbereDB] Locking database
 Node r6c3cn07 changing status: CERBST_UNKNOWN -> CERBST_READY_TO_SERVE
 [CerbereDB] Unlocking database
 [CerbereDB] Locking database
 [CerbereDB] Nb to clone: 0, nb waiting producers: 0
 [CerbereDB] No more potential consumer while node r6c3cn07 looking for one
 [CerbereDB] Unlocking database
 [CerbereServer] No more consumer for console
 [CerbereNB] Netboot cleaning (/etc)
 [CerbereNB] Netboot opens /etc/exports
 [CerbereNB] Removing netboot tags in file /etc/exports
 [CerbereDB] Database report:
 [CerbereDB] 	                               | cloned | error | unknown
 [CerbereDB] 	                          r6e1 |      0 |     0 |       0
 [CerbereDB] 	                         Total |      0 |     0 |       0
 [CerbereDB] no node in error

Furthermore, cloning works, but I've noticed an issue with it installing the grub2 bootloader. In the log above it mentions "special device /dev/sda1 does not exist" which happens to be the /boot partition. Nevertheless, I managed to get it to work by explicitly adding the following code to reconf.sh:

      partition_boot="$(cat ${CMU_RCFG_PATH}/etc/fstab |awk '$2 ~ /^\/boot$/ {print $1}'|sed 's;^/dev/;;')"
      disk_root="${partition_boot:0:-1}"
      if ! mount -l |grep -q ${CMU_RCFG_PATH}/dev ; then
        mount -o bind /dev ${CMU_RCFG_PATH}/dev
      fi
      if ! mount -l |grep -q ${CMU_RCFG_PATH}/sys ; then
        mount -t sysfs none ${CMU_RCFG_PATH}/sys
      fi
      if ! mount -l |grep -q ${CMU_RCFG_PATH}/proc ; then
        mount -t proc none ${CMU_RCFG_PATH}/proc
      fi
      if ! chroot ${CMU_RCFG_PATH} mount -l |grep -q ${CMU_RCFG_PATH}/boot ; then
        chroot ${CMU_RCFG_PATH} mount /dev/${partition_boot} /boot
      fi
      chroot ${CMU_RCFG_PATH} grub2-install --recheck --no-floppy /dev/${disk_root}
      chroot ${CMU_RCFG_PATH} grub2-mkconfig -o /boot/grub2/grub.cfg
      sync
      sleep 3
      for mount in /boot ; do
        if chroot ${CMU_RCFG_PATH} mount -l |grep -q $mount ; then
          chroot ${CMU_RCFG_PATH} umount $mount
        fi
      done
      for mount in ${CMU_RCFG_PATH}/proc ${CMU_RCFG_PATH}/sys ${CMU_RCFG_PATH}/dev ; do
        if mount -l |grep -q $mount ; then
          umount $mount
        fi
      done

I'm wondering:

Where is the "rm /etc/mtab" done, and can I avoid it from doing that, or can I restore it somehow?
Do I have to do the grub2-install steps myself in reconf.sh or is that taken care of by the CMUCrbrInstallBootLoader process?
Where I can find information on what exactly the CMUCrbrInstallBootLoader process does? (Is it a script file some where?)

Many thanks for your reply,

Regards,

Franky

Re: Cloning + grub2 + mtab issue

TTr — Thu, 04 May 2017 12:57:03 GMT

>...it fails to properly restore /etc/mtab which causes several things in the boot process to fail

What are the things that fail? They probably have nothing to do with mtab. The mtab file is excluded from the cloning process on purpose. The mtab file is a dynamic file which gets generated during boot up as the kernel mounts each the disks in /etc/fstab or any other custom start-up scripts that mount disks. Then while the server is running, mtab gets dynamically updated if you manually mount or unmount any disks.

Re: Cloning + grub2 + mtab issue

Abhishekc — Fri, 05 May 2017 06:28:09 GMT

Hello,

RHEL 7/CentOS 7 is not supported in CMU v7.2. Regarding the cloning failures you are seeing are about missing grub configuration file. In RHEL 7/CentOS 7 the grub configuration file has been changed from /boot/grub/grub.conf to /boot/grub2/grub.cfg. And, CMU v7.2 doesn't have support for grub2. Hence the failures.

CMU v7.2 is very old version and is no more supported. Please upgrade to latest version of CMU v8.1 which has support for RHEL/CentOS 7.x along with many more new features and to get the official support.

Regards,

Abhishek Chintala

Re: Cloning + grub2 + mtab issue

CalcUA — Thu, 11 May 2017 08:12:09 GMT

Hi,

I have upgraded/installed CMU v8.1 and I can confirm that the cloning works properly (i.e. the /etc/mtab issue is solved). Thanks for your advice!

Regards,

Franky

topic Re: Cloning + grub2 + mtab issue in Server Clustering

Cloning + grub2 + mtab issue

Re: Cloning + grub2 + mtab issue

Re: Cloning + grub2 + mtab issue

Re: Cloning + grub2 + mtab issue