Recently, I participated in a project that required applying all my skills in creating a custom operating system image. Depending on the server group, different static settings needed to be applied to each server. Additionally, each server group had different storage requirements. Since there aren’t many descriptions online on how to make an image meet expectations, I decided to share my experience.
To give you an idea, in less than 2 hours, more than 60 DL 385 gen 11 servers were deployed using AWX. All that was required from the client was to enter the ILo IP address in the inventory and specify variables for the static IP.
AWX WorkflowThe AWX workflow consists of the following steps:
- Create an image based on the user-data configuration with variable substitution.
- Mount the disk via SSH (ILo) (boot after next restart).
- Restart the server.
- Wait 20 minutes for the OS to install and restart.
- Execute the post-install script.
The most interesting part is the user-data configuration. Cloud-init is essentially the standard for configuring Linux OS. However, when configuring Ubuntu, Canonical decided to expand the capabilities and introduce a richer set of features.
The structure of the user-data is as follows:
#cloud-config
autoinstall:
version: 1
locale:
identity:
hostname:
ubuntu-pro:
interactive-sections:
early-commands:
ssh:
refresh-installer:
keyboard:
source:
network:
proxy:
apt:
package_update:
package_upgrade:
timezone:
kernel:
packages:
oem:
codecs:
drivers:
storage:
user-data:
late-commands:
error-commands:
shutdown: reboot
It is important to note that three syntaxes are used here: subiquity (for all except user-data and storage), curtin (for storage), and cloud-init (for user-data).
StorageUsing the curtin syntax, we have the ability to create LVM, DM_Crypt, RAID (software), and more. Here is an example configuration using RAID:
storage:
config:
- type: disk
id: disk-0
ptable: gpt
path: /dev/vda
wipe: superblock-recursive
grub_device: true
- type: disk
id: disk-1
path: /dev/vdb
wipe: superblock
- type: disk
id: disk-2
path: /dev/vdc
wipe: superblock
- type: partition
id: part-0
device: disk-0
size: 1048576
flag: bios_grub
- type: partition
id: part-1
device: disk-0
size: 21471690752
- id: raid-0
type: raid
name: md0
raidlevel: 1
devices: [disk-2, disk-1]
ptable: gpt
- type: partition
id: part-2
device: raid-0
size: 10737418240
- type: partition
id: part-3
device: raid-0
size: 10735321088,
- type: format
id: fs-0
fstype: ext4
volume: part-1
- type: format
id: fs-1
fstype: xfs
volume: part-2
- type: format
id: fs-2
fstype: ext4
volume: part-3
- type: mount
id: mount-0
device: fs-0
path: /
- type: mount
id: mount-1
device: fs-1
path: /srv
- type: mount
id: mount-2
device: fs-2
path: /home
The partition size can be specified with the size key. Sizes must be given with an appropriate SI unit, such as B, kB, MB, GB, TB, or using just the appropriate SI prefix, i.e. B, k, M, G, T… Also possiple to user -1 for provide all available storage for part
It is also important to pay attention to the wipe: superblock-recursive parameter. This parameter is useful to ensure that embedded superblocks on a disk aren’t rediscovered during probing, but in practice, this is not always the case. In a real project, we had to use a command to guarantee 100% success during installation, as the system previously encountered errors.
Example command for early-commands:early-commands: dd if=/dev/zero of=$DISK_NAME bs=1M count=10
In this case, you can perform a complete system configuration to ensure the installation happens with a single click. While we were stabilizing the system and understanding the intricacies of user-data configuration, we performed more than 30 test installations, not counting the number of installations on a virtual machine. In our case, we eliminated all interactive moments of the system installation, as it was necessary for everything to be automated. As a result, we received positive feedback from the client.