Install, configure, and manage a custom virtual machine image.
Due on Monday 07/11/2022 at 11:59PM
You will install and configure a custom virtual machine image locally. You will use a tool called Packer, which allows you to create custom virtual machine images from one json file for multiple providers (e.g., Azure, AWS, Digital Ocean), hypervisors (e.g., Virtual Box), or tools designed for managing virtual machine environments (e.g., Vagrant). You will also manage your virtual machine environment using Vagrant.
In this lab, you will learn how to create a custom VM image (CentOS) for VirtualBox that can be used with Vagrant. The VM image must have the following installed:
- Apache httpd web server version 2.x
- PHP version 7.2
- nano text eitor
- MariaDB version 10.x
- firewalld
Setup/Prerequisites
- To follow this lab tutorial, you will need to have the following tools installed on your local machine:
- VirtualBox
- packer
- Vagrant.
- CentOS 7 ISO image file (Minimal ISO).
Step One - Generate SSH keys for SSH key-based authentication
We need to authenticate with the custom image using key-based SSH authentication. Create an ssh key-pair using
ssh-keygenon your local machine:$ ssh-keygen -t rsa -b 4096 -C "vagrant-key"Generating public/private rsa key pair. Enter file in which to save the key (/Users/khalid/.ssh/id_rsa): /Users/khalid/.ssh/vagrant_key Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/khalid/.ssh/vagrant_key. Your public key has been saved in /Users/khalid/.ssh/vagrant_key.pub. The key fingerprint is: be:c7:d1:bc:bd:75:7c:df:d7:a3:95:80:f5:66:13:33 vagrant-key The key's randomart image is: +--[ RSA 4096]----+ | | | | | . E | | o . +| | S + . = | | . . o +.o| | .. . o o*| | .o . ooB| | .. .o.=| +-----------------+
Step Two - Creating a Packer template and provisioning scripts
Packer uses a template file in JSON to create a virtual machine. The template file contains a set of properties and values. The main properties are: builders, provisioners, and post-processors.
- builders are tasks that produce an image for a single platform. It can be for example virtualBox, AWS, or Azure.
- provisioners are sections in Packer for running multiple scripts before launching the VM image (e.g., custom bash scripts to install or configure software and tools).
- post-processors are sections in Packer for running multiple scripts after the machine image has been created (e.g., converting a VirtualBox image into a suitable image or box for Vagrant).
Packer supports two builders for building an image for VirtualBox:
- virtualbox-iso - This builder is useful when you want to start from an existing ISO file. It creates a new VM image from the ISO file for VirtualBox, provisions the VM with your software and tools, and exports the VM to an image.
- virtualbox-ovf - Takes an input file in the Open Virtualization Format (ovf or ova) and runs provisioners on top of that VM, and exports that machine to create an image. In order to use this builder, you need to export the existing VM in your hypervisor into an open virtualization format (.ovf) archive file. To export a VM in VirtualBox, go to the File menu in VirtualBox, select Export Appliance, select the VM to export (e.g., CentOS), and export the VM file as .ova.
In this lab, will use the former builder, virtualbox-iso.
Building an image from an ISO using the virtualBox builder (virtualbox-iso)
Step 2.1 Adding Builders:
- Create a file named
centos-7-virtualbox.jsonin your local machine. This JSON will be considered the template for Packer:Create a directory and a JSON file on your local machine:
$ mkdir ~/packer-template $ cd ~/packer-template $ nano centos-7-virtualbox.jsonAdd the following content:
{ "builders": [ { "type": "virtualbox-iso", "guest_os_type": "RedHat_64", "iso_url": "/path/to/CentOS-7-x86_64-Minimal-xxxx/CentOS-7-x86_64-Minimal-xxxxx.iso", "iso_checksum": "fabdc67ff3a1674a489953effa285dfd", "iso_checksum_type": "md5", "ssh_username": "{{user `user`}}", "ssh_password": "{{user `password`}}", "ssh_timeout": "20m", "disk_size": "8192", "vm_name": "packer-centOS-7", "http_directory": "http", "boot_wait": "5s", "boot_command": [ "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos-7-kickstart.cfg<enter><wait>" ], "shutdown_command": "echo '{{user `password`}}' | sudo -S shutdown -P now", "vboxmanage": [ [ "modifyvm", "{{.Name}}", "--memory", "512" ], [ "modifyvm", "{{.Name}}", "--cpus", "1" ] ] } ] }You need to change the
iso_urlto the URL or the path to the .iso file you downloaded, and theiso_checksumcontains the md5 checksum of the downloaded .iso file. See below for more details.- The value of the
iso_checksumproperty is the the md5 checksum of the file. You can also run the command line toolmd5sumormd5against the .iso file to get checksum value. - We specified the disk size to create for the virtual machine in the
disk_sizekey with a value of 8192 in megabytes (8GB). - We select the amount of CPU and memory for this virtual machine as an array of commands to the VirtualBox’s vboxmanage command
- The value of the
This template has a provisioner of type shell that executes a file named
startup.sh. This file will be executed by the value of theexecute_commandproperty.- The
execute_commandproperty has the following value:echo 'root-password-here' | sudo -S sh '{{ .Path }}'. This pipes the root password into the stdin of sudo and execute the shell script as a root user.
- The
Step 2.2 Adding a kickstart installation file
When installing an OS, you will be presented with various options in the user interface. You will have to follow the on-screen installer steps and answer all the questions until the installation is complete. However, we would prefer to use an automated/unattended installation method to speed up and automate the process. Thus, we will create a Kickstart file containing the answers to all the questions that would normally be asked during a typical installation. CentOS’ Kickstart provides a way for users to automate a CentOS Linux installation with. That’s it, a kickstart file will let us install CentOS automatically and answer the installer’s questions without any user interaction.
The template file has a key named
http_directory, which contains the path to a directory that will be served using an HTTP server. The files in this directory will be available over HTTP and can be requested from the virtual machine. We need to create this directory and create our kickstart file inside this directory.Create a directory named
httpin the same directory that your Packer template file is in, and create a kickstart file namedcentos-7-kickstart.cfginside it:$ mkdir http $ nano http/centos-7-kickstart.cfgAdd the following content to the file
# Install OS instead of upgrade install cdrom # System authorization information auth --enableshadow --enablemd5 # Use text mode install text # Firewall configuration firewall --disabled firstboot --disable # Keyboard layouts keyboard us # System language lang en_US.UTF-8 # Network information network --bootproto=dhcp --device=eth0 --activate network --hostname=localhost.localdomain # Reboot after installation reboot # Root password rootpw vagrant # SELinux configuration selinux --enforcing # System services services --enabled="chronyd,NetworkManager,sshd" # Do not configure the X Window System skipx # System timezone timezone UTC --isUtc user --name=vagrant --password=vagrant --groups=vagrant,wheel # System bootloader configuration bootloader --location=mbr # Clear the Master Boot Record zerombr # Partition clearing information clearpart --all --initlabel autopart %packages --instLangs=en --ignoremissing -- @Base @Core @Development Tools openssh-clients sudo openssl-devel yum-utils readline-devel zlib-devel kernel-headers kernel-devel net-tools bash-completion vim curl rsync %end %post ## Install sudo and configure it to allow passwordless sudo for the "vagrant" user # This is important to allow vagrant to install and configure network and other tools yum install -y sudo echo "vagrant ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vagrant sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers yum clean all %end
Step 2.3 Adding Provisioners
We need to perform actions before launching the image. We want to upload our public key that we created in step 1 into the virtual machine image. We also want to execute a shell script to install tools on our vm image.
Open the Packer template file and add the following provisioners after the end of the builders array:
"provisioners": [ { "type": "file", "source": "{{user `public_key`}}", "destination": "{{user `key_destination`}}" }, { "type": "shell", "script": "./installer.sh", "execute_command": "chmod +x '{{ .Path }}'; echo '{{user `password`}}' | sudo -S sh '{{ .Path }}'" }, { "type": "shell", "script": "./set_ssh.sh", "execute_command": "chmod +x '{{ .Path }}'; echo '{{user `password`}}' | sudo -S sh '{{ .Path }}' {{user `user`}} {{user `key_destination`}}" } ]Create a shell script file that contains the commands to install and configure software in the VM image:
- Create a file named
installer.shin the same directory that your Packer template file is in. Edit it in your text editor and add the following content:
#!/bin/bash # You must execute this shell script as a root user # Exit immediately if any command exits with a non-zero exit status. set -e echo "Installing software" yum update -y # Install nano text editor yum -y install nano # Install Apache httpd yum -y install httpd # Change Apache root directory ownership chown apache:apache /var/www/html # SELinux security context chcon -Rt httpd_sys_content_t /var/www/html setsebool -P httpd_can_network_connect on # Install PHP 7.2 yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm yum -y install yum-utils yum-config-manager --enable remi-php72 yum -y install php # Install MariaDB 10.1 sudo printf "[mariadb]\nname = MariaDB\nbaseurl = http://yum.mariadb.org/10.1/centos7-amd64\ngpgkey = https://yum.mariadb.org/RPM-GPG-KEY-MariaDB\ngpgcheck = 1\n" | sudo tee /etc/yum.repos.d/MariaDB.repo yum -y install MariaDB-server MariaDB-client # Clean the cache yum clean all- Create another file named
set_ssh.shin the same directory that your Packer template file is in. Edit it in your text editor and add the following content:
#!/bin/bash # Exit immediately if any command exits with a non-zero exit status. set -e # usage if [[ $# -ne 2 ]] then echo "Error: Usage $0 user_name public_key" exit 1 fi # Disable SSH password-based authentication sed -n 'H;${x;s/\#PasswordAuthentication yes/PasswordAuthentication no/;p;}' /etc/ssh/sshd_config > new_sshd_config cat new_sshd_config > /etc/ssh/sshd_config rm new_sshd_config # move the public key mkdir -p /home/$1/.ssh mv $2 /home/$1/.ssh/authorized_keys chmod 700 /home/$1/.ssh chown -R $1:$1 /home/$1/.ssh chmod 644 /home/$1/.ssh/authorized_keys- Create a file named
Step 2.4 Adding post-processors
We need to perform actions once the image has been created. We want to convert our virtual machine image into a Vagrant box.
Open the Packer template file and add the following post-processors after the end of the provisioners array:
"post-processors": [ { "type": "vagrant", "compression_level": "7", "output": "builds/{{.Provider}}-centos7.box" } ]
Step 2.5 Adding sensitive user variables
We want our template to use some common variables and receive sensitive data such as keys, passwords, and other user data dynamically from the command line.
Finally, add the following user variables at the beginning of the template file:
"variables": { "key_destination": "/tmp/vagrant_key.pub" }, "sensitive-variables": [ "user", "password", "public_key", "private_key" ]
Step Three - Building the Virtual Machine with Packer
If you have followed the previous steps, you should end up with a packer template file that looks like the one below:
{
"variables": {
"key_destination": "/tmp/vagrant_key.pub"
},
"sensitive-variables": [
"user",
"password",
"public_key",
"private_key"
],
"builders": [
{
"type": "virtualbox-iso",
"guest_os_type": "RedHat_64",
"iso_url": "/path/to/CentOS-7-x86_64-Minimal-xxxx/CentOS-7-x86_64-Minimal-xxxx.iso",
"iso_checksum": "fabdc67ff3a1674a489953effa285dfd",
"iso_checksum_type": "md5",
"ssh_username": "{{user `user`}}",
"ssh_password": "{{user `password`}}",
"ssh_timeout": "20m",
"disk_size": "8192",
"vm_name": "packer-centOS-7",
"http_directory": "http",
"boot_wait": "5s",
"boot_command": [
"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos-7-kickstart.cfg<enter><wait>"
],
"shutdown_command": "echo '{{user `password`}}' | sudo -S shutdown -P now",
"vboxmanage": [
[
"modifyvm",
"{{.Name}}",
"--memory",
"512"
],
[
"modifyvm",
"{{.Name}}",
"--cpus",
"1"
]
]
}
],
"provisioners": [
{
"type": "file",
"source": "{{user `public_key`}}",
"destination": "{{user `key_destination`}}"
},
{
"type": "shell",
"script": "./installer.sh",
"execute_command": "chmod +x '{{ .Path }}'; echo '{{user `password`}}' | sudo -S sh '{{ .Path }}'"
},
{
"type": "shell",
"script": "./set_ssh.sh",
"execute_command": "chmod +x '{{ .Path }}'; echo '{{user `password`}}' | sudo -S sh '{{ .Path }}' {{user `user`}} {{user `key_destination`}}"
}
],
"post-processors": [
{
"type": "vagrant",
"compression_level": "7",
"output": "builds/{{.Provider}}-centos7.box"
}
]
}
Step 3.1 Validate the template
- Before building the image, we want to check that our template is valid. We need to pass the user name, password, and public key into packer from the command line as user values.
$ packer validate -var "user=vagrant" -var "password=vagrant" -var "public_key=/path/to/public/key" -var "private_key=/path/to/private/key" ./centos-7-virtualbox.json
Step 3.2 Building the image from the template
$ packer build -var "user=vagrant" -var "password=vagrant" -var "public_key=/path/to/public/key" -var "private_key=/path/to/private/key" ./centos-7-virtualbox.json
This may take a few minutes or more to build the image, start the kickstart script, and install the provisioning scripts.
The action defined in the post-processor section of the template will take the build and convert it into a Vagrant box stored at builds/virtualbox-centos7.box.
Step Four — Start and provision the Vagrant box
You should have a Vagrant Box built by Packer. Vagrant Box is the package format for Vagrant environments that include the base image and the additional tools installed on top of the image. This box can be used by anyone to create an identical virtual environment or shared on the public Vagrant box repository.
Step 4.1 - Create a Vagrantfile
You need to create a file named
Vagrantfileat the same working directory that you created the Packer template at.$ cd ~/packer-template $ nano VagrantfileAdd the following content to the Vagrantfile
# -*- mode: ruby -*- # vi: set ft=ruby : user_name = ENV['vagrant_user'] or 'vagrant' private_key_path = ENV['vagrant_private_key'] public_key_path = ENV['vagrant_public_key'] if private_key_path == '' puts("Missing private key") exit(1) elif public_key_path == '' puts("Missing public key") exit(1) end Vagrant.configure("2") do |config| config.vm.box_check_update = true config.vm.box = "centos-7-vagrant" config.vm.box_url = "file://./builds/virtualbox-centos7.box" config.vm.hostname = "virtualbox-centos7" config.ssh.host = "127.0.0.1" config.ssh.port = 2222 config.ssh.private_key_path = [private_key_path] config.ssh.insert_key = false config.vm.network "forwarded_port", guest: 22, host: 2222, host_ip: "127.0.0.1", id: 'ssh' config.vm.provision "shell", path: "./set_ssh.sh", args: "#{user_name} #{public_key_path}" config.vm.provider :virtualbox do |vbox| vbox.gui = false vbox.customize ["modifyvm", :id, "--natdnshostresolver1", "on"] vbox.customize ["modifyvm", :id, "--ioapic", "on"] vbox.name = "centos-7-vagrant" end end
Step 4.2 - Create, start, and access the vagrant box
Store the path to the public and private keys in two environment variables
vagrant_public_keyandvagrant_private_keythat are defined in the Vagrantfile.From the command line, export the following environment variables:
On a Unix-like system such as macos or Linux, run:
export vagrant_private_key=/path/to/private/key export vagrant_public_key=/path/to/public/keyOn Windows, export may not work, so run:
set vagrant_private_key=/path/to/private/key set vagrant_public_key=/path/to/public/key
Validate the Vagrantfile
vagrant validateCreate your Vagrant Box and provision the vagrant environment in VirtualBox.
vagrant upYou will be prompted to enter the passphrase you chose in the first step. Vagrant needs this to provision the created box. If you have encountered an error, refer to the notes section below1.
Step 4.3 - Connect to the vagrant box via SSH and manage it using vagrant
Log in to your VM instances using SSH
vagrant sshYou will be prompted to enter the passphrase you chose in step 1.
Check the installed software on your vagrant box
php -v mysql -vStop the vagrant machine. You need to exit from the connected VM and then execute
vagrant halt.exit vagrant halt
Notes
- On Windows, if you get an error message that says “unknown encoding name”, try to change the system locale on your system:
- Control Panel » Region » Administrative tab » change system locale to English (United States).
- On CentOS, there’s a different tool called packer included in the PATH, so when installing packer, you need to resolve the name conflict. Run
which -a packerto see if you get an error which means you have a name conflict. Then, rename the installed packer tool into a different name such as packer.io, call it using the absolute path name, or create a symbolic link with a different name (e.g.,ln -s /usr/local/packer /usr/local/bin/packer.io). Please refer to this link for more information.
Submission
Submit your answers with screenshots showing the commands you executed as a PDF file by the due date.