In the previous post, I covered getting existing VMs out of VMWare ESXi and getting SmartOS ready to launch. Now it's time to boot up SmartOS and start converting our vmdk files.
Once SmartOS boots up, it will check for an existing "zones" zfs volume, which is where the VMs are stored. Since this is the first boot on this hardware, we don't yet have a zones volume. The helpful setup script runs and prompts for networking info, root password, and then asks us on which devices our global zone should exist. It also lists the available devices, using classic Solaris notation: c0t0d0
For the uninitiated, this breaks down to Controller 0, SCSI Target 0, Disk 0. This site covers the notation in more detail, for those interested. Back to the topic at hand!
As mentioned, the setup script is asking us which devices should be used for the global zone. Not so helpfully, we only have the device notation and no other information, like the output of format
. Either have only the drives meant for the zones volume connected at first boot or guess the right devices. The setup script will determine which zfs pool layout to use based on the number of drives given. For example, two devices will be put into a mirror, while five will be put into a raidz. It will not allow for raidz2 or raidz3. Luckily, the docs explain how to change this after the fact. The setup script creates the pool and reboots.
_____ ____ ____ . . _____ _____ | .-. . . .-. :--. |- __ __ ;| || |(.-' | | | _____ _____ `--' `-' `;-| `-' ' ' `-' ____ ____ / ; Joyent Live Image v0.147+ _____ `-' build: 20131213T023304Z 00-1e-2a-b7-57-4e wscons login:
The ASCII art logo is a nice touch. Log in or ssh in as root and bask in the Solaris-bred goodness. Now it's time to take the VMDKs extracted earlier and pull them into SmartOS. This is where networked storage is helpful to keep the large VMDKs accessible. Let's start with a simple CentOS 6.4 system.
We need to know how large of a disk the VM expects.qemu-img
provides this sort of info.
[root@00-1e-2a-b7-57-4e ~]# qemu-img info -f vmdk /zones/temp/centos6-disk1.vmdk image: /zones/temp/centos6-disk1.vmdk file format: vmdk virtual size: 40G (42949672960 bytes) disk size: 6.9GNext, we need a JSON document describing the attributes of the VM. We'll use the
virtual size
value to set the disk's size
value in megabytes.
{ "brand": "kvm", "vcpus": 1, "ram": 512, "resolvers": "192.168.240.10", "disks": [ { "boot": true, "model": "virtio", "size": 40960 } ], "nics": [ { "nic_tag": "admin", "model": "virtio", "ip": "192.168.240.50", "netmask": "255.255.255.0", "gateway": "192.168.240.1", "primary": 1 } ] }
Pay attention to the value of resolvers. It's slightly more complicated to update that value later, so you want to point it to the correct DNS server. It is worth noting that some operating systems, such as any Windows, do not support virtio without extra drivers. Check out the docs to see all valid attributes.
We can validate our json withvmadm validate
.
[root@00-1e-2a-b7-57-4e ~]# vmadm validate create -f centos6.json VALID 'create' payload for kvm brand VMs.With that written and validated, rerun the command without validate.
[root@00-1e-2a-b7-57-4e ~]# vmadm create -f centos6.json Successfully created 2c922230-7878-11e3-981f-0800200c9a66
SmartOS uses UUIDs to access its VMs. Without copy/paste, this is quite cumbersome. Fortunately, by creating persistent dot files and appending this wrapper function, vmadm can use unique alias names in place of the UUID for access.
We can check that the vm was actually created with eithervmadm list
or zfs list
.
[root@00-1e-2a-b7-57-4e ~]# zfs list NAME USED AVAIL REFER MOUNTPOINT ... zones/2c922230-7878-11e3-981f-0800200c9a66 82K 10.0G 82K /zones/2c922230-7878-11e3-981f-0800200c9a66 zones/2c922230-7878-11e3-981f-0800200c9a66-disk0 40.1G 1.36T 40.1G - ...It's there! At this point we have an empty VM with hardware characteristics and an IP. The VM is ready to be migrated with help from
qemu-img
:
[root@00-1e-2a-b7-57-4e ~]# qemu-img convert -f vmdk -O host_device wobble-disk1.vmdk /dev/zvol/rdsk/zones/2c922230-7878-11e3-981f-0800200c9a66-disk0 ... lots of time ... [root@00-1e-2a-b7-57-4e ~]#
This command takes quite a while and, unfortunately, the version of qemu-img
in SmartOS does not have the patched in -p
flag to indicate progress. Relax, have some tea, and document things while you wait. Is it done? Good, let's prove it.
vmadm
command. vmadm list
will list all VMs.
[root@00-1e-2a-b7-57-4e ~]# vmadm list UUID TYPE RAM STATE ALIAS a608f729-2569-40b0-9b22-2d41734a73c3 KVM 512 running mongoweb c9475323-2dec-462f-bc89-c1f1d5274c3c KVM 512 running pxe f873e475-dc43-45b8-8128-b88ef112a2c6 KVM 512 running dbbackup 91e4473d-13fb-4c96-a64f-3ac5aba9e50b KVM 1024 running testweb ac070f31-8f5a-4211-9ee3-e876937e42ad KVM 1024 running puppet e5476ad0-1f69-4eb9-b974-bd862a52cc11 KVM 1024 running kms 62d7e737-629e-45d5-9e80-0556b16e2972 KVM 2048 running db01 b354b725-342f-4300-93c2-b9297cb7553b KVM 2048 running ad01
This is my current setup. If the centos6 VM is the only VM in your environment, it will display just that VM and, unless specified in the JSON definition, no alias. There are lots of options to alter output of this command to include more, less, or different information, detailed in the documentation.
There may be some gotchas here to solve before you can ssh in. SmartOS provides VNC access for each VM with a randomized port.[root@00-1e-2a-b7-57-4e ~]# vmadm info 2c922230-7878-11e3-981f-0800200c9a66 ... "vnc": { "host": "192.168.240.8", "port": 57755, "display": 51855 }, ...
Using the vnc client of your choice, connect to your system. You may need to send ctl-alt-del to restart, as it might have attempted to boot before the image copied over.
When I moved CentOS 6.4 VMs from ESXi, their ethernet interface changed from eth0 to eth1. Fix by doing the following.
[root@00-1e-2a-b7-57-4e ~]# cd /etc/sysconfig/network-scripts/ [root@00-1e-2a-b7-57-4e ~]# cp ifcfg-eth0 ifcfg-eth1Then, modify the MAC address line. However, that's not scalable, so the better option is to deploy SmartOS and create the empty VMs first, grab the MAC address out of
vmadm get UUID
, and then create ifcfg-eth1
before extracting the VM from ESXi. I should add that to list of "things to sketch out."
For Windows Server VMs, you will need to alter their JSON description file to use disk model ide
instead of virtio
. Grab the correct vmtools iso here and place it in the root zone directory for the windows VM. Now, boot the VM with:
[root@00-1e-2a-b7-57-4e ~]# vmadm boot UUID order=cd,once=d cdrom=/vmtoolsXYZ.iso,ide
Alright, now SmartOS should be serving our ESXi VMs and, hopefully, there should be no problems at all. Now we can dive into SmartOS goodness and really make use of the Solaris-bred technologies that make it so great.