This summer, I was tasked with a workshop: spend roughly 8 hours of the day teaching Ansible. The workshop went great. I got good feedback to improve and good remarks. I have grown to love to teach, and I always appreciate what people think about how I do. As a part of that workshop, I wanted to have VMs available for any of the people there that couldn’t virtualize on their own computers. The group was small, with 10 members in the workshop so I felt I could take my extra Thinkpad T480 and throw SmartOS on there and let that be the virtualization platform. I configured it at home for the trip.
My first step in the workshop was to show the attendees a working playbook that they could copy/paste and then run on my server to spin up virtual machines. I was ready, flew to the conference, and the next day was showtime.
The Troubles Begin
Before the conference, I requested a block of IPs on the same network so that we could all be on the same network as the virtualization host. I also needed an ethernet port. Some of the network issues this day included the port in the wall not being connected to the right port on the patch panel and switch so there was no access, and then come to find out the cable I brought decided to die along the way. Thankfully there were extras, but it was fun to find these things out as the workshop was starting. Additionally, The DHCP addresses were not correct, so static IPs were needed.
Showing My Ignorance
As I was trying to figure these things out, SmartOS was spending an eternity booting. I could not get the system to boot, only single user mode! I thought maybe I had dorked something up, and I didn’t know what the fix was. So I figured the best thing was to reinstall the OS. What I later found out is SmartOS will get stuck booting for an eternity if it is configured to get a DHCP address but there is no DHCP answer. This situation left me with a questionable network, no hypervisor, and the clock was ticking, it was my teaching time.
The Resolution
With my Framework laptop handy, I was able to confirm that the networking was complete
and functional. By then, it was too late. I had already chosen the path of destruction.
I was already reinstalling SmartOS. The bad news though was because there existed a
previous zpool on the disk, I had to dd the beginning/end of the disk to reinstall.
I was able to wipe the beginning/end of the disk at this point and was plugging through
the installer. As I was doing that, I was teaching the workshop, getting up to the point
of running our first playbook. In about 10 minutes or so after starting the SmartOS installer,
I had a running system! I knew which Ubuntu 22.04 image I wanted to pull down, so
I grabbed it with imgadm and was then able to tell the users to add their ssh pubkeys
to the authorized_keys file on the SmartOS server. They ran their first playbook, which
contained community.general.vmadm and installed the new Ubuntu 22.04 BHYVE zones.
Great success!
The Failure and Success
This could be counted as a failure if this was any other operating system. If I had to nuke Debian or FreeBSD or something else, it would take a lot more for me to get from 0 to deploying VMs. I didn’t have to nuke my install, but in a hurry (and at the time not knowing the best way to fix the dhcp issue), I knew my setup was simple and would not require much effort to put things back together. As for the success, things timed out just right despite reinstalling mid-presentation and everything ran smoothly from there. I told the attendees to update their VM, take a snapshot, and then from there if there was ever a need, it was easy for us to rollback and fix mistakes. It was daring for me to take the OS out on the road and hook it up to an unfamiliar network, but all was well and the workshop was a success.