Vagrant Virtual Machines on M1 Arm Macs

Setting up an alternative workflow for Vagrant & Virtual Box on the new ARM chip Macs.

Got a lovely new M1 ARM Mac? All going great until you did vagrant up? That’s exactly what happened to me. I knew there would be some teething problems - (and at the risk of sounding old) I have been here before when Apple transitioned from Motorola to Power PC , and then to Intel 10 years later.

So, I did the sensible thing, waited a year or so for everyone to catch up and for Apple to iron out the bugs. And to be fair, it was pretty painless, until I tried to run our local dev server using Vagrant and Virtual box.

What’s the issue?

In a nutshell, it turns out that Virtual Box is built to run Virtual machines that support the Intel chip set — there is no emulation going on, it uses the host hardware as-is.

So the VM’s it runs will never run on ARM based macs.

What are the options ?

There are various ways to go; using an emulator like UTM, or VMWare’s tech preview for Mac M1, or even set up docker to emulate x86 images and launch it with vagrant.

UTM doesn’t play well with Vagrant at the moment, so that was out the question, and I didn’t want to go over all the fiddly set up of docker as an emulator and coercing it in to running like a VM.

And VMWare’s latest tech preview for M1 Macs stood out to me as the best option — it’s a native app for M1 Mac’s, runs ARM based VM images with no emulation, and is fully compatible with Vagrant.

So here we go….

Setting up VMWare

1. Download VMWare tech preview

First off you need the latest tech preview of VMWare Fusion: https://customerconnect.vmware.com/downloads/get-download?downloadGroup=FUS-PUBTP-2021H1

You will need to sign up to get the preview licence — and no doubt pay for it once its released.

Download and install the preview.

2. Remove / Rename the old VMWare app (if you have it)

Vagrant looks for the VMWare app by name — “VMware Fusion”, and the Tech Preview is called “VMware Fusion Tech Preview”.

So you need to remove/rename the original VMWare Fusion app.

3. Create a Alias / Sim link to the Tech Preview

Then create a sim link in the applications folder called “VMWare Fusion” pointing to “VMware Fusion Tech Preview”.

ln -s "/Applications/VMware Fusion Tech Preview.app" "/Applications/VMware Fusion.app"

Now Vagrant will find it.

Grab a ARM Linux distro

Next you need an ARM based linux base box for vagrant , preferably one that has been created for use with VMWare.

Vagrant has lots listed here

I went with Ubuntu Server 20.

Configure Vagrant

The vagrant file will pretty much stay the same, but there are a few tweaks to make.

1. Update the config.vm.box property

Update config.vm.box to point to the new ARM box.

2. Update any other OS specific settings

As we had come from Centos to Ubuntu, a few of the config.vm.synced_folder mount options had to be tweaked, mainly adding the allow_other option.

3. Update the config.vm.provider settings

The core part to change is the config.vm.provider settings.

vmware.vmx['ethernet0.pcislotnumber']

When booting the box for the fist time, Vagrant may warn you about the vmware.vmx['ethernet0.pcislotnumber'] setting. It maps PCI slots between the host and the guest — but thankfully Vagrant will tell you want to set it to!

WARNING: The VMX file for this box contains a setting that is automatically overwritten by Vagrant
WARNING: when started. Vagrant will stop overwriting this setting in an upcoming release which may
WARNING: prevent proper networking setup. Below is the detected VMX setting:
WARNING: 
WARNING:   ethernet0.pcislotnumber = "33"
WARNING: 
WARNING: If networking fails to properly configure, it may require this VMX setting. It can be manually
WARNING: applied via the Vagrantfile:
WARNING: 
WARNING:   Vagrant.configure(2) do |config|
WARNING:     config.vm.provider :vmware_desktop do |vmware|
WARNING:       vmware.vmx["ethernet0.pcislotnumber"] = "33"
WARNING:     end
WARNING:   end
WARNING: 
WARNING: For more information: https://www.vagrantup.com/docs/vmware/boxes.html#vmx-allowlisting

vmware.allowlist_verified

Even when you have set the pcislotnumber you will still be warned about it each time you do a vagrant up . To suppress this message you can set vmware.allowlist_verified to true

vmware.gui

I did find one issue, when launching with vagrant up it hangs waiting for the server to start, before it SSH’s into it to provision the box.

Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Checking if box 'bytesguy/ubuntu-server-20.04-arm64' version '1.0.0' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Starting the VMware VM...
==> default: Waiting for the VM to receive an address...

The only way round this so far is to either have VMWare running in the background already, or set the vmware.gui property to true — which means vagrant will launch VMWare for you. Not ideal — but at the moment that is still a known issue.

config.vm.provider

The final config looks something like this

config.vm.provider :vmware_desktop do |vmware|
 vmware.vmx["ethernet0.pcislotnumber"] = "33"
 vmware.vmx["memsize"] = "4096"
 vmware.vmx["numvcpus"] = "2"
 vmware.gui = true
 vmware.allowlist_verified = true
end

Now you can run vagrant up as before and use your shiny ARM based VM on a Mac M1.