- Be able to host many VMs of different OS (well, VirtualBox is exactly that :) )
- Except for the gateway VM, the guests VMs should require only minimal configuration (eg, single interface, no manual routes, manual IP+GW+single DNS acceptable) to access:
- Host machine
- LAN resources
- Internet (client only, optionnal port forwarding for servers not covered here)
- Each other
- A unique centralized DNS (on host or gateway VM) to resolve eachother (Dynamic DNS via DHCP not madatory, /etc/hosts or other on single machine acceptable), and forwarding DNS for internet names resolving. (optional use of proxy host resolver for LAN names resolve)
- The guests VMs should have either static, or DHCP assigned IPs (or a mix of both), but the Host Only subnet keeps stable addressing whatever network the host connects to. Guest musn't be impacted whether the host connect to LAN and/or internet via Ethernet,WiFi,even dialup or any other as long as ther's IP connectivity (eg a virtual network / test and development environment on a laptop).
- Optionally, the gateway VM can host various VPN clients and also act as a gateway to remote sites for both the other guests, and the host itself. (can help avoid multiple different VPN clients installed on a Windows host)
First a big spoiler before diving into the TL;DR details: This can't yet (as of 2022-02) be achieved by some clever settings in VirtualBox alone.
The trick is setting up a minimalist Debian (or other to your preference) guest VM with both a Host-only network interface, and a NAT interface, acting as a gateway for the other VMs, in place of the built-in integrated gateway (and optionally DNS and DHCP) service of VirtualBox.
So involves significantly more effort (not this much, as the GW VM is a pretty minimal one but still), so if both you don't need a full virtual network as a simulated test environment, and you don't plan to have more than 3-4 guest VMs, it'll obviously be simpler going with multiple network interfaces on each guest instead.
Problems using VirtualBox standard configurations:
The "easiest" configuration that gives access to everything listed, is obviously using VirtualBox's bridged interface.
Though it has its long lot of problems, that aren't this terrible for a fixed wired infrastructure, but a total deal-breaker for use on a roadwarrior:
- Might sometimes cause problems with MAC filtering things (WiFi APs)
- Reliant on a single physical network interface : don't know if the connectivity between guests and host would still work when interface not physically connected.
- And so would need frequent reconfiguration if frequently switching between Ethernet and Wifi or other, but even:
- As user scottgus1 pointed out in https://forums.virtualbox.org/viewt..., WiFi and bridged mode doesn't mix well at all.
Sure enough, having both multiple test VMs, and a complex simulated virtual network, all on a development laptop is not a really common use case, though can happen if you're a freelance working on network applications development :)
The requirement to isolate the guests configuration from the details of the physical connection totally points towards using a routed solution instead of a bridged one.
Leaving us with basically two other network options for VirtualBox guests:
- Host-only network (internal is the same, except without host <=> guests connectivity) satisfies 1, 4 and 6
- NAT network (simple NAT is the same without guests access to eachother) satisfies everything except host <=> guests connectivity.
Well, while host to guests access could be set-up for specific connections with port forwarding, and that would be a preferred method on a servers hosting infrastructure, for development, testing and simulations, i wanted to keep the ability to use the host as a client to any of the guest server VMs, exactly the same (at IPv4 level) as if it was connected on the same network.
Or using two network interfaces on each guest, if you have not too many of them, and you don't mind having a network configuration different to production servers on simulated ones in VMs.
Hence, trying to solve those topics:
TODO: some history about the flawed and unreliable solution using a custom bridged-behind-NAT gateway
The solution : Making a custom NAT+host-network gateway VM
TODO: VirtualBox network configuration.
Guest VM gateway installation:
Obviously any other Linux distro would do, just better use one that can install without graphical desktop environment to avoid wasting hardware resources and slowing its start. For those familiar with Linux router distros like pfsense or OpenWRT, i suppose those ones would be even better suited, but as many of my guests are Debian, i don't need web interface for config, and Debian is my OS of choice, for consistency i used this one, so instructions provided here, while easily translatable to other Linux'es, are for a Debian Bullseye. Use the minimal Debian (netinst) ISO for latest stable: https://deb.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/
Create a minimal guest with 1 CPU, 512MB RAM, every useless things disabled (floppy, sound, USB, 16MB video RAM only), 8GB HDD with no swap partition, two para-virtual network interfaces: Host-only network interface, and NAT network.
Install OS with default options, except you disable all packages selection except "standard system tools".
Log in as root.
optional trim-down: disable IPv6, grub timeout 1 second, and configure APT to not install recommended packages by default. TODO: commands
Network interfaces names:
Here, we'll use SystemD to configure the network interfaces names to something meaningful.
Alternately, you can skip this step, and note in the output of
ip a command, and note which enp0sX is the nat, and which one is the host-network, and us them in next step into the .network files as Name parameters instead of nat0 and host0.
First, create two .link files into
/etc/systemd/network/ for naming the interfaces:
10-nat0.link: Here, put the MAC address of the NAT interface in VirtualBox guest configuration:
[Match] MACAddress=08:00:27:aa:aa:aa [Link] Name=nat0
10-host0.link: Here, put the MAC address of the Host-only network interface in VirtualBox guest configuration:
[Match] MACAddress=08:00:27:bb:bb:bb [Link] Name=host0
If you want to check this works before following: reboot, and use the command
ip a to check the interfaces have correctly been named host0 and nat0 and not enp0sX anymore.
On previous Debian versions, i was using /etc/network/interfaces (or interfaces.d), but on Debian 11 i use SystemD's network configuration.
Not sure if this is a sensible option, as contrary to old static-only /etc/network/interfaces, it seems to keep a "systemd-network" active.
TODO: find adressing of NAT by command-line on the host, or using dchpcd on the guest.
Then, again into
/etc/systemd/network/, create two .network files:
nat0.network: This one connects to the NAT, to access everything outside the host machine: LAN and WAN:
[Match] Name=nat0 [Network] Address=10.0.3.15/24 #Address=10.0.3.4/24 # NON NON NON Cette adresse est déjà utilisée par VirtualBox Gateway=10.0.3.2
host0.network: This one is connected to the Host-only network:
[Match] Name=host0 [Network] Address=10.127.0.4/24
TODO: /etc/hosts and ping tests
Then confirm you can ping:
another guest the host internet
Enable Masquerading NAT:
First, we enable ip forwarding:
Edit the file
/etc/sysctl.conf and uncomment the line with
net.ipv4.ip_forward=1, or add a specific file with:
echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/80-ip_forward.conf
TODO: nftables configuration
Internal and forwarding DNS server with dnsmasq:
TODO: dnsmasq configuration
TODO end: service ping host (or use as network-online.target in systemd?)