04 / Network data path

A frame leaves the guest. A packet leaves the host.

Virtio-net moves Ethernet frames. TAP connects them to Linux. Optional NAT adds IP addressing, a gateway, DHCP and masquerading—separate jobs at separate layers.

Follow an outbound packet—and its reply

The net device uses receive queue 0 and transmit queue 1. Buffers include a virtio-net header inside the guest-facing queue; the TAP backend exchanges Ethernet frames, not that virtio metadata.

Virtio-net TAP and host NAT data path The guest TX queue sends a frame through a Go virtio-net handler into TAP. Linux forwards and masquerades the IP packet. Replies return through connection tracking, TAP and the RX queue. Guest networkTX 1 / RX 0 virtio-netstrip TX header Linux TAPEthernet frames Route + NAThost → uplink Return trafficconntrack → TAP → RX L2: guest ↔ virtio ↔ TAP L3: host routing + masquerade
  1. The driver publishes a TX descriptor chain and notifies the device. The handler reads the virtio header and frame across the guest's buffers.
  2. The Go device writes the frame to the backend. TAP injects it into the host networking stack as traffic arriving on that virtual Ethernet interface.
  3. In NAT mode, Linux forwards the IP packet and iptables MASQUERADE translates its source on egress. The host's routes determine the uplink.
  4. Connection tracking reverses the translation. A TAP read obtains the reply frame; virtio-net writes it, with a virtio header, into guest-posted RX buffers and completes them.
The NAT path is shown here. Plain TAP mode stops at the interface; routing or switching beyond it is supplied separately.

TAP is an attachment point, not a network policy

kube-vm opens /dev/net/tun and requests IFF_TAP | IFF_NO_PI. TAP carries full Ethernet frames; unlike TUN, it is not an IP-only interface. Disabling the extra packet-info prefix keeps the backend frame-oriented.

With --tap tap0, you provide the surrounding topology: bring the interface up and arrange addressing, a bridge or another suitable network integration. Merely creating a TAP does not provide DHCP, Internet access or a layer-2 fabric. See kube-vm-switch and L2 networking for the distinction.

Userspace data path today. The --vhost-net flag is currently rejected as not implemented. This path uses the Go virtio device handlers, not kernel vhost-net acceleration. TAP access requires suitable host permissions.

NAT adds a small per-VM network

--nat creates a TAP and derives a deterministic /30 subnet from the VM UID within 10.249.0.0/16. The host gets the first usable address; the guest gets the second. The embedded DHCP service offers that guest address, gateway and DNS configuration.

  • Bring up the TAP and assign its gateway address.
  • Enable host IPv4 forwarding.
  • Add forwarding rules for guest egress and established/related return traffic.
  • Add a subnet-scoped POSTROUTING MASQUERADE rule and DHCP rules.
  • Start the per-interface DHCP service; tear down its resources and rules when the backend closes.

This is not a shared Ethernet segment. Each NAT backend has its own tiny subnet. Deterministic UID hashing is not a globally coordinated IP allocator: check for conflicts with other guests and host routes. The host firewall and routing still govern connectivity, and NAT is not a complete security boundary.

Unsolicited inbound access from another machine needs an explicit routing or forwarding arrangement; this setup does not automatically publish every guest service. From the host, use the guest address reported by examine, or kube-vmctl ssh <uid> once the guest runs SSH.

Choose one backend

With the daemon running, a guest whose initrd configures networking can request NAT:

./bin/kube-vmctl start --uid net-demo --detach \
  --kernel ./bzImage --initrd ./initrd.cpio.gz --nat

./bin/kube-vmctl examine --uid net-demo

Or use --tap tap0 instead of --nat. They are mutually exclusive. NAT requires the host's ip and iptables commands and permission to configure interfaces, forwarding and firewall rules. The guest must actually run a DHCP client or configure its interface; a virtual NIC alone does not do that.

Queues can exert backpressure

The guest must keep RX buffers available. When a frame arrives without a receive chain, it may be dropped rather than stored indefinitely. Backend counters expose RX/TX bytes and packets: RX means host/backend → guest; TX means guest → backend.

Virtio-netTAP backendNAT and DHCPBackend selection