Direct Routed (L3) guest networks: route public IPv4/IPv6 to VM from Host - #13758

Description

This Pull Request adds a new guest network type in which the hypervisor performs L3 routing for the Instance: no Virtual Router, no NAT and no DHCP. Each Instance receives a public IPv4 address as a /32 and/or an IPv6 address as a /128, with a shared, host-independent link-local gateway (169.254.0.1 and fe80::1) that every hypervisor carries on the network's bridge. All addressing reaches the Instance exclusively via ConfigDrive/cloud-init; a routing daemon on the host (FRR, BIRD, ...) advertises the addresses to the fabric and is deliberately out of scope for CloudStack.

Direct Routed networks live on their own physical network, carrying the new ROUTED isolation method. This is what makes the feature adoptable in existing environments: an operator adds one physical network to an existing zone and gains L3 networks next to everything already running there — no changes to existing physical networks, offerings or Instances.

IPv6-only networks are fully supported. Nothing in this network type depends on IPv4 (no DHCP, no password/metadata service), so each address family is optional: IPv4 is a subnet (cidr=2.57.59.0/24, or netmask + start/end IP), IPv6 is an ip6cidr alone — IPv6 addresses are computed from the subnet and the NIC MAC with EUI-64, so no range and no pool exist. No gateways are declared or stored for either family, saving those addresses; Instances always use the shared link-local gateway.

Management server

• GuestType.L3; the guest_type column is char(32), so no schema change. BroadcastDomainType.Routed: each network carries a routed://<id> broadcast domain whose id names the per-network bridge on every host — allocated from the ROUTED physical network's vnet range, or chosen by the operator (specifyVlan offerings) so bridge names are plannable before the network exists.
• Offering validation: UserData via ConfigDrive is mandatory, Dns optional but ConfigDrive-only, SecurityGroup permitted (now allowed for L3 alongside Shared, including the deploy-time gate), Dhcp rejected as not supported and not needed. Network mode, specifyVlan-as-required and VPC use are rejected. A DefaultL3NetworkOffering is created on install and upgrade.
• DirectRoutedNetworkGuru subclasses DirectNetworkGuru, inheriting the Shared-network address lifecycle. After allocation the NicProfile is forced into host-route form (/32 or /128 + link-local gateway), which is also the signature by which ConfigDrive recognises these NICs.
• Zone-wide IPv4 overlap validation for L3 ranges: all L3 subnets share one host routing table and one fabric, so an overlap is an address conflict. The IPv6 vlan check was already zone-wide.
• SystemVMs work on routed public IP ranges (createVlanIpRange with vlan=routed://<id>): their public NICs take the same host-route form, with the IPv6 address derived EUI-64 from the range's subnet.

ConfigDrive

• Network data is always generated for a direct routed NIC; the historical gate (Dhcp or Dns supported) would leave these NICs with no addressing at all.
• A direct routed IPv4 network emits a network-level gateway key instead of a default-route entry: cloud-init derives the required on-link flag only from that key (netplan renderer since 23.1, networkd since 24.2), never from routes-list entries. This sets the guest requirement: cloud-init >= 23.1 with netplan, or >= 24.2 with networkd.

KVM agent

• One uplink-less bridge per network, brdr-<routed id>, created and removed by the new modifybrdr.sh (flock'd, idempotent, refuses to remove a bridge still in use). The bridge carries the gateway addresses, forwarding, strict rp_filter, and a MAC derived deterministically from the routed id — identical on every hypervisor, so a live-migrated Instance's ARP/NDP cache stays valid and cutover is seamless. Separate bridges make isolation between networks topological rather than a filtering concern.
• BridgeVifDriver plugs direct routed NICs into their brdr bridge and runs the existing modifymacip.sh hook per NIC to install the static neighbour entry and host routes (v4 and v6), regardless of the host-wide EVPN property, whose meaning is unchanged.
• security_group.py gains a --directrouted dispatch: rules for these Instances match by ipset destination rather than bridge port on the return path. Because a routed packet between two Instances on one host enters on one brdr bridge and leaves on another, the L3 framework runs two passes from a single FORWARD hook (shared BF-L3 chains): the source Instance's egress rules mark the packet, the destination Instance's ingress rules give the verdict, so both are always evaluated regardless of bridge creation order. A golden-file test harness pins the classic (bridged) rule stream byte-for-byte, proving existing deployments are untouched, and walks packets through the generated L3 rules.
• Host protection: an Instance's packets enter the host's own IP stack, so modifybrdr.sh installs once per host an INPUT chain for brdr-+ that admits only conntrack replies, ICMP echo and IPv6 neighbour discovery, plus an ip6tables rpfilter rule as the IPv6 counterpart of rp_filter=1. Forwarding towards management and storage subnets remains the operator's host firewall policy, like the routing daemon.

root@hv-138-d03-22:~# ip addr show dev brdr-578
15: brdr-578:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0e:00:00:00:02:42 brd ff:ff:ff:ff:ff:ff
    inet 169.254.0.1/32 scope global brdr-578
       valid_lft forever preferred_lft forever
    inet6 fe80::c00:ff:fe00:242/64 scope link
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link
       valid_lft forever preferred_lft forever
root@hv-138-d03-22:~#

Other changes this PR makes to CloudStack

Fixes and improvements that ride along because the feature exposed them, but apply generally:

• createNetwork gains an optional cidr parameter (L3-only for now, purely additive).
• Auto-allocated ids are exempt from the "VLAN tag is already being used for dynamic vlan allocation" check in createVlanAndPublicIpRange — previously Shared networks were exempted one layer up but not there.
• A failed VM start (uncaught RuntimeException) and a failed PrepareForMigration on the destination now unplug the NICs they plugged; previously bridges (and here: host routes) leaked.
• canUseForDeploy() treats Shared and L3 alike and counts the real IPv4 pool; IPv6-only networks are deployable from the UI wizard.
• The zone-wide IPv6 overlap check keys on ip6_cidr instead of ip6_gateway (what it actually compares).
• UI: L3 network creation form, ROUTED in the zone wizard and physical network forms, and the Instances list shows active IPv4/IPv6 addresses — IPv6 shortened with the tail visible, click copies the full address.
• SystemVMs (CPVM/SSVM) receive their IPv6 address and gateway as boot arguments, as the VR already did, and common.sh installs the IPv6 default route statically from ip6gateway instead of relying on a router advertisement. This also applies to classic VLAN public networks with IPv6, where systemvms previously had no IPv6 default route unless a fabric router sent RAs.
• The design document's §15 lists review findings that were consciously deferred (failed-migration rollback on the destination, root-admin-only network creation, KVM-only enforcement, vlan range CRUD on gateway-less rows, and others) so they are tracked rather than forgotten.

The design document, including the decision log and the verification notes behind each choice, is added under docs/design/.

This implements issue #12210

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

The feature was tested end to end on real hardware in a datacenter in September 2026: a KVM zone with a ROUTED physical network, SystemVMs on a routed public range, dual-stack and IPv6-only L3 networks, Instances with security groups, and live migration between hosts. Evidence of all tests is recorded, and the lab is available for remote access on request for anyone who wants to validate the feature without building their own environment.

Verified guest behaviour: Ubuntu 26.04 with cloud-init (26.1) and ConfigDrive as the datasource works as expected — IPv4 default route installed with on-link, EUI-64 IPv6 configured, dual-stack and IPv6-only.

Unit tests cover the guru, offering validation, ConfigDrive generation, the deploy gates and the address-family validation; the security_group.py golden harness pins the classic rule stream; integration tests (test_l3_networks.py) cover creation (subnet forms, operator-chosen ids, IPv6-only, gateway-ignoring, overlap and incomplete-family rejection) and deployment.

Pending lab validation for the review follow-up commits (September 2026): the two-pass security-group rules (cross-account Instance-to-Instance traffic on one host, iptables -C idempotency of the MARK rules), the host protection chain (Instances must still resolve and ping 169.254.0.1 / fe80::1) and the new unplug order. Design doc §12.2 and §13 list exactly what to check.