Daniels Blog
30Dez/140

Debian Jessie LXC networking. Containers with public and NATed IPs

It took me some time to get this working so it's time for a blog post:

Scenario
This was a setup for a server in a data center with a public IP address. The server has one physical interface with a public routable IP address. Additionaly I ordered another public IP address for the server to be used in one of the LXC containers.

I have two containers.

Container A
"A" gets a public routable IP-address to be reachable from the internet without NATing

Container B
"B" gets a private IP address and can only be reached thru NAT and port-mappings

Host
Host has 5.5.5.1 as main public IP
Container A has 5.5.5.2 as "virtual" IP
Container B has 10.10.10.1 as NATed private IP

HOST SETUP:



#NETWORKING CONFIG ON HOST
#/etc/network/interfaces

auto lo
iface lo inet loopback


allow-hotplug eth0
iface eth0 inet manual
   pre-up   ifconfig eth0 up
   pre-down ifconfig eth0 down


auto  br0
iface br0 inet static
  address   5.5.5.1
  broadcast broadcast.ip
  netmask   netmask.ip
  gateway   gateway.ip
  bridge_ports eth0
  bridge_fd 0
  bridge_maxwait 0


auto  br1
iface br1 inet static
  address   10.10.10.100
  netmask   255.255.255.0
  bridge_fd 0
  bridge_maxwait 0
  pre-up brctl addbr br1
  up iptables -t nat -F POSTROUTING

  # Exclude boxes with static IPs from Natting
  up iptables -A PREROUTING -t nat -i br0 -p tcp -s 5.5.5.2 -j ACCEPT


  # Enable Forwarding for NATed boxes
  up iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -o br0 -j MASQUERADE

  # example PORT FORWARDINGS FOR Mailserver
  up iptables -A PREROUTING -t nat -i br0 -p tcp --dport 25 -j DNAT --to 10.10.10.1:25
  up iptables -A PREROUTING -t nat -i br0 -p tcp --dport 465 -j DNAT --to 10.10.10.1:465
  up iptables -A PREROUTING -t nat -i br0 -p tcp --dport 587 -j DNAT --to 10.10.10.1:587

  # example PORT FORWARDINGS FOR Webserver
  up iptables -A PREROUTING -t nat -i br0 -p tcp --dport 80 -j DNAT --to 10.10.10.2:80
  up iptables -A PREROUTING -t nat -i br0 -p tcp --dport 443 -j DNAT --to 10.10.10.2:443

  post-down iptables -F
  post-down iptables -t nat -F
  post-down brctl delbr br1



#IP forwarding must be enabled in the kernel as well (don't forget reboot)
#/etc/sysctl.conf
net.ipv4.ip_forward=1

CONTAINER A Setup (static virtual public IP):

lxc.utsname = containershostname
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0

# This is the MAC for the public IP i got from my provider
# container gets IP by providers DHCP
lxc.network.hwaddr = 00:11:22:33:44:55

CONTAINER B Setup (static NATed private IP):

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br1
lxc.network.ipv4= 10.10.10.1
lxc.network.ipv4.gateway = 10.10.10.100