Aviatrix Packet Walk: Internet Access via Egress Firenet

In the last post, I covered the Firenet architecture and the security options it enabled on an Aviatrix secure cloud network. This time I’ll focus on Egress Firenet, a specific feature of the Firenet deployment that provides secure internet access for network resources without public addressing.

Combined or Split Firenet?

It’s possible to use a single Firenet to do both east/west, VPC-to-VPC firewalling, and also have it do egress firewalling as well. In the interests of explanation and being able to focus on one thing at a time, I’m electing to split Egress Firenet into its own VPC. There are design considerations to running it in a combined or split setup and pros/cons to both. Scale is tougher when combining the Firenet into a single point of inspection, but cost is also lower. The good news is that it’s very simple to start with a combined Firenet and then split out as needed.

The first step is to create an Egress Transit VPC into which we will deploy the Firenet. As before, we set it as a Transit Firenet VPC so it will create the necessary subnets for the firewall deployment.

The next step is to deploy the Firenet gateways into their respective subnets inside the new VPC.

In this case, we’ll deploy the HA gateway to another availability zone.

Now to turn on the Egress Firenet function on these gateways. This is actually a different feature than the standard Transit Gateway behavior, because the egress gateways will originate a default route into the Aviatrix fabric. We’ll see that in a bit.

Once the Egress Firenet is stood up, it’s time to attach the spoke gateways. The spokes can be attached to multiple transits at once, and a common design is to have the best transit used for the destination. VPC to VPC traffic can use the standard East/West Transit Firenet, while the Egress Firenet will be used for traffic leaving the cloud towards the Internet. Before we attach the spokes, let’s first look at the native VPC route table:

Notice there is no default route present, just the standard Aviatrix-programmed RFC 1918 routes to the gateway. Let’s look at the Aviatrix spoke gateway route table next.

No default route here either, just the other VPC routes connected to Aviatrix transit.

Now let’s attach the spoke gateways in the Spoke 1 VPC to the Egress Firenet and look again.

Here is the native VPC route table after attachment:

A default route was added to the VPC route tables pointing at the Aviatrix spoke gateway, programmed by the controller. Let’s see what the gateway itself is using:

I omitted the entire route table as we are only interested in the default. The routes follow the standard Aviatrix ActiveMesh route preference: ECMP to the primary and HA Egress Transit gateway and a less-preferred backup route to the local HA gateway.

What Next?

Now that the Egress Firenet is running, let’s attach firewalls to perform the inspection and NAT to the Internet via the egress interface. In production, we’d have all this done prior to attaching the spoke gateways so as not to potentially impact traffic (injecting a default route when no actual egress exists yet), but for purposes of demonstration I wanted to focus on a flow that builds up from basic to complex.

We covered the deployment of Palo Alto firewalls in the last post so I won’t rehash that here. Each Firenet has its own set of firewalls, so we are using different firewalls for this function. I’ll focus on how we turn the firewalls into an egress firewall because we covered how to deploy firewalls into a Firenet last time, . We need to add a NAT translation for traffic from within the Aviatrix network to the Internet:

According to this rule, any RFC 1918 addresses coming from inside the network and destined for the WAN interface per the firewall routing table will have a NAT performed, translating the source address into the egress interface and using a dynamic port for session tracking. This PAT is common for giving multiple clients access to the Internet using a single address.

In order to update the route table on the Palo Alto to use a default route for this, we use the Vendor Integration feature in the Aviatrix controller:

The next-hop listed here for the default route is the address of the VPC router in the egress subnet in which the eth1/1 ENI resides.

The ENI (Elastic Network Interface) has a public EIP (Elastic IP) attached, so the VPC router will pass the traffic to the IGW (Internet Gateway) of the VPC, use the EIP of the ENI to perform its own NAT and send traffic to the Internet.

We also need to add a firewall rule to allow the traffic from the inside to outside:

The last piece is to test connectivity from a host within a private subnet that has no public IP or Internet Gateway to use.

Using CURL on a website, and pinging Google DNS is a success! If we inspect the Palo Alto we should see the traffic.

We can see the traffic go through the firewall and be allowed. Each firewall in the Egress Firenet has its own subnets, interfaces, EIPs and because the NAT uses the interface IP, return traffic from the Internet will come back to the correct firewall.

Here’s the packet walk perspective, step by step:

1. The workload in a private VPC needs to access the Internet. The workload sends a packet to 8.8.8.8. The native VPC route table is consulted first and a default route is found with a next hop of the ENI of the Aviatrix spoke gateway.
2. Packet arrives at the spoke gateway and the gateway routing table is consulted. A default route exists using an IPSec tunnel to the Egress Firenet gateways (and a backup to the HA spoke gateway). Packet is forwarded following ECMP rules to one of the two Egress Firenet gateways.
3. Packet arrives at the Egress Firenet tunnel interface, is decapsulated and the Egress gateway route table is consulted. A default route exists to either eth2 (the firewall in the same subnet) or eth3 (the tunnel interface to the HA gateway). As explained before, the hashing algorithm takes care of forwarding traffic to either firewall.
4. Assuming the hashing algorithm determines the traffic will use the local firewall, the packet is forwarded to out of eth2 into the shared subnet between the firewall and gateway.
5. The firewall accepts the packet on its eth1/2 LAN interface, consults firewall rules and routing tables and decides the packet is allowed and should use the default route out the WAN zone and eth1/1 interface.
6. NAT is performed, mapping the inside private address to the egress interface. The source port is changed to a random port and the session tracked. The packet is forwarded to the VPC router per the route table.
7. The VPC router forwards the traffic to the VPC IGW, which sends the packet onward to the Internet.
8. Packet arrives at the destination, and if a reply is coming, its destination IP will be the EIP attached to the firewall interface. The packet will come back through the IGW, be forwarded by the VPC router to the firewall outside interface, which will de-NAT the packet, and forward it back to the host using the Aviatrix cloud network.

I hope that this packet walk has been helpful to understand the kind of granularity, control, security and visibility that Aviatrix gives to engineers. The cloud is a bit of a black box, which works fine for development, but is a real danger as critical workloads get moved to the cloud. Next time we’ll add in an on-premises connection, simulating a physical data center connection to allow access to cloud resources.