AWS – Flow Logs and Secondary IP’s on a ENI

Sometimes it may be necessary to add a secondary IP onto an elastic network interface on your EC2 instances. This can be for many reasons such as hosting multiple websites secured with certificates on a single server where each certificate is mapped to an IP, having application or appliances that have rules based on the target IP address and also having a IP address that you could move between different EC2 instances. The following example may help you better understand what you need to watch out for when using flowlogs in this scenario.

Although this is a fairly simple process to add a secondary IP, troubleshooting these IP’s with cloudwatch’s flowlogs can be a little more tricky (or more precisely, confusing, as they do not show the expected entries). Its important to understand that in flowlog entries on the ENI with multiple addresses, traffic to the “secondary IP” will not show as the secondary IP… it will actually show as the primary IP of the ENI within the entry.

In order to demonstrate this, our example infrastructure, we have 2 instances:

  • “Server” EC2 instance with multiple IP addresses assigned to an ENI (in subnet 1)
  • “Client” EC2 instance with a single IP address (in subnet 2)

The server EC2 will have the primary IP address of “172.16.0.10” and the secondary IP address of “172.16.0.20”. This instance will have IIS installed, simply to provide the default webpage for us to access/query. As you can see below, we have added a secondary IP to the only ENI attached to the server instance. It is also registered within windows on the interface. Doing “ipconfig” on the windows OS confirms our setup has the secondary address.

The client EC2 instance will have the primary IP address of “172.16.1.40”, this is where we will initiate the connection to our test webpage from.

From our client machine, I have started both a telnet session on port 80 and also browsing using internet explorer to the secondary/additional IP (172.16.0.20) of our server instance. As you can see, the telnet session is active, and also the default IIS webpage has been returned from the secondary address.

Now, lets examine our flowlogs. Starting with our client instance, we can see traffic with the source address of our “client”, 172.16.1.40 destined for the secondary IP of our “server”, 172.16.0.20 on port 80. We can also see return traffic from 172.16.0.20 destined to 172.16.1.40.

Lets now look at the “server” ENI flow logs, we can see the corresponding request on port 80 from our client’s IP, 172.16.1.40 and also the return traffic to the client. However, as you can see below, the destination for incoming traffic and the source for outgoing traffic is not the secondary IP (172.16.0.20), but the primary IP of the ENI (172.16.0.10) .

Hopefully the above information will help you when using flowlogs to troubleshoot or trace network traffic to secondary IP addresses on a single ENI