It’s the Outbound Stupid!

As nice as the word “Egress” may sound, I like “Outbound” better. Outbound is hardcore firewalling for the traffic that was initiated on the “trusted” side of the firewall, and is destined OUT to the unknown. However, over the last 20 years network administrators have put much more emphasis on what can come IN to their networks, rather than to worry about what comes OUT of them. There are many reasons for that, but the key one is that if nothing can come in, then we need not worry about what comes out. Right? Yea, right…

Fast Forward 2014 we have firewalls everywhere. On our smartphones, laptops and home broadband wifi routers. In the cloud, any typical IaaS setup has a firewall for every group of instances, usually called Security Group and every instance has its own, internal firewall that is controlled by the instance OS. We are starting to see more and more people that use them as a core building block, in a  network security architecture, and as a fundamental element of it. But as most of you (us) actually spend time, energy and concern about inbound firewall rules, only 15% of you (us) even bother with the Outbound. That’s true, and its about time we change that.

Plan. Plan for Success.

A well planned inbound firewall policy can give the bad guys a true challenge, but what happens when they do get in and there’s no outbound policy in place? Its an all-you-can-eat buffet. With a well planned Outbound policy, theres a strong chance they will go away, or be easily discovered – and that’s a great ROI for your time!
It all starts with a written (English, not Erlang or Java) security policy that outlines the way groups of servers interact with each other and with the outside world. First for the inbound/ingress which should be pretty easy to formulate. Then comes the outbound/egress tricky part – lets take a closer look at an application server.
Usually application servers sits in one of the middle tier, trusting traffic coming from the web-tier and initiating requests to the database tier before sending the replies back to the web servers.
Inbound: Allow traffic from Web Servers, usually only for HTTP or HTTPS depending on the Web<>APP settings.
Outbound: Allow DB traffic to the Databases on the specific DB port, for example, TCP3306 for MySQL.

Don’t forget the monitoring

Any monitoring solution you may be using, affects your security policy. An agent-based monitoring solution installed on the machine will have to be allowed to connect out, where an external monitoring service, that probes your servers from the outside, will have to be allowed in to do its job.
Agent Based Monitoring: Allow Outbound on specific protocol to a specific IP or a list of IPs
External Monitoring: Allow Inbound for specific protocol and from a specific IP or list of IPs

What else on the Outbound? The Killer (No) Rules

Our app server runs Ubuntu so it may needs access to the ubuntu update services and also needs DNS to live happily. Now the common theme would be to allow access to update services for outbound on https (TCP443) and to allow outbound DNS (UDP53). Right? WRONG. Why not limit only to the specific repository services that our OS needs? The hardline DevOps will kill updates altogether, stating that any part of your application should not update itself. You can always bake/launch a fresh updated one instead of messing with a working instance.
Result: No Outbound HTTPS TCP443
Now for DNS. If you are a security concerned it is recommended to stop using DNS altogether and use /etc/hosts instead. A less drastic approach would be to allow DNS access only to a DNS server that you trust (E.g. Google’s 8.8.8.8), this is done to make sure that data leaked using DNS queries could only reach Google, and they already know everything.
Result: No Outbound DNS  UDP53 or Allow DNS (UDP53) to 8.8.8.8 if you must.

Your Feedback Loop: The Network Logs

After setting this strict Outbound policy, we can finally identify the services that were left hanging, as we just severed their lifeline. It could be a non-critical agent that we should uninstall, or, god forbid, a rogue piece of code some bad guy wrote and now can’t connect to its C&C server – more on that next time.

Happy Firewalling,
Zohar