firewall-wizards@listserv.icsalabs.com
To subscribe or unsubscribe via the World Wide Web, visit
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
or, via email, send a message with subject or body 'help' to
firewall-wizards-request@listserv.icsalabs.com
You can reach the person managing the list at
firewall-wizards-owner@listserv.icsalabs.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of firewall-wizards digest..."
Today's Topics:
1. Re: Firewall rules order and performance (Carson Gaspar)
2. Re: Firewall rules order and performance (david@lang.hm)
3. Re: Firewall rules order and performance
(lordchariot@embarqmail.com)
4. Re: Firewall rules order and performance (Marcus J. Ranum)
----------------------------------------------------------------------
Message: 1
Date: Sat, 18 Jul 2009 06:09:07 -0700
From: Carson Gaspar <carson@taltos.org>
Subject: Re: [fw-wiz] Firewall rules order and performance
To: Firewall Wizards Security Mailing List
<firewall-wizards@listserv.icsalabs.com>
Message-ID: <4A61C973.7020201@taltos.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Pierre Blanchet wrote:
> This is a well known idea that the rules order is important for the best
> performance of a firewall. However, nowadays: 1. Stateful firewalls use their
> stateful engine for existing connections to allow traffic. That means that
> their performance is more related to the number of existing sessions rather
> than the number of rules, or more exactly it is tied to the ratio
> new/existing sessions. 2. Some firewalls no longer parse the configuration
> line by line but use hardware-based or tree-based model. Again, the number
> of rules has less effect on the performance.
>
> I'm looking for benchmarks/ideas that could prove I'm right or wrong. I know
> for sure that FW-1 and IOS depend on the rules order but what about the
> others ? Google didn't give any information one way or the other.
Define "performance"...
For state keeping packet filters, the general rules (in my experience) are:
- New connections / s is generally limited by ruleset size and complexity. As I
recall, several years ago Lucent had an O(1) algorithm for packet filtering on
some of their high end routers (OC-12 +) that leveraged some tricky algebra, but
it was (at the time) limited to 256 not very complex rules. I suspect some other
folks may have figured out similar optimizations. A simple test is to insert
10,000+ dummy rules followed by a rule you'll actually hit. Test connection
setup rate that way, and with your matching rule first. You'll quickly determine
if there's anything "interesting" going on. Be sure to make the dummy rules
annoyingly similar to the real one, but not quite the same (say, filter on TCP
flags)
- Throughput tends to be limited by packet rate and # of existing sessions. This
is why every vendor specifies throughput based on large packets - ask them for
64-byte packet throughput and watch them squirm.
- Throughput can also be limited by how much protocol parsing is done and how
complex it is. If you just have to forward packets, you can do it faster than if
you have to look for embedded port numbers in FTP, SIP, etc. Include NAT in the
complexity calculation.
- The above variables can also effect _latency_ which can be more or less
important that simple throughput, depending on your application.
Application proxies have similar performance correlations, but they're not as
straightforward.
--
Carson
------------------------------
Message: 2
Date: Fri, 17 Jul 2009 18:38:10 -0700 (PDT)
From: david@lang.hm
Subject: Re: [fw-wiz] Firewall rules order and performance
To: Firewall Wizards Security Mailing List
<firewall-wizards@listserv.cybertrust.com>
Message-ID: <alpine.DEB.1.10.0907171835300.9159@asgard>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
On Fri, 17 Jul 2009, Pierre Blanchet wrote:
> This is a well known idea that the rules order is important for the best performance of a firewall. However, nowadays:
> 1. Stateful firewalls use their stateful engine for existing connections to allow traffic. That means that their performance is more related to the number of existing sessions rather than the number of rules, or more exactly it is tied to the ratio new/existing sessions.
> 2. Some firewalls no longer parse the configuration line by line but use hardware-based or tree-based model. Again, the number of rules has less effect on the performance.
>
> I'm looking for benchmarks/ideas that could prove I'm right or wrong. I know for sure that FW-1 and IOS depend on the rules order but what about the others ? Google didn't give any information one way or the other.
this is going to depend on which firewall you look at, and potentially
which release of the software.
ordering the rules by how frequently they are used doesn't hurt
performance on systems that do tree-based rules internally, so the only
possible thing that you would gain is in the orginization of the rulesets,
and I'm not sure that that's enough to worry about trying to keep track of
which releases of which firewalls have which behavior.
David Lang
------------------------------
Message: 3
Date: Sat, 18 Jul 2009 13:04:25 -0400
From: <lordchariot@embarqmail.com>
Subject: Re: [fw-wiz] Firewall rules order and performance
To: "'Firewall Wizards Security Mailing List'"
<firewall-wizards@listserv.icsalabs.com>
Message-ID: <000001ca07c9$ce8c1bb0$6ba45310$@com>
Content-Type: text/plain; charset="us-ascii"
I've done this in the past with professional test equipment like SmartBits
or Web Avalanche that was able to measure performance, latency, connection
rates, etc.
The challenge was establishing unique connections from multiple MAC & IP
addresses to emulate real endpoints, and not just alias multiple IPs on the
same NIC. Mostly because of the ARP process prior to making a connection.
It's a lot faster making 65,000 connections from 1 MAC/IP to another than 10
connections to 6,500 unique MAC/IP/Src/Dest combinations.
The test gear could simulate real hosts with unique MAC & IP addrs.
We tested 1, 10, 100, 1000 & 10000 rules, all with different IP/port
combinations. UDP & TCP with different packets sizes, etc.
I was representing a now defunct product at the time, but the product faired
pretty well because the rule matches where a tree lookup to select the rule.
The overall difference between number of rules on our product was pretty
negligible, but it did a lot better than checkpoint at the time.
AIR, the rule selection wasn't the bottleneck, the number of already
established connections in the kernel was the primary factor. You'd plateau
after a certain point as new connections were trying to allocate the memory.
-erik
> -----Original Message-----
> From: firewall-wizards-bounces@listserv.icsalabs.com [mailto:firewall-
> wizards-bounces@listserv.icsalabs.com] On Behalf Of Pierre Blanchet
> Sent: Friday, July 17, 2009 10:52 AM
> To: Firewall Wizards Security Mailing List
> Subject: [fw-wiz] Firewall rules order and performance
>
> This is a well known idea that the rules order is important for the best
> performance of a firewall. However, nowadays:
> 1. Stateful firewalls use their stateful engine for existing connections
> to allow traffic. That means that their performance is more related to the
> number of existing sessions rather than the number of rules, or more
> exactly it is tied to the ratio new/existing sessions.
> 2. Some firewalls no longer parse the configuration line by line but use
> hardware-based or tree-based model. Again, the number of rules has less
> effect on the performance.
>
> I'm looking for benchmarks/ideas that could prove I'm right or wrong. I
> know for sure that FW-1 and IOS depend on the rules order but what about
> the others ? Google didn't give any information one way or the other.
>
> --
> Pierre Blanchet
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards@listserv.icsalabs.com
> https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
------------------------------
Message: 4
Date: Fri, 17 Jul 2009 22:21:35 -0400
From: "Marcus J. Ranum" <mjr@ranum.com>
Subject: Re: [fw-wiz] Firewall rules order and performance
To: Firewall Wizards Security Mailing List
<firewall-wizards@listserv.icsalabs.com>
Message-ID: <4A6131AF.6080800@ranum.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Pierre Blanchet wrote:
> This is a well known idea that the rules order is important for the best
> performance of a firewall.
It's also sometimes significant from a policy standpoint. I know
you know that, but I cannot avoid mentioning it.
> However, nowadays:
> 1. Stateful firewalls use their stateful engine for existing connections
> to allow traffic. That means that their performance is more related
> to the number of existing sessions rather than the number
> of rules, or more exactly it is tied to the ratio
> new/existing sessions.
Maybe. It may be tied to the implementation of the engine; whether
or not they are doing any L7 snooping or whether they are just
ripping control information out of raw packets. A "stateful"
firewall could have state-table entries that are very light
weight (e.g.: practically nonexistent or hardly more than
relying on the SYN flag - a single bit). It's possible that a
"stateful" firewall might have a cruddy stream lookup
algorithm, in which case performance might have to do with
number of sessions, or it might have a mediocre rules matching
algorithm, in which case it would depend on rule quantity,
or complexity. I've seen some firewalls that parse rules tables
into parse-trees and optimize them and I've seen other firewalls
that re-interpret the rules tables as strings, each time something
comes in. I kid you not; i.e.:
if(strcmp(rp,"permit")) {
...
In other words, I think you're dramatically oversimplifying things.
"It's more complicated than that."
> 2. Some firewalls no longer parse the configuration line by line
> but use hardware-based or tree-based model. Again, the number
> of rules has less effect on the performance.
Hardware does nothing but run software; so if the algorithms
burned into the hardware are mediocre, you have a fast version
of a mediocre algorithm and aren't necessarily any faster. I
know of one firewall that does some clever stuff to reorder
rules into something much like a yacc-generated parse tree,
to guarantee minimal cycle use in matching. That's a software
implementation but it'll kick the living snot out of a
linear search implemented in hardware. That's without even
taking into account the fact that a 3+Ghz CPU is one hell of
a fast engine, even running software, and might kick the
snot out of a decent piece of code running in a slower ASIC.
Some firewalls do as you say; perhaps some do not. It's hard
to know for sure and I only believe in code I've reviewed,
anymore. :D
> I'm looking for benchmarks/ideas that could prove I'm
> right or wrong.
There's a paper by Molitor, Kostick, and myself that lays
out these problems fairly clearly. The upshot is that, if
you really want to test firewall performance, you need to
hit firewalls with live streams of application data that
match your expected deployment. Anything else is pointless.
You might also want to search up a paper I wrote on IDS
benchmarking back in the NFR days; the problem there is
similar.
"Stateful" firewalls nowadays also often have bags on the
side that attempt to do various L7 processing. So, for example,
if you're just dealing with 10,000 streams/minute of
dummy HTTP traffic it's going to perform extremely
differently when you turn on URL scanning and the flow
path cuts from the hardware (which basically hardly looks
at each packet) to the CPU, which has to sequentially
handle packets. How do you test this? Feed real application
data through it.
If your benchmark is measured in packets/second, I can
guarantee you that you're doing it wrong. BUT that doesn't
mean I know what "right" is; other than that it's simulating
as accurately as possible what your real traffic will
look like. (The IDS benchmarking paper also points out
problems with using packet replays of streams off a hard
disk; a correctly functioning IDS or firewall will not
love seeing the same TCP session happen over and over again)
(Corollary: there are very very very few firewalls, by
that definition, that I'd consider "correctly functioning")
>I know for sure that FW-1 and IOS depend on the rules
>order but what about the others ? Google didn't give
>any information one way or the other.
Firewall vendors are typically very close-mouthed
about such implementation details. Because they know
that if most of you knew how little the firewalls
actually do (but gosh! they do it fast!), you'd scream.
I know this isn't the answer you wanted to hear, but it's
the truth.
mjr.
--
Marcus J. Ranum CSO, Tenable Network Security, Inc.
http://www.tenablesecurity.com
------------------------------
_______________________________________________
firewall-wizards mailing list
firewall-wizards@listserv.icsalabs.com
https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
End of firewall-wizards Digest, Vol 39, Issue 7
***********************************************
No comments:
Post a Comment