Layer7 on newer 2.0 snapshots

Hi all!

Sorry for the lack of news lately, but the holidays got in the middle. First of all, I want to wish all our visitors a stupendous 2009!

Now, regarding pfSense and Layer7, I’m glad to announce that the code is already in the newer 2.0 snapshots. You can find the interface for layer7 rules generation in Firewall->Traffic Shaper. Documentation on how to operate with the layer7 is currently being constructed, and I hope to have it posted soon on the pfSense devWiki.

There are possibly some problems with divert implementation+ipfw-classifyd, so I can not guarantee that it’s going to work right now. But at least, you can have already a look at the work that has been done.

A quick tutorial on how to construct a layer7 container and have it applied to a rule:
(1) Go to Firewall->Traffic Shaper->Layer7
(2) Create a new rule, assigining it a name, setting it to active and defining what to do with the protocols. For example, there are 3 kinds of structures, but Limiter and Queue will only appear if you have already defined any limiter or queue in the other tabs, next to the layer7 one.
(3) After you create the rule, assign it to a pf rule in Firewall->Rules. Create a new LAN rule and assign the Layer7 Container you created previously (it’s almost in the end of the page).
(4) Report back your problems.

I encourage to report your problems and/or sugestions in the pfSense official forum, and I’ll be very thankfull for the help you all can give in testing this implementation!

P.S: We are working on getting the current shaper wizards to also produce layer7 rules. Stays tunned for more info on this.

Layer7 interface finished

Well, the work still in progress. Since the last time, we have more development done.

The Layer7 shaper interface is finished. No significant differences were done to the interface since the last post. Now, we can create rules by simple select the protocol, structure and element. The protocol field contains a list of protocols that is builted dynamically from protocols patterns that come default in the pfSense instalation; the structure field allows the user to choose what type of structure will be applied: “action”, “queue” or “limiter”; the behaviour field is dynamically filled accordingly to the structure that was chosen. If the selected structure was “action”, the possible behaviours are only two, “allow” or “block”; if it was “queue”, the possible behaviours are all the altq queues already defined; if it was “limiter”, the possible behaviours are all the dummynet pipes and queues already defined too.

Some errors situations are controlled, such as the case of creating a rule with the same protocol chose more than once.

A new feature that we introduced was the possibility to add new protocol patterns.

Add Pattern functionality

 There are some patterns that come with a default pfSense installation, but we have the possibility to add more from the interface. It’s a simple gui interface, that uploads a protocol pattern file to the right location, but very useful if we want to upgrade our system with non default layer7 patterns. 🙂

Another option that we introduced was the capability to specify a layer7 rules container when editing the LAN firewall rules. Now, we can also specify to the filter if we want to apply the layer7 rules that we created.

Layer7 Dropbox in Firewall Rules Edit

Also some error situations are controlled such as the possibility to apply a layer7 rule only to TCP, UDP or TCP/UDP protocols.

The next step is creating a wizard to facilitate the configurations. Some ideas are thought but we still brainstorming. 🙂

The layer7 interface

Just a little snapshot of the layer7 tab to tease you 😉

 

Layer7 Tab in pfSense Shaper

Development is advancing. We are still having some problems with the protocol definition but we hope to sort them out soon.

This is just a place to create what we call layer7 containers. Each container can have multiple protocols with diferent definitions. After you create a container you will be able to assign it to a firewall rule! At least this is the idea. More to come soon 🙂

Implementation so far…

We are still doing some tests with ipfw-classifyd and pf. This is not an easy task as we’ve been finding some dificulties along the way.

In the mean time, we are also already working on the code base to integrate the layer 7 capabilities in pfSense. For now, I would say that the core code base is in pretty advanced form already, at least the part responsible for the creation of ipfw-classifyd configuration files. Essentially, this part of the code is responsible for storing what we call “layer 7 rules containers” in the $config array and ultimately generating the ipfw-classifyd config file for each of the “layer 7 rules container”. A rules container works like an alias and, for example, it can take the following shape:

http = action pass
bittorrent = action block
pop3 = dnpipe 1
sip = queue voIP

But as stuff in the ipfw-classifyd is still being tested we are not sure if this is going to be the final version of things. A “layer 7 rules container” also has a divert port automatically assigned to it in the 40000-60000 range. The idea is that someone might assign this container to a rule in the “Firewall -> Rules” section, like you assign, for example, a queue. How is this going to be exposed to the user has not yet been completely decided, but we are working on it.

The graphical interface for the creation of these so called “layer 7 rules container” is already being built. It’s taking us sometime since our html and javascript coding capabilities are a little rusty, but we are already improving our skills 🙂 This is being made in a way such that it will integrate seamlessly with the other shaper tabs.

As said before, we are still deciding on how to integrate thing on firewall_rules_edit.php and filter.inc However this does not seem to be difficult once we decide the road to take and how to expose this new option.

We expect to post more on the ipfw-classifyd tests we are doing as soon as we finish them.

ipfw-classifyd

ipfw-classifyd is an userland application for FreeBSD, capable of tagging traffic flows by their layer7 protocol. It uses application patterns in order to identify the application protocol. Ermal Luçi has done a great job in improving the original ipfw-classifyd and, thanks to its efforts, it is now compatible with pf. In order for it to receive the traffic flows, the flows are diverted to the application by using divert sockets.

The configuration file for ipfw-classifyd is very straight forward and is capable of using queues defined in the pf configuration. The pf present in pfSense 2.0 Alpha is also capable of using dummynet, once again thanks to the efforts of Ermal Luçi. As a consequence, ipfw-classifyd is capable of using pipes and queues from dummynet, queues from AltQ and actions from pf itself.

In order to divert traffic from pf, one has to write the following rule:

pass in on $int from any to any divert $divertport keep state(max-packets 5)

It’s just an example rule and keep state can have different options. One option that seems to be very important is “max-packets 5”. This is necessary in order to limit the overhead of the kernel to userland switch.

The syntax for ipfw-classifyd configuration can be seen in the above example:

$protocol1 = dnpipe 20
$protocol2 = dnqueue 30
$protocol3 = queue $queue_name
$protocol4 = action pass|block

The syntax is pretty simple. Let’s take a closer look at it. If $protocol1 is detected, than the ip flow will be assigned to the dummynet pipe 20. This can be usefull, for example, to limit the available bandwith for $protocol1. If $protocol5 is detected, we can allow the traffic to pass as it is or, we can decide to block the traffic. You can imagine the possibilities of such configuration: (i) port hoping will stop being a problem since the flow will be identified not by port but by application protocol; (ii) you can define a network where only certain protocols are allowed, no matter what port (for example, a network where you only authorize http traffic); (iii) block p2p traffic or any other unwanted traffic. These are only some examples, the possibilities are enormous.

Now, the next step will be to test ipfw-classifyd and report the results. Being an userland application we will have to see if it works well under heavy load. To do so, we will setup a simple ruleset and see how it behaves. I will post back our findings and configurations soon!

pfSense 2.0 Alpha

In the words of its creators:

“pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router. In addition to being a powerful, flexible firewalling and routing platform, it includes a long list of related features and a package system allowing further expandability without adding bloat and potential security vulnerabilities to the base distribution.”

Our work will be directed towards the 2.0 release. It’s still in alpha stage and is expected to come out in the last quarter of 2009. A lot of changes have already been made to it, like a new shaper, a new load balancer, and a lot more. We have a few goals for the next three months that we expect to accomplish:

(i) integrate ipfw-classifyd in pfSense and make the necessary code changes to integrate it seamlessly in the gui, exposing its functionality to the user in a simple way, while simultaneously keeping advanced options for advanced users;
(ii) Develop some wizards that would make faster and easier the creation of some “default” or “standard” configurations, based on the user needs, and adjust the QoS rules accordingly. The objective of this is to demystify this “black art” concept of QoS and bring it to the mainstream audience. To acomplish this, we will try to build wizards that will allow the user to create standard QoS configurations, while hiding the complexity associated with such process;
(iii) Due to the fact that ipfw-classifyd will be very important to the overall performance of the system, we will need to develop a tool that will allow the test of parallel patterns to evaluate the ipfw-classifyd performance.

Welcome

Welcome to my blog. With this blog I intend to document my main findings around QoS, most specifically, in the pfSense platform. Topics that are going to be studied are: pf, AltQ, dummynet and ipfw-classifyd. QoS is my main area of interest and right now I’m trying to extend the pfSense plataform with some more functionalities that, hopefully, will be present in the 2.0 release.

Welcome aboard and I hope we all can learn in the way 🙂