10Gbit Intel NICs and pfSense

As 10Gbit is becoming more and more common in large networks to cope with the ever increasing amount of data that is moved between service providers and users I have taken some time to look into what others have done when working with pfSense. I’ve limited my research to systems based on Intel 10Gbit NICs as they’re the most cost effective – high performance at low cost – and I’ve really good experiences with the 1Gbit Intel NICs.

First off the NICs have to be compatible with pfSense which is based on FreeBSD. When looking at [1] Intel provides a driver for the NICs based on the 82598 and 82599 chips, as well as the X540 cards. This indicates that there are some cards out there which provides 10Gbit capabilities for FreeBSD (from FreeBSD 7.3 and above) and therefore also pfSense. However when looking at the available configurations it only possible to get two interfaces in one NIC with multimode fiber or copper, but not with single mode fiber. However when looking at the price of the single mode fiber variations kind of exclude them as they’re twice the price of that copper variations; even with two interfaces. On a final note the Intel 10Gbit NICs all require PCIe 2.0 or higher as a minimum, which excludes a lot of servers that I’ve available, because they are based on the Intel 5000 chipset.

When looking at the pfSense forums there has been several dicussion recently on the Intel NICs, especially the X520 and X540, and from the debate it seems clear that running pfSense straight on hardware doesn’t work well with pfSense 2.1.x, there is a problem with the MBUF of the driver for the NICs [2]. A solution proposed to solve problem is to use VMware ESXi to hypervise the hardware and “mask” the NICs as VXNET3 interfaces and then install pfSense on top of that. A guide to do this can be found in [3]. The writer of the guide reports that transfering 7Gbit of traffic distributed over 132 VLANs and ~160 subnets without any problems [4]. From what I’ve been able to gather from the pfSense forum the MBUF error wouldn’t be corrected in the 2.1.x branch as it’s based on FreeBSD 8.3, but the 2.2 version of pfSense will address this problem as it will be based on FreeBSD 10.0, where the problem with the drivers have been corrected. Unfortunately pfSense 2.2 is only in Alpha at the time of writing and therefore unlikely to reach Stable status before the end of this year.

References