THE PFSENSE WALKTHROUGH PART 9: MULTIPLE PFSENSE FAILOVER

Thanks to http://thegeekninja.wordpress.com/category/security/

Welcome to the final episode in the pfSense Walkthrough.
Firstly I apologize for taking so long between episodes, as I’ve been a slight bit busy recently, but as promised here is the final episode where we will discuss having a secondary firewall in place for automatic failover in the event your firewall goes down.

If you have followed the pfSense Walkthrough from the beginning you should have a pretty awesome firewall setup by now that serves more than a single purpose, but with this you now have a single point of failure and were something to happen, such as hardware or software failure, you would then have a disruption in network connectivity to the outside world or even on your own network(depending on what features you are using) causing a large amount of downtime even if you have backed up the configs from pfsense for restoration, this downtime is not always acceptable as many network admins would know.
But there is an answer to this problem and that is to implement a secondary firewall to act as an automatic failover point in the event that something were to happen to your primary firewall.

Now you may think that building this secondary firewall and configuring it exactly the same as the primary firewall would take time and effort, along with the added time that would be spent having to implement changes twice whenever needed (once for the primary and then twice for the secondary) and sure you could just take a backup of the primary firewall and use it to build the secondary, but pfSense offers a simple and much better alternative using a combination of CARP, pfsysnc and XMLRPC sync to failover between two firewalls.

  • CARP – to failover the IP addresses
  • PFSYNC –  to sync firewall states and changes
  • XMLRPC SYNC – to sync settings between two firewalls

Multiple Firewalls with automatic failover:

So before we begin we need to decide on a IP address to be used for failover for each interface, this IP address will be the IP address that your internal network on each interface will talk to when trying to reach the firewall even if it goes down( in some environments it’s referred to as a Cluster IP Address) as well as the IP Address you will use as an external IP address for access from the outside world. Each one of your interfaces would need one of these IP addresses ( pfSense refers to them as Virtual IPs)  and the devices on your network will need to point to these virtual IP addresses instead if the actual IP address of each interface, along with your ADSL router or ISA device will point to the WAN interface/s virtual IP for NAT/Port forwarding purposes.
For the purpose of this tutorial we will only focus on the WAN interface and the LAN interface, but the same rules apply to all interfaces/VLANs you have configured.

Firewall 1:

  • WAN interface: 192.168.2.2/24
  • LAN interface: 192.168.1.1/24

Firewall 2:

  • WAN interface: 192.168.2.4/24
  • LAN interface: 192.168.1.3/24

Virtual IPs/Failover:

  • WAN Virtual IP: 192.168.2.3/24
  • LAN Virtual IP: 192.168.1.2/24

*NB: remember when you configure your devices they will point to the Virtual IP addresses to communicate to the firewalls, only use the actual IP address when accessing the individual Firewalls for making changes to settings.

Building the Primary Firewall:

You should by now have followed the previous episodes and built a multi-purpose firewall/internet gateway device. If you haven’t then you can proceed to the links below to setup a primary firewall by following episode to install and configure a standard firewall and then proceed to the other episodes depending on which features you want or need:

Building the Secondary Firewall:

Firstly follow the instructions in Episode 2: Install and Setup pfSense Firewall, build yourself a secondary firewall and assign the IP addresses to the various interfaces within the same subnet as the primary firewall’s matching interfaces(i.e. if the LAN interface of fw1 is 192.168.1.1, then assign fw2 LAN interface with 192.168.1.3 and so forth), then install any packages that you have installed on the Primary firewall, following the correct order (i.e. squidguard/dansguardian before Squid3) and don’t configure any settings for any of the packages, leaving them all on the default settings after the packages are installed completely.

Once the two firewalls have been built and the packages on the primary firewall has been configured you can now configure the firewalls for failover starting with your secondary firewall.

Secondary Firewall Settings:

On the secondary firewall there are only two things that need to be configured to allow for failover.

  1. Go to Firewall > Virtual IPs and click on the CARP Settings tab:
    carp1
    Check the Synchronize States check box and set the Interface (we are using the LAN interface in this example)**NB: Some people have recommended using a dedicated interface with a dedicated switch, which is not needed as the load generated by the sync is not so much that it would disrupt network traffic on the interface used. I would, however recommend using a different VLAN or interface such as your DMZ interface/VLAN if you have one installed/configured just to separate traffic for the sake of the firewall rules and management simplicity.
  2. Next we need to create a firewall rule to allow communication between the two firewalls:
    Go to Firewall > Rules and click on the interface you will be using for sync and click the (+) icon to add a new rule
    carp2
    Set protocol to any
    Source to the IP address of the Primary Firewall interface
    Destination to the interface address
    click Save and then apply changes

That’s all that needs to be configured on the Secondary Firewall no move on to the Primary Firewall.

Primary Firewall Settings

On the primary firewall there are a few more things that need to be configured.

We start by configuring the Virtual IPs for each interface:

  • Go to Firewall > Virtual IPs and click on the (+) icon to add a new Virtual IP
    carp3
  • Click the CARP radio button under Type.
  • Select the interface the virtual IP will belong to.
  • Set the virtual IP address and subnet bit
  • Enter a Virtual IP Password
  • Leave the other settings as is and set a description.
  • click Save

Do this for each interface you have, then hit Apply changes.

Next we configure the failover and sync settings:

  • Staying on the Virtual IPs page, click on the CARP Settings tab.
    carp5_1
    carp5_2
  • Click the Synchronize states check box.
  • Select the Interface we will be using for sync.
  •  Enter the interface IP address for the secondary firewall which will be used for sync in the pfSync Synchronize Peer IP as well as the Synchronize Config to IP boxes.
  • Enter the secondary firewall’s Username and Password used for the web interface.
  • Click each check box for every item you want synchronized to the primary firewall, ensuring that the Synchronize Virtual IPs is selected for failover to work.
  • Click on Save to save any changes and your firewall will now start syncing whatever options you have selected to the secondary firewall.

Lastly as an option most packages, such as Squid3, SquidGuard, Dansguardian, etc, have a sync/XMLRPC sync tab that allows you sync the configuration settings of each package to another firewall, saving you the hassle of re configuring  each package on each firewall as well as saving you the hassle of making changes on two separate systems instead of one as you would only need to configure this settings on the primary firewall.
below is a screenshot of the DansGuardian sync tab as an example. most packages sync tabs are the same.
carp6

The END…sort of….

So the way it works now is that a multicast broadcast gets sent between the two firewalls you configured and the minute those broadcasts stop, like in the event of a failure, then the other firewall will take over the virtual IPs and will hand back control of those virtual IPs when the broadcasts resume.
So at the end of the day if your primary firewall fails you will have drop in connectivity for a few seconds, mostly not even noticed, as control switches between firewalls thus allowing for downtime to occur on the primary firewall while you perform repairs without disrupting connectivity or productivity.

The End…really this time:

If you have followed all the episodes till this point then you should have a single firewall performing various  roles, which in my opinion is the whole idea and main drive behind pfSense, and should have a setup similar to the one in the diagram below:
carp_net_layout

I hope you have enjoyed the pfSense walkthrough and that you have found the episodes helpful and don’t fret as this may be the final epsiode in the pfSense walkthrough series, but it won’t be the last of the pfSense posts that I will be creating as I will be posting various other information with regards to pfSense and as usual you can drop us a comment below if you have any questions or head on over to the forum to discuss any issues you may have or request further posts on pfSense.

Until next time cheers fellow Ninjas