0

Configuring High Availability in pfSense Firewall (Clustering)

Howdy good people. In this lab we are going to configure High Availability in pfSense using the Common Address Redundancy Protocol (CARP) and pfsync protocol.

References: https://docs.netgate.com/pfsense/en/latest/book/

I have to 2 pfsense firewall setup in my lab using EVE-NG and both are connected to a multi-layer switch. This lab assumes that you have already installed and configured basic firewall settings such as IP address assignments, both WAN and LAN.

You can as well set it up on Virtualbox or Vmware or any other virtualization software of your choice.

Follow along to learn how to configure pfSense firewall High Availability using the two protocols mentioned above. Make sure to have read The pfSense Book from the above link and understood our objective

Configuring HA in pfsense firewall

Introduction

Pfsync – this is an open source protocol designed to synchronize the state tables of the firewall between cluster nodes. Any configuration change made on the master node are automatically sent to the backup node over the sync interface using the pfsync protocol.

CARP – Common Address Redundancy Protocol works similar as Cisco’s Hot Standby Router protocol (HSRP). This protocol will allow multiple nodes to share the same ip address for the sole purpose of failover redundancy. If the master node fails, then traffic will pass through the backup node.

In this scenario, when the master node fails, pfsync will ensure that users are not aware of the failover and traffic will go through the backup node in a seamless fashion. The pfsync will have been aware of the failure before the users. This provides seamless redundancy and high availability. We are going to configure XML-RPC synchronization to synchronize our configurations to the backup node. Then configure Virtual IPs for our LAN and WAN. We will then reconfigure NAT outbound as well as modify our DHCP and DNS Servers

Interface Assignments

  • SHARED WAN CARP VIP -> 190.168.183.143
  • SHARED LAN CARP VIP -> 172.16.80.5
  • SYNC INTERFACE -> 10.10.10.0/30
  • Primary FW LAN GW -> 172.16.80.1
  • Primary FW WAN GW -> 192.168.183.131
  • Secondary FW LAN GW -> 172.16.80.2
  • Secondary FW WAN GW -> 192.168.183.132

NB This is a lab environment and my WAN IP is a private IP. If you are in a production environment your WAN IP will most likely be a public IP given to you by your ISP.

Next, head over to the GUI for the primary firewall at http://172.16.80.1

We can configure the basic settings such as hostname as well as set up our ntp server and the admin password on the primary firewall following through the setup wizard:

Hostname > primaryfw

Domain > vmlab.corp

Ntp server > [ntp server pool nearest to you or if you have a dedicated ntp server in your production or lab environment]

Secondary Firewall at http://172.16.80.2

Hostname > secondaryfw

Domain > vmlab.corp

Ntp server > [ntp server pool nearest to you or if you have a dedicated ntp server in your production or lab environment]

You can set your own password.

The firewall will reload and you will be redirected to the dashboard.

We will need to change from HTTP to HTTPS on port 443. Therefore we are going to head over to System > Advanced > Check the box for HTTPS

Do this for both firewalls as it is a requirement for the pfsync to work.

Once you have checked the box for HTTPS, the page will reload and you will be redirected to the secure page.

Configure SYNC Interface

You can now configure the sync interfaces from the Interface menu.

N.B Pfsync protocol uses multicast address by default. For environments with only two cluster nodes, you will need to explicitly specify a /30 ip address for the sync interface to force unicast updates. The reason for this is for security and performance.

Set them as below.

  • Primary Firewall Sync Interface  -> 10.10.10.1/30
  • Secondary Firewall Sync Interface -> 10.10.10.2/30

XMLRPC Configuration.

As a first step, we are going to configure xmlrpc to synchronize our primary cluster firewall configurations with the secondary node.

To do this head over to System > High Aval. Sync and set the configuration as below

Check the box for Synchronize states then set the Synchronize Interface as the SYNC Interface we had set earlier. The pfsync Synchronize Peer IP is the SYNC interface IP of the secondary node which is 10.10.10.2 in my case. Do this for both the primary node and the secondary node. On the secondary node set the pfsync Synchronize Peer IP as 10.10.10.1 which is the Sync interface IP of the primary node.

N.B: For the XMLRPC Configurations options, do not set them on the secondary node, the sync will not work.

XMLRPC Configurations options on the primary node. Ensure that the same admin user with the same password exists on the secondary node.

Toggle all the options to synchronize and save your changes.

Make some firewall changes to allow traffic on the sync interface between the two nodes. We will override this setting letter on.

Allow traffic from secondary node on the SYNC interface with this rule:
Source Any, Destination Any
to avoid the below error.

Save your changes.

We can then create a user on primary firewall and check to see if we have the changes synchronized to our secondary firewall.

On the secondary node. We can view the user john

Once you have confirmed the configuration synchronization, we can then make some firewall changes on the primary firewall node. We can put our focus on the primary firewall since every change we make here will be synchronized to the secondary node.

Set the firewall changes as below

The changes will be overridden in our previous firewall change we made on the secondary firewall.

N.B Any change you make on the secondary will be overridden since we are using the primary firewall throughout.

Next, we are going to configure our failover to ensure that if our primary node goes down, the secondary will act as the main gateway for the users.

To do this, we are going to configure our Common Address Redundancy Protocol (CARP) virtual IP address for both WAN and LAN as previously stated in interface assignments.

Navigate to Firewall > Virtual IPs and click on Add

We will begin with our SHARED WAN CARP VIP

Set it as below

We have set the VHID Group as 143 to reflect our last digit of our IP for the sole purpose of administration otherwise you can set the VHID Group the number you wish.

The skew will be set to 0 to make our primary firewall node be the Master Node. From the pfSense book, a primary node will be set to 0 or 1, secondary node will typically be above 100. The values will be handled automatically by XMLRPC synchronization.

You can leave the rest as default.

Ensure the passwords are the same then save your changes.

Set the SHARED LAN CARP VIP as below

Once all that is done, you can add CARP status to your dashboard.

And on our secondary node.

As you can see, our primary firewall is the Master Node and our secondary

firewall is the Backup Node.

Configure NAT Outbound

We are now going to configure Outbound NAT for our WAN CARP VIP address.

Navigate to Firewall > NAT > Outbound tab

Change the setting to Manual Outbound NAT Rule generation.

Save your changes.

Below, you will see a set of rules. Edit the rules on the WAN with 172.16.80.0/24 and change the address on the Translate Option to 192.168.183.143 (SHARED WAN CARP VIP).

Save your changes.

Next, we will modify our DHCP Server as below:

Gateway > 172.16.80.5 (SHARED LAN CARP VIP)

DNS Server > 172.16.80.5 (SHARED LAN CARP VIP)

Failover Peer IP > 10.10.10.2 (SYNC INTERFACE SECONDARY NODE)

Testing

To test our failover we will use tracepath on our linux-host to trace the route to 8.8.8.8 (Google DNS)

With the primary firewall node up:

As it stands, traffic passes through our primary firewall node (primaryfw.vmlab.corp) as intended.

We will shut down our primary firewall and rerun the tracepath command once again to ensure the failover works as intended.

Traffic now passes through our backup firewall node (secondaryfw.vmlab.corp) as expected.

We have now come to the end of this lab.

I hope you picked up one or two from this.

bl4ckwidow

Co-Founder of Labing Overload. I am a Web Developer/Network Engineer turned CyberSecurity Engineer. FOSS enthusiast. Cisco Technologies enthusiast. Network Penetration Tester.

Leave a Reply

Your email address will not be published. Required fields are marked *