You can configure two appliances as an HA (high availability) pair to provide hardware redundancy for core network services and Infoblox Advanced DNS Protection. For more information, see Infoblox Advanced DNS Protection. An HA pair can be a Grid Master, a Grid Master candidate, a Grid member, or an independent appliance. The two nodes that form an HA pair—identified as Node 1 and Node 2—are in an active/passive configuration. The active node receives, processes, and responds to all service requests. The passive node constantly keeps its database synchronized with that of the active node, so it can take over services if a failover occurs. A failover is the reversal of the active/passive roles of each node; that is, when a failover occurs, the previously active node becomes passive and the previously passive node becomes active. You can configure an HA pair in either IPv4, IPv6, or in dual mode. An IPv4 HA pair uses IPv4 as the communication protocol between the two nodes and an IPv6 HA pair uses IPv6 as the communication protocol between the two nodes. But in a dual mode HA pair, you can select either IPv4 or IPv6 as the communication protocol between the two nodes. Note that when you add a dual mode HA member to a Grid, the communication protocol between the two nodes of an HA pair must the same as the Grid communication protocol.

Note: HA Grid Master and HA Grid Master candidate configurations are not supported when Threat Protection licenses are installed on the appliance.

When you configure an HA pair using the IB-4030 (Rev-1 or Rev-2) appliance for DNS cache acceleration, the passive node does not operate with a pre-loaded cache or hot cache during a failover; it builds up the DNS cache over time. For more information about HA and other limitations for the IB-4030 appliances, refer to the Infoblox DNS Cache Acceleration Application Guide.
For Infoblox , only the active node in an HA pair handles DNS traffic. The passive node is in a standby mode ready to take over if a failover occurs.
The appliance uses the following components in the HA functionality:

• bloxSYNC: An Infoblox proprietary mechanism for secure, real-time synchronization of the database that maintains the data, system configuration, and protocol service configuration between the two nodes. With bloxSYNC, the nodes continuously synchronize changes of their configurations and states. When a failover occurs, the passive node can quickly take over services. For information, see About HA Failover
• VRRP (Virtual Router Redundancy Protocol): An industry-standard, MAC-level HA failover mechanism. VRRP utilizes the concept of an active and passive node that share a single VIP (virtual IP) address. When the active node that owns the VIP becomes unavailable, the passive node takes over the VIP and provides network core services. For information about VRRP, refer to RFC3768,Virtual Router Redundancy Protocol (VRRP) and VRRP Advertisements.

Using bloxSYNC and VRRP combined, if the active node fails or is taken offline for maintenance purposes, the passive node assumes the VIP and continues to respond to requests and services with minimal interruption. You can deploy an HA pair as a Grid Master, a Grid member, or an independent HA. To deploy an independent HA pair, see Deploying an Independent HA Pair. To deploy an HA Grid Master, see Creating a Grid Master.

Planning for an HA Pair

To achieve high availability, the HA and LAN1 (or VLAN) ports on both the active and passive nodes are connected to switches on the same network or VLAN. Both nodes in an HA pair share a single VIP address and a virtual MAC address so they can appear as a single entity on the network. You can also assign IPv6 addresses for each of the active and passive nodes, in addition to the IPv6 VIP address.

Tip: Infoblox uses VRRP advertisements for the active and passive HA design. Therefore, all HA pairs must be located in the same location connected to the highly available switching infrastructure. Any other deployment is not supported without a written agreement with Infoblox. Contact Infoblox Technical Support for more information about other deployment support.

Note: You can enable ARP on the passive node of an HA pair and monitor its status externally. To enable ARP on the passive node of an HA pair, see Enabling ARP on the Passive Node of an HA Pair.

As illustrated in Figure 5.9, the VIP and virtual MAC addresses link to the HA port on each node. Select five IP addresses on the same network before you configure an HA pair, as follows:

• VIP: For core network services and for management purposes when the MGMT port is disabled. Both nodes share the same VIP.
• Node 1 HA (active): Source IP for the VIP and VRRP advertisements
• Node 1 LAN1 (active): For management through SSHv2 and listens for VRRP advertisements from the HA port
• Node 2 HA (passive): Listens for VRRP advertisements
• Node 2 LAN1 (passive): Source IP for SSL VPN to the VIP of the active node and receives bloxSYNC from the VIP

Figure 5.9 HA Pair
 

When you deploy a vNIOS HA pair, ensure that the port connection allows for more than one MAC address per vNIC. For example, if you deploy a vNIOS HA pair in VMware vSphere, the port-profile to which the vNIOS HA and LAN ports connect should allow for more than one MAC address per vNIC. You can do this by changing the security settings of the port-group to accept "MAC address changes" and "Forged transmits," as illustrated in Figure 5.10.

Figure 5.10 Configuring port-profile in VMware vSphere

About HA Failover

The appliance supports HA through bloxHA™, which provides a robust failover mechanism. As described in Planning for an HA Pair, both nodes in an HA pair share a single VIP address and a virtual MAC address. The node that is currently active is the one whose HA port owns the VIP address and virtual MAC address. When a failover occurs, these addresses shift from the HA port of the previous active node to the HA port of the new active node, as illustrated in Figure 5.11.

Note: For a vNIOS HA pair, you must configure both LAN1 and HA interfaces to operate. When there is a notification about failure in any one of the port, make sure that both of these ports are working. If one of the port is down and another port is still working, the HA pair believes its peer is active. But, there will be connectivity issues as one of the port is down. An HA failover occurs on vNIOS appliances only when both of these ports are down. For details about configuring these virtual NICs, refer to the Infoblox Installation Guide vNIOS for VMware.

Figure 5.11 VIP Address and Virtual MAC Address and HA Failover

Enabling ARP on the Passive Node of an HA Pair

You can enable ARP (Address Resolution Protocol) on the passive node of an HA pair and monitor its status externally. For example, when the active node of an HA pair fails over to the passive node, you can ping the passive node from an external location and monitor its status. By default, ARP is disabled on the passive node of an HA pair. ARP settings on an HA member are preserved during a system restart or reboot, HA switch over, and upgrade. In addition, you do not need to restart the appliance when you modify ARP settings. When the active node becomes passive during an HA failover, ARP on an HA member inherits the settings configured in the database.
You can view detailed status for both nodes of an HA pair through the Detailed Status panel. To view the Detailed Status panel, from the Grid tab, select the Grid Manager tab -> Members tab -> Grid_member check box -> Detailed Status icon in the horizontal navigation bar. In the Detailed Status panel, you can view ARP connectivity status for the passive node of an HA pair (Green = The passive HA node is connected to the local router; Yellow = The passive HA node fails to connect to the local router; Gray = ARP is disabled on the passive node of an HA pair). The passive HA node uses arping to test the ARP connectivity with the local router. If the local router is not configured, you may see false warnings even if the ARP connectivity is fine. In case of an ARP connectivity failure, the appliance sends an SNMP trap and an email notification, if configured.
Note that the ARP setting is not preserved on a passive HA node when you reset the appliance using the CLI command
reset all or reset the database using the CLI command reset database. To enable ARP on an HA passive node:

1. From the Grid tab, select the Grid Manager tab -> Members tab.
2. Select an HA member and click the Edit icon.
3. In the Grid Member Properties editor, select the Network tab -> Advanced tab and complete the following:
   • Enable ARP on HA Passive Node?: Select one of the following:
     • Disable (default): Select this to disable ARP on an HA passive node. This is selected by default.
     • Enable (not recommended): Select this to enable ARP on an HA passive node.

WARNING: Enabling ARP on the passive node of an HA interface might affect VRRP on the local network and could cause the firewall to send false alerts.

   4. Save the configuration and click Restart if it appears at the top of the screen.

HA Failover on DNS Nameservers

When an HA failover occurs on NIOS, there is an approximate 4-5 second time interval in which the network is adjusted for the new active node and the new passive node. During this failover period, the active node becomes unresponsive. After the new active node comes up on the network,  the DNS service loads all Response Policy Zone (RPZ) files if RPZ is configured. The larger the RPZ files, the longer it takes to load them, and the longer it takes the DNS service to start serving DNS. For example, on a TE-1425 with RPZs that contain 15 million resource records, it can take approximately one and a half minutes to start serving DNS.

If your nameserver uses Grid replication to keep internal zones up to date and is not configured to use RPZ, then the delay before the DNS service starts serving DNS is slightly longer that it is for the HA failover itself. 

VRRP Advertisements

VRRP advertisements are periodic announcements of the availability of the HA node linked to the VIP. The two nodes in an HA pair include a VRID (virtual router ID) in all VRRP advertisements and use it to recognize VRRP advertisements intended for themselves. Only another appliance on the same subnet configured to use the same VRID responds to the announcements. The active node in an HA pair sends advertisements as multicast datagrams every second. It sends them from its HA port using the source IP address of the HA port (not from the VIP address) and the sour ce MAC address 00:00:5e:00:01:vrrp_id. The last two hexadecimal numbers in the source MAC address indicate the VRID number for this HA pair. For example, if the VRID number is 143, then the source MAC address is 00:00:5e:00:01:8f (8f in hexadecimal notation = 143 in decimal notation).
The destination MAC and IP addresses for all VRRP advertisements are 00:00:5e:00:01:12 and 224.0.0.18 (00:00:5e:00:02:12 and FF02::12 for IPv6 only configurations). Because a VRRP advertisement is a multicast datagram that can only be sent within the immediate logical broadcast domain, the nodes in an HA pair must be in the same subnet together.
As illustrated in Figure 5.12, when you configure an HA pair, only the appliance configured to listen for VRRP advertisements with the same VRID number processes the datagrams, while all other appliances ignore them. The passive node in an Infoblox HA pair listens for these on its HA port and the active node listens on its LAN1 or LAN1 (VLAN) port. If the passive node does not receive three consecutive advertisements or if it receives an advertisement with the priority set to 0 (which occurs when you manually perform a forced failover or request the active node to restart, reboot, or shut down), it changes to the active state and assumes ownership of the VIP address and virtual MAC address.
If both nodes go offline, the one that comes online first becomes the active node. If they come online simultaneously, or if they enter a dual-active state—that is, a condition arises in which both appliances assume an active role and send VRRP advertisements, possibly because of network issues—then the appliance with the numerically higher VRRP priority becomes the active node. The priority is based on system status and events.
If both nodes have the same priority, then the appliance whose HA port has a numerically higher IP address becomes the active node. For example, if the IP address of the HA port on Node 1 is 10.1.1.80 and the IP address of the HA port on Node 2 is 10.1.1.20, then Node 1 becomes the active node.
For more information about VRRP, see RFC 3768, Virtual Router Redundancy Protocol (VRRP).

Figure 5.12 VRRP Advertisements with a Unique VRID

Note: For a dual mode (IPv4 and IPv6) HA Master or HA member, you can set either IPv4 or IPv6 for VRRP advertisements.