Loopback Aliasing Alternatives when using MAC Forwarding for WebSphere Edge Load Balancer on Linux

Technote (troubleshooting)


Problem(Abstract)

When using the MAC forwarding method for Load Balancer on Linux,
the Linux system behavior must be altered to make it compatible.

If the back-end servers (cluster members) are hosted on Linux, then they must be configured for ARP suppression or to NAT the incoming packets when using the MAC forwarding method for Load Balancer.

There are several different solutions, but may require the additional configuration steps shown below.

Symptom

All cluster traffic might be directed to a single back-end server in an indeterminate manner.


Cause

Why is configuring MAC forwarding for WebSphere Edge Load Balancer different on Linux?

The Linux operating system employs a host-based model of advertising hardware addresses to IP addresses using Address Resolution Protocol (ARP).

This model is incompatible with the back-end server or collocated server requirements for the Media Access Control (MAC) forwarding method with Load Balancer.

The Linux operating system issues an ARP response for any IP address that is configured on the machine for any interface that is present on the machine.

For any ARP who-has queries, the Linux operating system chooses a source IP address from the IP addresses that are present on the machine, regardless of the interfaces on which those addresses are configured.

When using the MAC forwarding method with Websphere Edge Load Balancer, you must ensure that cluster-addressed traffic can be accepted by the stacks of the back-end servers.

In a non-Linux environment, this is typically done by adding the Cluster IP to the loopback device on the back-end servers as shown below:



This requirement also includes the Collocated High Availability Standby machine when both High Availability and Collocation are in use.

This is because a Standby Load Balancer machine with a cluster member collocated is essentially a back-end server while it is not the Active Load Balancer.

In a non-Linux environment, this is typically done by adding the Cluster IP to the loopback device on the Standby Load Balancer in the goStandby script:



In a Linux environment, the Linux operating system broadcasts ownership of every IP address that is configured on a back-end machine, including the IP address for the cluster that is aliased on the Loopback device.

Therefore, all cluster traffic might be directed to a single server in an indeterminate manner.


Resolving the problem

Note: Be advised of the Load Balancer configuration limitations with zSeries and S/390 platforms.
Also, be aware that there might be distribution support implications for running a custom kernel.


To configure the Linux operating system to not advertise addresses on the Loopback, you can use any one of the following four solutions:

  1. Use a kernel that does not advertise the addresses.
    This option is preferred, as it does not incur a per-packet overhead and it does not require per-kernel reconfiguration. There are two ways to accomplish this:
    1. United Linux 1 / SLES8 with SP2(x86) or SP3 (all other architectures) and higher contains the Julian ARP hidden patch. Verify that it is always in effect before aliasing the cluster address with the command:

      # sysctl -w  net.ipv4.conf.all.hidden=1 net.ipv4.conf.lo.hidden=1

      Clusters can then be aliased in the normal way, such as:

      # ifconfig lo:1 $CLUSTER_ADDRESS netmask 255.255.255.255 up

    2. Use the arp_ignore sysctl function that is available in 2.4.25 and 2.6.5 and higher, but note that some distributions back-port features.
      1. Verify that the arp_ignore sysctl function is enabled by entering the following command at the prompt:

        # sysctl -w   net.ipv4.conf.all.arp_ignore=3 net.ipv4.conf.all.arp_announce=2

        Note: When you use the sysctl feature, ensure that these settings are retained by the operating system by adding the settings to the install_path/etc/sysctl.conf file.

      2. Alias the clusters by entering following command at the prompt:

        # ip addr add $CLUSTER_ADDRESS/32 scope host dev lo


        High Availability Collocation Notes for this method:

        In High Availability Collocation configurations, the goActive script should add the Cluster IP to the interface and remove the Loopback alias.

        An example of the goActive script modification for this:

        ip addr del $CLUSTER_ADDRESS/32 scope host dev lo
        ifconfig $INTERFACE $CLUSTER netmask $NETMASK up


        In High Availability Collocation configurations, the goStandby script should add the Loopback alias and remove the Cluster IP from the interface.

        An example of the goStandby script modification for this:

        ifconfig $INTERFACE down
        ip addr add $CLUSTER_ADDRESS/32 scope host dev lo


        Note: High Availability and goScripts only apply to WebSphere Edge Load Balancer "IPv4" for V7.0.0. Colocation is not supported on "IPv4 and IPv6" Load Balancer  

  2. Use IP tables to redirect all incoming cluster traffic to the local host.
    If you use this method, do not configure the loopback adapter with an alias.

    Instead, use the command:

    # iptables -t nat -A PREROUTING -d $CLUSTER_ADDRESS -j REDIRECT

    This command causes the Linux operating system to do a destination network address translation
    (NAT) on each packet by converting the cluster address to the interface address. This method has throughput impact of 6.4% connections-per-second. This method works on any supported stock distribution; a kernel module or kernel patch+build+install is not needed.

  3. Apply the noarp module Version 1.2.0 or later.
    The kernel source must be available and properly configured, and development tools, such as gcc, gnu make, and so on, must be available. You must build and install the module every time the kernel is upgraded. The module is available at http://www.masarlabs.com/noarp/. Because the kernel code itself is not modified, it is much less intrusive than the last solution, which is available later in this document, and is much less prone to error. It also must be configured before any cluster address is aliased on the loopback.

    For example:

    # modprobe noarp
    # noarpctl add $CLUSTER_ADDRESS
    nic-primary-addr

    where the nic-primary-addr variable is an address in the same subnet as the cluster address. You can then alias clusters in the normal way, such as:

    # ifconfig lo:1 cluster-address netmask 255.255.255.255 up

    Note:
    For high availability collocation configurations, place the noarpctl adds command and the noarpctl dels commands in the go* scripts to ensure that the active Load Balancer can perform an ARP for the cluster address. Also, placing these commands in the go* scripts ensures that the standby Load Balancer, which is acting as a server, does not accidentally or indeterminately begin to receive all of the cluster traffic. High Availability and goScripts only apply to WebSphere Edge Load Balancer "IPv4" for V7.0.0. Colocation is not supported on "IPv4 and IPv6" Load Balancer

  4. If you do not have the Julian ARP hidden patch, you can download the Julian patch from your Linux provider

    Follow your distribution instructions for patching and compiling a kernel that is suitable for use with that distribution. United Linux 1 / SLES8 with SP2(x86) or SP3 (all other architectures) and higher contain the Julian ARP hidden patch.

    Note: If this Load Balancer is a collocated high availability Load Balancer, ensure that the uname -r command matches the kernel that was supplied with the distribution, and ensure that you start with the .config file for the distribution kernel.

    After you build, install, and run your kernel with the Julian hidden patch, complete the following instructions to enable and use the patch:
    1. Ensure that patch is always in effect before aliasing the cluster address with the command:

      # sysctl -w net.ipv4.conf.all.hidden=1 net.ipv4.conf.lo.hidden=1

      Note: When you use the sysctl feature, ensure that these settings are retained by the operating system by adding the settings to the install_path/etc/sysctl.conf file.

    2. Alias the cluster in the normal way, such as:

      # ifconfig lo:1 $CLUSTER_ADDRESS netmask 255.255.255.255 up

Troubleshooting:
Here are some commands that may be helpful to see your current settings.
  • You can check your arp settings by issuing:

    # sysctl net.ipv4.conf.all.arp_ignore
    # sysctl net.ipv4.conf.all.arp_announce


    or a command like:

    # sysctl -a | grep arp

  • You can check your loopback and interface settings by issuing:

    # ifconfig -a

  • You can check your ip addr settings by issuing:

    # ip addr show

  • You can check your iptables settings by the following method:

    Issue the following command to determine if iptables are loaded:

    # lsmod | grep ip_tables

    The output from the preceding command might be similar to this:

    ip_tables         22400   3
    iptable_mangle,iptable_nat,iptable_filter

    Issue the following command for each iptable listed in the output to display the rules for the tables:

    # iptables -t <short name> -L

    For example:

    # iptables -t mangle -L
    # iptables -t nat    -L
    # iptables -t filter -L

  • You can check your sysctl contents by issuing:

    # cat /etc/sysctl.conf

Cross reference information
Segment Product Component Platform Version Edition
Application Servers WebSphere Edge Server Load Balancer Linux 6.1, 6.0, 5.1
Application Servers Runtimes for Java Technology Java SDK

Historical Number

207567

Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

WebSphere Application Server
Edge Component

Software version:

5.1, 6.0, 6.1, 7.0

Operating system(s):

Linux

Software edition:

Network Deployment

Reference #:

1177105

Modified date:

2004-08-16

Translate my page

Machine Translation

Content navigation