Configure LACP under CentOS along with Cisco

LACP setup

LACP setup

If a host has multiple physical network adapters, it is a common procedure to combine the particular links. Mostly, this configuration is chosen to enhance the maximal throughput. Another reason for this is to make network connections more reliable by defining alternative network paths. LACP (Link Aggregation Control Protocol, 802.3ad standard) combines those two approaches by grouping physical adapters logically. As long as enough physical uplinks are available, they are used to spread network traffic frame-wise (continous sessions are excluded from this behavior to ensure correct package orders). If a network path fails, the channel is automatically reconfigured – this is also called dynamic trunking. LACP needs to be supported by the switch as well as by the connected host because of automatic configuration streams.

Switch preparations

The switch configuration varies depending on the vendor and model. I’m using a Cisco SG300-28 L3 switch in my home lab. The following steps demonstrate the configuration using the web interface:

  1. Select Port Management > Link Aggregation > LAG Management in the menu on the left.
  2. Select a free LAG port from the list and click Edit.
  3. Enter a short description and select the used switch ports. Save the configuration by clicking Apply.
  4. For advanced settings (e.g port auto negotiation, optional), click LAG Settings in the menu on the left. Select the port and click Edit.
Cisco LAG settings

Cisco LAG settings

Configuration under CentOS

The Linux configuration varies depending on the distribution. I’m using CentOS – the following steps might need to be changed for other Linux distributions.

First of all, it is necessary to ensure that the kernel module for network teaming (bonding) gets loaded during the boot. An alias defining the network interface name is set – e.g. bond0 for the first bond. If you want to configure multiple bonds, you can define multiple entries – e.g. bond1 and bond2. Afterwards, the module is loaded:

# echo "alias bond0 bonding" > /etc/modprobe.d/bonding.conf
# modprobe bond0

The next step is to alter the configuration of participating network interface. Pre-existing IP information need to be removed and the interface is defined as  Slave. On Enterprise Linux-based Linux distributions, these configuration files are stored in the /etc/sysconfig/network-scripts directory:

# vi /etc/sysconfig/network-scripts/ifcfg-eno1
NAME="eno1"
DEVICE="eno1"
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes

ESC ZZ

This step is repeated for all other participating network interfaces. Afterwards, a configuration file for the bond interface is created:

# vi /etc/sysconfig/network-scripts/ifcfg-bond0
NAME="bond0"
DEVICE="bond0"
BONDING_OPTS="mode=4 miimon=100"
ONBOOT=yes
BOOTPROTO=static
...

ESC ZZ

The line BONDING_OPTS is important, it defines options for the bonding drive. The parameter mode defines the bonding mode, the value 4 enables LACP in this case. miimon defines the time frame in milliseconds, the network connection is probed for connectivity. The smaller the value, the faster network outages are recognized – – 100 is a decent default value.

Afterwards, the network is restarting – alternatively you can only bring up the new bond by using the ifup command:

# service network restart
# ifup bond0

If everything worked fine, the /proc file system now contains information about the bonding:

# less /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
 Aggregator ID: 1
 Number of ports: 2
 Actor Key: 9
 Partner Key: 1000
 Partner Mac Address: xx:xx:xx:xx:xx:xx

Slave Interface: eno1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: xx:xx:xx:xx:xx:xx
...
Slave Interface: eno2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: xx:xx:xx:xx:xx:xx
...

In the Cisco web interface the active links can be seen as well (Port Management > Link Aggregation > LAG Management):

LAG links

LAG links

The following screenshot and outputs demonstrate recognizing a failed link in the Cisco web interface (standby member) and the Linux system:

Standby LAG link

Standby LAG link

# dmesg
...
[277202.078630] tg3 0000:03:00.0 eno1: Link is down
[277202.079761] bond0: link status definitely down for interface eno1, disabling it
[277202.079796] bond0: first active interface up!
# less /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
...
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 1
...
Slave Interface: eno1
MII Status: down
Speed: Unknown
Duplex: Unknown
Link Failure Count: 2
...

Sharing is caring

Leave a Reply