{"id":14901,"date":"2020-12-28T16:57:34","date_gmt":"2020-12-28T15:57:34","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/"},"modified":"2020-12-28T16:57:34","modified_gmt":"2020-12-28T15:57:34","slug":"building-a-network-bonding-between-2-cards-on-oracle-linux","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/","title":{"rendered":"Building a network bonding between 2 cards on Oracle Linux"},"content":{"rendered":"

I recently needed to configure bonding between 2 network cards on a customer side and I wanted trough this blog to share my findings and how I built it showing some traces. I will also do a short comparison of what is possible or not on the ODA.<\/p>\n

<\/p>\n

Why should I use bonding?<\/h3>\n

Bonding is a technology which will allow you to merge several network interfaces, either ports of the same cards or ports from separated network cards, into a same logical interface. Purposes would be to have some network redundancy in case of network failure, called fault tolerance, or to increase the network throughput (bandwidth), called load balancing.<\/p>\n

What bonding mode should I use?<\/h3>\n

There are 7 bonding modes available to achieve these purposes. All bonding modes will guarantee fault tolerance. Some bonding modes will have load balancing functionnalities. For bonding mode 4 the switch will need to support links aggregation (EtherChannel). Link aggregation can be configured manually on the switch or automatically using LACP protocol (dynamic links aggregation).<\/p>\n\n\n\n\n\n\n\n\n\n
Mode<\/td>\nDescription<\/td>\nFault tolerance<\/td>\nLoad balancing<\/td>\n<\/tr>\n
0<\/td>\nRound-Robin<\/td>\nPackets are sequentially transmitted and received through each interfaces one by one.<\/td>\nYES<\/td>\nYES<\/td>\n<\/tr>\n
1<\/td>\nActive-backup<\/td>\nOnly one interface will be the active one. The other interface from the bonding configuration will be configured as backup. If the active interface will be in failure one of the backup interface will become the active one. The MAC address will only be visible on one port at the same time to avoid any confusion for the switch.<\/td>\nYES<\/td>\nNO<\/td>\n<\/tr>\n
2<\/td>\nBalance-xor<\/td>\nPeer connections are matched with MAC addresses of the slave interfaces. Once the connection is established the transmission of the peers is always sent over the same slave interface.<\/p>\nYES<\/td>\nYES<\/td>\n<\/tr>\n
3<\/td>\nBroadcast<\/td>\nAll network transmissions are sent on all slaves.<\/td>\nYES<\/td>\nNO<\/td>\n<\/tr>\n
4<\/td>\n802.3ad – Dynamic Link Aggregation<\/td>\nThis mode will aggregate all interfaces from the bonding into a logical one. The traffic is sent and received on all slaves from the aggregation. The switch needs to support LACP and LACP needs to be activated.<\/td>\nYES<\/td>\nYES<\/td>\n<\/tr>\n
5<\/td>\nTLB – Transmit Load Balancing<\/td>\nThe outgoing traffic is distributed between all interfaces depending of the current load of each slave interface. Incoming traffic is received by the current active slave. In case the active interface fails, another slave will take over the MAC address of the failed interface.<\/td>\nYES<\/td>\nYES<\/td>\n<\/tr>\n
6<\/td>\nALB – Adaptive Load Balancing<\/td>\nThis mode includes TLB (Transmit Load Balancing) and will use RLB (Receive Load Balancing) as well. The load balanced for the received packets will be done through ARP (Address Resolution Protocol) negotiation.<\/td>\nYES<\/td>\nYES<\/td>\n<\/tr>\n<\/table>\n

In my case, our customer wanted to guarantee the service in case of one network card failure only. No load balancing. The switch was not configured to use LACP. I then decided to configure the bonding in active-backup mode, which will guarantee redundancy only.<\/p>\n

Bonding configuration<\/h3>\n

Checking existing connection<\/h4>\n

The server is composed of 2 network cards having each of the card 4 interfaces (ports).
\nCard 1 : em1, em2, em3, em4
\nCard 2 : p4p1, p4p2, p4p3, p4p4 <\/p>\n

There is no bonding currently existing as shown in below output.<\/p>\n

\n[root@SRV ~]# nmcli connection\nNAME  UUID                                  TYPE      DEVICE\np4p1  d3cdc8f5-2d80-433d-9502-3b357c57f307  ethernet  p4p1\nem1   f412b74b-2160-4914-b716-88f6b4d58c1f  ethernet  --\nem2   0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3   d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4   52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2  12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3  0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4  a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
Checking existing configuration<\/h4>\n
The server was configured only with one IP address on the p4p1 network interface.<\/p>\n
\n[root@SRV network-scripts]# pwd\n\/etc\/sysconfig\/network-scripts\n\n[root@SRV network-scripts]# ls -l ifcfg*\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em1\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em2\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em3\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em4\n-rw-r--r--. 1 root root 254 Aug 19  2019 ifcfg-lo\n-rw-r--r--. 1 root root 378 Sep 21 17:09 ifcfg-p4p1\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p2\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p3\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p4\n\n[root@SRV network-scripts]# more ifcfg-p4p1\nTYPE=Ethernet\nPROXY_METHOD=none\nBROWSER_ONLY=no\nBOOTPROTO=none\nDEFROUTE=yes\nIPV4_FAILURE_FATAL=no\nIPV6INIT=yes\nIPV6_AUTOCONF=yes\nIPV6_DEFROUTE=yes\nIPV6_FAILURE_FATAL=no\nIPV6_ADDR_GEN_MODE=stable-privacy\nNAME=p4p1\nUUID=d3cdc8f5-2d80-433d-9502-3b357c57f307\nDEVICE=p4p1\nONBOOT=yes\nIPADDR=192.168.1.180\nPREFIX=24\nGATEWAY=192.168.1.1\nDNS1=192.168.1.5\nDOMAIN=domain.com\nIPV6_PRIVACY=no\n<\/pre>\n
<\/br><\/p>\n
Creating the bonding<\/h4>\n
Purpose is to create a bonding between the 2 network cards for fault tolerance. The bonding will then be composed of the slave interfaces p4p1 and em1.
\nThe bonding mode selected will be the mode 1 (active-backup).<\/p>\n
\n[root@SRV network-scripts]# nmcli con add type bond con-name bond1 ifname bond1 mode active-backup ip4 192.168.1.180\/24\nConnection 'bond1' (7b736616-f72d-46b7-b4eb-01468639889b) successfully added.\n\n[root@SRV network-scripts]# nmcli conn\nNAME   UUID                                  TYPE      DEVICE\np4p1   d3cdc8f5-2d80-433d-9502-3b357c57f307  ethernet  p4p1\nbond1  7b736616-f72d-46b7-b4eb-01468639889b  bond      bond1\nem1    f412b74b-2160-4914-b716-88f6b4d58c1f  ethernet  --\nem2    0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3    d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4    52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2   12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3   0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4   a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
Updating the bonding with appropriate gateway, dns and domain information<\/h4>\n
\n[root@SRV network-scripts]# cat ifcfg-bond1\nBONDING_OPTS=mode=active-backup\nTYPE=Bond\nBONDING_MASTER=yes\nPROXY_METHOD=none\nBROWSER_ONLY=no\nBOOTPROTO=none\nIPADDR=192.168.1.180\nPREFIX=24\nDEFROUTE=yes\nIPV4_FAILURE_FATAL=no\nIPV6INIT=yes\nIPV6_AUTOCONF=yes\nIPV6_DEFROUTE=yes\nIPV6_FAILURE_FATAL=no\nIPV6_ADDR_GEN_MODE=stable-privacy\nNAME=bond1\nUUID=7b736616-f72d-46b7-b4eb-01468639889b\nDEVICE=bond1\nONBOOT=yes\n\n[root@SRV network-scripts]# vi ifcfg-bond1\n\n[root@SRV network-scripts]# cat ifcfg-bond1\nBONDING_OPTS=mode=active-backup\nTYPE=Bond\nBONDING_MASTER=yes\nPROXY_METHOD=none\nBROWSER_ONLY=no\nBOOTPROTO=none\nIPADDR=192.168.1.180\nPREFIX=24\nDEFROUTE=yes\nIPV4_FAILURE_FATAL=no\nIPV6INIT=yes\nIPV6_AUTOCONF=yes\nIPV6_DEFROUTE=yes\nIPV6_FAILURE_FATAL=no\nIPV6_ADDR_GEN_MODE=stable-privacy\nNAME=bond1\nUUID=7b736616-f72d-46b7-b4eb-01468639889b\nDEVICE=bond1\nONBOOT=yes\nGATEWAY=192.168.1.1\nDNS1=192.168.1.5\nDOMAIN=domain.com\n<\/pre>\n
<\/br><\/p>\n
Adding slave interface em1 in the bonding bond1<\/h4>\n
Each slaves needs to be added to the master bonding.<\/p>\n
We will first delete existing em1 slave :<\/p>\n
\n[root@SRV network-scripts]# nmcli con delete em1\nConnection 'em1' (f412b74b-2160-4914-b716-88f6b4d58c1f) successfully deleted.\n<\/pre>\n
<\/br><\/p>\n
We will then create new em1 interface part of the bond1 bonding configuration :<\/p>\n
\n[root@SRV network-scripts]# nmcli con add type bond-slave ifname em1 con-name em1 master bond1\nConnection 'em1' (8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b) successfully added.\n<\/pre>\n
<\/br><\/p>\n
And we can check the interfaces :<\/p>\n
\n[root@SRV network-scripts]# nmcli con\nNAME   UUID                                  TYPE      DEVICE\np4p1   d3cdc8f5-2d80-433d-9502-3b357c57f307  ethernet  p4p1\nbond1  7b736616-f72d-46b7-b4eb-01468639889b  bond      bond1\nem1    8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b  ethernet  em1\nem2    0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3    d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4    52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2   12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3   0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4   a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
Activating the bonding<\/h4>\n
We need to first activate the first configured slaves :<\/p>\n
\n[root@SRV network-scripts]# nmcli con up em1\nConnection successfully activated (D-Bus active path: \/org\/freedesktop\/NetworkManager\/ActiveConnection\/4)\n<\/pre>\n
<\/br><\/p>\n
We can now activate the bonding :<\/p>\n
\n[root@SRV network-scripts]# nmcli con up bond1\nConnection successfully activated (master waiting for slaves) (D-Bus active path: \/org\/freedesktop\/NetworkManager\/ActiveConnection\/5)\n<\/pre>\n
<\/br><\/p>\n
We can check the connections :<\/p>\n
\n[root@SRV network-scripts]# nmcli con\nNAME   UUID                                  TYPE      DEVICE\np4p1   d3cdc8f5-2d80-433d-9502-3b357c57f307  ethernet  p4p1\nbond1  7b736616-f72d-46b7-b4eb-01468639889b  bond      bond1\nem1    8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b  ethernet  em1\nem2    0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3    d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4    52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2   12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3   0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4   a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
Adding slave interface p4p1 in the bonding bond1<\/h4>\n
We will first delete existing p4p1 slave :<\/p>\n
\n[root@SRV network-scripts]# nmcli con delete p4p1\nConnection 'p4p1' (d3cdc8f5-2d80-433d-9502-3b357c57f307) successfully deleted.\n\n[root@SRV network-scripts]# nmcli con\nNAME   UUID                                  TYPE      DEVICE\nbond1  7b736616-f72d-46b7-b4eb-01468639889b  bond      bond1\nem1    8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b  ethernet  em1\nem2    0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3    d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4    52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2   12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3   0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4   a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
We will then create new p4p1 interface part of the bond1 bonding configuration :<\/p>\n
\n[root@SRV network-scripts]# nmcli con add type bond-slave ifname p4p1 con-name p4p1 master bond1\nConnection 'p4p1' (efef0972-4b3f-46a2-b054-ebd1aa201056) successfully added.\n<\/pre>\n
<\/br><\/p>\n
And we can check the interfaces :<\/p>\n
\n[root@SRV network-scripts]# nmcli con\nNAME   UUID                                  TYPE      DEVICE\nbond1  7b736616-f72d-46b7-b4eb-01468639889b  bond      bond1\nem1    8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b  ethernet  em1\np4p1   efef0972-4b3f-46a2-b054-ebd1aa201056  ethernet  p4p1\nem2    0ab78e63-bde7-4c77-b455-7dcb1d5c6813  ethernet  --\nem3    d6569615-322f-477b-9693-b42ee3dbe21e  ethernet  --\nem4    52949f94-52d1-463e-ba32-06c272c07ce0  ethernet  --\np4p2   12f01c70-4aab-42db-b0e8-b5422e43c1b9  ethernet  --\np4p3   0db2f5b9-d968-44cb-a042-cff20f112ed4  ethernet  --\np4p4   a2a0ebc4-ca74-452e-94ba-6d5fedbfdf28  ethernet  --\n<\/pre>\n
<\/br><\/p>\n
Activating the new p4p1 slave interface<\/h4>\n
We can now activate the next recently added slaves :<\/p>\n
\n[root@SRV network-scripts]# nmcli con up p4p1\nConnection successfully activated (D-Bus active path: \/org\/freedesktop\/NetworkManager\/ActiveConnection\/11)\n<\/pre>\n
<\/br><\/p>\n
Restart the network service<\/h4>\n
We will restart the network service to have the new bonding configuration taking into account :<\/p>\n
\n[root@SRV network-scripts]# service network restart\nRestarting network (via systemctl):                        [  OK  ]\n<\/pre>\n
<\/br><\/p>\n
We can check the IP configuration :<\/p>\n
\n[root@SRV network-scripts]# ip addr sh\n1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\n    inet 127.0.0.1\/8 scope host lo\n       valid_lft forever preferred_lft forever\n    inet6 ::1\/128 scope host\n       valid_lft forever preferred_lft forever\n2: em1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n3: em3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4e brd ff:ff:ff:ff:ff:ff\n4: em2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:51 brd ff:ff:ff:ff:ff:ff\n5: em4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4f brd ff:ff:ff:ff:ff:ff\n6: p4p1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n7: p4p2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:31 brd ff:ff:ff:ff:ff:ff\n8: p4p3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:32 brd ff:ff:ff:ff:ff:ff\n9: p4p4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:33 brd ff:ff:ff:ff:ff:ff\n11: bond1:  mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n    inet 192.168.1.180\/24 brd 192.168.1.255 scope global noprefixroute bond1\n       valid_lft forever preferred_lft forever\n    inet6 fe80::b4f9:e44d:25fc:3a6\/64 scope link noprefixroute\n       valid_lft forever preferred_lft forever\n<\/pre>\n
<\/br><\/p>\n
Check IP configuration files<\/h3>\n
We are now having our bond ifcfg configuration file :<\/p>\n
\n[root@SRV ~]# cd \/etc\/sysconfig\/network-scripts\n\n[root@SRV network-scripts]# pwd\n\/etc\/sysconfig\/network-scripts\n\n[root@SRV network-scripts]# ls -ltrh ifcfg*\n-rw-r--r--. 1 root root 254 Aug 19  2019 ifcfg-lo\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p4\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p2\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em4\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em3\n-rw-r--r--. 1 root root 277 Sep 21 17:09 ifcfg-p4p3\n-rw-r--r--. 1 root root 275 Sep 21 17:09 ifcfg-em2\n-rw-r--r--. 1 root root 411 Oct  7 16:45 ifcfg-bond1\n-rw-r--r--. 1 root root 110 Oct  7 16:46 ifcfg-em1\n-rw-r--r--. 1 root root 112 Oct  7 16:50 ifcfg-p4p1\n<\/pre>\n
<\/br><\/p>\n
The bonding file will have the IP configuration :<\/p>\n
\n[root@SRV network-scripts]# cat ifcfg-bond1\nBONDING_OPTS=mode=active-backup\nTYPE=Bond\nBONDING_MASTER=yes\nPROXY_METHOD=none\nBROWSER_ONLY=no\nBOOTPROTO=none\nIPADDR=192.168.1.180\nPREFIX=24\nDEFROUTE=yes\nIPV4_FAILURE_FATAL=no\nIPV6INIT=yes\nIPV6_AUTOCONF=yes\nIPV6_DEFROUTE=yes\nIPV6_FAILURE_FATAL=no\nIPV6_ADDR_GEN_MODE=stable-privacy\nNAME=bond1\nUUID=7b736616-f72d-46b7-b4eb-01468639889b\nDEVICE=bond1\nONBOOT=yes\nGATEWAY=192.168.1.1\nDNS1=192.168.1.5\nDOMAIN=domain.com\n<\/pre>\n
<\/br><\/p>\n
p4p1 interface will be one of the bond1 slave :<\/p>\n
\n[root@SRV network-scripts]# cat ifcfg-p4p1\nTYPE=Ethernet\nNAME=p4p1\nUUID=efef0972-4b3f-46a2-b054-ebd1aa201056\nDEVICE=p4p1\nONBOOT=yes\nMASTER=bond1\nSLAVE=yes\n<\/pre>\n
<\/br><\/p>\n
em1 interface from the other physical network card will be the next bond1 slave :<\/p>\n
\n[root@SRV network-scripts]# cat ifcfg-em1\nTYPE=Ethernet\nNAME=em1\nUUID=8c72c383-e1e9-4e4b-ac2f-3d3d81d5159b\nDEVICE=em1\nONBOOT=yes\nMASTER=bond1\nSLAVE=yes\n<\/pre>\n
<\/br><\/p>\n
Check bonding interfaces and mode<\/h3>\n
\n[root@SRV network-scripts]# cat \/proc\/net\/bonding\/bond1\nEthernet Channel Bonding Driver: v3.7.1 (April 27, 2011)\n\nBonding Mode: fault-tolerance (active-backup)\nPrimary Slave: None\nCurrently Active Slave: em1\nMII Status: up\nMII Polling Interval (ms): 100\nUp Delay (ms): 0\nDown Delay (ms): 0\n\nSlave Interface: em1\nMII Status: up\nSpeed: 1000 Mbps\nDuplex: full\nLink Failure Count: 1\nPermanent HW addr: bc:97:e1:5b:e4:50\nSlave queue ID: 0\n\nSlave Interface: p4p1\nMII Status: up\nSpeed: 1000 Mbps\nDuplex: full\nLink Failure Count: 1\nPermanent HW addr: 3c:fd:fe:85:0d:30\nSlave queue ID: 0\n[root@SRV network-scripts]#\n<\/pre>\n
<\/br><\/p>\n
Test the bonding<\/h3>\n
Both network cables are plugged into em1 and p4p1. Both interfaces are UP. :<\/p>\n
\n[root@SRV network-scripts]# ip addr sh\n1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\n    inet 127.0.0.1\/8 scope host lo\n       valid_lft forever preferred_lft forever\n    inet6 ::1\/128 scope host\n       valid_lft forever preferred_lft forever\n2: em1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n3: em3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4e brd ff:ff:ff:ff:ff:ff\n4: em2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:51 brd ff:ff:ff:ff:ff:ff\n5: em4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4f brd ff:ff:ff:ff:ff:ff\n6: p4p1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n7: p4p2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:31 brd ff:ff:ff:ff:ff:ff\n8: p4p3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:32 brd ff:ff:ff:ff:ff:ff\n9: p4p4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:33 brd ff:ff:ff:ff:ff:ff\n15: bond1:  mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n    inet 192.168.1.180\/24 brd 192.168.1.255 scope global noprefixroute bond1\n       valid_lft forever preferred_lft forever\n    inet6 fe80::b4f9:e44d:25fc:3a6\/64 scope link noprefixroute\n       valid_lft forever preferred_lft forever\n<\/pre>\n
<\/br><\/p>\n
Pinging the server is OK :<\/p>\n
\n[ansible@linux-ansible \/ ]$ ping 192.168.1.180\nPING 192.168.1.180 (192.168.1.180) 56(84) bytes of data.\n64 bytes from 192.168.1.180: icmp_seq=1 ttl=64 time=0.206 ms\n64 bytes from 192.168.1.180: icmp_seq=2 ttl=64 time=0.290 ms\n64 bytes from 192.168.1.180: icmp_seq=3 ttl=64 time=0.152 ms\n64 bytes from 192.168.1.180: icmp_seq=4 ttl=64 time=0.243 ms\n<\/pre>\n
<\/br><\/p>\n
I have plug out the cable from the em1 interface. We can see em1 interface DOWN and p4p1 interface UP :<\/p>\n
\n[root@SRV network-scripts]# ip addr sh\n1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\n    inet 127.0.0.1\/8 scope host lo\n       valid_lft forever preferred_lft forever\n    inet6 ::1\/128 scope host\n       valid_lft forever preferred_lft forever\n2: em1:  mtu 1500 qdisc mq master bond1 state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n3: em3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4e brd ff:ff:ff:ff:ff:ff\n4: em2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:51 brd ff:ff:ff:ff:ff:ff\n5: em4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4f brd ff:ff:ff:ff:ff:ff\n6: p4p1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n7: p4p2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:31 brd ff:ff:ff:ff:ff:ff\n8: p4p3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:32 brd ff:ff:ff:ff:ff:ff\n9: p4p4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:33 brd ff:ff:ff:ff:ff:ff\n15: bond1:  mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n    inet 192.168.1.180\/24 brd 192.168.1.255 scope global noprefixroute bond1\n       valid_lft forever preferred_lft forever\n    inet6 fe80::b4f9:e44d:25fc:3a6\/64 scope link noprefixroute\n       valid_lft forever preferred_lft forever\n<\/pre>\n
<\/br><\/p>\n
pinging the server is still OK :<\/p>\n
\n[ansible@linux-ansible \/ ]$ ping 192.168.1.180\nPING 192.168.1.180 (192.168.1.180) 56(84) bytes of data.\n64 bytes from 192.168.1.180: icmp_seq=1 ttl=64 time=0.234 ms\n64 bytes from 192.168.1.180: icmp_seq=2 ttl=64 time=0.256 ms\n64 bytes from 192.168.1.180: icmp_seq=3 ttl=64 time=0.257 ms\n64 bytes from 192.168.1.180: icmp_seq=4 ttl=64 time=0.245 ms\n<\/pre>\n
<\/br><\/p>\n
I have then plug in the cable in em1 interface again and plug out the cable from the p4p1 interface. We can see em1 interface now UP again and p4p1 interface DOWN :<\/p>\n
\n[root@SRV network-scripts]# ip addr sh\n1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000\n    link\/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\n    inet 127.0.0.1\/8 scope host lo\n       valid_lft forever preferred_lft forever\n    inet6 ::1\/128 scope host\n       valid_lft forever preferred_lft forever\n2: em1:  mtu 1500 qdisc mq master bond1 state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n3: em3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4e brd ff:ff:ff:ff:ff:ff\n4: em2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:51 brd ff:ff:ff:ff:ff:ff\n5: em4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:4f brd ff:ff:ff:ff:ff:ff\n6: p4p1:  mtu 1500 qdisc mq master bond1 state DOWN group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n7: p4p2:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:31 brd ff:ff:ff:ff:ff:ff\n8: p4p3:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:32 brd ff:ff:ff:ff:ff:ff\n9: p4p4:  mtu 1500 qdisc mq state DOWN group default qlen 1000\n    link\/ether 3c:fd:fe:85:0d:33 brd ff:ff:ff:ff:ff:ff\n15: bond1:  mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link\/ether bc:97:e1:5b:e4:50 brd ff:ff:ff:ff:ff:ff\n    inet 192.168.1.180\/24 brd 192.168.1.255 scope global noprefixroute bond1\n       valid_lft forever preferred_lft forever\n    inet6 fe80::b4f9:e44d:25fc:3a6\/64 scope link noprefixroute\n       valid_lft forever preferred_lft forever\n<\/pre>\n
<\/br><\/p>\n
pinging the server is still OK :<\/p>\n
\n[ansible@linux-ansible \/ ]$ ping 192.168.1.180\nPING 192.168.1.180 (192.168.1.180) 56(84) bytes of data.\n64 bytes from 192.168.1.180: icmp_seq=1 ttl=64 time=0.159 ms\n64 bytes from 192.168.1.180: icmp_seq=2 ttl=64 time=0.219 ms\n64 bytes from 192.168.1.180: icmp_seq=3 ttl=64 time=0.362 ms\n64 bytes from 192.168.1.180: icmp_seq=4 ttl=64 time=0.236 ms\n<\/pre>\n
<\/br><\/p>\n
And what about the ODA?<\/h3>\n
This configuration has been setup at one customer system running DELL servers. I have been deploying several ODAs by other customers and the questionning of having fault tolerance between several network cards is often coming. Unfortunately, and albeit the ODA are running Oracle Linux operation system, such configuration is not supported on the appliance. The Appliance will only support active-backup between ports of the same network cards. Additionnal network cards will be used on the ODA to have additionnal network connections. Last but not least, LACP is not supported on the appliance.<\/p>\n","protected":false},"excerpt":{"rendered":"
I recently needed to configure bonding between 2 network cards on a customer side and I wanted trough this blog to share my findings and how I built it showing some traces. I will also do a short comparison of what is possible or not on the ODA.<\/p>\n","protected":false},"author":48,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[59],"tags":[315,317,79,802],"type_dbi":[],"class_list":["post-14901","post","type-post","status-publish","format-standard","hentry","category-oracle","tag-bonding","tag-network","tag-oda","tag-oracle-linux-7"],"acf":[],"yoast_head":"\nBuilding a network bonding between 2 cards on Oracle Linux - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building a network bonding between 2 cards on Oracle Linux\" \/>\n<meta property=\"og:description\" content=\"I recently needed to configure bonding between 2 network cards on a customer side and I wanted trough this blog to share my findings and how I built it showing some traces. I will also do a short comparison of what is possible or not on the ODA.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-12-28T15:57:34+00:00\" \/>\n<meta name=\"author\" content=\"Marc Wagner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marc Wagner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"18 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/\"},\"author\":{\"name\":\"Marc Wagner\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/225d9884b8467ead9a872823acb14628\"},\"headline\":\"Building a network bonding between 2 cards on Oracle Linux\",\"datePublished\":\"2020-12-28T15:57:34+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/\"},\"wordCount\":972,\"commentCount\":2,\"keywords\":[\"Bonding\",\"Network\",\"ODA\",\"Oracle Linux 7\"],\"articleSection\":[\"Oracle\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/\",\"name\":\"Building a network bonding between 2 cards on Oracle Linux - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\"},\"datePublished\":\"2020-12-28T15:57:34+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/225d9884b8467ead9a872823acb14628\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/building-a-network-bonding-between-2-cards-on-oracle-linux\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building a network bonding between 2 cards on Oracle Linux\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/225d9884b8467ead9a872823acb14628\",\"name\":\"Marc Wagner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"caption\":\"Marc Wagner\"},\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/author\\\/marc-wagner\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Building a network bonding between 2 cards on Oracle Linux - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/","og_locale":"en_US","og_type":"article","og_title":"Building a network bonding between 2 cards on Oracle Linux","og_description":"I recently needed to configure bonding between 2 network cards on a customer side and I wanted trough this blog to share my findings and how I built it showing some traces. I will also do a short comparison of what is possible or not on the ODA.","og_url":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/","og_site_name":"dbi Blog","article_published_time":"2020-12-28T15:57:34+00:00","author":"Marc Wagner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Marc Wagner","Est. reading time":"18 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/"},"author":{"name":"Marc Wagner","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628"},"headline":"Building a network bonding between 2 cards on Oracle Linux","datePublished":"2020-12-28T15:57:34+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/"},"wordCount":972,"commentCount":2,"keywords":["Bonding","Network","ODA","Oracle Linux 7"],"articleSection":["Oracle"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/","url":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/","name":"Building a network bonding between 2 cards on Oracle Linux - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2020-12-28T15:57:34+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/building-a-network-bonding-between-2-cards-on-oracle-linux\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Building a network bonding between 2 cards on Oracle Linux"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628","name":"Marc Wagner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","caption":"Marc Wagner"},"url":"https:\/\/www.dbi-services.com\/blog\/author\/marc-wagner\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/14901","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/48"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=14901"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/14901\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=14901"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=14901"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=14901"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=14901"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}