Using multiple NICs 1094

In a message on Tue, 2 May 2006 16:04:10 -0500, wrote :

On re-reading it *sounds* like you might be thinking of having two networks, one with the NFS server and one without the NFS server and a machine with two NICs, one on each network. (I thought at first you wanted to separate NFS reads from NFS writes to-from the same server.) This is trivial to do:

Lets say your NFS server is 192.168.1.100 and is thus on network 192.168.1.0. Your data is meant to land on some machine with an IP of 192.168.2.199 and your compute box has two NICs, eth0 and eth1. eth0 has an IP of 192.168.1.99 and eth1 an IP of 192.168.2.27. You also have two ethernet switches, A and B. You plug the ethernet cable from the NFS server into switch A along with a cable from the compute box's eth0 card. You connect the machine with an IP of 192.168.2.199 to switch B along with a cable from the compute box's eth1 card. You export your NFS share to 192.168.1.99 and mount 192.168.1.100:-whatever on your compute box. The NFS traffic will go through eth0 and on switch A. If your output is headed to 192.168.2.199, it will go through eth1 and through switch B. (It is possible to use a single switch, but I am not sure if you will get burned by network contention -- it would depend on how smart the switch is.)

If you only have one network (eg one network switch) and a machine with two NICs, then what you want to do is bonding. You can have a machine with two IP numbers (on the same network), one per NIC, but things are strange: what would the default route be? With bonding, the kernel merges the traffic on the two NICs into one logical data path, so you have one logical network interface with single IP address and a single route. It just has twice the traffic capacity.

Imagine the difference between a four lane highway, with two lanes in each direction vs two parallel two lane roads (buttumes full duplex NICs). The two *separate* roads get you the same place, but you can't shift lanes from one road to the other, should one become congested. Bonding has the advantage of sharing the load on a demand basis. You can't be sure that the NFS read traffic will *exactly* be only 1-2 of the total traffic. It might work out to only be 1-3 of the traffic (or 2-3). With bonding, the difference is made up so that you get to use all of the available bandwidth (buttuming your machines can really push that much data). If you do the two network 'trick' above, one NIC could end up being underutilized and the other will be maxed out and total thoughput will be less than the max possible. With bonding you get to use ALL of the available bandwidth, even if the traffic is not even.

Robert Heller -- 978-544-68 plus 133 Deepwoods Software -- Linux Installation and Administration