Difference between revisions of "Logical Domains with IPMP"

Revision as of 06:10, 24 April 2012

IP Multipathing (IPMP) is a way to bring redundancy into your Solaris networking. In essence, you can connect one physical, or, in this case, virtual server to multiple switches. In the event that one network interface should go down, then the server will still be accessible over the network. The idea is that you take two or more interfaces and make them part of an interface group. You assign an IP to one or both of those interfaces. In the event of a link failure on one interface, the other interface in the group continues to advertise the IP of the other. For a full explanation of IPMP, see the Oracle Solaris System Administration Guide : IP Services

There are two types of IPMP, Link based and Probe based, and both can be set up as either active/active or active/standby. You can use either for physical servers, but for LDOM's, you can only use Probe based IPMP. In Probe based IPMP, one or more of the interfaces are given a real IP which the server can use for external communication. Each interface in the IPMP group also has a virtual interface assigned to it which is given a test IP. The test IP is used to probe the network, by sending out ICMP ping packets on the virtual test interface, to either manually specified or automatically generated targets. In the event that one of the test IP's fails in it's probing, then that interface is marked as failed, and then one of the other interfaces in the IPMP group picks up the 'real' IP.

In the sample 'ifconfig' output below, you can see there are two interface, vnet0 and vnet1:

 vnet0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
         inet 10.10.10.40 netmask fffffc00 broadcast 10.10.10.255
         groupname testing
         ether 0:14:4f:fa:a2:6b
 vnet0:1: flags=209040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,CoS> mtu 1500 index 2
         inet 10.10.10.42 netmask fffffc00 broadcast 10.10.10.255
 vnet1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 3
         inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
         groupname testing
         ether 0:14:4f:fa:87:50
 vnet01:1: flags=209040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,CoS> mtu 1500 index 3
         inet 10.10.10.43 netmask fffffc00 broadcast 10.10.10.255

vnet0 has the external IP of 10.10.10.40 and the test IP of 10.10.10.42 on it's virtual interface, vnet0:1. vnet1 has the dummy IP 0.0.0.0, as we only want to use one external IP for data. However, vnet1 has a test IP assigned on vnet1:1 of 10.10.10.43. This is an active/active setup, which in this case means, that both interfaces can be used for outbound traffic, so in theory you can double your outbound throughput. In the event that vnet0 should fail, then vnet1 will create a new virtual interface and assign it the IP 10.10.10.40, thereby keeping the box on the network. In the event that vnet1 should fail, then only outbound traffic will be affected as in this case, vnet1 has no 'real' IP.

So how do you do this? Here's how:

1. Edit the /etc/hosts file and add entries for the real and test IP's

 10.10.10.40   ldmtest
 10.10.10.42   ldmtest-0
 10.10.10.43   ldmtest-1

2. Edit the file /etc/hostname.vnet0 and add the following:

 ldmtest netmask + broadcast + group testing up addif ldmtest-0 netmask + broadcast + -failover deprecated up

This creates the real interface vnet0 with the IP specified by the hostname ldmtest as part of the group testing. It also creates the test interface, vnet0:1, with the IP specified by the hostname ldmtest-0. The -failover option tells the box not to failover the test IP. The deprecated option tells the box not to use the test interface for any application data. The netmask and broadcast options tell the box to use the default netmask and broadcast values for these interfaces.

3. Edit/create the file /etc/hostname.vnet1 and add the following:

 group testing up addif ldmtest-1 netmask + broadcast + -failover deprecated up

This makes the vnet1 interface part of the IPMP group testing and creates a test interface vnet1:1 with the IP specified by the hostname 'ldmtest-1'. Again, the '-failover' option tells the box not to failover the test IP and the 'deprecated' option tells the box not to use the test interface for any application data. In this case, the vnet1 interface has no IP and does not get any incoming traffic, but can send outgoing traffic. If you want this interface to have an IP and accept inbound traffic, then add this to /etc/hostname.vnet1 instead:

 ldmtest-sec netmask + broadcast + group testing up addif ldmtest-1  netmask + broadcast + -failover deprecated up

and add an entry for 'ldmtest-sec' in the /etc/hosts file.

4. Reboot the box (you could also use ifconfig to assign all this without rebooting. Figure that out yourself :-)

That's it.