Solaris Networking: IP Multipathing Protocol

Solaris Networking: IP Multipathing Protocol

In this article we are going to go over some basics of using IPMP in Solaris.  We will discuss this on the latest versions of Solaris 10 and 11 on the SPARC platform, these instructions should also work on the x86 platform, but will be dependent on the listed version numbers, as the networking stack has changed alot of the past few minor versions of Solaris 10 and 11.

Solaris 11.1

Create an IPMP Group

The first step is to create an IPMP group, in this case named ipmp1.

solaris11~# ipadm create-ipmp ipmp1

Add Interfaces to an IPMP Group

Once the group is created we can add interfaces to it, in this case net1 and net2.

solaris11~# ipadm add-ipmp -i net1 -i net2 ipmp1

Create an IPMP Group and Add Interfaces

We can also combine the above steps so that we create the group and add the interfaces in one step.

solaris11~# ipadm create-ipmp -i net1 -i net2 ipmp1

Set Interface as Standby in an IPMP Group

In the event that you prefer to have a standby interface you can change an interface to be a standby, by default all interfaces are active.

solaris11~# ipadm set-ifprop -p standby=yes net2

Remove Interfaces from an IPMP Group

Prior to deleting an IPMP group you must remove all interfaces first.

solaris11~# ipadm remove-ipmp -i net1 -i net2 ipmp1

Delete an IPMP Group

Now we can delete the IPMP group.

solaris11~# ipadm delete-ipmp

Assign an Address

After we created everything then we can assign an address.  We will need an address for the IPMP group interface, in addition if we want to use probe-based IPMP then we will need to assign them on the underlying interfaces as well, otherwise you will be using the link-based version of IPMP.

Here is an example for probe-based IPMP.

solaris11~# ipadm create-addr -T static -a 192.168.1.1/24 ipmp1/v4
solaris11~# ipadm create-addr -T static -a 192.168.1.2/24 net1/v4
solaris11~# ipadm create-addr -T static -a 192.168.1.3/24 net2/v4

Here is an example for link-based IPMP.

solaris11~# ipadm create-addr -T static -a 192.168.1.1/24 ipmp1/v4

Multipath Configuration File

If you want to be able to control some of the specifics of multipath behavior, like failback for example, you can do so using the mpathd configuration file.

solaris11~# grep -v "^#" /etc/default/mpathd
FAILURE_DETECTION_TIME=10000
FAILBACK=yes
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

That completes our configuration on Solaris 11.1.

Solaris 10 1/13

The Solaris 10 version of these instructions are very similar to the way you would configure an IP on a normal interface, but with some additional parameters.

Interface Configuration Files (Active-Active) Link-based IPMP

In Solaris 10, we control everything with the interface configration files, named hostname.interfacename where the interface name is net1 and net2 in this environment, please change your link names or configuration file names to match your server.

solaris10~# cat /etc/hostname.net1
machinehostname netmask + broadcast + group ipmp1 up
solaris10~# cat /etc/hostname.net2
group ipmp1 up

Interface Configuration Files (Active-Active) Probe-based IPMP

solaris10~# cat /etc/hostname.net1
machinehostname-net1 depracated -failover netmask + broadcast + group ipmp1 up addif machinehostname netmask + broadcast + up
solaris10~# cat /etc/hostname.net2
machinehostname-net2 depracated -failover netmask + broadcast + group ipmp1 up

Interface Configuration Files (Active-Standby) Link-based IPMP

The only time that I see a value in an Active-Standby configuration is when you have to guarantee a level of service, even in a failed state.  In my environments the increased performance is of a much higher value than the non-reduced workload.

solaris10~# cat /etc/hostname.net1
machinehostname netmask + broadcast + group ipmp1 up
solaris10~# cat /etc/hostname.net2
group ipmp1 standby up

Interface Configuration Files (Active-Standby) Probe-based IPMP

solaris10~# cat /etc/hostname.net1
machinehostname-net1 depracated -failover netmask + broadcast + group ipmp1 up addif machinehostname netmask + broadcast + up
solaris10~# cat /etc/hostname.net2
machinehostname-net2 depracated -failover netmask + broadcast + group ipmp1 standby up

Hosts File

solaris10~# grep machinehostname /etc/hosts
192.168.1.1      machinehostname.localdomain machinehostname
192.168.1.2      machinehostname-net1.localdomain machinehostname-net1 # only needed for probe-based ipmp
192.168.1.3      machinehostname-net2.localdomain machinehostname-net2 # only needed for probe-based ipmp

Netmasks Configuration File

solaris10~# grep 192.168.1.0 /etc/netmasks
192.168.1.0       255.255.255.0

Multipath Configuration File

solaris10~# grep -v "^#" /etc/default/mpathd
FAILURE_DETECTION_TIME=10000
FAILBACK=yes
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes