ProCurve / ProVision-Based
cancel
Showing results for 
Search instead for 
Did you mean: 

VSF config with MAD

chintanpintu
Occasional Contributor

VSF config with MAD

Hi,

I think have read enough on 5400 series chassis VSF and MAD setup. But in my view there really is not a lot of good documentation out there. So I going to post one more topic on VSF and MAD - so I can get some help from experts and maybe help others.

I have currently have two - 5412 chassis that I want to setup VSF. I have setup VSF on them and it was pretty straight forward. I am using a 10 Gig DAC cable for VSP between chassis.

Here are my questions:

1. VSF Domain ID - if I am going to have let's say 10 closets with VSF - do I need 10 different Domain ID setup?

     eg. vsf enable domain 1, domain 2, domain 3 .......

2. MAD "assist device" - why is this needed? I know for split-brain purpose but what else?

3. Can I configure VSP without MAD? If so - how do I do this?

4. My current setup is two - 5412 chassis configured with VSF via 10 Gig DAC cable. If I add another - 10 Gig DAC to between the two chassis for redudancy - does this make up for "MAD".

5. I have seen OOBM ports used for "MAD" - can someone share documentation on this - how to setup?

6. Can I do 2930F stacking? If yes - is this also by VSF and having a MAD device setup?

 

Just have to say - Cisco stacking was so much easier.

 

Thanks for you help

 

P

 

6 REPLIES
parnassus
Honored Contributor

Re: VSF config with MAD

From VSF Configuration Guide, slightly rephrased:

  1. VSF Domain ID: uniquely identifies VSF Stack and prevents interfering with other VSF Stacks; must match on VSF Members of the same VSF Stack; It's configurable. So the answer to your question is Yes, you do.
  2. What do you mean with "...what else?", that's not Nespresso (joke)...if you deploy LLDP MAD you need a third supported device (the assist device) to act as a (mostly silent) arbiter, it helps because it lets the VSF Members to recognize the Split Brain scenario and to react to that accordingly avoiding (better: mitigating) its immediate consequences.
  3. Yes, as you did (didn't you?). MAD mechanism (no matter how it is implemented for supported methods: OoOB MAD, LLDP MAD, BFD MAD, etc.) isn't mandatory di-per-sè...but definitely it's a recommended addition to have a VSF Stack more resilient.
  4. Rephrase your question. If you add another 10G link (DAC) between two VSF Members...probably you create a loop, you should instead add that new link to the VSF Port on each VSF Member to add resiliency (so the VSF link is going to be formed by 10G+10G links aggregated together).
  5. There is a nice OoBM MAD [*] example here applied to Aruba 5400R zl2 in VSF Stack (or read this whole Thread to reach that link). Latest documentation has examples on doing that, I'm pretty sure about that.
  6. Are you asking if a VSF Member (let me say an Aruba 2930F) can be concurrently a VSF Stack Member and also a MAD assist device for the whole VSF Stack that switch is also member of? pretty strange desire. Isn't it?

Really can't comment on Cisco: too low (VSS/StackWise) experience with those ones.

[*] I'm not sure (I should check) that Aruba 2930F supports OoBM MAD since I believe it doesn't support OoBM as the Aruba 5400R zl2 does.

Vince-Whirlwind
Honored Contributor

Re: VSF config with MAD

I have installed plenty of VSS/IRF/VRF type things and I've never understood the point of the MAD link.

Instead of using interfaces for a MAD link, just add it to the redundant links between the chassis. The only way a split brain is gong to happen is if every single link between the chassis is down. The more links, the less likely this is to happen (and module diversity is a must of course). And really, anything that is going to take out all your links is going to take out the MAD link as well, or the whole chassis.

So why waste a link on MAD when it can be an additional redundant link instead?

 

 

chintanpintu
Occasional Contributor

Re: VSF config with MAD

Hi,

Question for you Vince or anyone.

So I have setup 5412's with VSF. I have a (2) 10 gig DAC ink' s for redudancy on different modules. This is what you mean by having more links between the chassis - correct? If one DAC cables fails - the other one takes over.

I have configured OOBM ports for MAD purposes. Does any know if these can be on private networks. What is I mean is - do they need to on the same network as the switches?

Thanks,

P

 

 

itcoop
Occasional Advisor

Re: VSF config with MAD

I have also configured oobm for MAD - identical setup.  Iv'e set them up on private, unrouted network in my lab.  When I unplug the VSF cables to simulate a failure, both switches go active.  

Here is my conifig:

vsf
   enable domain 2
   member 1
      type "J9850A" mac-address 1c98ec-000000
      priority 255
      link 1 1/F21,1/F24
      link 1 name "I-Link1_1"
      exit
   member 2
      type "J9850A" mac-address 00fd45-000000
      priority 128
      link 1 2/A7-2/A8
      link 1 name "I-Link2_1"
      exit
   oobm-mad
   port-speed 10g
   exit
no rest-interface
ip routing
snmp-server community "public" unrestricted
oobm
   ip address 192.168.66.5 255.255.255.0
   vsf member 1
      ip address 192.168.66.6 255.255.255.0
      exit
   vsf member 2
      ip address 192.168.66.7 255.255.255.0
      exit
   exit
HP-VSF-Switch(vsf)# show vsf

 VSF Domain ID    : 2
 MAC Address      : 1c98ec-bbba3f
 VSF Topology     : Chain
 VSF Status       : Active
 Uptime           : 0d 1h 5m
 VSF MAD          : OOBM
 VSF Port Speed   : 10G
 Software Version : KB.16.03.0003

 Mbr
 ID  MAC Address   Model                                  Pri Status
 --- ------------- -------------------------------------- --- ---------------
  1  1c98ec-000000 HP J9850A Switch 5406Rzl2              255 Standby
 *2  00fd45-000000 HP J9850A Switch 5406Rzl2              128 Commander
HP-VSF-Switch(vsf)# sho vsf top

VSF member's interconnection with links:

Stby Cmdr
+---+ +---+
| 1 |1==1| 2 |
+---+ +---+
HP-VSF-Switch(vsf)# sho vsf link detail

VSF Member: 1 Link: 1

Port State
-------- ------------
1/F21 Up: Connected to port 2/A7
1/F24 Down


VSF Member: 2 Link: 1

Port State
-------- ------------
2/A7 Up: Connected to port 1/F21
2/A8 Down

HP-VSF-Switch(vsf)#
HP-VSF-Switch(vsf)# ping source oobm 192.168.66.6
192.168.66.6 is alive, time = 1 ms
HP-VSF-Switch(vsf)# ping source oobm 192.168.66.7
Request timed out.
HP-VSF-Switch(vsf)# ping source oobm 192.168.66.5
192.168.66.5 is alive, time = 1 ms
HP-VSF-Switch(vsf)# sho oobm ip detail

 Internet (IP) Service for OOBM Interface

Global Configuration

  IPv4 Status    : Enabled
  IPv6 Status    : Disabled

  IPv4 Default Gateway :
  IPv6 Default Gateway :

 Origin     | IP Address/Prefix Length                    Status
 ---------- + ------------------------------------------- -----------
 manual     | 192.168.66.5/24                             preferred


VSF Member 1

  IPv4 Status    : Enabled
  IPv6 Status    : Disabled

  IPv4 Default Gateway :
  IPv6 Default Gateway :

 Origin     | IP Address/Prefix Length                    Status
 ---------- + ------------------------------------------- -----------
 manual     | 192.168.66.6/24                             preferred


VSF Member 2

  IPv4 Status    : Enabled
  IPv6 Status    : Disabled

  IPv4 Default Gateway :
  IPv6 Default Gateway :

 Origin     | IP Address/Prefix Length                    Status
 ---------- + ------------------------------------------- -----------
 manual     | 192.168.66.5/24                             preferred


HP-VSF-Switch(vsf)#

Questions:

1) For OOBM-MAD, what show command tells me that MAD is functioning?

2) Does routing need to be enabled on my 5406 for MAD to be effective?

3) Should the switch be able to ping oobm of member 2/1?

chintanpintu
Occasional Contributor

Re: VSF config with MAD

itcoop - Good questions. I am also wating to know.

A question for you. Are you just testing MAD setup? Also - your thoughts on have a redudant link's between switches and taking out MAD/OOBM totally from the setup - because I really not seeing a great advantage to having it.

Thanks,

P

itcoop
Occasional Advisor

Re: VSF config with MAD

Read my lips: do not deploy vsf/irf without configuring MAD.

 I rolled out several dozens of 5500HI's across 5,000 square miles thinking MAD was overkill.  And it was - right up to the first time I needed it: during an ISSU firmware update.  OSPF failed miserably due to duplicate routerID's - split brains were everywhere.  It required a site visit to every location to manually reboot each switch in the stack.  Never again...

Right now I am testing VSF 5406R's in a stack.  We have ten 5406R's - two in each closet - that I want to collapse using VSF.  I don't want the nightmare to repeat; thus I am labbing this configuration out and testing it.

I'm going to answer one of my questions above:

I cannot find a show command that describes whether MAD is functioning while oobm-mad is configured.  The theory of MAD is to disable the switch ports when a VSF link failure results in an orphaned switch (this is good).  My test was as follows:

1) Configure oobm-mad

2) Physically fault (unplug) all configured VSF links

3) Verify that MAD disables all of the orphaned switch ports

Here's what happened in the log file of the ACTIVE switch when I unplug the VSF links (logs obtained via console port):

HP-VSF-Switch# sho log
 Keys:   W=Warning   I=Information
         M=Major     D=Debug E=Error
----  Event Log listing: Events Since Boot  ----
I 05/10/17 00:41:48 00184 mgr: ST1-CMDR: Log cleared as a result of  'clear
            logging'  command
I 05/10/17 00:42:06 04992 vsf: ST1-CMDR: VSF port 1/F24 is in error state
I 05/10/17 00:42:06 04992 vsf: ST1-CMDR: VSF link 1 is down
W 05/10/17 00:42:06 03258 stacking: ST1-CMDR: Standby switch with Member ID 2
            removed due to loss of communication
I 05/10/17 00:42:06 03272 stacking: ST1-CMDR: Stack fragment active
I 05/10/17 00:42:06 03271 stacking: ST1-CMDR: Topology is a Standalone
I 05/10/17 00:42:06 00077 ports: ST1-CMDR: port 2/B5 is now off-line
I 05/10/17 00:42:06 00077 ports: ST1-CMDR: port 2/B9 is now off-line
I 05/10/17 00:42:06 04992 vsf: ST1-CMDR: VSF port 1/F24 is down
I 05/10/17 00:42:06 00406 ports: ST1-CMDR: port 1/F24 xcvr hot-swap remove.
----  Bottom of Log : Events Listed = 10  ----

The log file of the STANDBY switch:

HP-VSF-Switch# sho log
 Keys:   W=Warning   I=Information
         M=Major     D=Debug E=Error
----  Event Log listing: Events Since Boot  ----
I 05/10/17 00:41:48 00184 mgr: Log cleared as a result of  'clear logging'
            command
I 05/10/17 00:41:48 00184 mgr: Log cleared as a result of  'clear logging'
            command
I 05/10/17 00:42:06 04992 vsf: VSF port 2/A8 is in error state
I 05/10/17 00:42:06 04992 vsf: VSF link 1 is down
W 05/10/17 00:42:06 03258 stacking: Commander switch with Member ID 1 removed
            due to loss of communication
I 05/10/17 00:42:06 03278 stacking: Member 2 (00fd45-000000) elected as
            commander. Reason: Standby takeover
W 05/10/17 00:42:06 03270 stacking: Topology is a Chain
I 05/10/17 00:42:06 03272 stacking: Stack fragment active
I 05/10/17 00:42:06 03271 stacking: Topology is a Standalone
I 05/10/17 00:42:06 04992 vsf: VSF port 2/A8 is down
I 05/10/17 00:42:06 03267 stacking: Failover occurred
I 05/10/17 00:42:06 00061 system: -----------------------------------------
I 05/10/17 00:42:06 02712 console: USB console cable disconnected
I 05/10/17 00:42:07 03272 stacking: Stack fragment inactive
I 05/10/17 00:42:07 02682 OOBM: OOBM - Enabled globally.
I 05/10/17 00:42:07 00110 telnet: telnetd service enabled
I 05/10/17 00:42:07 00110 telnet: telnetd service enabled
I 05/10/17 00:42:08 03125 mgr: Startup configuration changed by SNMP.  New seq.
            number 20
I 05/10/17 00:42:08 00803 usb: port enabled.
I 05/10/17 00:42:08 03401 crypto: Function POWER UP passed selftest.
I 05/10/17 00:42:08 03261 stacking: Member active
I 05/10/17 00:42:08 03260 stacking: Member booted
I 05/10/17 00:42:08 00260 system: Mgmt Module 1 Active
I 05/10/17 00:42:08 00077 ports: port 1/A1 is now off-line
I 05/10/17 00:42:08 00077 ports: port 1/B2 is now off-line
I 05/10/17 00:42:08 00077 ports: port 1/F6 is now off-line
I 05/10/17 00:42:08 00077 ports: port 2/B5 is now off-line
I 05/10/17 00:42:08 00077 ports: port 2/B9 is now off-line
I 05/10/17 00:43:16 00179 mgr: SME CONSOLE Session - MANAGER Mode

What is ambiguous is that the logs on the STANDBY switch state 

Member 2 (00fd45-000000) elected as commander. Reason: Standby takeover

Both the ACTIVE and the STANDBY logs are telling me that I have two ACTIVE switches.  This is not good.  

Here is what was wrong with my test:  I didn't have any edge devices linked up.  After bringing up some L2 devices and staging the VSF failure, the ports on the STANDBY switch shutdown.  When I clear the VSF fault, the STANDBY reboots and all is good.

So, it is working; however, I still do not have access to a show command that tells me the status of oobm-mad.  There are show status commands for lacp-mad and lldp-mad; however, there is nothing you can see for oobm-mad.