Storage Boards Cleanup
To make it easier to find information about HPE Storage products and solutions, we are doing spring cleaning. This includes consolidation of some older boards, and a simpler structure that more accurately reflects how people use HPE Storage.
HPE StoreVirtual Storage / LeftHand
cancel
Showing results for 
Search instead for 
Did you mean: 

FOM fails to remain online

Peter J West
Frequent Advisor

FOM fails to remain online

Hi,

 

I've just finished configuring a multi-site P4000 installation which consists of a total of 6 nodes.

 

The best practice recommendation is that I create two managers on each site and then have a fifth running as a FOM in a third site.

 

I've tried installing the FOM under ESX using local storage but it's not at all stable.

 

The VM boots up and initially the manager starts to run, but then within a few moments it goes offline and you're unable to get it working again (it appears you can't right click a FOM and choose Start Manager like you can on a normal storage node).

 

This problem also means I can't apply the upgrades which are waiting.  On the FOM console there are some errors being shown relating to dbd_manager?  Is this expected or part of the issue?

 

I've tried searching for a later copy of the FOM but so far as I can see only v9.0 is available.

 

Grateful for any help that yoiu can offer.

 

Regards

 

Pete

 

11 REPLIES
Jitun
HPE Pro

Re: FOM fails to remain online

dbd_manager is the manager service on the unit.
Could you please share the screenshot.

Is the FOM seen in CMC ?
Have you been able to add it to the management group ?
If yes, does it say Manager Starting/Stopping ?
--------------------------------------------------------------
How to assign points? Click the KUDOS! star!
Bart_Heungens
Honored Contributor

Re: FOM fails to remain online

First of all: can you ping it? Is the network OK? VLAN-ing?

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Peter J West
Frequent Advisor

Re: FOM fails to remain online

Hi,

 

The network side of things appears to be ok - I can ping the FOM and there's no dropping of traffic.

 

This morning when I came in it wouldn't even let me view the configuration of the manager and it simply kept saying it couldn't find a system with that ID (and then displays the MAC address).  I did a restart on the FOM and searched for the system and it found it. 

 

I've attached an image file to this post that show's the status - initially it comes up fine and it IS part of the management group.  But within 60 seconds it goes offline and then there doesn't appear to be any way to bring it back online.

 

The second image attached shows the messages on the console.

 

Any clues would be really welcome. :)

 

Pete

 

Bart_Heungens
Honored Contributor

Re: FOM fails to remain online

Something strange seems to happen. Never saw this issue before...

 

Can you remove the FOM from the MG, delete it on ESX level and start all over with a new clean FOM? Re-aasign an IP etc...

 

If this doesn't help, I would suggest to log a call at HP... Unless other peaople have other ideas...

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Jitun
HPE Pro

Re: FOM fails to remain online

Obsolete socket error message can be ignored.

I would suggest the same as Bart.

Remove the FOM from management group.
Delete the FOM from ESX and use a new FOM.
--------------------------------------------------------------
How to assign points? Click the KUDOS! star!
Peter J West
Frequent Advisor

Re: FOM fails to remain online

Hi again,

 

Thanks for the suggestions - i'm trying out a couple of things at the moment.

 

One question though.

 

If all of my storage nodes are using jumbo frames and flow control then am I correct in assuming that the FOM should also have the same settings?

 

Thanks

 

Pete

 

Bart_Heungens
Honored Contributor

Re: FOM fails to remain online

Hi,

 

If you use Jumbo Frames (which I don't recommend), yes all devices in your SAN network (which is actually a LAN) must be configured for Jumbo Frames...

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Peter J West
Frequent Advisor

Re: FOM fails to remain online

Hi,

 

Thanks for your comments - but can I ask why you don't recommend jumbo frames?

 

In our environment we are running the SAN nodes on a totally seperate infrastructure to the data network - of course there is a link - there has to be so we can manage to nodes from our normal data network.  But the VLAN configuration means we don't have SAN traffic crossing over to the normal data network and in this way we can keep the two systems seperate.

 

I think I may have identified the cause of my problem - i'm testing at the moment and if so then i'll report back with what eventually fixed the issue.

 

Kind regards

 

Pete

 

Jitun
HPE Pro

Re: FOM fails to remain online

If you use Jumbo Frames... every other device in you SAN should support Jumbo Frames.

 

If you have Jumbo frames enabled on the nodes .. try disabling Jumbo Frames on the switch port...

Manager of that node would go offline... It will not be able to communicate with other Managers.

 

I recommend using Default Packet as well. The difference is performance is not worth the trouble, unless you have to have Jumbo Frames enabled.

--------------------------------------------------------------
How to assign points? Click the KUDOS! star!
Bart_Heungens
Honored Contributor

Re: FOM fails to remain online

You 'can'  have better performance, but like Jitun says you can/will have more issues setting it up and troubleshooting afterwards than you will have advantages...

 

I have also done som serious testing with it and I came to a magical improvement of 1.5%!

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Peter J West
Frequent Advisor

Re: FOM fails to remain online

Hi everyone,

 

Again thanks for taking the time to comment - it's really appreciated.

 

I have to agree - setting up Jumbo Frames and Flow Control has been a bit of a PITA, but in this case it wasn't the cause of my problem.

 

The issue actually turned out to be down to me not thinking about the VM Environment i'm using to host the FOM and how the network there was configured.

 

Our Virtual Environment has multiple vSwitches defined so that we can group vmNIC's together for the different kinds of traffic.  Unfortunately what i'd neglected to do was add a Port Group to the SAN vSwitch.  This meant that I was trying to use an IP from outside of the SAN IP Range and worse still it was routing it via switches which werent' configured for Flow Control or Jumbo Frames.

 

If i'd not have had these options enabled then of course the configuration would have worked but the routing being used wouldn't have been the most efficient.

 

I started by modifying the VMWare configuration and then reinstalling the FOM using an IP for the SAN range.  This worked but I have to admit that the headaches caused by Jumbo Frames and Flow Control ultimately have led to me turning it off.  The final straw came when I tried to enable the options on the FOM only to find that it simply wouldn't enable the options.

 

We're not particularly stressing our environment at the moment so I figure we don't really need the small gains it's going to give us - if we get further down the line and we hit a problem then of course we can return to the options then.

 

Thanks again for all of your help.  The system is working quite happily now with 2 managers in each of the sites and a FOM sat in a third location (using direct-attached storage on one of the ESXi Servers).

 

Pete