Unexcpected reboots Aruba JL357A (2540)

[KSTOOLS]

Hallo everybody,

i have the following issue. We have changed all our switches to new Aruba JL357A (2540) switches. So far so good. After we changed all our switches we've got the following issue that the main switch (same type and model) is rebooting unexpectly to different times. On the main switch we've connected our Sophos Firewall (which also manage our network vlans), our vmware esx server and our internetline. The main switch has a 10g fibre uplink to our second datacenter where is also a aruba switch used, and on this aruba switch is also one vmware esx server connected. The main switch and the switch in datacenter two allows the communication throug fibre with 10g. 

As the problem start, we first think that the hardware for the main switch has an problem, so we changed the switch to another one which is also the same model and type. also we change the power cable and the power connection to make sure that we have no power issue. this steps didn't help and unfortunetly we had these unexpected reboots still again. It takes around 7 - 10 hours after the switch is booting again. We also try to use another (one step older) firmware. this is also not solving our issue. 

Does anybody has an idea what kind of problem this is?

[parnassus]

Are you so kind to provide some more diagnostic information? the output of show logging -r (in reverse order = newer logs first)? sanitized running configurations would help too...you named various switches (what you called "main" shows the unexpected reboots issue)...is the issue hitting just that switch? is the IPv4 routing performed by the Firewall and all involved switches are just performing Layer 2? few more details (exact software version) would help us to help you...

[KSTOOLS]

Main switch means, that this is the first switch after Firewall. All other switches from the 1st datacenter are uplinked to this switch. Unfortunetly at the moment only by CAT not by fiber. there is a second datacenter on our campus which is connected via 10g from the main switch uplink to the switch from datacenter 2. 

The ip4 routing is performed by the sophos firewall.  We are using software version YC.16.10.0001 on this switch. 

On the events of the switch i can see that on port 8 are Excessive Multicasts. There is no more message on the events display.

And on the crash log data it says:

...Crash Log File Header..........................
Product: Aruba ID-15
Name: Aruba Module ID 15
Date: Dec 28 2019 05:36:15
Build: 261
Version: YC.16.10.0001
[24;1H[2K[24;1H[1;24r[24;1HDirectory: /ws/swbuildm/rel_ajanta_qaoff/code/buil
CPU: cortexa9
SerialNum:
crash rec size = 1396
crash rec index = 0
boot type = POWER ON
willBootType = UNKNOWN BOOT
..................................................
No Saved Crash Information


CRASHData

slot a:
-------
--- Length 26
--- Rec Ptr 46dc
No Crash data available!

poe_status_port all

[parnassus]

I saw at least two "Operator reboot from WEB session" logged messages...so what was going on?

[Brooks-O]

@KSTOOLS , @parnassus 

We've had this issue happen a acouple locations so far.  We have a moderate deployment with switches across 80+ locations. All the switches are on the same version. I just received a report that the "network" (switch) "has been rebooting once or twice a day since the new switch was installed".

When i look in the logs I am just seeing that the switch appears to have been power cycled.

Our SNMP server seems to support the claims as we are seeing the device go "down" about twice a day.

The timing is totally random. The reboot occured after 12 hours, then 7, then 13 then 10 then 9. All over the place.

Everything in our rack connects to the same PDU which has redundant power sources.  I've checked all the devices in the rack and none of the other devices appear to have power cycled. They all show uptime of over 100 days. So the issue appears to be isolated to the switch. And again, this isn't the first time we've had this issue.

I am curious if anyone is aware of any known issues.  Maybe a bad batch of pwr supplies? Maybe a bad batch of some IC chips somewhere?

We'll probably get an RMA, so I am mainly just annoyed that I have to swap out hardware and disturb my customers.

I am curious because I want to know if there is something we can look for to avoid this in the future. Maybe a range of serial numbers or date codes that we should look out for. Any insight would be helpful