Web and Unmanaged
cancel
Showing results for 
Search instead for 
Did you mean: 

HP 1920 - extreme slow VLAN routing

SOLVED
Go to solution
yurezg2
Occasional Advisor

HP 1920 - extreme slow VLAN routing

i have new HP 1920-24G Switch JG924A, Software Version 5.20.99 Release 1106

there are 2 vlans on it - vlan200 (servers) and vlan201 (workstations)

i want to route between these vlans and assign vlan ip to each of vlan, i can see routing table

all servers have default gateway as vlan200 ip

all workstations have default gateway as vlan201 ip

if i plug some workstation in vlan201 access port, then i have transfer speed between this ws and some server 100kbps-1mbps

if i change this port to vlan200 access port (server vlan), i have transfer speed on the same workstation between ws and server - 100Mbps.

 

also i have several workstations in vlan201 without this transfer speed problem.

 

my network topology is a little bigger then i describe here, but i think its not important at this time

 

have you any idea, why in some cases i have extremely slow network speed?

 

thanks

18 REPLIES
yurezg2
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

p.s.: enabling/disabling/deleteing antivirus software on both sides takes no effect

 

johnk3r
Respected Contributor

Re: HP 1920 - extreme slow VLAN routing

Good problem. Have you done the analysis using wireshark / tcpdump?

**************************************
ATP FLEXNETWORK V3 | ACSA
yurezg2
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

started on both sides, on workstation and server, wireshark shows the same pare Source-Dest (dest-source on the other side) with the same traffic ant layer 2 and at layer 3

 

vlan 1 -> vlan200 - speed id 100kbps-1mbps

 

just only changing vlan on access port of workstation (to vlan200, the same as destination server vlan) takes me to full speed

 

after 2 weeks testing i think its routing software error on l3 switch 1920-24g

 

also, another WS in vlan1, plugged at the same switch as previous WS, has no transfer speed problem while communitating wuth vlan200

 

yurezg2
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

i found a lot of records with "TCP segment of a reassembled PDU" in wireshark running n "slow" ws

trying to find any info about it

Apachez-
Trusted Contributor

Re: HP 1920 - extreme slow VLAN routing

Could you paste current config along with port counters?

 

Could be faulty cable or interface (either at switch side or client side).

yurezg2
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

files you are asked for are attached

its config of switch that route my subnets

at fact i just began to employ vlans on my old running network, create some vlans and attach some servers / workstations to them

but routing is slow and its no matter, if server and ws attached to the same phisical switch (but different vlans) or to different switches

Achim_B
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

We have exactly the same Problems at our Office. We have 4 different 1920-24g (JG924A) and on all of them the same problems with Layer 3 Routing.
Is here at least one person who has “normal” speed on routed connections thru this Switch?
(on more than 2 VLAN and more than 2 computers)

eggcurry
Visitor

Re: HP 1920 - extreme slow VLAN routing

Having this problem too. We have an HP 1920 24G with a small number of VLANS configured. Initially speeds are as you would expect but they drop off significantly after about 24 hours. Rebooting the switch seems to rectify the issue but speeds drop away again to a paltry 1-2 MB/sec.

 

Our ADSL router can do better than this and is significantly cheaper.

 

Anyone considering buying one of these ( HP 1920 24G ) please save yourself the hassle. They do not work properly and support is non-existant, just look at this forum.

Achim_B
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

We are folloing some ARP issues at the moment...after deleting the dynamic ARP entrys speed goes up again...

eggcurry
Visitor

Re: HP 1920 - extreme slow VLAN routing

So at the moment this switch requires daily intervention just to do what it is meant to. Well done HP.

 

I'm guessing there is no way to schedule flushing the dynamic ARP table?

 

Achim_B
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

It is possible to reduce the aging time, but I don`t know if it helps.

we are currently in contact with the HP Support to geht that solved...


ARP is just suspected to be THE Problem...

mp_AK
Occasional Visitor

Re: HP 1920 - extreme slow VLAN routing

Hey guys,

just wanted to share my experience on this case:

we got two 1920-16G Switches with several VLANS and are also experiencing similar problems.
Copying a big file from one VLAN to another with < 1MB/s.
Inside the same VLAN we can use the full bandwith.

After restarts or manually deleting the dynamic ARP entries, the speed goes up again.

So I can defenitely approve Achims assumption regarding some sort of ARP related issue.

eggcurry
Visitor

Re: HP 1920 - extreme slow VLAN routing

Ok I have managed to get a little further.

 

The ARP table on the HP 1920 24G (JG924A) can only reliably handle 60 entries. Go over this and you will start to experience the loss in performance reported here.

 

In English, If you have more than 60 devices across all of your VLANS you will have this problem. This is a pretty significant limitation for what is meant to be a layer 3 switch. I also note than this is not mentioned in any of the devices specifications.

 

It is possible to set the maximum age for ARP entries quite low and this may help mitigate the problem if you are only a few devices over the 60 limit.

 

To set the maximum ARP age:

 

1. log on to switch via console or SSH

***next step will invalidate your warranty unless you are doing so under the instruction of an HP engineer***

2. enable the command line mode ( _cmdline-mode on ) with password ( Jinhua1920unauthorized )

3. enter system mode ( system )

4. set the ARP timer ( arp timer aging XXXX ) where XXXX is the value in minutes up to a maximum of 1440

 

For me 10 minutes seems to work ok. It doesn't get rid of the problem but it does move it about throughout the day and should give preference to those with heavier network usage demands.

 

If you haven't bought one of these don't.

 

If I were to write a review of the HP 1920 24G (JG924A) it would be scathing.

 

We are currently talking to our supplier about returning our 1920 24G as it is not fit for purpose. It was sold as a Layer 3 switch. The specifications issued by HP make no reference to only supporting a maximum of 60 ARP entries (the ARP table can actually hold many more).  If you contact HP support you eventually get this information but they will make you jump through hoops before they admit it.

 

This is a real shame as I have had excellent experiences with HP's networking devices up until now. Their attitude over this switch is beyond contempt though.

 

 

 

 

16again
Respected Contributor
Solution

Re: HP 1920 - extreme slow VLAN routing

See:

http://h20565.www2.hpe.com/hpsc/doc/public/display?sp4ts.oid=7399514&docId=mmr_kc-0130409&docLocale=en_US

 

My comments:
-On 1920-24G I managed to reproduce this slow routing speed condition
-On 1920-24G there is no syslog event generated when ARP table grows beyond 64 entries.
-On 1920-24G the ARP table can't grow beyond 256 entries, so no CPU based routing for host #257! This condition does generate a  syslog message:"Oct 31 08:25:14 2015 HP1920G %%10TPMB/4/ARP TABLE FULL(t):   1.3.6.1.4.1.25506.2.38.1.2.4.1: ARP table is full and number of items is 256."
-HP doesn't specify IPv6 max neighbor table size,  so we can assume same limits apply. 
Note: On IPv6 number of hosts does not equal to neighbor table size! A host uses multiple addresses (with its link-local and privacy extension addresses)
On 1920-24G I did managed to get this IPv6 event at 256 entries:
"Oct 31 09:11:02 2015 HP1920G %%10TPMB/4/ND TABLE FULL(t):   1.3.6.1.4.1.25506.2.38.1.5.4.1: Neighbor table is full and number of items is 256."
-Switch reboot is seems useless to me , to make neigbors time out earlier use commands like:
 arp timer aging 4    (minutes, default=20)
ipv6 neighbor stale-aging 1    (hours! default=4)
After ARP table size is back within safe limits (60) , hardware speed returns

yurezg2
Occasional Advisor

Re: HP 1920 - extreme slow VLAN routing

what a shame +1 !

Guys, thanks for reply.

 

jbwoodoo
Occasional Visitor

Re: HP 1920 - extreme slow VLAN routing

Hi Folks

I had the same problem in just a small environment, just 40 VM’s and 25 workstations. I did not came across your blog about this issue, so I have struggled over a week on this.

Reading always helps a lot!

I just changed the switch to old Cisco 3750 and everything works fine. On all workstation I got file transfer throughput of more than 30 Mbyte/s between routed Vlans.

So what is this HP 1920 actually for? It says it is layer 3, but partially it is not!  

We are going to give back the switch to the supplier.  

Thanks & Regards,

Boris  

killergooj
Occasional Visitor

Re: HP 1920 - extreme slow VLAN routing

Hi, All

How can I see current ARP routing usage.
I've same problem with my customer, but my team think problem is not ARP routing resource. How can I discuss this with my team.

Sorry for bad English,

Thank you in advance.

parnassus
Honored Contributor

Re: HP 1920 - extreme slow VLAN routing

By generating Diagnostic Information file?

Maybe on that file there will be ARP usage info...not totally sure if reported ARP info will help your team but look at this example (generated by HPE OfficeConnect 1920-48G):

=============================================================
  ===============display arp specification===============
=============================================================
ARP Specification:
Permitted Static ARP Number : 64
Permitted Total ARP Number : 256
Permitted Max Number perIf Configed: 256
Default Permitted Max Number perIf: 256
============================================================
  ===============display arp error slot 1===============
============================================================
Dump Errors in ARP processing:
 Error(s) in initialization: 1
 Error(s) in malloc memory:  0
 Error(s) in func parameter: 207
 Error(s) in Send QueueMsg:  0
 Error(s) in write taskevent: 0

 Error(s) in malloc IPC msg: 0
 Error(s) in malloc RPC msg: 0
 Error(s) in Create IPCGroup: 0
 Error(s) in Join IPCGroup: 0
 Error(s) in Reg WaitList: 0
 Error(s) in Get WaitList: 0
 Error(s) in Set WaitList: 0
 Error(s) in Create Timer: 0

 Error(s) in Send ARP Request: 0
 Error(s) in match static Entry: 0
 Error(s) in Add Arp Entry: 0
 Error(s) in Get Arp Entry: 0
 Error(s) in Delete Arp Entry: 0
 Error(s) in Hitting Arp Entry: 0
 Error(s) in Add Entry For RenewTime: 0

 Error(s) in Create IFCB: 0
 Error(s) in Create Interface Link: 0
 Error(s) in Create Interface LinkNode: 0
 Error(s) in Create Unprobe node: 0
 Error(s) in Create ref Node: 0
 Error(s) in Add Reference : 0

 Error(s) in Slot Insert : 0
 Error(s) in Slot Remove : 0
 Error(s) in IfIndex Insert : 0

 Error(s) in Add Drv Dummy: 0
 Error(s) in Del Drv Dummy: 0
 Error(s) in Add Drv : 0
 Error(s) in Del Drv : 0
 Error(s) in Mod Drv : 0
 Error(s) in Add ArpHost Drv : 0
 Error(s) in Del ArpHost Drv : 0
 Error(s) in Mod ArpHost Drv : 0

 Error(s) in Get Vrf index: 0
 Error(s) in Call IF_GetDataByIfIndex: 0
 Error(s) in Get Device Data: 0
 Error(s) in Get Vlan Data: 0
 Error(s) in Get Lacp Data: 0
 Error(s) in Get Interface Mac: 0

 Error(s) in fast synchronization: 0
 Error(s) in **bleep** Sing: 0
 Error(s) in smoothing process: 0
 Error(s) in Receive wrong HA Msg: 31882
 Error(s) in Process event: 0
 Error(s) in Syn event: 0
 Error(s) in Common: 28533945
===========================================================
  ===============display arp info slot 1===============
===========================================================
ARP Information:
Learning switcher: 1 (1 on, 0 off)
CockSingStatus: 0
ARPCockEndStatus: 0x00000000000000000
ARP DataCollectState:
   ulState: 2 (0 syn, 1 asy, 2 end)
   LinkHeadWaitListHandle: 0
   ListEntryWaitListHandle: 0
   pstArpLinkHead: 0x0
   pstLinkNode: 0x0
   usArpNumSent: 0
ARPSmoothFlag: 0
ARP KeepConsistent:
    WalkState: 0 (1 start  0 end)
    LinkHeadWaitListHandle: 0
    ListEntryWaitListHandle: 0
    pstArpLinkHead: 0x0
    pstLinkNode: 0x0
ARP **bleep** Age:
    WalkState: 0 (1 start  0 end)
    LinkHeadWaitListHandle: 0
    ListEntryWaitListHandle: 0
    pstArpLinkHead: 0x0
    pstLinkNode: 0x0
    pstLinkNode: 0x0
    WalkState: 0 (1 start  0 end)
    LinkHeadWaitListHandle: 0
    ListEntryWaitListHandle: 0
    pstArpLinkHead: 0x0
    pstLinkNode: 0x0
ARP BlackHoleEnable: 0
ARP Default BlackKeep: 0
ARP BlackHoleKeep: 0
ARP ProbeLink Max-Len: 0
ARP Reload Timer: 10000 ms
ARP SendSeq: 0
ARP IPC Queue Len: 0
ARP Detection Information
    Max Vlan ID: 4094
    Max config Vlan number: 1024
    Current configed Vlan number: 0
    Valid check config: 0
    Vlan bit map length(byte): 512
    Vlan bit map address: 0x83d3c820
ARP RateLimit Information
    Default rate limit: 4294967295
    Min rate limit: 5
    Max rate limit: 100
ARP ActiveAck Information:
    Enable: 0
    Current ActiveAck Link Count: 0
ARP SrcMAC ValidCheck: 0
ARP SrcMACFixed Information:
    Fixed Mode: 0
    Current AgeTime: 300
    Current Threshhold: 50
    Current HashNode Count: 0
    Current AttackLink Count: 0
    Current ExcludeMac Count: 0
    Max ExcludeMac Count: 10
    Current DropMac Count: 0
    Max DropMac Count: 200
   Function Form DHCPC: 0x80aee270
   Function Form DHCPR: 0x81535330
   Function Form PORTAL: 0x0
ARP SlotNotifyInfo(0x840d3bd0) slot( )
=================================================================
  ===============display arp statistics slot 1===============
=================================================================
Dump ARP Statistics Data:
All Valid ARP Number: 46
All Static ARP Number     : 0
Unresolved Incomplete Static  ARP Number : 0
Resolved Incomplete Static  ARP Number : 0
Valid complete Static  ARP Number : 0
Invalid complete Static  ARP Number : 0
All Dynamic(unresolved + resolved) ARP Number : 46
Unresolved(Not Valid) Dynamic ARP Number : 0
Resolved Dynamic ARP Number   : 46
Multiport ARP Number          : 0
Permitted Static ARP Number : 64
Permitted Multiport ARP Number : 0
Permitted Total ARP Number : 256
Permitted Max Number perIf Configed: 256
Default Permitted Max Number perIf: 256
Permitted Dynamic ARP Number : 0
ARP Input Counter         : 31672644
ARP Resolved Counter      : 32411
Timer Entering Counter    : 0
Failed Update Fdb Counter : 0
Static Reload Timer       : 0
ARP max ipc queue len     : 0
Last Error Code           : 0
ARP Drop Counter          : 0
ARP Static Item Counter   : 0
ARP Dynamic Item Counter  : 0
ARP Dummy Counter         : 0
ARP ProbeNode Counter     : 0
ARP Probe Request Counter : 0
ARP Mbuf Num              : 0

 Remembering that HPE OfficeConnect 1920 Switch series is Comware 5 operating system based (even if with limited features WRT other more expensive and features rich Comware 5 HPE platforms).