Jitter for PTP packets (DSCP: EF) too high

[svhelden]

Hi, we are using a Spectralink DECT system that uses PTP (Precision Time Protocol) to sync its base stations. All base stations are connected to HP ProVision-based switches (2510, 2530) with a 5406zl2 core switch. The time-critical PTP packets are marked with DSCP EF (Expedited Forwarding), which is mapped to priority 7 in the switches. The same DSCP is used for VoIP traffic (Skype for Business).

According to Spectralink, the jitter between base stations is "too high", sometimes it goes up to 1200 ns, usually it is around 300 ns. Sometimes it is even above 400 ns between two stations on the same switch, and sometimes there are also huge differences for stations on the same switch (like, station on port 8 has 500 ns jitter to a remote station, but station on port 10 of the same switch has 800 ns to the same remote counterpart).

Another DECT vendor says that ProCurve switches are just perfect for syncing DECT stations with PTP, while Spectralink says that our jitter is way too high and no wonder we have issues (should be < 400 ns).

These are the relevant lines from our config files:

qos dscp-map cs5 priority 7 name "Lync-40"
qos dscp-map ef name "Lync-46"
qos type-of-service diff-services
vlan 30
name "DECT"
ip igmp
exit

Is there anything I can do from the configuration? Or are the Spectralink bases just too picky?

[parnassus]

Hi, really interesting.

Does your DECT Network belong to a separate high priority dedicated VLAN (let's say you defined a IP-DECT dedicated VLAN) traversing all involved Switches? yep, I see, it does (VLAN 30).

That seems to be an essential requirement (a mandatory one) for similar IP DECT deployments (as example, the one offered by Siemens - now Unify - with its HiPath Cordless IP V1 system) and there is also a quite common recommandation about the maximum supported number of cascaded Switches between the BSIP Sync Master and BSIP Sync Salves (so it's important where the Sync Master BSIP is located/defined); also Layer 3 Routing is not supported between the DECT Server (represented by a BSIP Base Station operating in IWU mode or by a bare metal Server which operates as Server IWU) and BSIP Base Stations operating in BSIP only mode...but this, I think, you know yet.

How is your Synchronization Topology (where is the BSIP Sync Master with respect to the BSIP Sync Slaves)?

How is your Network Topology in terms of Trunk Uplinks between the Switches?

Do all your involved Switches manage high traffic situations?

As example the Unify HiPath Cordless IP V1 (which is basically an Ikon GmbH based IP-DECT system) reports a maximum of 700 ns for Jitter (Variation of Delay), stating that higher values will definitely lead to re-synchronizations (so IP-DECT calls go KO).

Regarding the LAN Synchronization (IEEE 1588 PTP) I've to admit that the Siemens solution uses a method that was developed using the IEEE 1588 as a base: as someone at Siemens told me four years ago, to overcome the issue represented by Low End non IEEE 1588 compliant Switches or by High End IEEE 1588 compliant Switches that show high Jitter values even with low traffic situations, they developed a method that let the "...Master BS to generate UDP messages with time stamps and send them to the Slave BS(s). The Slave BS(s) are then able to read the time stamps and so to deduct the flying time needed between Slave and Master. If this is successful the different time bases in the different DECT IP Base stations are kept in sync...during the development was recognized that Jitter variations are most critical, if the Jitter is constant the developed methodology can adapt this Jitter, but if the Jitter varies very much, this methodology is not able to adapt the Jitter to a fix value. The LAN synchronization will not work in this case."

Indeed LAN Synchronization, in contrast to the DECT Synchornization (Over the Air), requires a PSR (Project Specific Release) approved by Siemens/Unify.

Don't know much about Spectralink IP Dect solution and its requirements (apart of what is reported in their Spectralink DECT/IP-DECT Server 400/6500/2500/8000 Synchronization and Deployment Guide here) so I can't judge if their requirements are too strict or not (at some point they state that "The total forwarding jitter added by switches on any path through the deployment network should be less than one microsecond and preferably less than 100 nanoseconds." but I haven't any means to understand if this requirment is too strict or not even with HP ProCurve Switches)...but, hey, first:

• Do HP ProCurve Switches support IEEE 1588 Version 2 (PTPv2) or IEEE 802.1AS? <- I found NO reference for that (on the contrary the HPE FlexNetwork family seems to support it) in any QuickSpecs or Configuration Manuals.
• What are the HP ProVision 6th Generation ASICs specs (related to HPE 5400R zl2) regarding the delay time (in nano-seconds or fraction of micro-seconds) added to process/manage PTP traffic? <- really don't know but, personally, I expect at least that an High End Switch to perform significantly better than a Low End one!

Edit: consider, I'm reading it now on a IEEE 802.3AS six years old presentation, that one of the 802.3AS golas was to keep the time at any two LAN-attached stations accurate to within +/- 500ns (assuming 7 newtork hops) and that 802.3AS goals were not to:

• Improve the latency of media packets
• Improve the delivery jitter of media packets
• Improve latency or delivery jitter of 802.1AS packets
• Provide clock which all media must be synchronized to
• Solve time synchronization over 802.3 only (or .11 only)

Also, be careful when looking for IEEE 1588 PTP support on various HPE Switches QuickSpecs...I found now that HPE reports it incorrectly as RFC 1855 PTP (gosh!) so when doing a research specifically for 1588 I initially found nothing even if the Switch appears to support it (see here for HPE FlexNetwork 5900 Switch Series QuickSpecs, just as example).

[svhelden]

Thanks for the comprehensive reply!

Spectralink specifically say that they do NOT support such thing as 'PTP-compliant switches' ;)

In our topology, we have few cascaded switches, but it's not more than 5 switches between any base stations. All switches are interconnected with 1 G links (copper or fiber), and all involved ports have zero failures/drops.

Could it be related to VoIP traffic that shares the same priority?

[parnassus]

---

@svhelden wrote:

In our topology, we have few cascaded switches, but it's not more than 5 switches between any base stations. All switches are interconnected with 1 G links (copper or fiber), and all involved ports have zero failures/drops.

---

I've read that only the Store and Forward Latency of an Ethernet Switch for the maximum size Ethernet frame (1500 bytes) at 100 Mbps is about 120 μs...for comparison...the minimum size frame (64 bytes) at Gigabit speeds has a latency of just 0.5 μs (but 0.5 μs are still 500 ns, aren't?)...this without considering Switch Fabric Latency, Wireline Latency (really trascurable in LAN) and Queueing Latency (sensible to Network load).

So when I re-read the Spectralink 100 ns critical threshold...well...I start thinking their one is very strict (is it really possible to stay below the 100 ns threshold within the same Switch?).

---

@svhelden wrote:

Spectralink specifically say that they do NOT support such thing as 'PTP-compliant switches' ;)

Yep, also the Siemens HiPath Cordless IP V1's Product Manager (at time, so 4 years ago) stated something very similar...they didn't rely on IEEE 1588 Switch capabilities to permit the LAN Synchronization...but actual documentation seems to say the contrary (see below).

The recommandation about the maximum supported number of cascaded Switches admitted between the BSIP Sync Master and any BSIP Sync Salve is only 3 for the HiPath Cordless IP solution (your is higher but you have Spectralink...on the other hand it's also clear that you still have issue within the same Switch and not only between them...but...it's important to understand where is located - in the network topology - the BSIP Sync Master with respect to all BSIP Sync Slaves <- Star Synchronization Topology only).

And just a configuration example:

As you see the example above starts from a very simplified topology (basically no cascaded Switches).

---

@svhelden wrote:

Could it be related to VoIP traffic that shares the same priority?

---

Probably: Siemens recommends to separate VoIP VLAN from DECT Network VLAN and give to this one the highest priority (see above).

Don't forget how much high traffic situations - within a Switch or between them - can impact the Jitter value!

Note that Siemens placed all the BSIP(s) Base Stations (IWU too) within the control of just a single Switch! <-- that's a simply too ideal scenario in my opinion.

Real world cases, like your one, are a little bit more complex (or a little bit less simple).

[Vince-Whirlwind]

IP Precedence 7 is probably wrong. Just because it is a higher number than "6" doesn't mean it will get you better performance than "6".

Generally 7 is for quiet protocols where reliability is important (so no drops) but not latency.

[Vince-Whirlwind]

I just read that you are expecting sub-microsecond latency. Provision switching latency is 10X higher than this.

You need ultra-low latency switches. Try Arista, or IBM.

[svhelden]

I just read that you are expecting sub-microsecond latency. Provision switching latency is 10X higher than this.

Absolute latency doesn't matter. It's about Jitter (latency variation).

[svhelden]

Thanks parnassus!

From your comments, it seems that there is nothing much I can do configuration-wise. Maybe I could try giving VoIP traffic a lower priority, so that PTP packets are still treated with higher priority than these.

I'm also still waiting for information from Spectralink, maybe our issue is not related to Jitter at all. Their first response to our ticket was basically 'You have high jitter, of course it cannot work', but in the meantime they are also considering other issues..

The recommandation about the maximum supported number of cascaded Switches admitted between the BSIP Sync Master and any BSIP Sync Salve is only3 for the HiPath Cordless IP solution (your is higher but you have Spectralink..

Spectralink does not support a specific number, but their documentation says that they tested it with 5 switches, and with that we would comply.

it's important to understand where is located - in the network topology - the BSIP Sync Master

We tested with multiple stations as master, without notable difference. 

Anyway, thank you very much for the profound reply! I will post my future findings here ...

[svhelden]

One more note, I noticed that older HP switches had an option "ip igmp high-priority-forward", which seems to be gone from newer models. Is there any replacement for this command, or is the functionality gone completely?

[Vince-Whirlwind]

You can't get jitter that is an order of magnitude lower than the latency is. 

Put it this way - if your ruler is graduated in centimetres, you can't generate measurements of sub-millimetre precision.