802.1x not opening port to VLAN on 2600 switches

[Steve Woodward_2]

I am having an issue with 802.1x on 2600 switches and I’m hoping someone has either run into it or maybe has some thoughts as to the cause.
We have roughly 100 2600 series switches and 100 2610 series switches deployed in 50 different locations.
We are currently rolling out 802.1x one site at a time, with a guest VLAN, and for the most part it works as expected.
We have configured our switches with a guest VLAN to accommodate devices like printers that are not able to authenticate. The guest VLAN (if we ever get this working properly) will have very limited access to our network resources, while the default VLAN will have full access, in an effort to prevent outside devices from being able to do too much damage. The idea behind this being that we can than configure most switch ports identically, so when people just plug things in, it should automatically be assigned to the appropriate VLAN whether it’s a PC or a printer, etc. It also helps prevent holes in our security where someone may hijack a switch port or introduce a hub.
It’s not all devices that have the 802.1x problem by any means. Most non-802.1x devices work properly. There are a few devices that we’ve encountered that every time they are connected, 802.1x doesn’t open the port to the default or guest VLANs even though the port is up. There are other devices that work for a period, and then the port enters this state. This is similar to if the logoff-period expired, but there’s no way these devices have gone 1,000,000 seconds (what our timer is set for) without generating traffic.
Some of these devices can be “fixed” with a firmware upgrade. Some devices are simply not upgradable. It’s not even specific to the type of device, as I have two Jetdirect cards, with the same part number and the same firmware. One has this problem with 802.1x as soon as it’s connected and the other doesn’t.
If you turn 802.1x off the port and device work fine.
This issue only occurs on the 2600 switches and not on the 2610 switches. The configs between the switches are nearly identical. We use the two switches interchangeably at sites. A wire closet may have a mix of 2600 and 2610s. A device with this problem can be moved to a 2610 and the problem is gone. This leads me to believe it’s a bug in the 2600 software. I’ve had a case open with HP for a few weeks and so far they are stumped. I have a Jetdirect box in the mail to them which has this issue and hopefully that will produce some progress. What seems weird is that apparently no one has run into this issue before.
This issue has made some unhappy users, and made some very unhappy IT employees who don’t appreciate the extra trouble this is causing them.

The relevant config:

************************************************************
vlan 903
name "Default"
untagged 48
tagged 50
exit
vlan 126
name "Guest"
tagged 50
exit
aaa authentication port-access eap-radius authorized
radius-server key (removed)
radius-server host 192.168.132.10
dhcp-snooping
no dhcp-snooping option 82
dhcp-snooping vlan 126 903
aaa port-access authenticator 48
aaa port-access authenticator 48 unauth-vid 126
aaa port-access authenticator 48 logoff-period 1000000
aaa port-access authenticator 48 client-limit 1
aaa port-access authenticator active
loop-protect 48
loop-protect trap loop-detected
loop-protect disable-timer 600
arp-protect
arp-protect trust 50
arp-protect vlan 126 903
************************************************************


Part 2, next reply.

[Steve Woodward_2]

When a device is connected that is not 802.1x aware this is what you expect to see, and most devices work as expected:

************************************************************
2650-test# show port-access authenticator 48

Port Access Authenticator Status

Port-access authenticator activated [No] : Yes
Allow RADIUS-assigned dynamic (GVRP) VLANs [No] : No

Current Current
Port Status VLAN ID Port COS
---- ------ -------- -----------
48 Closed 126 No-override

2650-test# show mac-address 48

Status and Counters - Port Address Table - 48

MAC Address
-------------
0001e6-0239a5
*************************************************************




When some (not all) non-802.1x aware devices are connected, 802.1x doesn't open the port to any VLAN as seen below.

*************************************************************
2650-test# show port-access authenticator 48

Port Access Authenticator Status

Port-access authenticator activated [No] : Yes
Allow RADIUS-assigned dynamic (GVRP) VLANs [No] : No

Current Current
Port Status VLAN ID Port COS
---- ------ -------- -----------
48 Closed 903 No-override

2650-test# show mac-address 48

Status and Counters - Port Address Table - 48

MAC Address
-------------

2650-test(config)# show interfaces 48

Status and Counters - Port Counters for port 48

Name :
Link Status : Up
â ¦

Once the port is in this state where 802.1x didnâ t open it to a VLAN, it doesnâ t matter what non-802.1x aware device is connected, none will be able to communicate on the port. Turning 802.1x off and back on for the port will revive it until it enters this state where no VLAN is assigned to the port.
no aaa port-access authenticator 48
aaa port-access authenticator 48

Most of the devices that the switch has this issue with are HP or Oce/Imagistics printers. Weâ ve also seen a few others like Wyse Winterms have this issue.

This issue has been seen on the latest software (or at least as of a couple weeks ago) H.10.99.
If you have run across this issue or have some thoughts, your reply will be appreciated.
Thanks,
Steve

[Arran Cudbard-Bell]

There are a number of things this could be. Is there any chance you could use a minihub to trace between the supplicant the switch, then attach the trace file to this thread?

Look for differences between working and non-working clients, things like src/dst mac-addresses, packet types, and packet ordering can all expose weird issues.

If the devices you're connecting are 802.1X capable and are only responding with 802.1X packets, then you could see this issue. Depending on the switch model and firmware version ProCurve switches sometimes require an additional non 802.1X frame to trigger unauth-vid/VLAN assignment.

Your best bet is probably going to be to contact ProCurve customer support and log this as a bug, but as the 2600 is EOL you may not have much luck.

If you have a decent RADIUS server like FreeRADIUS or RADIATOR, you can actually emulate the functionality of the unauth-vid using a combination of mac-authentication and 802.1X. Both can run on the port concurrently, and 802.1X will take precedence if the client that comes on is actually 802.1X enabled.

Just assign your unauth-vid to any mac-auth clients coming on the port. That way it also allows you to fully block problem clients whereas unauth-vid doesn't.

[Steve Woodward_2]

Sorry for the late reply.  I was having trouble with the old site (I guess they were in the middle of transitioning?).  The new one seems to be working though.

HP Support after weeks or looking into this and after me sending them a device that causes this problem to occur has narrowed down the issue to something in the client-limit.  Turning off the client limit makes the issue go away.  The problem with that is it weakens the 802.1x implementation, so someone could throw a hub on a port where a PC has authenticated and ride on that authenticated port with their unwanted device. 

The issue has gone up to level 3 where they'll hopefully come up with a fix. 

Thanks,

Steve

[Mike_Stachnicki]

Steve, I found your post yesterday whilst looking for solution to what sounds like an identical problem. I was wondering if you ever got a response from HP support and have now resolved the problem ?

We also have quite a large number of 2626 and 2650 switches and were looking to deploy 802.1x for PCs in public areas to secure the ports. The first area I tried to deploy a pilot service was connected to a 2626 and I had PC's not waking up from power save correctly and ending up on neither the auth VLAN or the unauth VLAN - just stuck in a void somewhere.

Any news on how you resolved this would be most appreciated.

Regards,

Mike

[Arran Cudbard-Bell [HP]]

Hi Mike,

Could you expand on what you mean by not waking up from power save correctly? Do you mean that the PCs came up but could not communicate with other devices on the network?

If the device coming up never TXs any packets, then they would not be placed in any VLAN or even registered by the security subsystem. Bringing the port up electrically is not enough to trigger the process whereby the PVID of the port is altered to the unauth-vid/or auth-vid.

If the problem you're having is that devices can't be woken up using WOL packets. My advice is to set control-direction to in only, then designate a WOL VLAN and statically untag it on your 802.1X authenticated ports.

When devices go to sleep the NIC card almost always cycles the interface, meaning that any temporary assignment to the auth or unauth VID is lost.