Hosts unable to get IP addresses

[Adnan004]

Greetings,

I have one HP Aruba switch 3810M-24GPoE+ one my switch there are two VLANs configured as per below details,

vlan 1
name "USER"
untagged 1-24
ip address 100.91.x.x 255.255.255.0

vlan 9
name ".ENTRY."
no ip address

DHCP Snooping is configured as per below, 

dhcp-snooping
dhcp-snooping authorized-server x.x.x.x
dhcp-snooping authorized-server x.x.x.x
dhcp-snooping authorized-server x.x.x.x
dhcp-snooping authorized-server x.x.x.x
no dhcp-snooping option 82
dhcp-snooping vlan 9

We are using aaa port-access authenticator and aaa port-access mac-based on all 24 ports 

Some of my connected devices are not getting IP addresses when aaa port-access authenticator and aaa port-access mac based are enabled on these ports. 

I am looking for  help how can I fix this issue, as per my requirements aaa port-access authenticator and aaa port-access mac-based is required always.

Is there a way all devices connected can get IPs (Note: All devices are configured for DHCP assignment)

[Emil_G]

Hello,

You are using aaa port-access authenticator and aaa port-access mac-based on all 24 ports. This authentication methods are also colled 802.1x and MAC authentication and they are known as Layer 2 authentication method. Which means that the end devices is able to obtain an IP address only after succesful authentication. Before the authentication is completed DHCP traffic will be blocked on the port.

So the first step would be to verify if the connected devices were authenticated succefully. You can use the command show port-access client detail <port-nr> to check detailed authentication status of the port. If the authentication has failed you should troubleshoot it on the client and on the RADIUS server. 

The other possibility is that the authentication is succesful but the client is placed in a wrong VLAN where you dont have DHCP server.  Or some other setting was enforced by RADIUS attribute which prevents the communicateion. You can see the VLAN  and other authorization information also in the output of show port-access client detail <port-nr>. Feel free to share the output of the command if you are not able to intepret it.

Please also share the interface configuration of the affected port, use the command show running-config interface <port-nr>.

Check the event logs with the port that has the issue. You can use the command show logging -r <port-nr>

Regarding your question if all connected device can get an IP address. By default only the authenticated devices can get an IP. If you want also unauthenticated device to get an IP there are some options depending on why the devices cannot authenticate -is the RADIUS server not reachable, is the client rejected by RADIUS etc.

hth

[Adnan004]

Here are the results,

Port-access client details

Client Base Details :
Port : 9 Authentication Type : 802.1x
Client Status : authenticated Session Time : 13179 seconds
Client name : host/BC581196.aram... Session Timeout : 43200 seconds
MAC Address : 98fa9b-3c8be7
IP : n/a

Access Policy Details :
COS Map : Not Defined In Limit Kbps : Not Set
Untagged VLAN : 1 Out Limit Kbps : Not Set
Tagged VLANs : No Tagged VLANs
Port Mode : 100FDx
RADIUS ACL List : No Radius ACL List

Client Base Details :
Port : 9 Authentication Type : 802.1x
Client Status : authenticated Session Time : 13223 seconds
Client name : 2150081912BTE3004914 Session Timeout : 43200 seconds
MAC Address : d46aa8-f9e3f1
IP : n/a

Access Policy Details :
COS Map : Not Defined In Limit Kbps : Not Set
Untagged VLAN : 1 Out Limit Kbps : Not Set
Tagged VLANs : No Tagged VLANs
Port Mode : 100FDx
RADIUS ACL List : No Radius ACL List

interface config

show running-config interface 9

Running configuration:

interface 9
mdix-mode mdix
untagged vlan 1
aaa port-access authenticator
aaa port-access authenticator reauth-period 21600
aaa port-access authenticator client-limit 3
aaa port-access mac-based
aaa port-access mac-based addr-limit 3
aaa port-access mac-based reauth-period 21600
exit

logs

show logging -r 9
Keys: W=Warning I=Information
M=Major D=Debug E=Error
---- Reverse event Log listing: Events Since Boot ----
I 08/01/21 17:37:07 00076 ports: port 9 is now on-line
I 08/01/21 17:37:06 00435 ports: port 9 is Blocked by AAA
I 08/01/21 17:36:49 00077 ports: port 9 is now off-line
I 08/01/21 17:36:40 00435 ports: port 9 is Blocked by AAA
I 08/01/21 17:36:18 00561 ports: port 9 Applying Power to PD.
I 08/01/21 17:36:18 00560 ports: port 9 PD Detected.
I 08/01/21 17:36:11 00565 ports: port 9 PD Removed.
I 08/01/21 17:36:11 00077 ports: port 9 is now off-line
I 08/01/21 17:34:20 00076 ports: port 9 is now on-line
I 08/01/21 17:34:19 00435 ports: port 9 is Blocked by AAA
I 08/01/21 17:34:01 00077 ports: port 9 is now off-line
I 08/01/21 17:33:53 00435 ports: port 9 is Blocked by AAA
I 08/01/21 17:33:31 00561 ports: port 9 Applying Power to PD.
I 08/01/21 17:33:31 00560 ports: port 9 PD Detected.
I 08/01/21 17:33:16 00565 ports: port 9 PD Removed.
W 08/01/21 17:33:16 00563 ports: port 9 PD MPS Absent indication.
I 08/01/21 17:33:15 00077 ports: port 9 is now off-line
I 08/01/21 13:36:49 00076 ports: port 9 is now on-line
I 08/01/21 13:36:48 00435 ports: port 9 is Blocked by AAA
I 08/01/21 13:36:30 00077 ports: port 9 is now off-line
I 08/01/21 13:36:22 00435 ports: port 9 is Blocked by AAA
I 08/01/21 13:36:00 00561 ports: port 9 Applying Power to PD.
I 08/01/21 13:36:00 00560 ports: port 9 PD Detected.
I 08/01/21 13:35:56 00565 ports: port 9 PD Removed.
W 08/01/21 13:35:56 00563 ports: port 9 PD MPS Absent indication.
I 08/01/21 13:35:55 00077 ports: port 9 is now off-line
I 08/01/21 12:25:30 00076 ports: port 9 is now on-line
I 08/01/21 12:25:29 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:25:11 00077 ports: port 9 is now off-line
I 08/01/21 12:25:03 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:24:41 00561 ports: port 9 Applying Power to PD.
I 08/01/21 12:24:41 00560 ports: port 9 PD Detected.
I 08/01/21 12:24:35 00565 ports: port 9 PD Removed.
I 08/01/21 12:24:34 00077 ports: port 9 is now off-line
I 08/01/21 12:23:26 00076 ports: port 9 is now on-line
I 08/01/21 12:23:25 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:21:26 00077 ports: port 9 is now off-line
I 08/01/21 12:21:18 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:20:55 00561 ports: port 9 Applying Power to PD.
I 08/01/21 12:20:55 00560 ports: port 9 PD Detected.
I 08/01/21 12:20:47 00565 ports: port 9 PD Removed.
I 08/01/21 12:20:47 00077 ports: port 9 is now off-line
I 08/01/21 12:12:43 00076 ports: port 9 is now on-line
I 08/01/21 12:12:42 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:12:24 00077 ports: port 9 is now off-line
I 08/01/21 12:12:15 00435 ports: port 9 is Blocked by AAA
I 08/01/21 12:11:53 00561 ports: port 9 Applying Power to PD.
I 08/01/21 12:11:53 00560 ports: port 9 PD Detected.
I 08/01/21 12:11:46 00565 ports: port 9 PD Removed.
I 08/01/21 12:11:46 00077 ports: port 9 is now off-line
I 08/01/21 11:53:34 00076 ports: port 9 is now on-line
I 08/01/21 11:53:16 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:52:58 00077 ports: port 9 is now off-line
I 08/01/21 11:52:50 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:52:28 00561 ports: port 9 Applying Power to PD.
I 08/01/21 11:52:28 00560 ports: port 9 PD Detected.
I 08/01/21 11:52:23 00565 ports: port 9 PD Removed.
W 08/01/21 11:52:23 00563 ports: port 9 PD MPS Absent indication.
I 08/01/21 11:52:22 00077 ports: port 9 is now off-line
I 08/01/21 11:39:04 00076 ports: port 9 is now on-line
I 08/01/21 11:38:46 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:19:02 00077 ports: port 9 is now off-line
I 08/01/21 11:18:54 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:18:32 00561 ports: port 9 Applying Power to PD.
I 08/01/21 11:18:32 00560 ports: port 9 PD Detected.
I 08/01/21 11:18:23 00565 ports: port 9 PD Removed.
I 08/01/21 11:18:22 00077 ports: port 9 is now off-line
I 08/01/21 11:03:45 00076 ports: port 9 is now on-line
I 08/01/21 11:03:45 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:03:27 00077 ports: port 9 is now off-line
I 08/01/21 11:03:19 00435 ports: port 9 is Blocked by AAA
I 08/01/21 11:02:56 00561 ports: port 9 Applying Power to PD.
I 08/01/21 11:02:56 00560 ports: port 9 PD Detected.
I 08/01/21 11:02:47 00565 ports: port 9 PD Removed.
I 08/01/21 11:02:47 00077 ports: port 9 is now off-line
I 08/01/21 10:48:39 00076 ports: port 9 is now on-line
I 08/01/21 10:48:39 00435 ports: port 9 is Blocked by AAA
I 08/01/21 10:48:18 00077 ports: port 9 is now off-line
I 08/01/21 10:48:09 00435 ports: port 9 is Blocked by AAA
I 08/01/21 10:47:47 00561 ports: port 9 Applying Power to PD.
I 08/01/21 10:47:47 00560 ports: port 9 PD Detected.
I 08/01/21 10:46:54 00565 ports: port 9 PD Removed.
I 08/01/21 10:46:53 00077 ports: port 9 is now off-line
W 08/01/21 10:44:51 00975 dipld: port 9 is now dhcp-snooping untrusted.
I 07/31/21 18:53:38 00076 ports: port 9 is now on-line
I 07/31/21 18:53:19 00435 ports: port 9 is Blocked by AAA
I 07/31/21 18:53:01 00077 ports: port 9 is now off-line
I 07/31/21 18:52:53 00435 ports: port 9 is Blocked by AAA
I 07/31/21 18:52:30 00561 ports: port 9 Applying Power to PD.
I 07/31/21 18:52:30 00560 ports: port 9 PD Detected.
---- Top of Log : Events Listed = 90 ----

How can we proceed further.

[Emil_G]

Hello @Adnan004 

Thanks for the outputs!

It seems like 2 devices are authenticated via 802.1x on port 9 at the moment.

98fa9b-3c8be ( vendor LCFC(HeFei) Electronics Technology co., ltd) is probably a Windows machine. It is assigned to VLAN 1. There is no RADIUS ACL or rate limits which could block DHCP.

The second device d46aa8-f9e3f1 (vendor Huawei Technologies Co.,Ltd) is also assigned to VLAN 1 (untagged) there are also no ACLs and rate limits.

The configuration allows up to 3 clients to authenticate on the port via 802.1x and mac-auth simoultaneously. Dont see any obvious issues with port 9 in the log. It looks like the device directly connected to the port is receiving power via PoE. Is this maybe an IP phone with a PC connected to it?

Which of this devices is not able to obtain an IP? Phone, PC or both of them? Or maybe you are connecting another device which is not visible in the show port-access output?

Should VLAN 1 be the correct VLAN for both devices? Or maybe they should be in different VLANs?

Are both devices sending untagged traffic? Phones typically send tagged traffic.

Let us check the port counters -command: show interface 9

DHCP snooping shouldnt be blocking because it is not even enabled at all in VLAN 1 and I dont see messages, nevertheless please save show dhcp-snooping stats.

[Adnan004]

Thanks for your quick reply,

Yes there are two devices connected on port 9 both using 802.1x authentication 

Right one is a windows workstation and another is an IP phone. IP phone is directlty connected with Switch port 9 and from IP phone PC port workstation is connected.

We face issues always on IP phones. IP phones are unable to take IP addresses while workistations connected thru IP phones are working fine.

VLAN1 is the correct VLAN for both devices while VLAN9 have no ip addresses.

All ports under VLAN1 are untagged.

vlan 1
name "DEFAULT_VLAN"
untagged 1-24
ip address 10.140.1.240 255.255.255.0
exit
vlan 9
name "ENTRY"
no ip address
exit

Here are the results of port 9 show interface 9 

Status and Counters - Port Counters for port 9

Name :
MAC Address : 70106f-8755f7
Link Status : Up
Totals (Since boot or last clear) :
Bytes Rx : 585,575,654 Bytes Tx : 1,250,023,444
Unicast Rx : 1,598,022 Unicast Tx : 2,508,924
Bcast/Mcast Rx : 139,983 Bcast/Mcast Tx : 492,198
Errors (Since boot or last clear) :
FCS Rx : 1 Drops Tx : 0
Alignment Rx : 0 Collisions Tx : 0
Runts Rx : 0 Late Colln Tx : 0
Giants Rx : 0 Excessive Colln : 0
Total Rx Errors : 1 Deferred Tx : 0
Others (Since boot or last clear) :
Discard Rx : 0 Out Queue Len : 0
Unknown Protos : 0
Rates (5 minute weighted average) :
Total Rx (bps) : 10,448 Total Tx (bps) : 25,288
Unicast Rx (Pkts/sec) : 1 Unicast Tx (Pkts/sec) : 4
B/Mcast Rx (Pkts/sec) : 0 B/Mcast Tx (Pkts/sec) : 0

Port 8 have the same issue now

Name :
MAC Address : 70106f-8755f8
Link Status : Up
Totals (Since boot or last clear) :
Bytes Rx : 1,002,765,547 Bytes Tx : 1,761,593,558
Unicast Rx : 2,732,721 Unicast Tx : 3,914,282
Bcast/Mcast Rx : 115,145 Bcast/Mcast Tx : 505,535
Errors (Since boot or last clear) :
FCS Rx : 0 Drops Tx : 0
Alignment Rx : 0 Collisions Tx : 0
Runts Rx : 0 Late Colln Tx : 0
Giants Rx : 0 Excessive Colln : 0
Total Rx Errors : 0 Deferred Tx : 0
Others (Since boot or last clear) :
Discard Rx : 0 Out Queue Len : 0
Unknown Protos : 0
Rates (5 minute weighted average) :
Total Rx (bps) : 54,976 Total Tx (bps) : 100,960
Unicast Rx (Pkts/sec) : 5 Unicast Tx (Pkts/sec) : 7
B/Mcast Rx (Pkts/sec) : 0 B/Mcast Tx (Pkts/sec) : 0
Utilization Rx : 0 % Utilization Tx : 00.01 %

port 8 port-access clients details

Client Base Details :
Port : 8 Authentication Type : 802.1x
Client Status : authenticated Session Time : 1632 seconds
Client name : host/windows  Session Timeout : 43200 seconds
MAC Address : 98fa9b-3e4614
IP : n/a

Access Policy Details :
COS Map : Not Defined In Limit Kbps : Not Set
Untagged VLAN : 1 Out Limit Kbps : Not Set
Tagged VLANs : No Tagged VLANs
Port Mode : 1000FDx
RADIUS ACL List : No Radius ACL List

Client Base Details :
Port : 8 Authentication Type : 802.1x
Client Status : authenticated Session Time : 206461 seconds
Client name : 2150081912DWQ3004585 Session Timeout : 43200 seconds
MAC Address : d46aa8-f9e42f
IP : n/a

Access Policy Details :
COS Map : Not Defined In Limit Kbps : Not Set
Untagged VLAN : 1 Out Limit Kbps : Not Set
Tagged VLANs : No Tagged VLANs
Port Mode : 1000FDx
RADIUS ACL List : No Radius ACL List

port 8 logs

I 08/03/21 06:32:29 00076 ports: port 8 is now on-line
I 08/03/21 06:32:28 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:32:26 00077 ports: port 8 is now off-line
I 08/03/21 06:32:00 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:31:57 00077 ports: port 8 is now off-line
I 08/03/21 06:31:55 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:31:54 00561 ports: port 8 Applying Power to PD.
I 08/03/21 06:31:54 00560 ports: port 8 PD Detected.
I 08/03/21 06:31:52 00565 ports: port 8 PD Removed.
W 08/03/21 06:31:52 00563 ports: port 8 PD MPS Absent indication.
I 08/03/21 06:31:51 00077 ports: port 8 is now off-line
I 08/03/21 06:31:38 00076 ports: port 8 is now on-line
I 08/03/21 06:31:37 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:31:35 00077 ports: port 8 is now off-line
I 08/03/21 06:31:09 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:31:07 00077 ports: port 8 is now off-line
I 08/03/21 06:31:04 00435 ports: port 8 is Blocked by AAA
I 08/03/21 06:31:03 00561 ports: port 8 Applying Power to PD.
I 08/03/21 06:31:03 00560 ports: port 8 PD Detected.
I 08/03/21 06:31:01 00077 ports: port 8 is now off-line
I 08/03/21 06:31:00 00565 ports: port 8 PD Removed.
W 08/03/21 06:31:00 00563 ports: port 8 PD MPS Absent indication.
I 08/02/21 21:51:40 00076 ports: port 8 is now on-line
I 08/02/21 21:51:39 00435 ports: port 8 is Blocked by AAA
I 08/02/21 21:51:37 00077 ports: port 8 is now off-line
I 08/02/21 21:51:12 00435 ports: port 8 is Blocked by AAA
I 08/02/21 21:51:09 00077 ports: port 8 is now off-line
I 08/02/21 21:51:06 00435 ports: port 8 is Blocked by AAA
I 08/02/21 21:51:06 00561 ports: port 8 Applying Power to PD.
I 08/02/21 21:51:06 00560 ports: port 8 PD Detected.
I 08/02/21 21:50:59 00565 ports: port 8 PD Removed.
W 08/02/21 21:50:59 00563 ports: port 8 PD MPS Absent indication.
I 08/02/21 21:50:59 00077 ports: port 8 is now off-line
I 08/02/21 11:03:07 00428 802.1x: 1 auth-failures for the last 60 sec.
I 08/01/21 11:01:06 00076 ports: port 8 is now on-line
I 08/01/21 11:01:05 00435 ports: port 8 is Blocked by AAA
I 08/01/21 11:01:03 00077 ports: port 8 is now off-line
I 08/01/21 11:00:37 00435 ports: port 8 is Blocked by AAA
I 08/01/21 11:00:34 00077 ports: port 8 is now off-line
I 08/01/21 11:00:31 00435 ports: port 8 is Blocked by AAA
I 08/01/21 11:00:31 00561 ports: port 8 Applying Power to PD.
I 08/01/21 11:00:31 00560 ports: port 8 PD Detected.
I 08/01/21 11:00:23 00565 ports: port 8 PD Removed.
I 08/01/21 11:00:23 00077 ports: port 8 is now off-line
I 08/01/21 10:48:09 00076 ports: port 8 is now on-line
I 08/01/21 10:48:08 00435 ports: port 8 is Blocked by AAA
I 08/01/21 10:48:06 00077 ports: port 8 is now off-line
I 08/01/21 10:47:40 00435 ports: port 8 is Blocked by AAA
I 08/01/21 10:47:38 00077 ports: port 8 is now off-line
I 08/01/21 10:47:35 00435 ports: port 8 is Blocked by AAA
I 08/01/21 10:47:35 00561 ports: port 8 Applying Power to PD.
I 08/01/21 10:47:35 00560 ports: port 8 PD Detected.
I 08/01/21 10:47:02 00077 ports: port 8 is now off-line
I 08/01/21 10:47:02 00565 ports: port 8 PD Removed.
W 08/01/21 10:44:51 00975 dipld: port 8 is now dhcp-snooping untrusted.
I 07/31/21 18:53:22 00076 ports: port 8 is now on-line
I 07/31/21 18:53:04 00435 ports: port 8 is Blocked by AAA
I 07/31/21 18:53:02 00077 ports: port 8 is now off-line
I 07/31/21 18:52:36 00435 ports: port 8 is Blocked by AAA
I 07/31/21 18:52:33 00077 ports: port 8 is now off-line
I 07/31/21 18:52:31 00435 ports: port 8 is Blocked by AAA
I 07/31/21 18:52:30 00561 ports: port 8 Applying Power to PD.
I 07/31/21 18:52:30 00560 ports: port 8 PD Detected.
---- Top of Log : Events Listed = 63 ----

This is happening on all IP phones ports randomly. 

[Emil_G]

Hello @Adnan004 

Thanks for the detailed response!

The interface statistiscs look good. There is a single FCS error on port 9. There are no other errors or dropped packets.

If the issue always happens with the IP phones and never with the Windows PC altough they are connected through the same port, I think the issue is probably specific to this IP phones. I dont think that we can see more from the CLI of the switch. 

Do you see any specific messages on the display of the phone when they are not getting IP? I have no knowledge about Huawei IP phones but maybe they can give us a general clue what exactly is happening with the phone.

Can we exclude that the phone starts sending tagged packets to the switch? Do you see the phone report any particular VLAN ID in the display when it is booting? Did you test what happens when you make VLAN tagged (just for test

Are the VLAN configured on the phone statically or dynamically (DHCP or LLDP-MED)

I think it would be helpful if you can save a Wireshark trace on the port at the time when the phone is not getting an IP address. Wireshark can show us if the phone is sending DHCP Discover or Request messages and if answers are coming from the DHCP server. 

The DHCP servers usually log all the DHCP packets arriving there. Do you see in the log any DHCP packets coming from the phone. We can also use Wireshark to check if DHCP packets from the phone are arriving and what answer is sent.

[Adnan004]

Sorry for replying late, I was thoroughly working on the phones.

I have tested three phones which were connected with Switch and then a workstation was connected thru phone. I tested both scenarios as per below details,

When workstation is connected with Phone

Workstation gets an IP and works fine while phone does not work due to no IP address.

When Phone is connected alone

Phone does not work as per logs phone is authenticated but there is no IP address obtained.

We are using Huawei eSpace 7910 phones which when power on or connected with network initialize as following,

1. Power up or LAN connected (With log in key pressed)
2. After 2 minutes we get message "IP address obtained successfully."
3. Then after another minute we have "802.1x authentication success."
4. Failed to login 
5. Retrying

Access Settings on Phone screen

802.1x authentication is on with (EAP-TLS)

LLDP-MED is on

VLAN is disabled (I tried on two phones two use our only one VLAN which is VLAN 1 with priority 1 but same results) No VLAN ID is displayed at screen in both cases.

 Access Settings on phone's web interface,

802.1x authentication is on with (EAP-TLS as authentication method)

LLDP-MED is on (120 seconds)

All these settings are identical on all phones working or not connected with or without workstations.

Some details from switch

BANIYAS-01-S# show port-access authenticator clients

Port Access Authenticator Client Status

Port-access authenticator activated [No] : Yes
Allow RADIUS-assigned dynamic (GVRP) VLANs [No] : No
Use LLDP data to authenticate [No] : No

Port Client Name MAC Address IP Address Client Status
----- --------------------- ------------- --------------- --------------------
6 2150081912BTE3005902 d46aa8-f9e82b n/a Authenticated
7 host/BC583137.aram... 98fa9b-33776d n/a Authenticated
7 2150081912BTE3005892 d46aa8-f9e809 n/a Authenticated
8 host/BC583136.aram... 98fa9b-3e4cf3 n/a Authenticated
8 2150081912BTE3006069 d46aa8-f9e8e4 n/a Authenticated
9 host/BC583135.aram... 98fa9b-4017ac n/a Authenticated
9 2150081912BTE3005896 d46aa8-f9e807 n/a Authenticated
10 2150081912BTE3006023 d46aa8-f9e8a7 n/a Authenticated
11 2150081912BTE3002152 d46aa8-f9d511 n/a Authenticated
12 2150081912BTE3006033 d46aa8-f9e89d n/a Authenticated

BANIYAS-01-S# show mac-address

Status and Counters - Port Address Table

MAC Address Port VLAN
------------- ------------------------------- ----
0080ae-13ab71 1 1
98fa9b-33776d 7 1
98fa9b-4017ac 9 1
d46aa8-f9d511 11 1
d46aa8-f9e807 9 1
d46aa8-f9e809 7 1
d46aa8-f9e82b 6 1
d46aa8-f9e89d 12 1
d46aa8-f9e8a7 10 1

BANIYAS-01-S# show lldp info remote-device

LLDP Remote Devices Information

LocalPort | ChassisId PortId PortDescr SysName
--------- + ------------------ ------------------ --------- ------------------
6 | 10.140.18.13 d4 6a a8 f9 e8 2b eth0 eSpace7910.(none)
7 | 98 fa 9b 33 77 6d 98 fa 9b 33 77 6d
7 | 10.140.18.11 d4 6a a8 f9 e8 09 eth0 eSpace7910.(none)
8 | 10.140.18.17 d4 6a a8 f9 e8 e4 eth0 eSpace7910.(none)
8 | 98 fa 9b 3e 4c f3 98 fa 9b 3e 4c f3
9 | 98 fa 9b 40 17 ac 98 fa 9b 40 17 ac
9 | 10.140.18.14 d4 6a a8 f9 e8 07 eth0 eSpace7910.(none)
10 | 10.140.18.6 d4 6a a8 f9 e8 a7 eth0 eSpace7910.(none)
11 | 10.140.18.16 d4 6a a8 f9 d5 11 eth0 eSpace7910.(none)
12 | 10.140.18.12 d4 6a a8 f9 e8 9d eth0 eSpace7910.(none)

In above scenario we have 7 phones only one at port 8 does not work while it is authenticated and LLDP info shows also the IP address.

Are the VLAN configured on the phone statically or dynamically (DHCP or LLDP-MED)

By default, the VLANs are disabled on phones and LLDP-MED is on

Did you test what happens when you make VLAN tagged (just for test)

Could you please elaborate more how this can be done?

Regarding Wireshark captures in our environment it is not possible currently I am working on it to get Wireshark traces.

Most importantly I have noticed one strange thing which may help us 

We have three types of switches / firewall devices

Juniper SRX-340 firewall no issues on phones with or without workstations.

HP 3500yl-24G-PoE+ with firmware version K.15.16.0012m no problems at all with or without workstations.

HP Aruba 3810M-24G-PoE+-1-slot with firmware KB.16.02.0013 (This issue is only here on this switch)

[Emil_G · Accepted solution]

Hello,

Thanks for the new information!

How exactly have you determined that the IP phone is not obtaining an IP address? The phone is producing the message "IP address obtained successfully” and it is also displaying the IP 10.140.18.17 in the LLDP Information.

1. Power up or LAN connected (With log in key pressed)
2. After 2 minutes we get message "IP address obtained successfully."
3. Then after another minute we have "802.1x authentication success."
4. Failed to login 
5. Retrying

Did you check if this IP address is active? Can you ping it? If you check the DHCP leases of the DHCP server can you confirm if and when it was assigned to the IP phone on port 8? Or maybe it was assigned to another device?

It is a bit strange that "802.1x authentication success." is displayed after "IP address obtained successfully." This should be in the opposite order, first 802.1x and then DHCP. But both processes are completed within seconds so maybe it is just a matter of how the messages are designed. Do you see the same sequence of messages on working phones?

I mentioned that I am not very familiar with IP phones. What exactly means a login failure from a networking point of view? You are entering account credentials but the phone is unable to verify them with the SIP server?

Based on this information I don’t think that testing with a tagged VLAN on the port will make sense. If VLAN is disabled the phone will use untagged traffic. LLDP-MED is enabled, but the switch is not configured to advertise a voice VLAN via LLDP-MED, so the phone cannot learn a VLAN ID and configure itself with some tagged VLAN. If you want to test you the port tagged in VLAN 1 with the following command.

Testing-Switch#config

Testing-Switch(config)# interface 8 tagged vlan 1

Testing-Switch(config)#exit

Keep in mind that this may affect the workstation. Some NICs accept tagged traffic and simply remove the tag but some may just drop the whole frame. Restoring the previous config

Testing-Switch#config

Testing-Switch(config)# interface 8 untagged vlan 1

Testing-Switch(config)#exit

You mentioned DHCP snooping before, I reviewed the configuration and don’t think it can be a problem. It is only enabled for VLAN 9 and  not for VLAN 1. So the switch shouldn’t inspect and block any DHCP traffic in VLAN 1.

Like you mentioned the port-access statistics indicate successful authentication so this also cannot be a reason for failing DHCP assignment or failing network communication of the IP phone. Client limit and address limit are set to 3 for both 802.1x and MAB authentication.

BTW initially you reported that the issue is resolved when you remove the port-access configuration from the port. Does this workaround work for port 8?

The MAC table is not having the MAC of the failing phone while it has the MAC addresses of all other phones. This is bit strange especially when we see that there is LLDP information on the switch. But this should not block the communication with the IP phone. Even if there is no MAC entry the switch will flood traffic destined to the phone out of all ports. Not sure in what state was the phone when show tech was taken, maybe it was simply inactive.

You are also writing that you have isolated the issue on this 3810 switch, HP 3500yl and Juniper Firewalls don’t have the issue. At the moment you have 1 phone that is not working on this switch, port 8 while 7 phones are working. But you mentioned 3 phones that you used for the test. Did you have the same issue with 3 phones on port 8 of the 3810? Or only with 1 out of the 3 test phones?

Did you see the issue only on port 8 of this switch or on multiple ports?

Is the phone with MAC d46aa8-f9e8e4 working fine when connected to the 3500yl or an SRX firewall?

Do you have multiple 3810 switches which all have this issue or only a single one?

Both HPE switches you mention have outdated firmware. But since you have issues with the 3810 maybe it would be a good idea to update it. The most current version is KB.16.10.0016

https://h10145.www1.hpe.com/downloads/SoftwareReleases.aspx?ProductNumber=JL073A

Sometimes an update can save you a lot of time of troubleshooting.

So all in all I cannot see obvious issues on the switch. If an update cannot help or is not possible, I think you need to go in depth with Wireshark and it will probably reveal which party is not behaving correctly, phone or switch.

[Adnan004]

Greetings,

My issue has been resolved after upgrading the firmware as per your valued advice.

I read the Fixes part on release notes for KB.16.10.0016 and the first bug with ID 255682 page 15 made it very clear what was the issue here are the little details,

Bug ID 255682

Category 802.1X

Symptom: The RADIUS accounting packets sent from the switch to the RADIUS server do not contain the correct client IP address.

Scenario: This issue occurred when both user authentication and MAC authentication were configured.

After upgrade the issue vanished

How exactly have you determined that the IP phone is not obtaining an IP address? The phone is producing the message "IP address obtained successfully” and it is also displaying the IP 10.140.18.17 in the LLDP Information.

I have checked on phone using display there is no IP addresses on phone but lldp info shows IP. This was random I worked on 15 Switch with problem sometimes lldp show 0.0.0.0 some lldp shows correct IP address but at the same time phone does not show any IPs.

1. Power up or LAN connected (With log in key pressed)
2. After 2 minutes we get message "IP address obtained successfully."
3. Then after another minute we have "802.1x authentication success."
4. Failed to login 
5. Retrying

Did you check if this IP address is active? Can you ping it? If you check the DHCP leases of the DHCP server can you confirm if and when it was assigned to the IP phone on port 8? Or maybe it was assigned to another device?

I tried to ping the assigned IPs as per lldp info but there was no response.

It is a bit strange that "802.1x authentication success." is displayed after "IP address obtained successfully." This should be in the opposite order, first 802.1x and then DHCP. But both processes are completed within seconds so maybe it is just a matter of how the messages are designed. Do you see the same sequence of messages on working phones?

Yes, this is strange for me also but this behavior is only with our HP and Aruba switches while Juniper firewall does authentication first and the assign IP addresses.

I mentioned that I am not very familiar with IP phones. What exactly means a login failure from a networking point of view? You are entering account credentials but the phone is unable to verify them with the SIP server?

Login credentials are fixed and cannot be modified. I have checked with SIP servers when 802.1x was disabled and credentials were verified.

Based on this information I don’t think that testing with a tagged VLAN on the port will make sense. If VLAN is disabled the phone will use untagged traffic. LLDP-MED is enabled, but the switch is not configured to advertise a voice VLAN via LLDP-MED, so the phone cannot learn a VLAN ID and configure itself with some tagged VLAN. If you want to test you the port tagged in VLAN 1 with the following command.

Testing-Switch#config

Testing-Switch(config)# interface 8 tagged vlan 1

Testing-Switch(config)#exit

Keep in mind that this may affect the workstation. Some NICs accept tagged traffic and simply remove the tag but some may just drop the whole frame. Restoring the previous config

Testing-Switch#config

Testing-Switch(config)# interface 8 untagged vlan 1

Testing-Switch(config)#exit

Never did this testing because issue resolved like there was not any issue after upgrading firmware.

You mentioned DHCP snooping before, I reviewed the configuration and don’t think it can be a problem. It is only enabled for VLAN 9 and not for VLAN 1. So, the switch shouldn’t inspect and block any DHCP traffic in VLAN 1.

I was just looking if we could find anything on DHCP snooping but from the very first day the issue was with 802.1x authentication

Like you mentioned the port-access statistics indicate successful authentication so this also cannot be a reason for failing DHCP assignment or failing network communication of the IP phone. Client limit and address limit are set to 3 for both 802.1x and MAB authentication.

BTW initially you reported that the issue is resolved when you remove the port-access configuration from the port. Does this workaround work for port 8?

Yes, it worked 100 % until I applied port-access again.

The MAC table is not having the MAC of the failing phone while it has the MAC addresses of all other phones. This is bit strange especially when we see that there is LLDP information on the switch. But this should not block the communication with the IP phone. Even if there is no MAC entry the switch will flood traffic destined to the phone out of all ports. Not sure in what state was the phone when show tech was taken, maybe it was simply inactive.

You are also writing that you have isolated the issue on these 3810 switches, HP 3500yl and Juniper Firewalls don’t have the issue. At the moment you have 1 phone that is not working on this switch, port 8 while 7 phones are working. But you mentioned 3 phones that you used for the test. Did you have the same issue with 3 phones on port 8 of the 3810? Or only with 1 out of the 3 test phones?

Actually, I have lot of 3810s with number of phones I faced this issue on 15 to 20 remote sites where people reported me the issues but there were more people who did not complained but they had issues.

Did you see the issue only on port 8 of this switch or on multiple ports?

I have seen this issue on all ports randomly one day 2 phones working next 4, somedays 1 and somedays not any single.

Is the phone with MAC d46aa8-f9e8e4 working fine when connected to the 3500yl or an SRX firewall?

Yes, the phone worked fine on 3500yl and Junipers without disabling 802.1x

Do you have multiple 3810 switches which all have this issue or only a single one?

I have a lot in actually in three digits, around 20 users complained and others user were nice or sleeping.

Thank your for your support and patience. I really appriciate that.