Internal Server Error when viewing OpenFlow Monitor page

[JGrider]

Hello,

I just upgraded to v2.2.5, transfered my license, and reconnected my switches. Now whenever I view the OpenFlow Monitor page I get the following error:

Server Error - 500: Internal Server Error

URL: /sdn/ui/ctl/app/rs/of/datapaths

View ID: ofmon

Type: error

Any help or guidance would be appreciated.

Thanks,

Jeremiah

[JGrider]

UPDATE:

It is only one particular switch that I have that is causing the issue. I have a HP 2920-24G, a Pica8 3290, 2 Extreme Networks Summit X440s, and 3 TPLink home routers running OpenWRT with OpenFlow extensions. All will connect to the controller without issue.

I have a Centec V350 that is an OF v1.3 switch that previously worked fine with the v2.0 controller. However, since upgrading to v2.2.5 everytime it attempts to the controller the OpenFlow Monitor page has the internal error. If I try to pull the datapaths JSON from /sdn/v2.0/of/datapaths I get a 'Resource not found' message.

Any further guidance as to what I can look at in the logs to see what is causing the issue would be appreciated.

Thanks,

Jeremiah

[ShaunWackerly]

Hi Jeremiah,

I don't believe that the v2.2.5 release supports OF 1.3 on any devices other than ProCurve devices (2920, 3500, 3800, 5400, 8200). Do you see any errors/exceptions in the controller's virgo log? Those might indicate the root cause of why the monitor is having a problem when this OF 1.3 device is connected.

Shaun

[JGrider]

Shaun,

I have a Pica8 3290 that negoitated OFv1.3, as well as an HP 2920, with the VAN controller. Previously on v2.0 the Centec V350, Pica8 and HP 2920 all negoitated OFv1.3. So the controller seems to be working with non-HP hardware. I did change the General/Configurations for com.hp.sdn.ctl.of.impl.ControllerManager so that hybrid.mode is false versus true and restarted the controller. I encountered the same error again.

Attached is the last log message in the virgo server log.log file. Looks like a parsing error and that the VAN controller was expecting a different header size.

My follow up to this would be, is this a controller issue or a switch/vendor issue? Since the switch previously worked with v2.0 controller software was this parsing changed in the v2.2.5 to ensure OF message specification compliance? Just need to know if I need to ask the vendor/Centec to change something or if Centec is compliant with OF messaging, does HP need to change something.

Thanks again,

Jeremiah

---

[sdnindia]

Hello Jeremiah,

Please check the firmware version you are using and upgrade to the new Firmware  release K/KA/WB.15.15.0006. The Web released software can be accessed from this link  www.hp.com/networking/support

Enter a product # or description – e.g 5406  check the box for the 5406

Click on “display selected”

Click on the link “software downloads”->It will take you to:

https://h10145.www1.hp.com/downloads/SoftwareReleases.aspx?ProductNumber=J9642A

Sign in with a passport account to download the software( specifically that mentions "Early Avaialbility") 

EA –  Early Availability – should be evaluated prior to moving it to a production environment

GA – General Availability – considered production ready.

Please do let us know if you are using the latest firmware already.

Thanks,

HP SDN Team

[ShaunWackerly]

Hi Jeremiah,

I think you'll need to contact the Centec V350 manufacturer because it looks like they're using vendor extensions and the header length isn't divisible by 8. From the log snippet you posted:

[2014-04-09 02:20:50.654] WARN  of-io-28-thread-3            hp.of.msglib                                                     DE0005I Exception was: TFF:java.nio.DirectByteBuffer[pos=146 lim=480 cap=65536] > com.hp.of.lib.MessageParseException: AF:java.nio.DirectByteBuffer[pos=146 lim=480 cap=65536] > com.hp.of.lib.MessageParseException: AF:java.nio.DirectByteBuffer[pos=146 lim=480 cap=65536] Unexpected experimenter header length (should be div by 8): 10 

This message in the virgo log indicates that the HP openflow library wasn't able to parse the experimenter header because the header length was 10. The openflow standard indicates that experimenter headers should have a length divisible by 8. Specifically, in the Openflow 1.3.2 spec you can see this at the very end of section 7.2.5:

/* Action header for OFPAT_EXPERIMENTER.
* The rest of the body is experimenter-defined. */
struct ofp_action_experimenter_header {
uint16_t type; /* OFPAT_EXPERIMENTER. */
uint16_t len; /* Length is a multiple of 8. */
uint32_t experimenter; /* Experimenter ID which takes the same
form as in struct
ofp_experimenter_header. */
};
OFP_ASSERT(sizeof(struct ofp_action_experimenter_header) == 8);

[JGrider]

Hi there HP SDN Team,

Thank you for taking the time to respond to my question.

However, I am not sure if you read my issue correctly. I am having issues with connecting a NON HP switch to the version 2.2.5 controller. The switch inparticular is from Centec Networks, model V350. It is a pure OpenFlow switch, in that there is no traditional L2/L3 networking software on it. I have 3 other NON HP switches connected to the version 2.2.5 controller. These are from Pica8, Extreme Networks, and TPLink. These are all functioning correctly.

The Centec Networks, model V350, switch previously connected to the version 2.0 controller without issue. Now at version 2.2.5 the OpenFlow Monitor webpage reports the internal server issue and the REST response is unavailable.

Looking into the logs there is a parsing error on one of the openflow messages. Not sure if/how this parsing handling changed between versions, so I need a little help to understand if the issue is with HP or with Centec. Moreover, regardless of whether or not the openflow message from the Centec Networks switch is non compliant, or if the parsing handling on HP's side was changed incorrectly; I would certainly expect the SDN controller to NOT loose some necessary capability. If not then it appears to be a perfect way to shutdown the controller by merely sending it a malformed OpenFlow message.

Thanks,

Jeremiah

---

@sdnindia wrote:

Hello Jeremiah,

Please check the firmware version you are using and upgrade to the new Firmware  release K/KA/WB.15.15.0006. The Web released software can be accessed from this link  www.hp.com/networking/support

Enter a product # or description – e.g 5406  check the box for the 5406

Click on “display selected”

Click on the link “software downloads”->It will take you to:

https://h10145.www1.hp.com/downloads/SoftwareReleases.aspx?ProductNumber=J9642A

Sign in with a passport account to download the software( specifically that mentions "Early Avaialbility") 

EA –  Early Availability – should be evaluated prior to moving it to a production environment

GA – General Availability – considered production ready.

Please do let us know if you are using the latest firmware already.

Thanks,

HP SDN Team

---

[JGrider]

Shaun,

Thank you very much for detailing the specification mismatch. I will work with Centec to get this corrected. Was this parsing check changed in the v2.2.5 software, because the switch worked with the v2.0 software? Not sure if you work for HP or not and can provide anymore information. Additionaly should there not be a better way to handle errors such as this, other than making controller resources unavailable? Curious if I can cause similiar issues with malformed openflow message for other messages types, such as flow, etc.

Thanks again, greatly appreciate the response!

Jeremiah

[ShaunWackerly]

Hi Jeremiah,

You are correct that the proper response to a malformed OpenFlow message should not be to make the TopologyViewer (or other controller services) unavailable. The proper response is to log the error and keep the service up. It sounds like the controller is not doing that from your original description, so there may also be a bug in the controller. Once the service becomes unavailable, what makes it become available again?

Shaun

[sdnindia]

Hello Jeremiah,

Could you please check the value of “msg.parse.strict” set in the ControllerManager properties?  You can find it under Configurations -> com.hp.sdn.ctl.of.impl.ControllerManager.

A value of “false” indicates it is not enabled.  However, if it has been set to “true”, we may see the failure reported since some switches don’t implement OF 1.3 completely.

Thanks,
HP SDN Team

[sdnindia]

Hello Jeremiah,

There have been no recent changes to the OpenFlow Message Library code between v2.0 and v2.2.5 with respect to the Experimenter actions.  It looks like something may have changed with how the switch formats its Experimenter actions because if the length in the header field had been 10 (not divisible by 8) (v2.0), we would have failed in v2.0 too.

Having a parse failure of a message should not break the UI.

 To investigate further, could you please  provide the following:

• The URL that produced the error message.  It should be something like:

https://<ip>:8443/sdn/ui/app/index#ofmon

https://<ip>:8443/sdn/ui/app/index#ofmon-dp-flows,00:00:00:00:00:00:00:01

• What browser was being used?
• What version of that browser was being used?
• The complete log.log file from the virgo directory.

Thanks,

HP SDN Team

[SudhaHP]

Hello Jeremiah,

HAs there been any update on the issue?

Could you also provide us the following information that 'sdnindia' requested

• The URL that produced the error message.  It should be something like:

https://<ip>:8443/sdn/ui/app/index#ofmon

https://<ip>:8443/sdn/ui/app/index#ofmon-dp-flows,00:00:00:00:00:00:00:01

• What browser was being used?
• What version of that browser was being used?
• The complete log.log file from the virgo directory.

We are looking into ways to prioritise and investigate the issue ? 

Thanks

Sudha Krishnamurthy

GPLM