Openflow HP ProCurve 6600-24g-4xg

[gojkog21]

Hi,

I'm Msc student on faculty of electrical engineering. I have to create an OpenFlow testbed which might be consisted of more Openflow instances. So, could anyone tell me is it possible to configure one HP ProCurve 6600-24g-4xg switch to work as more Openflow switches which can communicate to each other, and be viewed by the controller as separated openflow switches (although it is one phisical openflow switch)??

I found that is possible to implement more openflow instances using virtualisation mode, but if I understood correctly they cannot be configured to communicate to each other...
On the other hand, I read one research paper where it seems that authors managed to make data center topology from a single HP procurve switch.

Can you provide me some hints about how can I do the same, or just give me any feedback so that I can know is that possible or not?

Thanks.

[ShaunWackerly]

Hi gojkog21,

When using virtualization mode on the 6600, you'll be able to associate 1+ vlans with an openflow instance, then configure multiple openflow instances per switch. Depending upon how you interpret "configured to communicate with each other", it may be possible.

In a software-defined network, there is a control-plane (controller-to-switch communication) and a data-plane (end-user to end-user communication). When end-user traffic flows from one vlan to another (such as when the OF switch is a router), that will effectively make traffic flow from one OF instance to another (if those vlans are in different instances). From the controller's standpoint, the controller can also issue a packet-out message on one instance and it will receive a packet-in message from another instance (if the vlans are connected together). In this way, instances can communicate with eachother (send packets) over the data-plane.

Instances on the same switch will not communicate with eachother over the control-plane, since doing so would be non-optimal because they are implemented on the same switch CPU.

Does this answer your question? If not, could you please provide more specifics?

Shaun

[gojkog21]

Hi ShaunWackerly

First of all I want to say thank you, because your answer helped me!

I've created 5 VLANs, then assigned ports to that VLANs, and created 5 OF instances (every OF instance have 1 VLAN). When I connect switch with controller, it can see 5 instances. Hosts, which are connected on different instances, can communicate each other , but there is a problem: Controller gives a next error:

Error: OFPFMFC_EPERM (2)

I'm using pox controller, and I think that my controller wants to make flow entries in flow table 0, but hp doesn't give a permission for that.

Can you tell me your opinion about that problem, please?

All the best,

Gojko

[ShaunWackerly]

Hi gojkog21,

It is expected behavior that you're unable to write a flow to table 0. The only flow that HPE/Aruba switches allow in table 0 is a flow which matches all packets and uses a GOTO_TABLE instruction. We did this because when you change the pipeline-model (we have standard, ip-control, and custom) then it will change the first table in the pipeline. You can also enable other tables using the "openflow instance X dst-mac-grp-table" and "openflow instance X src-mac-grp-table" commands.

HPE/Aruba wanted static numbering between the standard and ip-control, so to achieve that we implemented table 0 being a branch point, rather than an actual forwarding table (to remain spec compliant).

Assuming you are using the standard pipeline-model, you'll want to write all of your flows into table 100. However, be sure to adhere to what the switch reports in its TABLE_FEATURES reply, which is where the supported matches and instructions are detailed by the switch. If anything is not supported in table 100, you may need to write to table 200 (which is a software-based table with more features but much lower performance).

Shaun

[Abhay_B]

Hi Gojko,

You can also change the table number for Policy Table from its default value of 100 to 0.

Please find the syntax to do the same.

switch(config)# openflow instance <instance-name> table-num policy-table 0

If your controller is installing a flow that cannot be accomodated in the Policy Table, then you can set the S/W table to be first table in the OpenFlow pipeline. It offers extra set of match and action capabilities at considerabily lower performance (a maximum of 10000pps).

Please find the syntax to do the same. 

switch(config)# openflow instance <instance-name> table-num sw-table-1 0

You can find more details on page 20 of the OpenFlow Administrator Guide for 16.02 release.

Thanks!

Abhay

[gojkog21]

Hi Abhay_B,

I tried with command you gave to me, but part table-num is not supported there...

HP-6600ml-24G-4XG(config)# openflow instance vlan10 table-num policy-table 0
Invalid input: table-num

P.S vlan10 is the name of instance... When I type ? after openflow instance vlan10, table-num is not listed..

Do you have any other instructions, or idea to solve that??

Best regards,

Gojko

[ShaunWackerly]

Hi Gojko,

Which version of firmware are you using? I believe Abhay's response above was assuming you were using a relatively recent firmware release. Our latest release is 16.02, so if you download that then you should see the 'table-num' option available.

Shaun

[gojkog21]

Hi Shaun,

I've updated firmware to K_15_18_0013. Now I can see table-num option, but when I try to change policy-table to 0, there arrived message: inconsistent number. 

I have other problems after upgrading firmware. Now I can't use telnet remote access unitl I login using serial connection and ping my lap top from switch first. The other problem is that hosts which are connected on the same, or different instances can't comunicate each other.  On old firmware that was possible.

Any solutions for that, please??

Gojko

[Abhay_B]

Hi Gojko,

Can you please try an "erase startup-config" on the switch for resolving the "Inconsistent number" issue..

Also, can you please share the output of the "show openflow instance <name> flows" command to investigate the issue with hosts not being able to talk to each other.

Thanks!

Abhay

[gojkog21]

Hi,

I've tried with erasing startup config, and I can type command for changing table-num.... But there is a lot of problems now.

I removed all instances, and create only one. I've connected 2 hosts on that instance, but thay can't communicate each other. I'll attach you error, which I see on my controller, and packet which I recorded on Wireshark. Controller can't create flow table in my switch, exactly it creates flow with action 'output port 65531 or 65535'.

This is result of show config command:

HP-6600ml-24G-4XG# show config
Startup configuration: 4

; J9264A Configuration Editor; Created on release #K.15.18.0013
; Ver #0d:10.00.34.00.24.6b.ff.f7.fc.7f.ff.3f.ef:c2
hostname "HP-6600ml-24G-4XG"
module 1 type j86wwa
module 3 type j92zza
snmp-server community "public" unrestricted
openflow
controller-id 1 ip 10.0.0.3 controller-interface vlan 1
instance "vlan10"
member vlan 10
controller-id 1
enable
exit
enable
exit
vlan 1
name "DEFAULT_VLAN"
no untagged 1-4
untagged 5-24,25-28
ip address 10.0.0.4 255.255.255.0
exit
vlan 10
name "VLAN10"
untagged 1-4
no ip address
exit

Flow table for that instance is in attach.

Any ideas??

All the best,

Gojko

[Abhay_B]

Hi,

One observation on your configuration. You have configured your instance to run OpenFlow 1.0. Is that what you want to use or do you want to use OpenFlow 1.3?

The table-num CLI I suggested is only applicable for 1.3 instances. 

The 6600 runs the v1 generation of ASICs on which we don't support the OFPP_FLOOD output action. The v2 and v3 generation of switches support this action. Also none of the HPE Aruba switches support the OFPP_NONE output action.

Which controller are you using? To enable the switch to forward traffic between the 2 hosts, you need to install appropriate rules on the OpenFlow flow table. You could start by adding two rules on the instance:

a. Match = Eth_type=ARP,  Action=OFPP_NORMAL and

b. Match = Eth_type=IP,  Action=OFPP_NORMAL

This should enable ARP resolutions and IP forwarding between the hosts to be processed by the switches non-OpenFlow pipeline.

Thanks!

Abhay

[gojkog21]

Hi,

I'm using pox controller which supports only OpenFlow 1.0. 

When I was on old firmware (that was K_15_10), I was able to connect switch with pox controller. Communication between hosts were able to, and controller was installing rules in flow table, but with error:

Error: OFPFMFC_EPERM (2)

This error message was something that I thought  to solve...

I'm on firmware K_15_18 now. I configured switch as like as I do it with old firmware, but now I can't connect switch with pox controller. In that way, controller can't create rules in flow table, I got error messages which I attached in last message, and hosts can't communicate.

I don't want to create static entires... Could you tell me is it possible to connect that switch with pox controller without that error messages? If it is possible, how to do that?

Thanks you!

Gojko

[Abhay_B]

Hello Gojko,

As I stated in my previous response, HPE-Aruba switches don't support the OFPP_NONE action. Also, the OFPP_FLOOD action is not supported in hardware on v1 generation of ASICs. I believe the pox controller is trying to program flows with these actions that aren't supported.

Will it be possible for you to attach the packet trace between the switch and the controller as a .pcap file. It would be easier for us to analyze the exact flows which are erroring out.

On a side note, is there a reason why you want to continue using pox as the OpenFlow controller? There are other OpenFlow controllers available in the market today that support OpenFlow 1.3 including ryu and HPE VAN.

Thanks!

Abhay

[gojkog21]

Hi,

I understood that FLOOD ACTION is not supported on hardware. That is how pox works, but I've tried with ONOS contrller, and I have the same problem. (That means that onos using flood??)

I've updated firmware to K_15_18_0013 version, as I described below, and I try with OF version 1.3, but it can't work.

So, I need to know, on which controller FLOOD ACTION can be escaped? As I understand, in every cases of configuration, FLOOD ACTION can't be added in hardware flow table, and switch will forward using software with too bed performances.

I have four instances, and with software flow table I can't reach 30 Mbits/s on Gigabit interfaces...

I have .pcapng files, but I can't attach it:

''The attachment's hp_1_inst.pcapng content type (application/x-pcapng) does not match its file extension and has been removed.''

All the best,

Gojko

[Abhay_B]

Hi Gojko,

As you understand now, there is no way to get rules with OFPP_FLOOD as an action to work in hardware on the 6600 irrespective of the version of OpenFlow used.

The choice of using OFPP_FLOOD is purely based on the controller implementation and would vary from controller to controller. I don't have sufficient idea on various controller implementations in the market today to comment on controllers which don't use the OFPP_FLOOD action. I know for a fact that the HPE VAN controller doesn't make use of it. You may want to consider it as an alternative.

If you want to use the POX and ONOS controllers, then you should evaluate an option to move to a newer model of an HPE Aruba switch (http://arubanetworks.com/products/networking/switches/). The v3 based switches offer a lot of new OpenFlow features in hardware including the capability to create an OpenFlow pipeline of your choice.

Thanks!

Abhay

[gojkog21]

Hi Abhay,

First of all, thanks for answring!

Could you tell me is it normal to have bandwidth only 30Mbit/sec when I use software flow table (in the case where packets are forwarding through three instances)?

And I need to know one more thing: If I modify control application on my controller such that flood action will not be used, would that resolve my problems? Can I configure switch to use hardware table than?

Thanks,

Gojko

[Abhay_B]

Hi Gojko,

Yes, it is expected that you see a much lower throughput when the flows are in software. The exact numbers would vary based on the software rate limit set per instance (which is in pps) and the packet size on the data plane.

There are a number of match fields and actions that are not supported on v1 hardware. So just changing the controller application to remove the OFPP_FLOOD action may not be enough if it also uses any of the other unsupported match or action fields.

v1 hardware supports the following match fields in h/w - {IN_PORT, VLAN_ID, ETH_TYPE, IP_SRC, IP_DST, IP_PROTO, IP_DSCP, TCP/UDP SRC, TCP/UDP DST}.

v1 hardware supports the following actions in h/w - {OUTPUT=PHYS_PORT/NORMAL, SET-FIELD {VLAN_PCP, IP_DSCP}}.

Also, in case of 1.0, if a rule acceleratable in hardware is inserted with priority lower than an already installed rule that is in software, the new rule will also sit in software with reason description stating a higher priority rule is in software.

Thanks!

Abhay

[gojkog21]

Hi Abhay,

I set limit software-rate to maximum (10000 pps) and checked speed between 2 hosts using iperf. Throughput is 30 Mbit/sec when packets're going through the three instances on TCP. On UDP it takes only 1Mbit/s.

Is it too big reduction, or expected?

P.S When switch works in L2/L3 mode, the bandwidth is around 1Gbit/s.

Thanks,

Gojko

[Abhay_B]

Hi Gojko,

I have never personally tried testing with iperf so I am not aware of the various factors that could contribute to the performance in the test runs.

Can you please share the flows installed in the instance during the test runs. It would be good to understand if the flows are installed in h/w or s/w. Also any difference between TCP and UDP based tests.

In pure L2/L3 mode, the traffic would not be switched/routed via CPU but would be done via the ASIC and hence the performance would be line rate.

Thanks!

Abhay

[gojkog21 · Accepted solution]

Hi,

That was my mistake on iperf for UDP. It's OK now.

I solved a problem with hardware flow-table with small modification on controll application. Flood actions weren't a only problem. That was math eth field too.

So, the problem is solved.

Thanks for support!