Building New Core Network for 10Gb iSCSI SAN - Recommendations

[stewea]

Hi,

I wonder if anyone can help me specify some new switches for a new project I'm working on.

I have been given the chance to build a new server and network infrastructure, so want to make sure its done right.

I will be installing a SAN a HP P2000 G3 10GbE iSCSI along with a few HP DL380s with 10GbE.

For core switching I wanted to go modular and have selected the HP 5406zl (J9866A).

For edge switching to the desktop I've selected 2920-24G Switch J9726A with HP 2920 2-port 10GbE SFP+ Module (J9731A) as they fit the budget. I'm looking at 4 2920s at present.

So the plan would be to have 10GbE iSCSI trunks to the servers and SAN and 10GbE data trunks from servers to core switch.

And 10GbE SFP+ fibre trunk between the 5400zl and 2920-24Gs HP 8-port 10GbE SFP+ v2 zl Module and using HP 2920 2-port 10GbE SFP+ Module (J9731A) over 50/125 OM3.

First off, is this a compatible configuration?

Second, can I apply different VLANS to the 2920s so I can:

1) Separate desktops from printers, with desktops getting one ip range from dhcp and printers getting another.

I know printers are notorious for generating lots of chatter, so I want to keep them separate.

Is it a case of ip helpers on the VLANS pointing to the dhcp server.

2) I'd like to use a 10.x.x.x class of address, but subnetting is not my strong point.

The site is never going to have more than 20 servers, 30 printers or 80 desktops.

I'd like servers in one range, desktops in another, and printers in another.

There are also 20 VPN users would I'd like in a different range.

Some recommendations on this would be gratefully received.

[Vince_Whirlwind]

Sounds fine - don't try anything fancy with the subnets, there's no point.

First, rack your switches and connect them up. You have a star topology, with a 5400 "Core" in the middle, and 4x 2920s patched in between each other using 10GB SFP+ transceivers.

Create three VLANs on your "Core".

VLAN 10 name SERVER

VLAN 20 name WORKSTATION

VLAN 30 name PRINTER

VLAN 99 name MANAGEMENT

Configure an IP address for each VLAN

VLAN 10    

   ip address 10.1.10.1 255.255.255.0

VLAN 20    

   ip address 10.1.20.1 255.255.255.0

   ip helper-address 10.1.10.42

VLAN 30    

   ip address 10.1.30.1 255.255.255.0

   ip helper-address 10.1.10.42

VLAN 99

   ip address 10.1.99.1 255.255.255.0

I assume the Servers are patched to the "Core"?

Configure the Server ports for the Server VLAN:

VLAN 10 untag A5,A6,A7,A8

Now add the VLANs to the "Core" uplinks to the switches

VLAN 20 tag A1,A2,A3,A4

VLAN 30 tag A1,A2,A3,A4

VLAN 99 tag A1,A2,A3,A4

Now get on each edge switch and create the WORKSTATION, PRINTER & MANAGEMENT VLANs.

On the edge switch uplink port, configure VLANs 20,30,99 as tagged.

Give each edge switch a management IP address in the 99 subnet.

That's your basic network.

You then need to add a default GW to your "Core" pointing at your gateway router/firewall.

Not sure how your VPN users fit into this. Presumably they are coming into your DMZ somewhere?

[stewea]

Hi Vince_Whirlwind,

Thanks for the info, your spot on with what I'm looking to do.

I have a couple of questions.

The ip address assigned to each VLAN, is that an address in the range, but not the broadcast address or an address in the dhcp range. So if your dhcp range was x.50 - x.253  you could use x.254 or x.1 like you have detailed.

The firewall is a ZyXEL ZyWALL USG 300.

As its currently in use on the production network, I'll look to setup a connection to the secondary interface.

Give it an address of 10.1.100.1 or something like that.

So just point the Default Gateway on the 5406 to 10.1.100.1

Regarding the VPN users, I'd like the firewall to manage that. I dont have access to it yet, but its a ZyXEL ZyWALL USG 300. Thats one of the last jobs. Prehaps I'll give it a 10.1.50.x /24.

You dont know anything about the MSA 2000 10Gb iSCSI SAN do you?

I believe the 10GbE iSCSI interfaces on the back of the controllers are SFP+.

Whats the best way to connect that into the 5406?

Also I take it I need to create VLANS for the iSCSI too?

Cheers,

Steve

[Vince_Whirlwind]

The IP address you assign to each VLAN is the "Router" for that VLAN's subnet. The DHCP scope has that "Router" address configured as an option. The PC's then receive that "Router" option as their "Default GW".

(Which reminds me, I forgot to mention you need a default GW address in the management VLAN configured on each of the Access switches).

Yes, create a new VLAN, VLAN100, and give it 10.1.100.9. Configure a port to be "untagged" in VLAN 100, and patch that port to the firewall, where you configure it as 10.1.100.1. Then configure the Core with default GW pointing at 10.1.100.1.

I don't know whether you need iSCSI VLANs, or whether that can all just go on the server VLAN. I can't picture how that is going to work. Just thinking about it, you have some servers on the network, using 10Gb ports, then their storage is also attached to the network on 10Gb interfaces? Maybe there are some whitepapers relating to that product which give an indication of a best practice for how to put it together.

You only have 8 10Gb interfaces to play with, right? It's not many. It also means your entire network depends on a single module in your switch. I've used those 8-port modules from very soon after they became available and I don't think I've ever had a failure.

[stewea]

---

@Vince_Whirlwind wrote:

The IP address you assign to each VLAN is the "Router" for that VLAN's subnet. The DHCP scope has that "Router" address configured as an option. The PC's then receive that "Router" option as their "Default GW".

Ok, that makes sense now.

---

@Vince_Whirlwind wrote:

(Which reminds me, I forgot to mention you need a default GW address in the management VLAN configured on each of the Access switches).

---

Can you explain this a bit more?

---

@Vince_Whirlwind wrote:

Yes, create a new VLAN, VLAN100, and give it 10.1.100.9. Configure a port to be "untagged" in VLAN 100, and patch that port to the firewall, where you configure it as 10.1.100.1. Then configure the Core with default GW pointing at 10.1.100.1.

---

Thinking about it now, I should bring all the VLAN ip addresses into line, so I might set the VLAN 100 address to 10.1.100.1 and set the firewall ip address to 10.1.100.10

---

@Vince_Whirlwind wrote:

You only have 8 10Gb interfaces to play with, right? It's not many. It also means your entire network depends on a single module in your switch. I've used those 8-port modules from very soon after they became available and I don't think I've ever had a failure.

I'm going to put in 2x 8 10GbT and 2x 10GbE SFP+ modules and split the servers and SAN controllers over both. 

Thanks for your advise so far.

I'm no network expert, just a jack of all trades.