SOLVED
G'day!
Your story sounds exactly like mine. We just purchased and installed a CS300 (36TB + 3.2TB), 10G iSCSI and 4x R630 VMware hosts all at 10G.
We already had a pair of Extreme Networks X670V switches (stacked @ 160Gbps) so we had plenty of 10G available. While Nick is correct, most traffic will probably suffice on 1Gb, where 10G truely shines is vMotion (& storage vMotion when migrating data onto the Nimble)
Yes, 10G is backwards compatible to 1Gb so I'd highly recommend making the host NICs 10G from day 1. You'll have to talk to your sales rep about the cost for 10G vs 1Gb cards in the Nimble controller, again, if the cost is minimal I'd suggest doing this from day 1 (having left over parts from upgrades is just another pain). This will just leave to to purchase some 10G switches further down the track.
We have CommVault already (this was doing our backups & replication). Had I not managed to convince the boss to buy CommVault, Veeam was our next choice 
Like you, I was very conscious of the 'whole eggs in one basket' concern but many, many, many hours of research convinced me I'd be fine. We're planning the purchase of another CS300 for a remote office (and replication site) now.
Can't comment on the spare parts, haven't had anything go wrong yet
pokermunkee wrote:
My VAR is recommending we use 1GbE instead of 10GbE since our requirements don't justify the costs of 10GbE....with having multiple 1GbE ports trunked and VMware's MPIO, performance will be fine they tell me. Is this sound advice or should I bump up to 10GbE? Could I order my servers with 10GbE but connect to 1GbE ports to future proof? I believe 10GbE is backward compatible, but is that OK for connecting to the iSCSI switches? Not sure how much more 10GbE controllers are over the 1GbE ones on Nimble. Upgrading on Dell servers is less than $1K.
I've found a lot of customers are stepping up to 10GbE switching when purchasing Nimble arrays. Depending on your vSphere licensing and Host config you could significantly cut down on the number of cables a Host requires with a dual nic card or two. In the past I've dealt with mixed 1GbE and 10GbE iSCSI and I'll be honest, it always left me suspicious when looking into latency or networking issues, I really wish I would have had consistent bandwidth (all 1GbE or all 10GbE) just to rule that out.
Having all of my eggs in the Nimble basket makes me nervous, won't lie. I have never had an outage in the 7 years I've managed this environment, all with local storage. Without replicating to another Nimble, what's the most cost effective way in having a Disaster Recovery Plan in the event of a complete Nimble outage? I'll probably switch from Unitrends to Veeam for our backup solution and I've heard Veeam has a decent replication feature at no additional cost. Does anyone use Veeam as their DR plan?
In the event a controller fails or we failover on purpose, is an outage created? Would SQL or Exchange have a problem?
Is there a single point of failure on a Nimble unit? I can't find any horror stories online with a unit going completely down. Has anyone seen one go down? What spare parts are good to keep onsite?
Unitrends and Veeam both have the ability to create NFS shared storage from their repositories so you can fire up protected VMs immediately without having to restore them to new storage first. Obviously you sacrifice some performance in order to gain this ability but its a life saver when you find yourself in that situation. I've used the feature more for testing and running through mock DR scenarios to ensure the recover-ability of my VMs - always a good idea to be prepared for a failure ahead of time!
Unitrends and Veeam both have replication features built into the base products but Veeam has a WAN accelerator that can be utilized with their Enterprise Plus licensing.
In my experience Nimble has provided the fastest controller failovers of all the arrays I've worked with. vSphere, MSCS, SQL, and Exchange have all continued to function with no hiccup when failing over controllers during controller maintenance. I think the key here is that you're following Nimble best practice guides and utilizing Nimble's integration kits when appropriate.
Congrats!
Sounds like you've got some projects to look forward to - good luck on those!
In the meantime I'd recommend reading up on Nimble best practice guides since it sounds like you'll have some fresh Switches and Hosts to work with - your VAR or Nimble SE should be able to get some material together for ya.
Well, look what showed up today! 
Still amazes me, so much power in just 3U