Hadoop + VMWare + Nimble = ?

[csatola128]

Has anyone had experience building a hadoop cluster using Nimble? We're building a pilot and testing now. Of course, most everything you read from Apache and Hortonworks tells you that it's designed for local disk commodity machines and that you shouldn't use SAN or Virtualization tech. Having said that, so far, our tests are looking very promising. Just curious if anyone else has started playing with this yet and what your experiences are.

[sdaniel47]

I have not set up Hadoop on Nimble, but I'm very interested in how it goes. As you have news, keep us posted.

Thanks.

[chris24]

Hello,

We have a few customers running hadoop ontop of VMware, however best practice is to present the Nimble volumes directly Nimble Storage for Hadoop 2.x on Oracle Linux and Red Hat Enterprise Linux 6

Many thanks,

Chris

[csatola128]

Thanks Chris. I'll give that a read. Feedback for you though... to download that paper, you need to fill out a few questions. One is "Who is your current storage vendor". I'm surprised Nimble isn't on the list... Had to select "Other"

To update, our pilot so far has been very successful. Our current "commodity hardware" stack is 2 Name Nodes and 10 data nodes with 32GB RAM and 4x 750GB 7200RPM SATA drives. Our "pilot" stack is 2 Name Nodes and 12 data nodes with 32GB RAM and a single 400GB data drive VMDK attached running in a CS700. With that configuration, our jobs run 12.5% faster on the virtual hardware than the physical.

Reading the "Best Practices Guide" above, it seems geared towards environments where the Nimble is being used with iSCSI. We're on Fiberchannel. Do you have any thoughts there? So far, even going against best practice, it's looking really good that virtual beats physical.

[chris24]

Marketing follow an odd line of logic I can not explain I will be sure to pass it on.

On infosight you will also find some additional resources Nimble Storage InfoSight in the document titled Nimble Storage for Hadoop®, Vertica®, Splunk® and MySQL™ on Linux this has custom settings defined.