software raid 0 or LVM

[iinfi1]

hi all
i am not sure whether this topic has been discussed here before...
well in our test servers we already have two disks shown from the SAN
one is 100G and another one is 50G on vRAID5 of eva4400
I dont want to touch the storage as of now. the dba has asked me if he can get a single 150G partition. this will be used for oracle test instance of Oracle Apps and Database. Performance is not an issue as only 10-12 tech consultants will be using it.
I am wondering whether I should do a software raid 0 of the two disks or i should use LVM.

please suggest.
in both cases even if one LUN fails the whole data is gone. isnt it?

thanks

[Emil Velez]

If the lun from the disk array is Raid 5 or 1 then if there is a failure the disk array will take care of it and you just replace the disk and be done with it.

No disk array or OS interruption.

if the lun in the disk array is raid 0 and there is a failure you will have to rebuild the raid set if there is a failure and then have a interruption at the OS.

Thats the nice thing about RAID.

Heck nowadays with the blades and boot from san you do not even have to mirror the boot disk any more since the Blades have a RAID controller and with boot from san the boot disk can be Raid 1 or 5

[iinfi1]

hi thank u for the nice explanation.
Could u also please throw more light on my first question?
Whether i am better off creating a LVM or software RAID in the above mentioned scenario

[Viktor Balogh]

Hi,

>we already have two disks shown from the SAN
one is 100G and another one is 50G on vRAID5 of eva4400

Since the LUNs are vRAID5 on the SAN level, you can go with LVM and build a simple non-mirrored volume from it.

Not really sure what you mean by software raid, with LVM or other volume manager you can achieve a software RAID.

Regards,
Viktor

[Alzhy]

Use LVM.

Just concat the 100 and 50 G "already RAIDED" physical volumes from your SAN as a single 150GB LVM LVOL.

No need to softwre RAID or any kind of RAID as the EVA already stripes your 100 and 50 GB RAID disks accross however many disks you have in your EVA4400.

[TwoProc]

Stripe across them. Use lvm with distributed stripes. If you use just raid zero, then your stripe sizes will be small. I think you'll be better off with the larger stripe sizes given by distributed stripes of lvm. An exception would be if the number of drives (physical drive count) that the EVA is giving you for the two containers is small then I'd go with just plain Raid 0 as your best chance. But! because you're on R5, with all your stuff on just two drive containers, it's gonna be slow regardless, IMHO. Unless, you've got a large chunk of ram in the servers you make, so that he can cache up a lot of I/O in the db_block_buffers area.

[Alzhy]

TP,
if he does a software raid0 (md) of the 150G and 50G luns - he'll end up with just 100GB. Plus I don't THINK it matters.

These EVA "disks" are already "striped" (RAIDED) behind the EVA -- albeit RAID5... it will still be "fast"

so PV1 - 150GB, PV2 50GB...

Carve an LVM VG out of this two.
Carve an LVOL of 150 or even 200GB out of this 2.

Maks no diffy.

[TwoProc]

Like I said, it depends how many physical hard drives are behind the array(s). If it is a low amount 10 or less, I'd definitely use a plain ol' stripe. If it is a lot of drives, that is multi chunks of multiples of 14 drives across three or four EVA multi racks, across duplexed controllers, etc. I'd follow the rules of the paper produced about 6 years ago which expound the idea of SAME, which would recommend large chunks being pulled, 1Mg or larger at a time.

Is it gonna matter for 12 technical consultants? Probably not nearly as much as how bad the consultants' code for their mods will probably be. This is because most of this development for the consultants (either in setup, or actual code mods) will run off of the "vision" demo database, which don't have enough rows in them to present any type of reality for what that system will look like when finally done.

Like most things, your mileage will vary.

The reason I answered in detail is, he bothered to ask. If he gets used to using a standard set of methods in deploying these, he will learn, bit by bit, how and why what works best for his environment as he moves toward go-live. It's a pretty bad thing to to go all the way to go live, and have had no experience or feel of sucesses/failures in setup for go live.

Therefore, only out of admiration for the sysadmin caring enough to take a decent starter shot at a set up, I thought I'd offer what I thought would be best for the solution, even if the results of the outcome for this single decision may be only be incremental.

Besides, even though small, its still how I attempt to set them up if and when I have the resources to setup test/dev systems. Sometimes, due to resource constraints, I just don't get to.

[Viktor Balogh]

Hello,

I'm with Alzhy: I don't see the point for setting another level of striping on the top of the RAID5-out-of-the-box. The throughput was already maximalized with RAID5, why overcomplicate it with another level of striping?

Regards,
Viktor

[iinfi1]

thank you all for ur detailed explanations. i will create a LVM out of the two disks.

>> Use lvm with distributed stripes?
i didnt understand this. i normally create LVMs with default options.

[Viktor Balogh]

>> Use lvm with distributed stripes?
> i didnt understand this. i normally create LVMs with default options.

LVM is capable of doing a RAID 0, aka. striped volume, he meant this. But as your LUNs are already striped between the physical storage disks, I don't think that you would profit anything by creating a striped LV. So, stay with the default values! ;)

To read about LVM striping, here is a doc:

http://docs.hp.com/en/B2355-90672/ch08.html

[Jayakrishnan G Naik]

Hi

No need to use striping here as the luns are of different size. And LVM is the best solution for you here as the Redundancy are met at the storage level.

You can create a volume group with these two luns and create a logical volume which is pretty simple.

Software raid levels are available but that don't have the flexibility that lvm can give.

Regards
Jayakrishnan G Naik

[iinfi1]

i created a LVM out of sda2 and sdb1.

/dev/sda1 55G 41G 6.7G 86% /ora1
/dev/sda2 48G 46G 1.5G 97% /ora2
/dev/sdb1 101G 78G 16G 83% /ora3

after a few hours the mountpoint on sda1 started misbehaving and since the storage admin didnt bother to create zoning in eva, it affected all the servers.
we restarted the server in single user mode and removed the mountpoints from /etc/fstab
and restarted the server. now the mountpoints from SAN are not mounted.

is there any relation to me creating an LVM out of sda2 and sdb1 and the mountpoint on sda1 being affected.

[Viktor Balogh]

Hi,

>after a few hours the mountpoint on sda1 started misbehaving

What do you mean by 'misbehaving'? How could a _mountpoint_ misbehave???

Regards,
Viktor

[iinfi1]

sorry for not being clear ...
df -h was showing the mount point, but i could not list the contents. it simply disappeared.

have a look at the status of the server in the image attached. something to do with this server and the storage but i dont know what.

just before this happened i had created that lvm which i was talking about and our dba was cloning the prod db on the lvm.
i want to know whether creating of an lvm of the two partitions i mentioned, caused this issue?

/dev/sda1 55G 41G 6.7G 86% /ora1
/dev/sda2 48G 46G 1.5G 97% /ora2
/dev/sdb1 101G 78G 16G 83% /ora3

sda1 and 2 are part of one block device and sdb1 is another.
sda2 and sdb1 are PVs of my LVM. would such a configuration have caused a direct impact on the sda1?
did that result in the server throwing too many I/Os on the SAN (with no zoning) affecting all servers?
i know my explanation will appear vague. but has anyone faced a similar situation.

[Alzhy]

Sir.

Do you know what you are doing?

/dev/sda1 55G 41G 6.7G 86% /ora1
/dev/sda2 48G 46G 1.5G 97% /ora2
/dev/sdb1 101G 78G 16G 83% /ora3

You claim the above to be LVM - it is not sir unless it is just lost in translation.

If the above is indeed true that you have mounted filesystems on individiual disks - then question will come out and mine will be -- are the above disks SAN (eva4400) disks!? Coz if they are -- then you are NOT using multipathing!!

I suggest you take a very deep breath and let us go over your problem again -- if you still want our help.

Shukran.

[iinfi1]

i will start again
/dev/sda1 55G 41G 6.7G 86% /ora1
/dev/sda2 48G 46G 1.5G 97% /ora2
/dev/sdb1 101G 78G 16G 83% /ora3

above was my partition before i did the LVM config.
i did the LVM config after umounting /dev/sda2 and /dev/sdb1
i am not sure about the multipathing thing as i didnt do the setup.
my question was
sda1 and sda2 are part of one block device and sdb1 is another.
sda2 and sdb1 are PVs of my LVM. would such a configuration have caused a direct impact on the sda1?
i was unable to access sda1 at all and the state of the server was as shown the JPEG in my previous post.
i have now restarted the server after disabling the SAN moutpoints from /etc/fstab
and now i have manually jus mounted /dev/sda1. all data is intact.

[iinfi1]

pardon my ignorance,
how do say multipathing is not working correctly?
infact an HP consultant who came yesterday told us the people who did the configuration (HP) have not done the cabling correctly!!

[Alzhy]

.ae

hmmmm... HP should have excellent people there.

I guess get your act together sir. Chase whoever manages the EVA4400 to make sure it is "zoned" and / or config'd correctly if you think or others think it is zoned incorrectly.

/dev/sdNN are not the correctr names of EVA multipathed devices!

[TwoProc]

Re: Hello,

I'm with Alzhy: I don't see the point for setting another level of striping on the top of the RAID5-out-of-the-box. The throughput was already maximalized with RAID5, why overcomplicate it with another level of striping?

If you go and read the paper on SAME for big arrays that are already striped you'll see that there is immense value in using Distributed striping on hardware Raid arrays, as I've indicated. Like I said before - feel free to ignore.

As far as something being "already maximized" because it's raid 5. Well, you've just missed the big truck leaving town. There's so many other things to consider - balance I/O, balancing cards, balancing controllers, balancing san ports, NOT using RAID 5 in certain areas. Just saying that something is "maximized" because it's R5 is just leaving so much other stuff out. Which is exactly what his question was, about the other things out there.

No one should ever consider:
"Already Maximized" = "R5"

That statement says a lot more about what's not being considered in setup than what has been.

[Rob Leadbeater]

Hi,

I'm confused... Looking at this output, both /dev/sda2 and /dev/sdb1 already have data on.

/dev/sda1 55G 41G 6.7G 86% /ora1
/dev/sda2 48G 46G 1.5G 97% /ora2
/dev/sdb1 101G 78G 16G 83% /ora3

> above was my partition before i did the LVM config.

What exactly did you do, when you did the LVM config ?

You cannot combine two existing file systems, on physical volumes into an LVM Volume Group...

Cheers,
Rob

[iinfi1]

sorry for confusing everyone ...
i didnt need the data in the drives i mentioned.
i will rephrase my query which i have in my mind now

I have 2 VDISKS (vRaid5) from eva 4400 pointed to Linux server.

my config after creating that LVM is as follows.
vDisk1:is partitioned into two at OS Level
part1:ext3
part2:PV1

vDisk2:is a single block device
part1:PV2

LVM1=VG1=PV1+PV2

in such a configuration can a LVM be created safely? Will creating LVM out of two VDISKS one partition of which is a ext3, cause any storage performance issues.

P.S.We found out later that there is no zoning in SAN, nor is multipathing working correctly.

[iinfi1]

i finally got everything up after isolating the servers from the storage and mounting the SAN drives manually.
we had a USB attached to a the server and someone accidentally pulled it for 2-3 hrs before the SAN mountpoint disappeared.
u can see it from the logs which i have attached.
can someone plz throw more light on these logs. and from OS point of view how I can come to know immediately if there is a situation like this.
any monitoring tool/script you folks use?