RAW Oracle DB Storage on HP-UX - Best Practices

[Alzhy]

Those of you who've done the right thing going RAW (whilst the AsynchIO promises of Filesystem remain a promise or with HP-UX 11.31?) - can you please share thine practices? To wit:

- Do you use uniformly sized raw devices?
- Do you just grow raw devices when tablespaces need growth or do you simply add raw devices?
- Do you use pointers to raw devices or do your instances use the actual raw device file names?
- The redo logs - did you use raw on them as well?

Backups
- What backup approach did you use? In-array split mirrors/BCV's?
- Do any of you use host based split mirror backups with raw device storage of Oracle filesystems?

Points will be rewarded generously...

Thank you.

[KapilRaj · Accepted solution]

Not on HPUX, but that makes little diffrence ..

So here goes my response.

- Do you use uniformly sized raw devices?

kaps:Yes

- Do you just grow raw devices when tablespaces need growth or do you simply add raw devices?

kaps:Add raw devices

- Do you use pointers to raw devices or do your instances use the actual raw device file names?

kaps:we use pointers ( basically a soft link to the raw device file ).

- The redo logs - did you use raw on them as well?

kaps: Yes, Infact we started with redologs.

Backups
- What backup approach did you use? In-array split mirrors/BCV's?

RMAN,RMAN and RMAN. We did a daily flashcopy ( like BCV) but they are not backed up to tape.

- Do any of you use host based split mirror backups with raw device storage of Oracle filesystems?

kaps:We never used.

Points will be rewarded generously...

kaps:This statement was unnecessary :(

Thank you.

kaps :U'r welcome.

[Alzhy]

King Kaps,

May I know on what platform you are running RAW? And what's the size of your DB and how large of a machine (# CPUs, Memory, # of Storage Channels).

What prompted you to go RAW?

And what was the improvement over cooked filesystems -- that is, if you started out cooked and shifted to raw.

I am encouraging my client to shift to raw to better scale the environments. The DBs are on average 1.6 - 2.0 TB and fast growing...

[KapilRaj]

The env was AIX and Oracle 9i. It was round about 1.5 TB when we moved redo logs into RAW. And when jfs2 gave a 'PIA' , we thought to reduce the filesystem overhead and went towards RAW. Acually I was not happy about going to raw but the performance did improve and I kept my mouth shut... Soon after I left the project So I have no idea how it is now ... But oracle and RMAN gives you the best backup so I do not see any issues going towards raw.

Regds,

Kaps

[Alzhy]

Why were you "unhappy" if you mind sharing your unhappiness going to raw? Was it just it made your life miserable managing raw devices? Did you use any volume manager at all apart from the built in VM of AIX which oculd have made your life easier?

[KapilRaj]

What made me unhappy ?.

1) No filesystem, so I have very less control on them

2) The way we implemented was something like this. /dev/oradatalv001 is a raw logical volume. This has a soft link from /oracle/data/dbase001/xyz.dbf. For database to work, I have to make sure that the permission of /dev/oradatalv001 is oracle:dba. Being on a cluster environment, whenever I add a disk or create / extend a logical volume , I need to re-import the Volume groups on the failover node. In aix , When you export/import a volume group, by fefault the device files ( LV & VG) are deleted and re-created. So this resets the permission on the failover node to root:sys. So I have to remember to change this back to oracle.dba too. Or the failover will not work.

We used the built in LVM nothing else.

So all them looked like we are bringing in more complexity.

Regards,

KapilRaj

[Alzhy]

I hope you bear with me for asking these questions.. I just would like to be prepared for the resentment expected.

You mentioned:

"
1) No filesystem, so I have very less control on them
"

I know we are all "control freaks" - but what specifically do you mean you will have no control over them? You will still have control on the raw devices. If you're using LVM or better yet VxVM -- managing raw devices are a breeze and you should have less steps to perform in storage provisioning..

Ain't it kool to have no filesystems? No mount points to be concerned about. No potential for corruption at the filesystem level. No filesystems to "fill" up or go 100%... no fsatab entries to maintain...

[KapilRaj]

It is just the 'feel good' factor mate. I am so much used to ls -l on any filesystem. Nothing else. I don't know if they can do it on raw files but on filesystems the DBAs sometimes take a database export and forget about it. So when they come to me for space the next time , I do an audit and make sure that they are not wasting any space :)

[Alzhy]

Thanks mate...

"Feel good seeing all me files/filesystems" I don't think will not stand out. As with raw devices, you can also feel good knowing all your raw devices are fine and dandy via LVm or VxVM commands to list them.

Thanks for your views though.

[Volker Borowski]

Hello,

I think mount options of modern filesystems almost provide you nearly the same performance as filesystems (convosync=direct or forcedirectio i.E.).

If you gain performance by migrating a FS-DB to a RAW-DB I'd say 90% of it comes from cleaning up internal fragmentation within the export/import procedure.
You would nearly get the same effect when going RAW -> FS for the same reason.

So yes, there will be an effect, but I doubt it will count that much.

We tried Redologs FS<->RAW on a heavy load SAP-System (16 Log-Groups of 900MB) which got switching 45-60 secs at peaktime.
That was on EMC boxes with striped/mirror volumes on solaris with veritas logical volumes against veritas filesystem in veritas logical volumes (so volume manager the same, just filesystem overhead for convosync=direct mounted log filesystems). The effect was near zero for our load profile, so we did decide not to migrate the DB (5TB) because benefit was too less.

As for the "nice view"...
When you have a big team and split responsibilities (dba<->os), it is easier, when a non-root-privileged dba can do some "bdf / df -k" to check if the datafilesystems are in place, even if the system to check is not the one you are working with day-by-day. This can happen for the guy who is on call.
No need to check out volume configuration and layout and resolve symlinks and play around with rarely used volumemanager commands for a not so frequently used system when getting a call at 2:30 am !

If the os guy is the dba guy as well, things might look diffrent :-)

Volker