Oracle 9i Rel2(9.0.2.0.1) raw disk or filesystem.

[Steven E. Protter]

You have to actually be using it in production or provide expertise and docs as to why I should go either way.

I just found out during install that I've been wrong about this database. You can now, contrary to what Oracle said at Hp-World 2002 choose between filesystem and raw disk.

This is a survey. How does your shop do it? Why?

7 points for any reasonable response, 8 for a compelling one.

Don't look for quick points assignment though, I just watched Kerry Wood's homer. I think I've caught a disease, called Cubs fanatisis. I don't recall being a cub fan though I lived here for a bit while I was 3. Came here in 1989.

Anyway, go cubs.

Lets here some good, thoughtful answers.

SEP

[Michael Tully]

Hi SEP,

Our shop uses filesystems for Oracle (9i release 2 9.2.0.3). Our DBA reckons there is not that much difference between the two (cooked V's raw) but prefers cooked because of the managability factor. He says that this choice has been around for a while. I like you were under the impression that cooked was always the preference.

Cheers
Michael
(sorry not a baseball fan)

[T G Manikandan]

SEP,

you should have a copy of this doc.

[Steven E. Protter]

I should mention I've been running Software AG adabas on raw disk for nearly a decade. I'm familiar with raw disk and there is an apparent advantage in performance.

Bummer about the cubs. I will now begin to make normal posts again.

SEP

[Ravi_8]

Hi, steven

We use filesystem as it got advantages over raw( though bit slow)

[Jean-Luc Oudart]

SEP,

we have Oracle8i (8174) on different platforms. We us e file system and we did benchmark the application.
The big plus is to use memory better and tune the application towards this direction. InWe found that if the data was loaded in a certain way we could save thousands of logical IO (therefore physical IO too !)

The pb with raw would be manageability.
One option would be to mix the 2 : what about redo on raw ?

my question a while ago :
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=193104


Rgds,
Jean-Luc

[aparna challagulla]

Hi SEP,

We have oracle 9i RAC on raw device for performance reasons. though we can have it on filesystems with Oracle managed Files the DBA guys are more comfortable with raw volumes
aparna

[Christian Gebhardt]

Hi

For most databases (about 140, from 7.x to 9.2.x) we are using filesystem, the local representant of HP said there is no great increase in performance with raw disks.

But:
Our DataWarehouse (HP Superdome SD-32, 32 processors, EMC Clarion CX-600, about 10 TB) uses raw-devices (Two stripesets about all disks, 10GB chunks)
With the old configuration (Solaris, Oracle 7, EMC) we found a remarkable increase in perfomance using raw devices. This raw devices stay during all upgrades till the actual configuration: HPUX 11.11, Oracle9.2

Chris

[Khalid A. Al-Tayaran]

Hi,

We have Oracle 8i (8.1.7.4). All Oracle data files are mounted on file systems. Raw file systems as I recall reading from this forum have the advantage of speed. But it is not much of a diifference. You can go with file systems with no worries.

More reading:
http://dbforums.com/arch/165/2001/10/810473

[Massimo Bianchi]

Hi SEP,
one of my customer has a 9.2.0.1, and choosed to use the FS layout.

His coiche was motivated for easyness of manageability, because every one can see a datafile if it is a file, but if it is a rlv there may be confusion. This is also because there are various level of persons in the support, the ones that do the day-by-day maintenance and the experts, for the problems.

tablespace managing is for the first level, with low skills, so the preferred choice is the one with the lowest risk.

Regarding performance: with hpux onlinejfs 3.3 and a proper tuning they are really indifferent, there is no noticeable gain in the use of raw device.

But if you plan carefully and decide to use the new feature of 9i, like external tables, the FS choice could be more consistent, avoiding you to keep different mind and performance effort between rawa devices and fs.

Massimo

[Yogeeraj_1]

hi,
-
To me, /dev/.... looks no different then /d01/oradata, they are just files after all.
-
I have been using both. For instance, if you consider online redo logs - You never touch online redo logs for anything -- so they are safe on raw (and benefit the most from it -- write intensive)...
-
I've also seen ardent fans who always prefer putting a system together to always go for raw. The reasons being:
-
a) a PIO as reported as Oracle is very likely to be a real PIO, so you get a true picture of whats going on
-
b) the buffering is controlled by Oracle so as a DBA I have more control over it, and Oracle should yield smarter buffering, since it knows thing like extent and segment boundaries
-
c) you typically get some benefits from the OS layer (kernelised aio etc)
-
d) you do *not* get autoextend. Seeing this as a benefit because it forces you to do some decent sizing and planning up front (sadly lacking in many databases)
-
The historical arguments against raw were always management, administrators, etc, but nowadays, every OS as a simple GUI or equivalent to resize them, move them, create/drop them as if they were file systems anyway.
-
Many people "afraid of raw" -- like it is "harder". Perhaps that is because the DBA needs SA help to set things up (needs root everynow and again) and they feel a loss of control. Or, they really don't understand what RAW means. A major "fear" for OPS/RAC adopters has always been "oh, i have to use raw". We had to create a cluster file system just to work around that!
-
hope this helps too!
-
regards
Yogeeraj

[Khalid A. Al-Tayaran]

Hi,

We have Oracle 8i (8.1.7.4). All Oracle data files are mounted on file systems. Raw file systems as I recall reading from this forum have the advantage of speed. But it is not much of a difference. You can go with file systems with no worries.

More reading:
http://dbforums.com/arch/165/2001/10/810473

[Graham Cameron_1]

We run 817 and 9024 under filesyestem.
In the past (remember OPS), we have used raw file systems but the unix management effort was too great, as was the difficulty in providing dev and test systems which truly mirrored live.
Technically it may not be compelling, but from a business point of view it is a no-brainer - the company would need a platoon oif unix guys to maintain all environemnts on raw. (Last count > 10 production databases and >100 pre-prod)
-- Graham

[Stefan Farrelly]

Weve got dozens of servers all running Oracle and we only use filesystems, not raw, for the simple reason its a lot easier to administer. If its a filesystem the Unix admins can look after it, if its raw the Oracle DBA's need to look after it, and ocassionally when we get a problem or corruption no more DBA's blaming admins and vice versea - better to keep it all under one teams control - the Unix admin team. If there is a problem is far easier for one team to control and diagnose and fix it.

I wouldnt trust our DBA's or even our junior admins with raw - too easy to make a mistake, think a volume is free when it isnt, and accidentally blast it. Ive seen it happen too many times...

[Steven E. Protter]

Very good Oracle fans. Its good to see your thoughts, and the more the merrier.

I'll have my dba look at the thread later today.

SEP

[A. Clay Stephenson]

I seem to be one of the few whacko's who actually believes in measuring the performance. I've done it and my results depended much more on the OS than on the Oracle version. In 10.20 there was about an 18-20% boost in using raw/io --- mainly because by decreasing the size of the buffer cache you could increase the size od the SGA on memory-limited boxes. On 11.0 boxes the benefits of raw vs. cooked were typically about half that of 10.20 (9-10%). On 11.11, the situation reversed and the best performance resulted from cooked files for everything.

If using raw/io, I've found that in all cases, do it for datafiles and indices but leave the archive and redo logs as cooked files.

By the way, there is a way to have your cake and eat it too and that is to use the OnlinewJFS options convosync=direct,mincache=direct,nodatainlog mount options. These have the effect of completely bypassing the buffer cache and the performance is all but indistinguishable from true raw i/o. You can then easily test the difference simply by changing the mount options and restarting Oracle.

NOTE: None of this applies to Oracle RAC. Although there is a Veritas filesystem designed to work across hosts I would never fully trust it and would only use raw/io.

[Sandro Schaer_1]

all our oracle databases are running on filesystems. it's far easier to manage. especially mirroring across multiple storage systems or moving from one to another.

to gain a bit of performance one really needs to know where to place the files on the raw devices. otherwise it might happen to overload such a devices. talk about io contention.

usually the peformance gain by tuning the application programm is more significant than any hardware tuning. somewhere on oracle's website they have a list on how to get better performance. hardware tuning (filesystems vs. raw devices) is #10.

[Mark Grant]

SEP

I don't have much to offer here apart from marvelling at Mr A Clay again except to say that our DBA's run all these version of oracle and very much insist we go the cooked route.

8.1.7 8.1.7.4 9.0.1.4 9.2.0.2.0 9.2.0.4.0
8.1.7.3 9.0.1 9.2.0.1.0 9.2.0.3.

Again, I believe it's mostly for administration purposes, we really like to be sure that we can determine what a disk is being used for by referencing LVM.

[Leif Halvarsson_2]

Hi,
If you are interested in Oracle performance you sould perhaps have a look at Veritas Database Edition for Oracle.

http://www.veritas.com/products/category/ProductDetail.jhtml?productId=oracleedition


From the product description:

"VERITAS Database Edition includes a unique database accelerator that delivers raw device speed with the administrative ease of traditional file systems."

[Claudio Cilloni]

We use datafiles on filesystems. Once we spent some time thinking about this choice, and the manageability of files over logical volumes was the key factor for this decision. Reading here that there isn't so much difference make me feel good :-).

Sometimes it is useful to copy datafiles for cloning/test/do some kind of experiments; easier than copying logical volumes (we haven't big databases - 10 or 20 GB).

Ciao
Claudio

[Jon Mattatall]

All DB servers 11.00.

We use some 8.0.5, a lot of 8.1.6, and are moving to 9.2.0.3. All use filesystems.

As far as I can tell, the choice was never based on performance. We have a lot of small instances (~15 per server) and it's just much easier to have filesystems so that we and the DBA's don't get things confused.

We're moving to 9.2.0.3 on AIX5.2, and again, it's planned to use filesystems. The probable reason? Inertia. We have nothing empirical to show superiority.

...and if the Series winds up Marlins-Yankees, I'd rather watch bowling.

[Steven E. Protter]

I'll keep handing out points, so long as you keep posting good data. I'm glad I created this thread, even if it did the cubbies no good.

Thanks for your posts so far.

A. Clay Stephenson: You're not a whacko, you are clearly in my opinion the most respected member of itrc. That is not a points measurement, its a subjective opinion. I appreciated the fact that you took the time to measure the data and now regret I didn't bump your score up to bunny.

I hope you come to Chicago next year.


SEP

[Stefan Farrelly]

No Clay - your not the only one who measures the performance, just that SEP never asked about performance in the original question so nobody mentioned it in their reply until you.

Any talk about using the mincache=direct etc. mount options to allow 'pseudo' raw i/o performance is a long discussion - needless to say you will NOT get the same performance as using raw i/o. There are lots of rules which must be obeyed to gain the benefit, and not all applications are written to do so (eg. i/o's on certain block size boundries). We estimated the performance benefit from this to only be <10% - which wasnt worth the effort to get it to work.

As for raw i/o being faster, of course it is, any raw dd will show its 100% faster than a dd on a filesystem, but nowadays with 2Gigabit fibres and storage systems running at 250MB/s you simply dont need to add all the complications of using raw i/o over using much simpler filesystems - which can be boosted with large buffer or SGA caches.

As youve pointed out, raw or mincache options only add a few % improvement - for a lot of effort. There are much easier and simpler ways to boost performance by changing to 2Gb fibre or add faster disks or boost your cache sizes or simply by striping your filesystem.

[Steven E. Protter]

Stefan,

You are also one of the most highly respected members here(My opinion again).


Thanks for all your input. The more the merrier. Even as this thread turns in different directions I lean more.

Thats what its about.

SEP

[Stan_17]

Steven,

We have few databases in raw via veritas quickio and some in cooked filesystem (all in 8.1.7.4)

Using cooked vs. raw IMHO, totally depends on the application you run with. If you have a busy oltp environment, then i would definitely go with raw for redologs at least to start with, as it normally suffers from partial block writes and you could also see log file sync waits followed by log file parallel write, log buffer space waits all alluding to scalability issues. so it's better to have the redolog on raw fs to get the benefits of direct/io as well as kaio.

In a typical DSS environment, tempfiles and datafiles are potential candidate for raw as most queries are unique there, which means lots of PIO's and raw is better with large io operations.

I've seen in most cases, with a well tuned apps and raw fs, performance go sky high. conversely, it could get worse if apps are bad. so the big bucks lies on tuning the apps first and then the fs.

If you are dubious about performance of the apps, then start with a fs supporting direct/io and threaded aio. see how it goes, tuned the apps and then contemplate moving into raw fs.

-
hth,
Stan