Disk Layout in HP-UX 11.31 for Oracle 10G ASM

[Chandan Laskar]

In our environment HP-Unix Server are connected with EMC CX4 storage. Fiber connectivity is there between server and storage.
We need to install Oracle 10G (10.2.0.4) with ASM.

Can any body suggest me the disk layout for this installation?

I am attaching a sample FileSystem layout for one of the Oracle server currently running in our environment.

Please revert if you need any further clarification.

[Jean-Luc Oudart]

Hi

Why do you require ASM ?
will you be using clustering (RAC)?
Is this only one instance ?

just trying to get the reason for ASM, especially if you have a single instance and no RAC.

Regards
Jean-Luc

[Chandan Laskar]

Hi,
First of all thanks for your queries.

Please find below the answer:
We are planning for ASM because of better I/O performance.

No we will not configure Cluster.

Yes this is only one instance which will use ASM.

Please revert if you need any further input/clarification.

Regards,
Chandan Laskar

[Viktor Balogh]

Hi,

here is a document on how to migrate the database to ASM:

http://sve.to/wp-content/uploads/2009/08/oracle10g_hpux_raw_to_asm.pdf

and this thread might help decide if whether ASM or LVM:

http://h30499.www3.hp.com/t5/Serviceguard/Oracle-11g-using-ASM-raw-or-slvm/m-p/4443864#M16779

Regards,
Viktor

[Chandan Laskar]

Hi Victor,

Thanks for your advice. But it is not exactly in-line with my requirement.

Please check the attachment.
Do I need to take different Physical Volume of all the mentioned space in excel file?

Can I use the same name (i.e. /redolog1, /redolog2, /redolog3,
/redolog4, /oraclebin, /oradata1, /oradata2,
/oradata3, /oradata4 & /oraarch) for ASM environment?

Please advice.

Regards,
Chandan Laskar

[Viktor Balogh]

>Do I need to take different Physical Volume of all the mentioned space in excel file?

AFAIK you should assign LUNs from your storage, and you need to create hard links to these luns with mksf for not to interfere with insf. (insf would clear the ownership from the disk device files)

>Can I use the same name (i.e. /redolog1, /redolog2, /redolog3,
/redolog4, /oraclebin, /oradata1, /oradata2,
/oradata3, /oradata4 & /oraarch) for ASM environment?

keep in mind that ASM doesn't work on the filesystem level. So you don't need mountpoints anymore. Just present LUNs which are equal in size to the existing filesystems, create hard links with mksf, and assign these to the oracle user with chown. It's as simple as that. But I suggest to find a howto on the oracle site and follow that guide.

Also keep in mind that ASM isn't able to store any kind of data: the database itself should be located on a normal filesystem, (which in turn can reside on LVM). But for this see the excellent post of VK2COT from the above linked thread.

[Viktor Balogh]

>Do I need to take different Physical Volume of all the mentioned space in excel file?

Physical Volume is a concept from LVM, which you would avoid with ASM.

[Chandan Laskar]

Hi Victor,
Many Thanks for your explanation. It really shows your detailed knowledge on ASM.

Only I need your further advice to understand some more areas as mentioned below:

-- You have mentioned that "Just present LUNs which are equal in size to the existing filesystems, create hard links with mksf, and assign these to the oracle user with chown".

-Q: How these LUNs will be identified? can it visible through BDF command? Has Operating System assigned any default name for those LUNs? And is it mksf command or mkfs command?

-- You have mentioned that "Also keep in mind that ASM isn't able to store any kind of data: the database itself should be located on a normal filesystem, (which in turn can reside on LVM)".

-Q: Can it possible to store Oracle Data files (.dbf) in LUNs which is managed by ASM?

Regards,
Chandan Laskar

[Viktor Balogh]

-Q: How these LUNs will be identified? can it visible through BDF command? Has Operating System assigned any default name for those LUNs? And is it mksf command or mkfs command?

The LUNs will be visible in the ioscan output, not in bdf output. In bdf you can see the filesystems mounted, and ASM is just one below the filesystem layer. The command is mksf, (memorize it like "MaKe Special File).

-Q: Can it possible to store Oracle Data files (.dbf) in LUNs which is managed by ASM?

AFAIK there is no need to have a datafile with ASM, the data will be stored directly as to a raw device

[Chandan Laskar]

Hi Victor,

Many Thanks. Your advices helped me a lot to understand the Oracle ASM preparation activity.

I am from Kolkata, India . Now it is 08:16 P.M... I need to pack up. Tomorrow, if needed, I may seek your further help.

Hope you will give me similar high quality technical advice.

Thanks again.

Regards,
Chandan Laskar

[Hein van den Heuvel]

Chandan,

There is a lot of infomation on ASM on the web, intro and deep-dive, oracle.com and elsewhere.
Google will quickly identify many such as:
http://www.oracle-base.com/articles/10g/AutomaticStorageManagement10g.php
http://www.youtube.com/watch?v=dxBnKQGTEUM

When going ASM, notably with HP-UX 11.31 as you indicate with native multypath, I would skip LVM. You don't need two cooks in the kitchen. You hand the raw devices/disks to ASM and let it manage the storage.

-Q: How these LUNs will be identified? can it visible through BDF command?

No. BDF is for file system info. When using ASM uou will have to ask ASM (SQL) about current sizes and usages.

-Q: Can it possible to store Oracle Data files (.dbf) in LUNs which is managed by ASM?

There will be no more data files. Instead of datafiles you will create tablespaces pointing to disk-groups in ASM.

Regards,
Hein

[Jean-Luc Oudart]

Hi again

please note that the oracle binaries will remain on file system. ASM is for the database.

You will also need to use RMAN for backup (and initial migration to ASM).

I invite you to read the following docs :
http://download.oracle.com/docs/cd/B28359_01/server.111/b31107/asmcon.htm
http://download.oracle.com/docs/cd/B19306_01/backup.102/b14191/rcmasm.htm
http://download.oracle.com/docs/cd/B28359_01/backup.111/b28270/rcmasmmi.htm

Regards
Jean-Luc

[Chandan Laskar]

Hi All,

I am summarizing the entire discussion.

According to the attached excel, if I need to configure Oracle ASM then I need to ask following layout from our Server & OS team:

==>/oraclebin --- 68 GB (1 no. of 68 GB LUN) -- For Oracle Binaries.

==>4 No. of 204 GB Physical Volumn for Oracle Tablesspaces.
==>1 No. of 204 GB Physical Volumn for Oracle Archive Log.
==>4 No. of 20 GB Physical Volumn for Oracle Redo Log.

I shall be grateful if you advice me the correctness of my understanding.

Regards,
Chandan Laskar.

[Jean-Luc Oudart]

Hi

As you will be using RMAN I would suggest you create and use the FRA (Flash Recovery Area).

The FRA can also be used to host your online and redo log files as well as your backup.

Please review documenation to size the FRA.

http://download.oracle.com/docs/cd/B19306_01/backup.102/b14192/setup005.htm

http://download.oracle.com/docs/cd/B28359_01/backup.111/b28270/rcmarchi.htm

Regards
Jean-Luc

[Chandan Laskar]

Hi Jean,

Thanks. In that case my proposed layout will be

==>/oraclebin --- 68 GB (1 no. of 68 GB LUN) -- For Oracle Binaries.

==>4 No. of 204 GB Physical Volumn for Oracle Tablesspaces.
==>2 No. of 204 GB Physical Volumn for Oracle Flash Recovery Area.
==>4 No. of 20 GB Physical Volumn for Oracle Redo Log.

is it OK?

Regards,
Chandan.

[Jean-Luc Oudart]

Hi

As I say sizing the FRA is important as you will need to use RMAN (Do you use RMAN currently ?).
Do you backup to tape ?
Do you have RMAN integration with your backup software ?

May be you can outline how you currently backup your database.

Also, do you have a test environment you can test the new solution (even with a smaller database you can extrapolate).

Regards
Jean-Luc

[Alzhy · Accepted solution]

Chandan,

First things first - are you a DBA or a SysAdmin? If the latter - do you already know what it entails to have your DB instance on ASM storage?

2nd, Have you or your SysAdmin or StorageAdmin verified that the CX4 is officially supported on HP-UX 11.31 -- if you are using its native multipathing? If you are using Powerpath - then it's a non issue. Give us the powerpath devices of the disks you mentioned and I can walk you through.


Let me know.

[Alzhy]

Well - I can assume you use PowerPath - so here it goes.

ASM needs to be fed easily identifiable/distinguishable "disk device names" so ASM_DISKSTRING can be set to this value. On HP-UX and other UNICES - the easiest would stil be to use a volume manager unless your storage management or multipathing software layer allows you to give them "friendlier names".

On HP-UX, you have 2 choices - LVM and VxVM. But there are strict rules. Each LVOL (LVM) or VOL (VxVM) must reside WHOLLY on each disk. Meaning:

/dev/SAPvg/rASMVOL001 should reside on /dev/rdsk/disk1234 (or the Powerpath device) in the case of LVM.

For VxVM:

/dev/vx/rdsk/SAPdg/ASMVOL001 should be wholly on /dev/rdsk/disk1234 (or the PowerPath device).

So first step is to set up your "ASM" disks on the volume management layer (LVM or VxVM) IF PowerPath does not give you that option to give it custom names. Beleieve me - it is easier to do it this way than just having +ASM be fed with specific disk strings for PowerPath devices.

Then your ASM DiskString can simply be: /dev/SAPvg/rASM* or /dev/vx/rdsk/SAPdg/ASM*

Your SysAdmin should be able to understand what I mean above and should be able to do this.

+ASM will then be able to recognise the "disks" as available for ASM to use.

[Alzhy]

I forgot to mention, ASM will do the "striping" of the disks for you in addition to the striping that the CX4 does on the backend. And depending on your needs.. redo and archive log storage *CAN* be on ASM storage but it may be wise for you to have archive logs on a simple filesystem.

Of course Oracle software tree will need to be on a simple filesystem as well -- but it should be on a dedicated, fast enough disk as it will also house your admin files, ASM, etc.

So - with your spreadsheet - you have 4x20GB disks for redo (ASM), a 68 GB disk for Oracle Software Home and 12x68GB Oracle Data disks (ASM) and 3x68GB for orararch (simple filesystem). This is how I would proceed (LVM recipe):

Set up LVM VG for Oracle software tree - name it say ORAvg with the 68 GB disk and carve an LVOL sized say initially 20GB and make a filesystem and mount as say /opt/oracle or whatever is your standard.

Set up the 4 redolog disks as say REDOvg with 4 wholly contained LVOLS named ASMREDO01 to 04 as I outlined in previous post.

Set up the 12 datafile disks aand name the VG say DATAvg with 12 wholly contained LVOLS named ASMDATA01 to 12 as outlined prior.

Set up the 3 other 68 GB disks as say ARCHvg with one striped filesystem and mount say as /oraarch or whatever your standard is.

Next.. install 10G and ASM, cssd.

[Alzhy]

Now that you have all your disks set up... of course add the oracle id and group (whatever your standard is) and set ownerships of the oracle filesystems AND the LVM raw vols of your ASM diskgroups.

Install 10GR2 (10.2.0.5)... this should be easy 'nuf and straightforward. Install the software only - do nto set up a starter DB!

Next set up ASM and CSSD. This is EASILY done by using "dbca" - so you need X-Windows environment here... You SHOULD know this already...

Launch dbca, and set up ASM. This will also set up the cssd service as well as create the +ASM DB engine for the ASM Layer. You will be asked of course to execute a certain script as root to set up cssd. Towards the end of your dbca ASM configuration session -- it will ask you to create your ASM DiskGroups -- you will now have the option to provide the ASM Disk Strings for your REDO ASM and DATA ASM disks for you to create 2 ASM DIskgroups. Specify EXTERNAL REDUNDANCY so +ASM will not bother mirroring, etc -- it will be only doing striping on the 4 and 12 disks comprising your 2 ASM DIskgroups.

Once that is done.. proceed with enshrining yuor Oracle and +ASM startup scripts and create your Database to peruse of +ASM disk storage.

And that is IT.. welcome to the wonderful world, easy manageabiity and performance of ASM stored Databases.


Caveat! I can't seem to remember if ASYNCIO is enabled by default on HP-UX 11.31 so watch out! and Google.

Hope this helps.

[Alzhy]

And now.. how about some Luv for the ITRCians who've helped you so far if you think you have been enlightened and helped.

Please let us know if you still want blow by blow details on how to do your LVM layer...

[Chandan Laskar]

Hi Alzhy,

Superb explanation. Let me summarized the disk layout again.

Mount Point Name -- Size---------Purpose
/oraclebin----------68 GB(68GB * 1) -- For Oracle Binaries ( Instead of 20 GB I am considering 68 GB)
/oraarch------------204 GB(68GB * 3) -- For Archive Log

/ASMREDO01----------20 GB -- -- For Redo Log
/ASMREDO02----------20 GB -- -- For Redo Log
/ASMREDO03----------20 GB -- -- For Redo Log
/ASMREDO04----------20 GB -- -- For Redo Log

/ASMDATA01-----------68 GB -- -- For Data
/ASMDATA02-----------68 GB -- -- For Data
/ASMDATA03-----------68 GB -- -- For Data
/ASMDATA04-----------68 GB -- -- For Data
/ASMDATA05-----------68 GB -- -- For Data
/ASMDATA06-----------68 GB -- -- For Data
/ASMDATA07-----------68 GB -- -- For Data
/ASMDATA08-----------68 GB -- -- For Data
/ASMDATA09-----------68 GB -- -- For Data
/ASMDATA10-----------68 GB -- -- For Data
/ASMDATA11-----------68 GB -- -- For Data
/ASMDATA12-----------68 GB -- -- For Data


Also Need one more clarification, Can I install 10GR2 (10.2.0.4) instead of 10.2.0.5 ?

Please reply whenever you have time. Based on your confirmation I shall communicate to my Server & OS team. As they need the disk layout to ready the server for Oracle installation.

I shall wait for your reply Alzhy.

Warm Regards,
Chandan Laskar

[Alzhy]

You can install 10.2.0.4 but I thought 10.2.0.5 includes sme security, bug and enhancement fixes which may or may not be apropo to your site.

And Yes sir on the disk configuration. Please STRESS the volume group recommendations as well as the STRICT one LVOL on one PV configuration of LVM. ASM will do the striping on the host layer once the rLVOLs are utilized as ASM diskgroups!

IF you are not familiar with ASM -- please seek assistance or peruse the wondeful tool of Google! I am a SysAdmin but I have perused Google and actually helped our DBA Team a lot along the way.

[Chandan Laskar]

THANK YOU VERY MUCH ALZHY!!!! I shall give you final update / share my experience once I shall complete the installation.