Serviceguard and Oracle 11g

[Dave Hutton]

Hi all, it's been 7+ years since I've touched Serviceguard at a different company in a Unix Admin role.

Recently we've started to implement SG here and my role has changed to the DBA side of things. Oracle's support can be less than stellar, so I'm asking here :)
Is there a good FAQ or "Recommended practices" when it comes to Oracle Patches?
My basic questions about patching are when it comes to code tree upgrades I recall having to basically apply the patch/upgrade to both node’s code trees. I’m okay with that, but is it more common practice to have the Oracle dbms code tree part of the package now so it reduces the amount of patching that happens? I’ve seen this being suggested in some web pages. Currently we’ve configured it so really only the internal oracle DB stuff .dbf files float between nodes and the code tree will be separate. Which I’m more familiar with but is that an older way of thinking about this?
The second question along those lines, OPatch. Since those tend to be internal to the DB. Do you continue with the same practices as the code-tree/upgrade type of patches? And opatch both nodes?


Thanks

[klb]

It's been even longer for me 12+ years on IBM HACMP/ Oracle OPS 7.x.

If I were to do it today, I would still want local-to-the-node software trees for Oracle and probably all other application software. Each node IMO needs to be capable of running system software outside of the "cluster" configuration and shared disk subsystems.

Makes things more straightforward especially during maintenance when we can remove a node from the cluster ( completely ) and go and do whatever patching and maintenance is required for that node.

Hth,

-klb

[Dave Hutton]

Sounds like the seperate dbms code trees are still the way to go. Even one of the guys from the unix team just completed the SG class and it sounded like it's still the common practice.

I guess going from there, my assumption for upgrades or any patches that do not use OPatch (runinstaller) is something along this route:
1) Stop the cluster or put it in maintenance mode.
2) Apply the patch to the primary node as I would for a standalone database.
3) Apply the patch to the secondary node using the selection "just upgrade the binaries".
4) start the cluster/package.

Where OPatch threw me a little, was I applied a patch prior to the DB being "packageized" and now when it's on the secondary node opatch lsinventory doesn't show it. Granted, I think for the most part, it's just reading from files and since it wasn't opatched from that server, it's not aware of it. My worry about opatching the secondary node is, that it's double patching the database. I'm not sure if thats a bad thing. Or even though it doesn't show the patch, is the DB still considered patched?

My enviornment is pretty simple really, no RAC, no clusterware just a Serviceguarded DB that is active/passive and really only there for hardware failover. HPUX 11.31 and the DB 11.2.0.2.
I'm kind of shocked on metalink and just searching the internet I couldn't find any answers. They seem pretty basic to me.
Maybe there isn't a standard practice? Gosh for something thats been around for a while, I would assume someone has a faq or some sort of doc.

[klb]

I'd think you should be able to find info on Metalink for sure. The search engine there is not that great, so you'll have to play around with various key words until you find the right combination.

I very much like to keep each Oracle installation ( whether in a cluster or not ) nice and clean so that everything is done via the installer or OPatch locally. This, I believe, is the only way Oracle will ~officially~ support the stuff since about 9i or so. No more copying the binary trees from one host to another. You gotta install it now!!!

If you've patched your DB once, it's done, even if your local software install tree doesn't ~know~ about it. Running the DB portion of the patch again probably wouldn't do much harm in any case, but you could check it with:

select * from dba_registry;

... That will show point releases but may not show one-off patches applied by OPatch ( these probably don't require DB changes anyway ).

[Dave Hutton]

Thanks for the imput, I continued to search high and low using different key phrases to see if I could get better hits then what I found prior to posting here. Also turned metalink upside down to see if I could get more information from it. I'm going to open an SR with them and see if they can respond back. So I'll see what they suggest.
I guess meanwhile I'm going to go forward with what feels to me to be over patching.

Stop the package, depending on what kind of patch either run opatch or runinstaller, swing the db disks over to the adoptive node and re-run opatch or the runinstaller. Swing them back and to the primary node and start it up.

The only reason it feels like over patching is because I'm guessing if it does anything internal to db, it'll be done twice.

Thanks anyway for your suggestions.

[klb · Accepted solution]

Dave,

The Installer will give you the choice whether or not to upgrade your local running database. OPatch I don't believe will do database changes.

Just do the database part once on the running "hot" side of your cluster. Then, without failing over to the other node, just log in the the other node and do the same thing there whether that's running the installer ( and choosing NOT to do the db part ) or running OPatch.

That should have you covered for everything: database and software binaries/scripts.

Hth,

-klb

[Dave Hutton]

Makes sense, I guess I wanted to be safe then sorry. Also wanted to make sure I was doing what is somewhat standard practice. As I joke with the others here, obviously we aren't the only ones who've done this.

Whatever I do now, will become standard practice here. I didn't want them to be learning the wrong way right off the bat as SG and clustering is new to the rest of the crew.

So opatch I'll just patch the hot side or active side and not concern myself with the fact that someday if it's on the cold/secondardy node/stand-by node whatever you want to call it might happen to run opatch lsinventory and it doesn't show infact there are patches applied that they don't get worked up over it.

[klb]

If your code trees are local to each node, you'd need to do patching always on both nodes.

Run any database patch requirements ( scripts manually or through the installer ) only on the side where the DB is running. Then run OPatch and the Installer ( no DB update ) patches also on the cold node.

This way you keep the code trees in sync so that the lsinventory would show the same on both nodes.
-klb

[Dave Hutton]

We do, the only part of the database that moves between the two servers is the .dbf another words the actual database files.

Like I said above, we are and with my prior experience setting it up in my prior job for the dba's was to keep local to each host the Oracle code trees. (How at that place they did their patching I have no idea, we very much had a seperation of duties). I've called some old contacts there, but they no longer are there to find out how they used to do it.

I'll have to look closer at the patches when applying them to see if it does any db updates. Sometimes it's hard to tell what exactly changes.