Operating System - HP-UX
1748129 Members
3700 Online
108758 Solutions
New Discussion юеВ

Reducing downtime for database patching

 
SOLVED
Go to solution
Clara Rowe
Frequent Advisor

Reducing downtime for database patching

Hi experts! I???m trying to find out how other companies are reducing the downtime required to apply Oracle database patches. We need our database up 24x7, but patching can take hours. We simply can???t afford the downtime to apply necessary patches. How are all of you handling this problem?

Clara
Take time to smell the roses.
16 REPLIES 16
Caesar_3
Esteemed Contributor

Re: Reducing downtime for database patching

Hello!

After we bring good working system we
try to not make patch update.
If we have no other option so we preparing
every thing for update and choose the best
time for and do it fast!

Another option is work in cluster so you
update one server, sync it and the you work
on the other.

Caesar
Steven E. Protter
Exalted Contributor
Solution

Re: Reducing downtime for database patching

We do the following:

1) Quickly down the databse, split the mirror its sitting, on bring it back up.

2) Handle the database upgrade on a non-production server.

3) Down the database again.

4) Transfer in the revised version of the database to production, along with all system tables and such that were modified for the upgrade. We don't transfer back the data which has gotten out of date due to new transactions while we were working. Only database instance system data is migrated back in.

5) Bring the database up, resynch the mirrors.

The two down times for the production database are only for how long it takes to bounce the instance.

This doesn't work if a patch or process requires all data to be upgraded, but it works for most patches that change the executables and data in the oracle database systems tables.

I don't think oracle supports this methodology, but we actually have done it, so it does work.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Michael Steele_2
Honored Contributor

Re: Reducing downtime for database patching

With a failover node in an MC/SG environment, of course. That's what 24x7 high availability is all about.

Sound's like an infra structure issue.
Support Fatherhood - Stop Family Law
Bryan D. Quinn
Respected Contributor

Re: Reducing downtime for database patching

Hey Clara,

On our production systems, as a general rule we do not make patch changes unless it is absolutely necessary. If we find that it is, the patch or patches are installed on our development and QA boxes first. Then those systems are monitored for any fluctuations in operation. If everything is sucessful then we perform the install on our production app servers first, if they are in need of the patch(s). Then we perform the install on our production box. As far as the time to do that, we have the luxury of a four hour maintenance window starting at 7pm every Sunday night. If there is no maintenance to be done, then we just reboot all of our boxes and let them go back to humming along. We the system was constructed and put into place, a maintenance window was one of the provisions that was bargained into the deal. That way we do not have to fight to schedule a downtime. We are supporting operations around the globe...so I am sure it is an inconvenience at some locations.

That is our spin on it.

-Bryan
Bryan D. Quinn
Respected Contributor

Re: Reducing downtime for database patching

ooops...

Sorry Clara,

I missed the Oracle patches part! duhhhhh!!!

I apologize...but I think we handle the Oracle patches in roughly the same manor I described. We are an extremely conservative crew here, so we only make changes if it is absolutely necessary. No ifs ands or buts about it. The Oracle changes are handled during our Sunday night outage also.


-Bryan
Yogeeraj_1
Honored Contributor

Re: Reducing downtime for database patching

hi clara,

patch only if it is necessary!

We prefer to install the patch on a test (development) server first. Test everything. After that, identify a suitable downtime period that would less affect our 24x7 operations. Then proceed as Mr. Protter mentioned about (well, in our own way ;) ).

We take no risks since our business depends on it.

regards
Yogeeraj
No person was ever honoured for what he received. Honour has been the reward for what he gave (clavin coolidge)
twang
Honored Contributor

Re: Reducing downtime for database patching

I assume that for now you want to patch it up to 8.1.7.3 and later you want to install 9i
and migrate 8.1.7.3 to 9i.
If you have enough hardware resources I would suggest one thing - Create a new database with the patched up software. Then export the running database and import into the new database. This would not require a downtime for the live database.But you'll have to plan to incorporate the changes that are made in the source after the export has been taken into the new database.And it'll involve more work also though it descreases the downtime required.
If you are migrating to 9i this is infact one of the migration methods.

Further patching time depends on your processor speed and all.If you can upgrade to a more powerful processor that'll help in reducing the downtime.
Massimo Bianchi
Honored Contributor

Re: Reducing downtime for database patching

Hi,
apart from others suggestion, each time we need to apply a patch we take all the time needed.

We know this means night or week-end, of even more easy on holidays, but we feel patch very risky and data alignig even more risky.

So, in case of updates, we take a test on develop and test, measure the time, and then we decide with the cutomer an appropriate time-frame.


This shall include, for every server if a cluster:
- backup offline of DB- executables
- install patch O.S. if needed
- kernel recompile if needed
- DB executable patching
- DB data (catalog, For example) updating
- DB backup tools test of backup POST- ACTIVITY
- test from key users

We cannot risk to forgot some data or to have some non-updated cluster.


You said that you cannot afford downtime for the patching.

The patch is necessary? Than take your time.

HTH,
Massimo
Clara Rowe
Frequent Advisor

Re: Reducing downtime for database patching

I don't think you guys are getting the whole picture here. We are applying Oracle application upgrades/patches. You can't resync mirrors, user Service Guard, or patch one instance and copy it to production because the upgrade/patch changes the database tables/schema. This means you can't successfully replay log files because the original tables have changed. So how can this be handled with little or no down time?
Take time to smell the roses.