Post patch bundle install testing

[AndyMueller]

All, I am curious to know what you guys test after completing an HP-UX patch bundle install? Do any of you have a "Test Plan" that you complete? We generally test ServiceGuard Failover, Ignite image creation, ssh, bastille, EMS & OVO functionality. Anything else we should be checking?

Thanks, Andy

[Ivan Krastev]

It depends from the server usage. At least syslog/rclog and applications log should be examined after the upgrade.

regards,
ivan

[Steven E. Protter · Accepted solution]

Shalom,

I have a written test plan of important server functionality.

This includes testing procedures, and expected results.

After a major patch install, I execute the plan and roll back if major functionality is missing.

Prior to any patch, I back up the vg00 boot volume group with make_tape_recovery or make_net_recovery

SEP

[AndyMueller]

Steven,

I know that depending on the server's function (application) test plans may differ. Can you drill down in little more detail, what specifically you test? (especially OS level related).
Thanks, Andy

[Steven E. Protter]

Since I love earning points, an example.

Lets say I have an oracle app.

Tests:
ps -ef | grep instance name
This insures oracle processes are up.

I run a script that reads data from oracle tables to make sure the tables are actually online.

I test any oracle apps using a test account, which I fight the programmers to have to insure the app is up.

I do an ssh login to make sure that service is up since updates tend to "bork" it.

Expected results vary with the test.

I don't test everything, but I pick enough tests to insure that critical systems are up when I head out of the office.

I'd like to know how you test bastille, the individual apps?

If there is a web server, I browse it to make sure it works.

SEP

[Bob E Campbell]

While this is not exactly a test, to me the key is having an IUX recovery image and/or a DRD clone of the original environment.

While it does not increase confidence it does limit the risks to a defined level.

Beyond that if multiple systems are involved I would look at a rolling update that brought the new environment into production in stages with the most important systems coming online last.

[UVK]

Andy,

Apart from the regular checks you mentioned. I also moniter the system performance after any sort of patching(as i have bad experience when bundle invloves any PHKL's) to see any unusual CPU, io, mem usage. And enguage the relevant apps teams to have them test their apps and take a sign off.

Cheers,
uvk

[AndyMueller]

Steven, for Bastille, I will un-bastille the server, then run bastille -b to read the current configuration back in. I make sure that that services we don't allow (ftp & telnet for instance) are still disabled.

Thanks for some of the items you had listed. Given by the limited responses, it appears that many may not conduct extensive "formal" testing, or don't follow a stringent test plan. I am developing this for my organization and was hoping for a bit more feedback to add to my procedures.

Andy

[AndyMueller]

UVK, excellent points, thank you!

[AndyMueller]

Thank you Bob, we are doing precisely that. Any formal testing you conduct AFTER the patch bundle has been installed?

Andy