Operating System - HP-UX
1748236 Members
3564 Online
108759 Solutions
New Discussion юеВ

Re: How to control the new program quality?

 
ericfjchen
Regular Advisor

How to control the new program quality?

Sometimes, a new program or code caused the production down or slow. How to control the new code quality? Should DBA inspect the program before it is deployed on production? Any good idea?

Thanks for any comment

Eric
8 REPLIES 8
Steven E. Protter
Exalted Contributor

Re: How to control the new program quality?

Shalom Eric,

Where I used to work there was the following review process.

Code was tested to see if it would work on a diferent server reserved for development.

After it passed a test plan that included expected results it went to the dba for sql review(this was oracle application server).

If necessary, the dba required/requsted changes in sql statements and did logging to gather data on the performance of the program.

When this policy was followed very little bad code hit production.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Jean-Luc Oudart
Honored Contributor

Re: How to control the new program quality?

Eric,

This is an important performance aspect.
I think this is good practice to have a program inspection for new /amended transaction. This is to protect your production environment. Obviously there is an overhead, but this has to be part of an acceptance process for any new program and/or change.

You will have to convince the right people. Sometimes developers don't accept this type of activity. Therefore you will have to take baseline performance before any new change and tell them about accountability for any performance hit (due to the new code).

To summarise : - take regular performance baseline
- don't accept "wild" change in production

Regards
Jean-Luc
fiat lux
Patti Johnson
Respected Contributor

Re: How to control the new program quality?

Eric,

One suggestion is to get the developers to learn to trace their code and review the output from tkprof. Since development systems often have less data and activity than production systems, code that runs OK in development will have a problem when run with production volumes. If your developers are taught to read the trace output and identify problems they can tune code before it moves to production. Also the formatted trace can be included with the documentation used to move the code and the dba can review.

Obviously this takes support from management and the developers need to be given the extra time necessary to complete and review the trace. But in the long run it pays off since they do not have so spend as much time "fixing" production performance issues.

Patti

A. Clay Stephenson
Acclaimed Contributor

Re: How to control the new program quality?

The best approach that I have found over many years of development is to intentionally use slow machines with limited resources as the test/development environment. The test data should be similar in volume to production. The idea is that if the algorithms are good enough to run on a dog they will also run well on your fast boxes. As a bonus, it will annoy your developers to no end to be forced to run on slow boxes. They will complain that compiles take forever and that is why everything taks so long to develop. The truth is that compile/link times even on a dog are much less than 1% of the development cycle. Over time the developers will come to see the wisdom of this approach because very little code than runs well in test on the dog will perform poorly
on the fast production machines.

Today's software tends to be so complex that human code reviews often do little good. There are some automated testing suites available that can throw transactions at a system and examine the results.


If it ain't broke, I can fix that.
ericfjchen
Regular Advisor

Re: How to control the new program quality?

Thanks for your kindly comments in advance. If my boss assigns DBA to take full responsibility to inspect the new/change code before, is there any good procedure to finish it? Do you have a SOP to inspect the new or change code?

Thanks a lot

Eric
Indira Aramandla
Honored Contributor

Re: How to control the new program quality?

Hi Eric,

When there is a new program / code to be promoted into production environment, it is good to have the DBA review the SQL and the object definitions (storage parameters├в ┬ж). But it is also required that the end-users are involved in the testing process on the Test Server with a similar volume of data at the QC stage of the change control procedure. And discuss with the Systems Administrators if any changes needs to be done on the Server. In my opinion apart from the DBA who can take care of the Database aspects, the other team members (Sys. Admins, application Developers and especially end-users also play an important role in the testing of a particular code / program before it can be promoted or approved to be promoted into production environment.


Indira A
Never give up, Keep Trying
Yogeeraj_1
Honored Contributor

Re: How to control the new program quality?

hi Eric,

Assuming the developers have designed their programs "well", it is also important that you keep statistics about the past. Maintaining a history of past performance is the single most important thing you can do to facilitate this.

There are many ways to maintain this history. Statspack is an excellent "low-bandwidth" method. It is very nonintrusive, and many other tools can use the information it collects. OEM with its diagnostic pack is another choice..

What i mean to say is that when you get an unexpected problem, it must be corrected immediately or sooner. You must also know what you have to do.

A typical todo list would include:
1. Find out what is different
2. change one thing at a time
3. have a sound reason for changins that one thing
4. be able to unchange things
5. Build a test case

hope this helps too!

kind regards
yogeeraj
No person was ever honoured for what he received. Honour has been the reward for what he gave (clavin coolidge)
Rory R Hammond
Trusted Contributor

Re: How to control the new program quality?

I agree with Clay about "dog" development systems and have expressed the same to management for years.

Many of these new programs are to complex to be "reviewed" by the DBA. Currently we are in a 3 year implementation process. We have Lots of releases using many business functions, the DBA would beable to catch a "few" problems.

Our development system actually is a pretty big box. But it runs 4 oracle databases, A "pristine", "development", "test" and "preproduction" enviornments. The pristine environment is the orginal unmodifed source code and test are run to see if the orginal source has bugs.
1. Developer first creates, runs and test their programs on the "development" system.
2. The Code is then functionally tested on the "test" system by the business process owners.
3. The Code is then Tested with all the other production code on the preproduction environment before it is rolled out to the production System.

With this process whe have fixed many problems before production. The DBA does monitors these systems and looks for and addresses issues. We generally know what the "hogs" are before going to production. Our produtions system has been generally stable.

The bad news is that we have four copies of the production database and we refresh the data quarterly. I don't like the space requirement, and the DBA has some complicated data refresh scripts. At first, the developers thought the process as tedious. After doing this for a couple years. The process is will accepted and worth the effort.

Rory
There are a 100 ways to do things and 97 of them are right