Change in Kernel, Now Oracle 7 No Longer Works

[R. Allan Hicks]

Just a couple of thoughts.....

You might try and break the problem into smaller pieces. From what I read, it looks like the server and the listener are having problems. I'd pick one and work on just it before moving to the other. To me the listener is the easier of the two.

The listener reads listener.ora in $TNS_ADMIN. I've had traps laid for me before where someone has come in and messed with my hosts file and the problem not appear until reboot. This would fit in that your problem may not be the kernel change, but in the re-boot.

Start the listener: lsnrctl start

Check the logs like before.

Your listener log file looks interesting. Is there anything significant about Host=WindowsPC? Your port number is interesting too. I know you can't use the standard one with two instances. You might want to make sure that someone else isn't using that port too. A reservation in /etc/services is always a good idea.

Another thing to consider. Is there enough memory for the SGA and the oracle processes. Sounds like your machine pretty heavily loaded maybe the server processes that are croaking are doing so because of resources. I know you put the kernel back like it was, but you might want to double check. Most of the time kernel issues show up as an ORA-0600. Although the ORA-7445 error would indicate a kernel issue. As you've probably figured out by now, ora-12000 series are network issues.

Sounds like a nasty problem. One last thing. I had a problem with my K class core dumping one of the main oracle processes with an Ora-7445. Turned out to be a bad memory controller. But since Oracle 8 is happy, there's a little chance that that is the culprit on this one.

-Good Luck

[Scott Buckingham]

Thanks again, everyone, for your guidance. Yes, today is a new day and I'm hoping for glorious things!

I did find out that my oracle0 (primary user of Oracle 7 instance) path setting wasn't set to /usr/ccs/bin first. I also didn't have the SHLIB_PATH set. I added these two items and tried the link again. Here are the results:

rm -f oracle imp exp sqlldr
cc +O2 +Ofastaccess +Oprocelim -Wl,-PF,linkorderfile -Wl,-aarchive -o oracle -L/
software/oracle/product/7.3.4.5/lib -L/software/oracle/product/7.3.4.5/rdbms/lib
opimai.o /software/oracle/product/7.3.4.5/rdbms/lib/config.o -lserver -lclient
-lcommon -lgeneric -lknlopt -lslax -lapps -lcog -lcox -lidl -lknlde -lpkg -lpls
-lsem -lsyn -licx -lndw1 -lserver -lclient -lcommon -lgeneric -lknlopt -lslax -l
apps -lcog -lcox -lidl -lknlde -lpkg -lpls -lsem -lsyn -lserver -lclient -lcommo
n -lgeneric -lslax -lapps -lcog -lcox -lidl -lknlde -lpkg -lpls -lsem -lsyn -lse
rver -lsqlnet -lncr -lsqlnet -lclient -lcommon -lgeneric -lsqlnet -lncr -lsqlnet
-lclient -lcommon -lgeneric -lepc -lepcpt -lnlsrtl3 -lc3v6 -lcore3 -lnlsrtl3 -l
core3 -lnlsrtl3 -l:libcma.sl -lcl -lm -l:libcl.a -L/usr/lib -lm -lstublm -lm -l:
libcl.a -l:libdld.sl /usr/lib/libnsl.sl -l:libnss_dns.1
(Bundled) cc: warning 480: The +O2 option is available only with the C/ANSI C pr
oduct; ignored.
(Bundled) cc: warning 480: The +Ofastaccess option is available only with the C/
ANSI C product; ignored.
(Bundled) cc: warning 480: The +Oprocelim option is available only with the C/AN
SI C product; ignored.
/usr/ccs/bin/ld: Can't create oracle
/usr/ccs/bin/ld: Permission denied
*** Error exit code 1

Stop.

When Oracle 7 was originally installed on this box, I'm guessing it was done as the oracle user, not oracle0 (I created oracle0 when I installed 8.1.7). Would this confuse the issue? I have changed the owner to oracle0 in the $ORACLE_HOME/bin directory.

As far as hosts and services changing between reboots, I would say no. There are only 2 others here that have that kind of access to this box and they both prefer the Windows environment!

[Scott Buckingham]

OK. I'm back where I started! I finally got the relink to work by making sure my oracle0 user was the owner of all 7.3.4.5 files. Once I did this, all was well. I was able to start up the instance and I checked the alert log. All was good! tnsping works! Then the moment of truth; connecting to a database within the instance. OUCH! Same result as before. I get a trc file in my $ORACLE_HOME/rdbms/log directory, a core dump in /home/oracle (not oracle0!), and a ora-12500 error on the client.

Back to the drawing board...

[Steven E. Protter]

I would now test my tnsping network configuration. I would say that changes to name servers on your network could be effecting your tns configuration and your listener configuration.

You really are banging away at this and I admire your drive. Maybe try a post on oracles forums.

SEP

[Volker Borowski]

This is good news Scott :-)

OK,

please attach sqlnet.ora, listener.ora from the database server and tnsnames.ora from the client !

Check the permissions again. If your Listener runs under 8.1.7, the user-id of this listener needs to create an oracle 7 proces for a connect form a client to this specific ORACLE_7_HOME. This is, why you need to have the ORACLE_7_HOME in the LISTENER.ORA of the 8.1.7 Listener.
Now if you changed the permissions for the re-link in ORACLE_7_HOME, it may be, that the oracle 817 user now can not execute the oracle7 binary any more.

Consider to run a seperate listener for Oracle 7. Might be easier anyway.

Volker

[Michael Schulte zur Sur]

Hi,

according to your trace file you got an ora-07445 error. Look in oracle metalink Note:52742.1. It is mentioned there.
Also look Note:61314.1

Michael

[Scott Buckingham]

Here are the files you requested, Volker. As far as having a separate listener for Oracle 7, I wasn't sure that was possible. It just needs to be set to listen to a different port? This is all so mind-boggling since everything was working fine up until my reboot last Saturday!

[Michael Schulte zur Sur]

Hi again,

are there any errors in the alertlog when you try to connect? Also look into /var/adm/syslog/syslog.log

Michael

[Scott Buckingham]

Michael,

Thanks for joining in! As far as your questions, the syslog.log reveals no errors, at least nothing obvious. The alert_log, well, I have two; one in $ORACLE_HOME/rdbms/log (which shows the 07445 error over and over again) and the one in the bdump directory. This one shows no errors whatsoever. It's acting as if everything is OK!

[Michael Schulte zur Sur]

Hi Scott,

thanks for your warm wellcome.

I see, that the Oracle 7 has the latest patch already.

can you post kmtune -l as attachment?

Michael

[Scott Buckingham]

Here ya' go! In the meantime, I'm checking out Metalink (just so you know I'm trying as well!)

[Volker Borowski]

Hi Scott,

first find some usefull information about the NET8 config at http://www.oriolecorp.com/SQLNet.html

Your attached sqlnet.ora appears to be the one from your running listener. As I notice, the DEFAULT-Domain is not set !

First of all check the timestams of the files to find out wich one was edited last.

The TNSNAMES.ORA from the client looks fine, but refers ALL names to the domain METRIX-INC.COM.

The LISTENER.ORA has a global DB_NAME "aquila." for the ORA 7 DB !

Your error Message from the listener.log shows, that you actually try to connect to
the global name "aquila_TST0.world", where "world" seems to be the oracle-default domain.

I think somewhere is a domain mismatch between ".world" and ".METRIX-INC.COM", but I can not tell you where, because this mixed domain setup is not clear to me. I am only used to standard ".world" setups for all DBs involved.

Backup all the .../network/admin/*.ora files on the client. Try to change
AQUILA_TST0.METRIX-INC.COM
to
AQUILA_TST0.world
first.

May be install an sqlplus and/or tnsping on the client, it is easier for testing than to reconfigure the application.
Check the sqlnet.ora on the client for the DEFAULT_DOMAIN Setting.
Try to tnsping from the client. If ths works, try to connect from sqlplus.

Concerning the kernel:
- reduce dbc_max_pct -> 20...30
- maxfiles seems low to me -> 1024
- nfile shows 4000 and nproc 500 (is this the old kernel ?)

Good luck
Volker

[Scott Buckingham]

I just thought of something else I did before the reboot. I read somewhere that process.dat, regid.dat, and collect.dat were just trace collection files and that if they get too big or are not useful, they can be deleted. Well, instead of deleting them, I nulled them out (for Oracle 7 only) as the biggest one was well over 100 MB. Since they still exist but have nothing in them, is this confusing Oracle in some way?

[Volker Borowski]

Yum,

these are related to oracle trace, and need to be disabled when not in use.

http://www.jlcomp.demon.co.uk/faq/slow_connection.html

credits to google(.de), which I asked with "regid.dat process.dat oracle" -> right the first hit !

Volker

[Scott Buckingham]

Well, I now feel like a complete doofus! The DAT files were my problem and everything is OK again. I remember researching the .DAT-thing awhile ago and I had it in my notes to do the next time I restarted the Oracle 7 instance. My notes were not complete enough.

Thanks to all who toiled through this with me and a special thanks to Volker for showing the site that corrected the error in my ways.

[Volker Borowski · Accepted solution]

Glad it works for you Scott !

The best thing from these problems is what you can learn from this (and in this case all in 28 hours)!

If I had to place a bet before, I would have put my money on the NET-8 config.

I never heard about this tracefile stuff and the related problems before, so this one is filed up as a "good to know"!

Just keep in mind that google is your friend :-)
Volker