- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: Received unhandled signal: 15
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 09:58 AM
тАО01-17-2006 09:58 AM
Received unhandled signal: 15
On Friday, 13th, for the first time ever, oracle crashed! Impossible? But True!
HPUX 11i on Rp5430 with reasonably up-to-date pathes and NO sign of any OS or HW problems what so ever.
-->swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 0 4194304 0% 0 - 1 /dev/vg00/lvol2
dev 4194304 1078712 3115592 26% 0 - 0 /dev/vg08/lvol01
localfs 1048576 0 1048576 0% 1048576 0 1 /u02/paging
reserve - 4738696 -4738696
memory 3300728 926748 2373980 28%
root@cis1: in /home/root
Suddenly, with out any warning, and with NOTHING logged anywhere (no alert.log, lsnr.log, anything), ALL the oracle processes, both system and user, terminated! Very drastic and severe!.
The only trace left behind were 100's small trace files in the udump and bdump directories, one for each of the terminated processes. These trace files all look like:
/u01/app/oracle/admin/csccis/bdump/csccis_s000_20366.trc
Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.6.0 - Production
ORACLE_HOME = /u01/app/oracle/product/920
System name: HP-UX
Node name: cis1
Release: B.11.11
Version: U
Machine: 9000/800
Instance name: csccis
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 20366, image: oracle@cis1 (S000)
*** 2006-01-17 10:11:46.631
Received unhandled signal: 15, code=800003ffbfff68e8
Terminating.
I've logged a call on Friday, but Oracle are still scratching there heads.
I'm not the only one in the world who has experienced this either - see metalink doc id's 608324.999, 573563.994, and several others. There are no useful responses to these metalink article.
Is there anyone out there with any clues?
Many Thanks
Leon Allen
Caboolture, Australia
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 10:12 AM
тАО01-17-2006 10:12 AM
Re: Received unhandled signal: 15
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 10:17 AM
тАО01-17-2006 10:17 AM
Re: Received unhandled signal: 15
I forgot to mention, the silly thing crashed again yesterday, exactly the same symptoms. That's twice now in as many (work) days. I'm stating to really worry now.
On Friday, it was a quite afternnon - not too much activity at all. Yesterday was a normal morning.
To restart after the crash, I have to do
sqlplus /nolog
connect / as sysdba
shutdown abort
startup open
.
.
lsnrctl start
agntctl start
ie the listener process and agent process seem to crash as well.
Cheers!
Leon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 10:26 AM
тАО01-17-2006 10:26 AM
Re: Received unhandled signal: 15
Given the none-specific times, I haven't suspected a cron job. (pm. one day, am. two days later).
I've check .sh history of root and oracle to see if there was any funny business going on - but did not detect anything.
I might do a cat or string * | grep -l kill (or something like that) to see if anything has been scripted, but that is most unlikely (there is really only me here).
I think the crashes are genuine (cf accidental or malicious)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 03:56 PM
тАО01-17-2006 03:56 PM
Re: Received unhandled signal: 15
did you install any patch or other software recently?
also, did you change any kernel parameters recently?
I would also check my syslog file and run a "analyze table ... validate structure" on the tables (if feasible) or do a full database export to make sure there is no form of data corruption somewhere....
also, make sure your backups are up to date and you can perform recovery successfully on your backup server.
take all your precautions
kind regards
yogeeraj
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-17-2006 04:42 PM
тАО01-17-2006 04:42 PM
Re: Received unhandled signal: 15
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-18-2006 09:43 AM
тАО01-18-2006 09:43 AM
Re: Received unhandled signal: 15
And thanks RAC; yes, Clays perspective on it was interesting, and did get me thinking. Thinking so much my brain started to hurt, and I did just yesterday turn on auditing of processes, including kill, via sam. I checked what accounts could potetially initiate a kill - we have a GIS account which is a member of the dba group, which through the gis application (ArcSDE) can execute a range of commands. I'm going to keep an eye on this, for if it happens again.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-18-2006 12:43 PM
тАО01-18-2006 12:43 PM
Re: Received unhandled signal: 15
IIRC there are ways to register signal handlers such that they can be given a siginfo_t structure that includes information about the origin of the signal. Perhaps Oracle could create a "bugcatcher" that does this if there is no convenient auditing mechanism available.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-18-2006 01:08 PM
тАО01-18-2006 01:08 PM
Re: Received unhandled signal: 15
The audit trail so far (oracle user)
-->audisp -c kill -u oracle /u03/.secure/etc/audfile4
users and aids:
oracle
12
Selected the following events:
37
All ttys are selected.
Selecting successful & failed events.
TIME PID E EVENT PPID AID RUID RGID EUID EGID TTY
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
060118 14:41:22 17288 S 37 17287 12 102 102 102 102 pts/tf
[ Event=kill; User=oracle; Real Grp=dba; Eff.Grp=dba; ]
RETURN_VALUE 1 = 0;
PARAM #1 (int) = 0
PARAM #2 (int) = 15
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
060118 14:41:22 17288 S 37 17287 12 102 102 102 102 pts/tf
[ Event=kill; User=oracle; Real Grp=dba; Eff.Grp=dba; ]
RETURN_VALUE 1 = 0;
PARAM #1 (int) = 0
PARAM #2 (int) = 26
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
root@cis1: in /home/root
-->
There are a lot more 'kill's' by root. I presue this is all normal (oracle hasn't crashed again yet), and I might see an 'exception' in this audit log if / when it does crash.
What does the above data tell me? PARAM #1 looks like a pid, and PARAM #2 looks like a signum? (opposite order for the actual kill comamnd parameters)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО01-18-2006 01:11 PM
тАО01-18-2006 01:11 PM