Simpler Navigation for Servers and Operating Systems - Please Update Your Bookmarks
Completed: a much simpler Servers and Operating Systems section of the Community. We combined many of the older boards, so you won't have to click through so many levels to get at the information you need. Check the consolidated boards here as many sub-forums are now single boards.
If you have bookmarked forums or discussion boards in Servers and Operating Systems, we suggest you check and update them as needed.
General
cancel
Showing results for 
Search instead for 
Did you mean: 

Oracle error after HP UX patch updation

SOLVED
Go to solution

Oracle error after HP UX patch updation

I recently updated my HP UX 11.23 to latest available patches. Everything seems to be working fine but on Oracle 9i following error occured while creating archives.

Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_arc1_557.trc:

ORA-19502: write error on file "/backup/archives/1_85.arc", blockno 1 (blocksize=1024)

ORA-27072: skgfdisp: I/O error

HPUX-ia64 Error: 22: Invalid argument

Can anybody have any clue?
21 REPLIES
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

You need to check the disk for the mountpoint /backup, check whether its online.

Also check syslog.log to see additional information for this error.
Jozef_Novak
Respected Contributor

Re: Oracle error after HP UX patch updation

Hello,

I did a quick search on this topic, all results suggest to check the file that is unable to be written to. This might include incorrect file permissions, filesystem full or a disk HW error.

Jozef

Re: Oracle error after HP UX patch updation

/backup filesystem is mounted. Space is also available and permission is 644 with owner is oracle.
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

ls -ld /backup
ls -ld /backup/archives

it should be 755 and not 644, please check.
Steven E. Protter
Exalted Contributor

Re: Oracle error after HP UX patch updation

Shalom,

Try relinking oracle.

That usually solvesproblems like this.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

Steven is right, relinking oracle should solve most of these issues unless you did not explicitly change permissions for these directories.

Re: Oracle error after HP UX patch updation

Yes. /backup have 755 and /backup/archive have 777.

rariasn
Honored Contributor

Re: Oracle error after HP UX patch updation

Hi,


Stop all Oracle process, and

$ relink all

rgs.

Re: Oracle error after HP UX patch updation

I relinked the oracle binaries but still facing same issue. If I make datafiles and archives < 4 MB it works fine. But when it exceeds this size again same issue occurs. Please advice as this is my production database.
Mikej_2
Advisor

Re: Oracle error after HP UX patch updation

Do you have space left in /backup/archives ?
What gives :
df -k /backup/archives
and :
bdf /backup/archives

?

Re: Oracle error after HP UX patch updation

I have checked it earlier also, there is no issue of free space. We have more than 10 GB of free space (I think :))
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

did you check the syslog.log and the oracle log files for additional information please.

Re: Oracle error after HP UX patch updation

My syslog.log show

Mar 17 23:32:56 pibaspr vmunix:
Mar 17 23:32:56 pibaspr vmunix: SCSI: Write error -- dev: b 31 0x010100, errno: 22, resid: 0,
Mar 17 23:32:56 pibaspr vmunix: blkno: 142735392, sectno: 4848128, offset: 146161041408, bcount: 1048576.
Mar 17 23:32:56 pibaspr vmunix: blkno: 142736416, sectno: 4843354, offset: 146162089984, bcount: 1015808.
Mar 17 23:32:56 pibaspr vmunix: blkno: 142751776, sectno: 98304, offset: 146177818624, bcount: 1048576.
Mar 17 23:32:56 pibaspr vmunix: blkno: 142752800, sectno: 82402, offset: 146178867200, bcount: 1048576.
Mar 17 23:32:56 pibaspr vmunix:
Mar 17 23:35:00 pibaspr above message repeats 3 times
Mar 17 23:32:56 pibaspr vmunix: SCSI: Write error -- dev: b 31 0x010100, errno: 22, resid: 0,
Mar 17 23:35:00 pibaspr above message repeats 3 times

Oracle Log: I have already sent.
T G Manikandan
Honored Contributor
Solution

Re: Oracle error after HP UX patch updation

grep 22 /usr/include/sys/errno.h points to **INVALID ARGUMENT**

Did you manually remount any of the mount points after the server restart.

I suspect if you had mount options like async for the file system.

Please let me know.

Re: Oracle error after HP UX patch updation

No I didn't mounted any filesystem after reboot. All were mounted automatically.

My fstab is

"/etc/fstab" 11 lines, 495 characters
/dev/vg00/lvol3 / vxfs delaylog 0 1
/dev/vg00/lvol1 /stand vxfs tranflush 0 1
/dev/vg00/lvol4 /tmp vxfs delaylog 0 2
/dev/vg00/lvol5 /home vxfs delaylog 0 2
/dev/vg00/lvol6 /opt vxfs delaylog 0 2
/dev/vg00/lvol7 /usr vxfs delaylog 0 2
/dev/vg00/lvol8 /var vxfs delaylog 0 2
/dev/vg00/oracle /oracle vxfs rw,suid,largefiles,delaylog,datainlog 0 2
/dev/dsk/c1t0d1 /oracledb vxfs rw,suid,delaylog,datainlog,largefiles 0 1
/dev/dsk/c1t0d2 /backup vxfs defaults 0 2
/dev/vg00/lv13 ... swap pri=1 0 2

Actually I forgot to look into fstab before applying patches but

# ls -lt /etc/fstab
-rw-r--r-- 1 root sys 495 Oct 12 00:40 /etc/fstab

Meaning it is not disturbed.

Re: Oracle error after HP UX patch updation

Also wanted to add that my disks are local scsi disks. I dont have SAN attached.
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

can you please attach the oracle alert log file please.

Thanks

Re: Oracle error after HP UX patch updation

Tue Mar 17 23:30:13 2009
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
SCN scheme 3
LICENSE_MAX_USERS = 0
SYS auditing is enabled
Starting up ORACLE RDBMS Version: 9.2.0.7.0.
System parameters with non-default values:
processes = 4000
sessions = 4405
timed_statistics = TRUE
resource_limit = TRUE
shared_pool_size = 637534208
sga_max_size = 4296498144
large_pool_size = 16777216
java_pool_size = 0
control_files = /oracle/home/oradata/pibassrv/control01.ctl, /oracle/home/oradata/pibassrv/control02.ctl, /oracle/home/oradata/pibassrv/control03.ctl
db_block_size = 8192
db_cache_size = 3254779904
compatible = 9.2.0.0.0
log_archive_start = FALSE
log_archive_dest = /backup/archives
log_archive_dest_1 =
log_archive_dest_2 =
standby_archive_dest = /oracledb/pibasdb/archive
log_archive_format = %t_%s.arc
db_file_multiblock_read_count= 16
fast_start_mttr_target = 300
undo_management = AUTO
undo_tablespace = UNDOTBS2
undo_retention = 10800
remote_os_roles = TRUE
remote_login_passwordfile= EXCLUSIVE
audit_sys_operations = TRUE
db_domain =
commit_point_strength = 200
instance_name = pibassrv
job_queue_processes = 10
hash_join_enabled = TRUE
background_dump_dest = /oracle/home/admin/pibassrv/bdump
user_dump_dest = /oracle/home/admin/pibassrv/udump
core_dump_dest = /oracle/home/admin/pibassrv/cdump
open_links = 200
open_links_per_instance = 200
audit_trail = DB
sort_area_size = 524288
db_name = pibassrv
open_cursors = 3000
star_transformation_enabled= FALSE
_no_or_expansion = TRUE
query_rewrite_enabled = FALSE
pga_aggregate_target = 209715200
PMON started with pid=2
DBW0 started with pid=3
LGWR started with pid=4
CKPT started with pid=5
SMON started with pid=6
RECO started with pid=7
CJQ0 started with pid=8
Tue Mar 17 23:30:15 2009
ALTER DATABASE MOUNT
Tue Mar 17 23:30:19 2009
Successful mount of redo thread 1, with mount id 1896047879
Tue Mar 17 23:30:19 2009
Database mounted in Exclusive Mode.
Completed: ALTER DATABASE MOUNT
Tue Mar 17 23:30:26 2009
alter database archivelog
Completed: alter database archivelog
Tue Mar 17 23:30:33 2009
alter database open
Tue Mar 17 23:30:33 2009
LGWR: Primary database is in CLUSTER CONSISTENT mode
Thread 1 opened at log sequence 1271
Current log# 4 seq# 1271 mem# 0: /oracle/home/oradata/pibassrv/log4.ora
Successful open of redo thread 1
Tue Mar 17 23:30:33 2009
SMON: enabling cache recovery
Tue Mar 17 23:30:33 2009
Successfully onlined Undo Tablespace 11.
Tue Mar 17 23:30:33 2009
SMON: enabling tx recovery
Tue Mar 17 23:30:33 2009
Database Characterset is WE8ISO8859P1
replication_dependency_tracking turned off (no async multimaster replication found)
Completed: alter database open
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02019: connection description for remote database not found
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
Tue Mar 17 23:31:18 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325

..................

Tue Mar 17 23:31:40 2009
Thread 1 advanced to log sequence 1272
Current log# 1 seq# 1272 mem# 0: /oracle/home/oradata/pibassrv/log1.ora
Tue Mar 17 23:31:40 2009
ARCH: Evaluating archive log 4 thread 1 sequence 1271
ARCH: Beginning to archive log 4 thread 1 sequence 1271
Creating archive destination LOG_ARCHIVE_DEST_1: '/backup/archives/1_1271.arc'
ARCH: Completed archiving log 4 thread 1 sequence 1271
Tue Mar 17 23:31:51 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02019: connection description for remote database not found
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
Tue Mar 17 23:31:51 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:

................................
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
Tue Mar 17 23:32:56 2009
create tablespace abc datafile '/oracledb/pibasdb/datafiles/vicky.dbf' size 50M logging extent management local segment space management auto
Tue Mar 17 23:32:56 2009
ORA-19502 signalled during: create tablespace abc datafile '/oracledb/pibasdb/...
Tue Mar 17 23:33:52 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02019: connection description for remote database not found
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
Tue Mar 17 23:33:52 2009
Errors in file /oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found



===================================================================================



/oracle/home/admin/pibassrv/bdump/pibassrv_reco_1997.trc
Oracle9i Enterprise Edition Release 9.2.0.7.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.7.0 - Production
ORACLE_HOME = /oracle/home/Ora9i
System name: HP-UX
Node name: pibaspr
Release: B.11.23
Version: U
Machine: ia64
Instance name: pibassrv
Redo thread mounted by this instance: 1
Oracle process number: 7
Unix process pid: 1997, image: oracle@pibaspr (RECO)

*** SESSION ID:(6.1) 2009-03-17 23:31:18.430
*** 2009-03-17 23:31:18.430
ERROR, tran=3.13.177816, session#=1, ose=0:
ORA-02019: connection description for remote database not found
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
*** 2009-03-17 23:31:18.433
ERROR, tran=5.31.204647, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.434
ERROR, tran=6.15.169919, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.435
ERROR, tran=7.16.172083, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.435
ERROR, tran=3.43.178069, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.436
ERROR, tran=3.24.178078, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.437
ERROR, tran=7.28.172198, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.439
ERROR, tran=1.18.295041, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.440
ERROR, tran=10.9.316890, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325
ORA-02019: connection description for remote database not found
*** 2009-03-17 23:31:18.452
ERROR, tran=11.29.52287, session#=1, ose=0:
ORA-02062: distributed recovery received DBID c63e2355, expected 25efd325








T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

This looks like a HPUX issue, the patch database points to PHKL_38574 for 11.31.

hmm you are running HPUX 11.23,checking for patch for 11.23
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

ok its patch PHKL_37840 for 11.23.
T G Manikandan
Honored Contributor

Re: Oracle error after HP UX patch updation

Please go the latest version of the above patch
PHKL_39259


/*(SR: QXCR1000806947)
SYMANTEC Incident Number: 1426690 (1415892)
The getacl(1)command may return "Invalid argument" (EINVAL)
error when operating on a directory with only default ACL
entries if chmod is used to make the class permission differ
from the owning group's permission.
*/


This is a kernel patch , I recommend you to prepare a ignite image for your server before you start installing this patch. I hope this would fix it.