topic Re: System hang after modification a tunable kernel parameter in Operating System - HP-UX

System hang after modification a tunable kernel parameter

Speedware — Fri, 20 Jul 2007 09:37:57 GMT

Hello,

I am currently having the following problem on A500 with 2 cpu 11.23 PA-RISC server with 2 gig of ram. After modify the following kernel parameter max_mem_window from 5 to 15 using kcweb interface. The server would hang at the following line /sbin/auto_parm is DHCP is disabled is disable see /etc/auto_parm.log. I waited like 30 minutes before I decided to reboot the server. Then trying to boot in single user mode, I saw the following message /etc/ioconfig and /stand/ioconfig are inconsistant and the system did a reboot itself. After that I was able to reboot in single user. Then I was able to launch init 1 init 2 and init 3, my system was booting correctly and was online. But my modification to the max_mem_window when back to what it was before. (I don't know why) I try to boot from the "backup" kernel before my modification, the same problem. Next I try to redo my modification to the max_mem_window. I had the same problem but the now the modification were kept.
I also by way I did the same modification on a 11.23 itanium server at the same time and I did not get the problem. Actually the value was set higher from 5 to 30 but I have a lot more ram on that system.

Any help at this point would be greatly appreciated.

Re: System hang after modification a tunable kernel parameter

Dave Hutton — Fri, 20 Jul 2007 09:58:35 GMT

Are you running out of space in /stand after you make the kernel adjustments. It sounds like it's going back to the previous kernel.

What does bdf /stand show?

Reading man 5 max_mem_window it doesn't sound like it could cause too many problems. It does suggest that more memory would be used by the kernel. With only 2 gigs of memory I doubt this is your problem being you get that inconsistant error.

Re: System hang after modification a tunable kernel parameter

Speedware — Fri, 20 Jul 2007 10:01:45 GMT

here is the output of bdf
# bdf -l
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 409600 334016 75152 82% /
/dev/vg00/lvol1 298928 92048 176984 34% /stand
/dev/vg00/lvol8 4710400 830704 3850936 18% /var
/dev/vg00/lvol7 3260416 1968368 1281968 61% /usr
/dev/vg00/lvol4 1024000 79368 937864 8% /tmp
/dev/vg00/lvol6 8192000 7518480 668312 92% /opt
/dev/vg00/lvol5 24576 17392 7184 71% /home

Re: System hang after modification a tunable kernel parameter

Fabian Briseño — Fri, 20 Jul 2007 11:52:30 GMT

Hello speedware.

is there any error message appearing in /var/adm/syslog/syslog.log ?

another option is to recover the system from a previous ignite backup.

Re: System hang after modification a tunable kernel parameter

Jaime Bolanos Rojas. — Fri, 20 Jul 2007 12:05:22 GMT

Speedware,

Going to the main error, "/sbin/auto_parm is DHCP is disable" what I would do is check the "/etc/rc.config.d/netconf file, compare it with the system that is working fine.

Regards,

Jaime.

Re: System hang after modification a tunable kernel parameter

Don Morris_1 — Fri, 20 Jul 2007 21:56:25 GMT

On 11.23, raising max_mem_window would cause *very* little extra memory to be allocated by the kernel -- we're talking about 80 bytes difference for the values given (exact values would depend on padding, cache line size, if the kernel gets a larger page which is then used for other allocations as well, etc. etc.). There'd be a higher cost if the extra windows are actually used, of course [around 1Mb per] -- but you only pay for what you're really using.. not at tunable change time.

It sounds to me like something else was pending (or at least kcweb thought it was) and the ioconfig and Kernel Registry (for the tunable change) were being written out at the time of the hang. When you TOC'd, the Registry wasn't written yet, and the ioconfig was in an inconsistent state.

To fix your ioconfig issue, it seems to me the following from ioinit(1M) applies:

If /etc/ioconfig is corrupted or missing when the system reboots, ioinitrc brings the system up in single-user mode. The user should then restore /etc/ioconfig from backup or invoke the ioinit with the -c option to recreate /etc/ioconfig from the kernel.

Hope this helps -- sorry it was a VM tunable you were happening to modify. I can assure you it doesn't have anything to do with DHCP on our end (and if they use Memory Windows, I'd be surprised to no end...)

Re: System hang after modification a tunable kernel parameter

Speedware — Mon, 23 Jul 2007 09:55:06 GMT

Fabian:
No error message in syslog because the daemon not started yet when in hang. And when I start with init 1, 2, 3, no error either in syslog. Everything normal.

Jaime: I already did with the itanium system running 11.23. they are the same.

Don: I will try that later on today with some plan downtime.

I will let you know if that work or now.
Thanks for the help.

Re: System hang after modification a tunable kernel parameter

Speedware — Tue, 24 Jul 2007 10:20:57 GMT

I try to restore from backup /etc/ioconfig (about a week before the problem arrive). I check the size and the cksum of the files they are exactly the same as the one that is currently in /etc. So I did not bother to replace the existing one with the backup

I also try to recreating the files using the following command ioinit -c to recreate the files. I rebooted the server but I still have the same problem.

I am also thinking of copying the default /usr/newconfig/etc/initttab over my current inittab. Do you think that would help ?

I added debug code in the rc script to see where I hanging exactly. I am planning a reboot tonight to try to get more information.

Re: System hang after modification a tunable kernel parameter

Speedware — Thu, 26 Jul 2007 11:20:23 GMT

/sbin/auto_parms: DHCP access is disabled (see /etc/auto_parms.log)

we are after auto_parms
new level
3
previous run level
S
BOOT
1
new run level is higher than old, so run start scripts in
all intermediate run levels and in new run level.
is_terminal_hp
LOGFILE=/etc/rc.log
return code from init_list: 1
after init_list HP-UX Start-up in progress
after init_list
add_list /sbin/rc1.d/S080crashconf start Configure system crash dumps
add_list /sbin/rc1.d/S091vxvm-nodes-check start VxVM device node check
add_list /sbin/rc1.d/S092vxvm-startup start VxVM general startup
add_list /sbin/rc1.d/S093vxvm-reconfig start VxVM reconfiguration recovery
add_list /sbin/rc1.d/S100localmount start Mount file systems
add_list /sbin/rc1.d/S320hostname start Setting hostname
add_list /sbin/rc1.d/S350kl start Start Kernel Logging facility
add_list /sbin/rc1.d/S400set_prvgrp start Set privilege group
add_list /sbin/rc1.d/S420set_date start Display date
add_list /sbin/rc1.d/S430pdcinfo start Copy processor logs to /var/tombstones
add_list /sbin/rc1.d/S440savecrash start Save system crash dump if needed
add_list /sbin/rc1.d/S450evp.init start Start evp
add_list /sbin/rc1.d/S500swap_start start Enable auxiliaryadd_list /sbin/rc1.d/S
520syncer start Start syncer daemon
add_list /sbin/rc1.d/S600utmpd start Start Utmp Daemon : manages User Accounting
Database
add_list /sbin/rc1.d/S900CreateUsbSf start Configuring USB-00 device special fil
es
add_list /sbin/rc2.d/S008net.init start Configure Loopback interfaces (lo0)
add_list /sbin/rc2.d/S096vxvm-recover start VxVM volume recovery start
add_list /sbin/rc2.d/S120swconfig start Configuring all unconfigured software fi
lesets
add_list /sbin/rc2.d/S130pfilboot start Configuring PFIL
add_list /sbin/rc2.d/S131ipfboot start Starting IPFilter
add_list /sbin/rc2.d/S135sec_mgmt start Configuring Install Time Security
add_list /sbin/rc2.d/S200clean_ex start Recover editor crash files
add_list /sbin/rc2.d/S202clean_uucp start Clean UUCP
add_list /sbin/rc2.d/S204clean_tmps start List and/or clear temporary files
add_list /sbin/rc2.d/S206clean_adm start Clean up old log files
add_list /sbin/rc2.d/S220syslogd start Start system message logging daemon
add_list /sbin/rc2.d/S230ptydaemon start Start pty allocator daemon
add_list /sbin/rc2.d/S290pci_olar start Configuring OLA/R interface
add_list /sbin/rc2.d/S300nettl start Start network tracing and logging daemon
add_list /sbin/rc2.d/S305hpigelan start Configure HP igelan Gigabit Ethernet int
erfaces
add_list /sbin/rc2.d/S306hpiether start Configure HP iether 100BT/Gigabit Ethern
et interfaces
add_list /sbin/rc2.d/S308hpgelan start Configure HP gelan Gigabit Ethernet inter
faces
add_list /sbin/rc2.d/S329hppcifddi start Create PCI FDDI device files.
add_list /sbin/rc2.d/S333hpbtlan start Configure HP 100BASE-T interfaces
add_list /sbin/rc2.d/S337vlan start Configure VLAN interfaadd_list /sbin/rc2.d/S
340net start Configure LAN interfaces
add_list /sbin/rc2.d/S340net-ipv6 start Configure LAN interfaces for IPv6
add_list /sbin/rc2.d/S356dhcpv6.server start Starting DHCPv6 Server daemon
add_list /sbin/rc2.d/S360dhcpv6.client start Configuring DHCPv6 Interfaces
add_list /sbin/rc2.d/S370named start Start name server daemon
add_list /sbin/rc2.d/S393secsh start Starting HP-UX Secure Shell
add_list /sbin/rc2.d/S400nfs.core start Start NFS core subsystem
add_list /sbin/rc2.d/S406nisplus.server start Start NIS+ server subsystem
add_list /sbin/rc2.d/S408nisplus.client start Start NIS+ client subsystem
add_list /sbin/rc2.d/S410nis.server start Start NIS server subsystem
add_list /sbin/rc2.d/S420nis.client start Start NIS client subsystem
add_list /sbin/rc2.d/S430nfs.client start Start NFS client subsystem
add_list /sbin/rc2.d/S440comsec start Start the Trusted Mode with NIS+ subsystem

add_list /sbin/rc2.d/S490mrouted start Start multicast routing daemon
add_list /sbin/rc2.d/S500inetd start Start Internet services daemon
add_list /sbin/rc2.d/S510gated start Start dynamic routing daemon
add_list /sbin/rc2.d/S520rdpd start Start router discover protocol daemon
add_list /sbin/rc2.d/S522ppp start Configuring PPP Interface
add_list /sbin/rc2.d/S523pppoe start Configuring PPPoE
add_list /sbin/rc2.d/S525rarpd start Start RARP protocol daemon
add_list /sbin/rc2.d/S530rwhod start Start remote system status daemon
add_list /sbin/rc2.d/S538rtradvd start Start IPv6 router advertisement daemon
add_list /sbin/rc2.d/S540sendmail start Starting mail daemon
add_list /sbin/rc2.d/S550ddfa start Starting outbound connection daemons for DDF
A software
add_list /sbin/rc2.d/S560SnmpMaster start Start SNMP Master Network Management d
aemon
add_list /sbin/rc2.d/S565OspfMib start Start OSPF MIB Network Management subAgen
t
add_list /sbin/rc2.d/S565SnmpHpunix start Start SNMP HP-UNIX Network Management
subAgent
add_list /sbin/rc2.d/S565SnmpIpv6 start Start SNMP IPv6 Network Management subAg
ent
add_list /sbin/rc2.d/S565SnmpMib2 start Start SNMP MIB-2 Network Management subA
gent
add_list /sbin/rc2.d/S565SnmpNaa start Start Native Adapter Agent
add_list /sbin/rc2.d/S565SnmpTrpDst start Start SNMP Trap Dest Network Managemen
t subAgent
add_list /sbin/rc2.d/S570dce start Start DCE daemons
add_list /sbin/rc2.d/S576SnmpFddi4 start Start SNMP PCI FDDI Network Management
subAgent
add_list /sbin/rc2.d/S590Rpcd start Start RPC daemon if needed
add_list /sbin/rc2.d/S600cimserver start Start CIM cimserver subsystem
add_list /sbin/rc2.d/S602icod start Initialize Instant Capacity
add_list /sbin/rc2.d/S605hpservices start Starting ISEE hpservices mad daemon.
add_list /sbin/rc2.d/S620xfs start Starting X Font Server at TCP port 7000
add_list /sbin/rc2.d/S630vt start Start vt daemon
add_list /sbin/rc2.d/S660xntpd start Start time synchronization
add_list /sbin/rc2.d/S700acct start Start accounting
add_list /sbin/rc2.d/S705pwgr start Starting the password/group assist subsystem

add_list /sbin/rc2.d/S710hparamgr start Starting HP Disk Array Manager daemons.

add_list /sbin/rc2.d/S720lp start Start print spooler
add_list /sbin/rc2.d/S730cron start Start clock daemon
add_list /sbin/rc2.d/S740spc start Run Security Patch Check Utility
add_list /sbin/rc2.d/S742diagnostic start Start diagnostic subsystem
add_list /sbin/rc2.d/S750envd start Start environment monitoring daemon
add_list /sbin/rc2.d/S760auditing start Start auditing subsystem
add_list /sbin/rc2.d/S770audio start Start audio server daemon
add_list /sbin/rc2.d/S780slsd start Start Distributed Single Logical Screen daem
on
add_list /sbin/rc2.d/S785hub start Start USB hub daemon
add_list /sbin/rc2.d/S800spa start SAM System administration configuration
add_list /sbin/rc2.d/S810mwa start MeasureWare performance software is being sta
rted.
add_list /sbin/rc2.d/S870swagentd start Initialize Software Distributor agent da
emon
add_list /sbin/rc2.d/S895gwlmagt start Starting the gWLM Agent
add_list /sbin/rc2.d/S900cifsclient start Starting CIFS Client
add_list /sbin/rc2.d/S900ciss start Configure HP RAID SA interfaces
add_list /sbin/rc2.d/S900cpumap2 start Copy processor logs to /var/tombstones
add_list /sbin/rc2.d/S900eloq6 start Starting Eloquence
add_list /sbin/rc2.d/S900ems start Starting Event Monitoring Service
add_list /sbin/rc2.d/S900emsa start Start EMS SNMP subagent
add_list /sbin/rc2.d/S900hpfcms start Configure HP Fibre Channel Mass Storage in
terfaces
add_list /sbin/rc2.d/S900mpt start Configuring Ultra320 SCSI Mass Storage interf
aces
add_list /sbin/rc2.d/S900roc_om_server start Starting Rhapsody Script: roc_om_se
rver
add_list /sbin/rc2.d/S900roc_om_web start Starting Rhapsody Script: roc_om_web
add_list /sbin/rc2.d/S900samba start Starting SAMBA Server
add_list /sbin/rc2.d/S910rstEMSlistener start Starting the RST EMS listener
add_list /sbin/rc2.d/S989fc_td start Configure HP Fibre Channel Tachyon TL/TS Ma
ss Storage interfaces
add_list /sbin/rc2.d/S990fc_fcd start Configuring HP Fibre Channel FCD Mass Stor
age interfaces
add_list /sbin/rc2.d/S994vxnm-vxnetd start VVR - Starting Communication Daemon
add_list /sbin/rc2.d/S999isisd start Starting VERITAS Enterprise Administrator S
ervice (vxsvc)
add_list /sbin/rc3.d/S100nfs.server start Start NFS server subsystem
add_list /sbin/rc3.d/S200tps.rc start Start X print server(s)
add_list /sbin/rc3.d/S823hpws_apache start Starting HP-UX Apache-based Web Serve
r
add_list /sbin/rc3.d/S823hpws_tomcat start Starting HP-UX Tomcat-based Servlet E
ngine
add_list /sbin/rc3.d/S823hpws_webmin start Starting HP-UX Webmin-based Admin
add_list /sbin/rc3.d/S823hpws_webproxy start Starting the HPUX Webproxy subsyste
m
add_list /sbin/rc3.d/S823hpws_xmltools start Starting HP-UX XML Web Server Tools

add_list /sbin/rc3.d/S828kwdbd start Start kwdbd
add_list /sbin/rc3.d/S829nddbcs start Start NDDB Comm Server
add_list /sbin/rc3.d/S870hpsmhd start Starting the System Management HomePage se
rver

This is were it hang after adding echo in the rc startup script ...