memory dump HPUX 11i v2

[bouazzaoui]

hello all,
I have an Itanium server (rx8620) and we add physical memory so now we have 92GB of RAM
the current configuration of SWAP, dump and crash is a s follow:
SWAP DATA
=========
4096 MB on /dev/vg00/lvol2 with Priority 1
40960 MB on /dev/vg00/lvol13 with Priority 0
89567 MB on memory
----
134623 MB Total



DUMP DISKS
==========
Dump: lvol2 on: /dev/dsk/c0t6d0s2, 0

DUMP DATA
=========
SAVECRASH: Disabled
SAVECRASH DIR: /var/adm/crash
SAVECRASH AVAILABLE SPACE: 15657MB

CRASHCONF: Enabled (referencing /etc/fstab)
Replace Option: CrashConf values REPLACE existing Kernel Definitions.
Included Pages:
Excluded Pages:
FSTAB File Entries:
CRASHCONF Configured MEMORY TO DUMP: 6259 MB

KERNEL SIZE: 57788152 Bytes.
TOTAL MEMORY TO DUMP: 89625 MB
DUMP DISK CAPACITY: 4096 MB

we had a crash last month and sent crash file to HP but they said that "crash dump is heavily truncated, due to insufficient dump space being configured on the system"
could you please help me to configure correctly my dump space so we can have enough to collect all memory dump?
we have no space available into internal disks but we can present some free space from an XP attached storage area

best regards,

[Mark S Meadows]

Hi,

It looks as though you are only utilising primary swap as a dump device.

Please run the following command as root and post the output :

crashconf -v

You can utilise the secondary swap device lvol13 as an additional dump device similar to the following :

# crashconf -v
Crash dump configuration has been changed since boot.

CLASS PAGES INCLUDED IN DUMP DESCRIPTION
-------- ---------- ---------------- -------------------------------------
UNUSED 21977226 no, by default unused pages
USERPG 18613247 no, by default user process pages
BCACHE 9584168 no, by default buffer cache pages
KCODE 10690 no, by default kernel code pages
USTACK 139267 yes, by default user process stacks
FSDATA 236 yes, by default file system metadata
KDDATA 6892157 yes, by default kernel dynamic data
KSDATA 7933 yes, by default kernel static data
SUPERPG 1438362 no, by default unused kernel super pages

Total pages on system: 58663286
Total pages included in dump: 7039593

Dump compressed: ON

Dump Parallel: ON

DEVICE OFFSET(kB) SIZE (kB) LOGICAL VOL. NAME
------------ ------------ ------------ ------------ -------------------------
1:0x000002 1447776 1048576 64:0x000002 /dev/vg00/lvol2
1:0x000002 2496352 7340032 64:0x000009 /dev/vg00/lvol9
1:0x000001 33555456 16777216 64:0x000011 /dev/vg00/swap2
1:0x000001 1024 33554432 64:0x00000c /dev/vg00/swap
------------
58720256
Persistent dump device list:
/dev/vg00/lvol2
/dev/vg00/lvol9
/dev/vg00/swap2
/dev/vg00/swap

Please see : man 1m crashconf.

Regards,

Mark

[bouazzaoui]

hello,
please find below the result command of crashconf -v:
crashconf -v
CLASS PAGES INCLUDED IN DUMP DESCRIPTION
-------- ---------- ---------------- -------------------------------------
UNUSED 2745297 no, by default unused pages
USERPG 17445092 no, by default user process pages
BCACHE 917167 no, by default buffer cache pages
KCODE 6928 no, by default kernel code pages
USTACK 39158 yes, by default user process stacks
FSDATA 1 yes, by default file system metadata
KDDATA 1711845 yes, by default kernel dynamic data
KSDATA 2897 yes, by default kernel static data
SUPERPG 60821 no, by default unused kernel super pages

Total pages on system: 22929206
Total pages included in dump: 1753901

Dump compressed: ON

DEVICE OFFSET(kB) SIZE (kB) LOGICAL VOL. NAME
------------ ---------- ---------- ------------ -------------------------
31:0x006000 940916 4194300 64:0x000002 /dev/vg00/lvol2
----------
4194300

[Mel Burslan]

It all depends on the capabilities of your FC card which connects your server to the XP frame. If your firmware level allows you to boot from and XP LUN as your primary OS drive, then you can create a dump device on XP. Otherwise, I'd suggest purchasing an additional 146GB drive to be added internally to be configured as dump/swap space. Disks are cheap nowadays. Maybe thy are not as cheap as going out the local computer store and buying a terabyte drive for $100 but relatively speaking, spending a thousand dollars on a drive which may save your crash dump, saving you a lot of future headache, is very well worth the price in my opinion.

[bouazzaoui]

hi Mel,
totally agree with you, except that we have already 2HD of 146GB on this machine and no empy space where we can add more disk :(
We use two Npars on this server and each of them use two HD of 146GB in order to work

please see the bdf for my server in case you have any suggestion :
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 491520 345792 144648 71% /
/dev/vg00/lvol1 414869 110297 263085 30% /stand
/dev/vg00/lvol7 12582912 669664 11820288 5% /var
/dev/vg00/lvol11 5242880 306293 4628054 6% /var/opt/perf/datafiles
/dev/vg00/lvol10 4096000 2012027 1953777 51% /var/adm/sw
/dev/vg00/lvol9 23068672 11867691 10500980 53% /var/adm/crash
/dev/vg00/lvol6 6160384 2590432 3542088 42% /usr
/dev/vg00/lvol5 6291456 2945440 3320120 47% /tmp
/dev/vg00/lvol12 20971520 20272025 656349 97% /sapcd
dev/vg00/lvol4 8192000 4808832 3367768 59% /opt
/dev/vg00/lvol8 2097152 308120 1775128 15% /home

[Mark S Meadows]

Hi,

Your current memory dump space requirement is :

>> Total pages included in dump: 1753901 <<

1753901 * 4 = 7015604

The current amount of dump space is 4194300, i.e :

Dump compressed: ON

DEVICE OFFSET(kB) SIZE (kB) LOGICAL VOL. NAME
------------ ---------- ---------- ------------ -------------------------
31:0x006000 940916 4194300 64:0x000002 /dev/vg00/lvol2
----------
4194300

I would look at utilising lvol13 as an additional dump device, but it will need to be configured as contiguous, i.e. :

gba70102:/home/markme# lvdisplay /dev/vg00/lvol9
--- Logical volumes ---
LV Name /dev/vg00/lvol9
VG Name /dev/vg00
LV Permission read/write
LV Status available/syncd
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 7168
Current LE 448
Allocated PE 896
Stripes 0
Stripe Size (Kbytes) 0
Bad block off
Allocation PVG-strict/contiguous <====
IO Timeout (Seconds) default

Regards,

Mark

[bouazzaoui]

hi Mark,
Thanks for you quick answer:
looking into config file I have from the server I found that dev/vg00/lvol13 is strict and bad block reloc is on but it's used as swap device
I tought that HPUX dumps it memory into swap devices, wish means he's using lvol13 so are you saying that lvol13 is not used for dumping?
I will try to answer myself: no it's not, because this lvol13 is not contiguous.
because I don't have access to this server (it's at customer site) I would like to now if there is no pb if:
-changing dev/vg00/lvol13 to strict-contiguous and set bad block relof to off will not harm the production server (and how to do that if possible)
- how can I ask hpux server to use the lvol13 as dump device?
thank you for your support

regards

[Mel Burslan]

If lvol13 is used as a *device* swap (not filesystem swap) space, it must already be contiguous. In order to designate it as additional dump space:

lvlnboot -d /dev/vg00/lvol13

before and after, run

lvlnboot -v

to be able to tell the difference.

[bouazzaoui]

hello,
this is the info I have from sysinfo regarding lvlnboot configuration:(sorry again but I don't have to the systems)
lvlnboot Data

Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0s2 (0/0/0/2/0.6.0) -- Boot Disk
/dev/dsk/c2t6d0s2 (0/0/0/3/0.6.0) -- Boot Disk
Boot: lvol1 on: /dev/dsk/c0t6d0s2
/dev/dsk/c2t6d0s2
Root: lvol3 on: /dev/dsk/c0t6d0s2
/dev/dsk/c2t6d0s2
Swap: lvol2 on: /dev/dsk/c0t6d0s2
/dev/dsk/c2t6d0s2
Dump: lvol2 on: /dev/dsk/c0t6d0s2, 0

regards

[Mark S Meadows · Accepted solution]

Hi,

Because a device has been configured as a secondary swap device does not always mean that it has been set up as contiguous. I often come across secondary swap devices in use, but set up as none contiguous.

Primary swap (lvol2) must be configured as contiguous and is primarily always set up by the operating system install and build scripts.

Currently lvol13 is not being utilised as a dump device and the only device being used at the moment is lvol2, which is insufficient in size for your system dump requirements - hence HP reporting that your crash dumps are being heavily truncated.

To see if lvol13 can be configured as contiguous run the following command :

lvchange -C y -r n /dev/vg00/lvol13

the command will fail if the logical volume is not in one contiguous chunk.

Another way to get around it if you have sufficient large enought space in vg00, which I probably doubt, is to unmirror lvol13 and re-mirror/pvmove to a contiguous area. This is how I have got around the problem in the past of making none contiguous lv's into contiguous.

Assuming lvol13 can be made contiguous and the "Bad block location policy" set to "off" the following is how to go about setting it up as an additional dump device.

1. Add lvol13 as an additional "dump" device in the /etc/fstab - it should already have an entry as a swap device :

/dev/vg00/lvol13 ... swap defaults 0 0 - currently present or similar

/dev/vg00/lvol13 ... dump defaults 0 0 - newly added

2. Add lvol13 as listed above in the /etc/fstab to the system dump configuration :

crashconf -a

3. Make lvol13 a new dump device persistant across future system reboots :

crashconf -s

Display the crash dump configuration as well as the list of persistant dump devices :

crashconf -v

Please note that the total memory dump space required on a system varies all the time depending upon the memory load on the system, i.e. it can go UP as well as Down.

Please find attached some documentation I have drawn up as an aid.

I hope the above is of help.

Regards,

Mark