Ruuning universe in 32 bit mode

[Jawed Akhtar]

We are running universe on hp-ux 11 64 bit OS.
But our univers is rnning in 32 bit mode as per uvconfig file parameter 64BIT_FILES is set to 0. Last month our F.JOURNAL file excedde 2GB file size limit and application stopeed to accept any more data.
1. what 64BIT_FILES parameter is all about. Is it to set the universe in 64 bit mode and will increase the performance of application.
2. Do we need to recompile all programs to after changing 64BIT_FILES to 1.
3. What are the suggested tunning parameteres for universe on HP_UX.

THanks in advance.

REgd
Jawed Akhtar

[Bill Hassell · Accepted solution]

Large files (more than 2Gb) do not require a 64bit application program. However, you must have a version of UniVerse that supports large files (contact IBM's web site or your sales rep). Secondly, the filesystems that have UniVerse datafiles must be changed to allow large files. This is done in two steps:

1. Run fsadm -o largefiles /your_filesystem
(this will convert the filesystem to allow large files to be created)

2. Change the mountpoint options in /etc/fstab to add largefiles as in:

/dev/vg03/lvol2 /db3 vxfs nosuid,largefiles,rw,delaylog 0 3

As with any changes to /etc/fstab, it is a good idea to un-mount any changed filesystems (after shutting down any apps that use these mountpoints), then re-mount them using just the mountpoint name. In the above example for fstab: mount /db3

If all goes well, there will be no error message with the mountpoint (which means that it will also work correctly on reboot). Check that the filesystem is largefile-aware with:

fsadm -F vxfs /db3

1. 64BIT_FILES from ibm.com: http://www-1.ibm.com/support/docview.wss?uid=swg21082324 and http://www-1.ibm.com/support/docview.wss?uid=swg21082132

Now it should be noted that the default model for 64bit programs is to enable large file support inside the program, but any 32bit program can be rewritten to use the largefile system calls. Contact IBM for information about your version of UniVerse.

2. see URLs in question 1.

3. Tuning HP-UX is fairly simple but unlike other databses, UniVerse may need hundreds to thousands of files. MFILES set to 50 will drastically slow down each copy of UniVerse as this rotating pool of file handles must be constantly changed to allow access to the required files. Changing it to 200 or even 500 works quite well *BUT* there are 3 kernel limits that may need to be adjusted: maxfiles, maxfiles_lim and nfile. Here are the rules:

- maxfiles and maxfiles_lim: this is the per-process limit for open files. The default is 60 (for all programs) which is way too small for UniVerse. However, maxfiles is a 'soft' limit. It can be changed by either the program (system call setrlimit, but I don't think UniVerse does that) or by changing the ulimit value in the shell prior to starting UniVerse. The commands:

ulimit -n
kmtune -q maxfiles -q maxfiles_lim

will report the maximum number of file descriptors allowed per process and will normally be the same as the kernel parameter maxfiles. To change the soft limit before starting UniVerse, use: ulimit -n 200 (or similar). The kernel parameter maxfiles_lim is a hard limit so your ulimit -n value can't exceed that value.

Now you can set kernel parameter maxfiles to 200 (or 500 or more if needed) and now you won't have to add ulimit to you UniVerse startup scripts. You can also increase maxfiles_lim to a larger value (default=1024). Some sysadmins don't care about the soft limit and will set maxfiles = maxfiles_lim. Now every program is allowed to open files up to the maxfiles limit.

Now for UniVerse to take advantage of maxfiles, the MFILES config parameter must be changed. If maxfiles is 500, then set MFILES to 490 (always choose MFILES to be less than maxfiles by 8 or more--10 is easier to remember). BUT: see the next parameter if you change MFILES.

- nfile
This indirectly related to maxfiles. nfile is the maximum number of files open on the entire computer. If maxfiles is set to 500, then any process can open up to 500 files each. Since UniVerse is the only process that might do this, you need to count the maximum number of UniVerse proceses that will be run. You'll have a license that controls this value so use the maximum license count unless you know there will never be more than a certain number running at the same time. The number of files needed just for UniVerse will be:

files = (MFILES+8)*maximum_UniVerse_clients

So if you have 100 clients running, and MFILES is set to 200, then UnivVerse will need at least 2800 file descriptors. This is in addition to the rest of the operating system. Set the kernel parameter nfile to 5000 as a start and then watch sar during the busiest time of the day:

sar -v 1

The last value: file-sz will report 2 numbers: used and max. If used is getting close to max (say 4900 out of 5000), you'll need to change nfile as soon as possible (like that evening) or stop some of the users from running extra copies of UniVerse.

To see the improvement by using a larger value of MFILES, use sar to check directory activity:

sar -a 1

Do this now before you change MFILES and then later after the new values are in place. MFILES prevents excessive open/close/reopen calls which are performance problems for UniVerse.