Golden System Image restoration hangs on reboot

[Ken O'Neil]

Using two identical rp2400s with server A as my I-UX server and B as my client, I created a golden system image of B on A using make_sys_image. After reinstalling B with the image I created, the system reboots and hangs for extended periods of time on each of these services:
NFS Client Subsystem
Mail Daemon
SNMP HP-UX Network Management Sub-Agent
Software Distribution Agent Daemon
HP Dual Port...Fiber Channel...PCI-X

The reboot process eventually moves past each of these as they timeout, but the boot process takes upwards of 30 minutes to complete. Once the system is up I can login, but all network services appear to be hung or extremely delayed. If I telnet from A to B for example, a connection is established by it takes 2-3 minutes for a login to appear on A. swlist hangs the system, and Ignite cannot reconnect to B to execute bootsys.

I have attempted to reload the image again by rebooting from the command line, interrupting that boot process and using boot lan.xxx.xxx.xxx.xxx install. However each time I repeat the process I get the same results.

Any ideas on what could be effecting the client?

Environment:
> 2 rp2400s - identical hardware
> HPUX 11.11 on both - identical factory builds (sept 2003)
> Ignite 4.4.30 on Server A, passed pax patch test, and configured per Ignite Administration manual chapter on Using Golden System Images.

[Sridhar Bhaskarla]

Hi,

I guess you have a problem with the network setup on the server. One or more of the following may be causing the issue.

1. Speed settings. If your interface is supposed to run at 100 MBPS, make sure it is set to full duplex.

lanadmin -x 0 -> to verify the speed/duplex on lan0
lanadmin -X 100FD 0 -> to set the speed on lan0

2. Make sure your name resolution is fine on the box.

nslooup some_server

Check /etc/resolv.conf, /etc/nsswitch.conf and /etc/hosts.

3. Look at any errors in /etc/rc.log.

-Sri

[Ken O'Neil]

Here is the info you mentioned, but I don't think these are related because I can't even get a normal response from ping localhost.

1. lanadmin -x 0 returns:
Current Config = 100 Full-Duplex AUTONEG

2. nslookup times out but DNS is setup but NIS is off:

/etc/resolv.conf.....
domain (domain_omitted).com
nameserver 10.0.1.151
nameserver 10.0.1.152

/etc/nsswitch.conf.....
#
# /etc/nsswitch.conf:
#
# @(#)B.11.11_LR
#

passwd: compat
group: compat
hosts: dns [NOTFOUND=return] files
networks: files
protocols: files
rpc: files
publickey: files
netgroup: files
automount: files
aliases: files
services: files

/etc/hosts.
# @(#)B.11.11_LRhosts $Revision: 1.9.214.1 $ $Date: 96/10/08 13:20:01 $
#
# The form for each entry is:
#
#
# For example:
# 192.1.2.34 hpfcrm loghost
#
# See the hosts(4) manual page for more information.
# Note: The entries cannot be preceded by a space.
# The format described in this file is the correct format.
# The original Berkeley manual page contains an error in
# the format description.
#

10.15.16.152 hq-cfghp2.(domain_omitted).com hq-cfghp2
127.0.0.1 localhost loopback

3. Here's /etc/rc.log:

I didn't notice too much unusual except for the removal of the /etc/nsswitch.conf file and the hostname, which is normal since we use hostnames longer than 8 characters, but I added a system name (uname -S) and put our standard nsswitch.conf file in place and rebooted with no difference in results.

Old /etc/rc.log moved to /etc/rc.log.old

**************************************************
HP-UX Start-up in progress
Wed Jan 21 14:34:08 PST 2004
**************************************************

Configure system crash dumps
Output from "/sbin/rc1.d/S080crashconf start":
----------------------------
EXIT CODE: 0

VxVM device node check
Output from "/sbin/rc1.d/S091vxvm-nodes-check start":
----------------------------

VxVM general startup
Output from "/sbin/rc1.d/S092vxvm-startup start":
----------------------------

VxVM reconfiguration recovery
Output from "/sbin/rc1.d/S093vxvm-reconfig start":
----------------------------
Mount file systems
Output from "/sbin/rc1.d/S100localmount start":
----------------------------
checking quotas

Update kernel and loadable modules
Output from "/sbin/rc1.d/S110kmbuild start":
----------------------------
"/sbin/rc1.d/S110kmbuild start" SKIPPED

Initialize loadable modules
Output from "/sbin/rc1.d/S112kminit start":
----------------------------
* All configured modules are registered successfully.

NOTE: Module status

Name ID Status Type
=====================================================
krm 1 UNLOADED WSIO

Setting hostname
Output from "/sbin/rc1.d/S320hostname start":
----------------------------
Nodename must be less than9 characters
EXIT CODE: 1
"/sbin/rc1.d/S320hostname start" FAILED

Start Kernel Logging facility
Output from "/sbin/rc1.d/S350kl start":
----------------------------
"/sbin/rc1.d/S350kl start" SKIPPED

Set privilege group
Output from "/sbin/rc1.d/S400set_prvgrp start":
----------------------------
"/sbin/rc1.d/S400set_prvgrp start" SKIPPED

Display date
Output from "/sbin/rc1.d/S420set_date start":
----------------------------
"/sbin/rc1.d/S420set_date start" SKIPPED

Copy processor logs to /var/tombstones
Output from "/sbin/rc1.d/S430pdcinfo start":
----------------------------
Before vpar check: Error 0
after vpar check: No such file or directory
Before PDCINFO_POST_10XX: No such file or directory
Before vpar check 2: No such file or directory

Save system crash dump if needed
Output from "/sbin/rc1.d/S440savecrash start":
----------------------------
savecrash directory not set; defaulting to: /var/adm/crash
EXIT CODE: 2 - savecrash found no core dump to save
"/sbin/rc1.d/S440savecrash start" SKIPPED

Enable auxiliary swap space
Output from "/sbin/rc1.d/S500swap_start start":
----------------------------
swap device(s) active

Start syncer daemon
Output from "/sbin/rc1.d/S520syncer start":
----------------------------
syncer started

Configure Loopback interfaces (lo0)
Output from "/sbin/rc2.d/S008net.init start":
----------------------------

VxVM volume recovery start
Output from "/sbin/rc2.d/S096vxvm-recover start":
----------------------------

Configuring all unconfigured software filesets
Output from "/sbin/rc2.d/S120swconfig start":
----------------------------
* The following files, which could not be closed by swinstall when they
were updated, are being removed now:
/usr/lib/#337libc.2
/configure3
/monitor_bpr

* Turning off all network based resolving services in
'/etc/nsswitch.conf'
* Removing the temporary '/etc/nsswitch.conf'.

Recover editor crash files
Output from "/sbin/rc2.d/S200clean_ex start":
----------------------------
preserving editor files (if any)

Clean UUCP
Output from "/sbin/rc2.d/S202clean_uucp start":
----------------------------
cleaning up uucp

List and/or clear temporary files
Output from "/sbin/rc2.d/S204clean_tmps start":
----------------------------
NOTE: Files in /tmp:
total 304
-rw------- 1 root sys 9908 Jan 9 15:09 cata02010
-r--r--r-- 1 root sys 1171 Jan 21 14:12 install.vars
-r--r--r-- 1 root root 0 Jan 14 17:26 llbdbase.dat
drwxr-xr-x 2 root root 96 Jan 9 14:43 lost+found
-rwxr-xr-x 1 root sys 95688 Jan 9 15:17 make_sys_image
-rw------- 1 root sys 7666 Jan 9 15:09 mana02010
-rw------- 1 root root 604 Jan 14 17:21 portmap.file
-rw------- 1 root root 2036 Jan 14 17:21 rpcbind.file
-rw-r--r-- 1 root root 99 Jan 14 17:27 sys.tst_systest
NOTE: Files in /var/tmp:
total 112
-rw-r--r-- 1 root sys 0 Jan 14 14:12 baaa02516
-rw-rw-rw- 1 root sys 0 Jan 14 17:39 cmd_res2459
-rw-rw-rw- 1 root sys 0 Jan 14 17:40 cmd_res2465
-rw-rw-rw- 1 root sys 0 Jan 9 15:39 cmd_res3453
-rwxr--r-- 1 root root 28 Jan 14 17:26 envd.action2
-rwxr--r-- 1 root root 28 Jan 14 17:26 envd.action5
-rw-rw-rw- 1 root sys 1308 Jan 9 15:24 excluded_files
drwxrwxrwx 2 root sys 8192 Jan 14 17:40 ign_configure
drwxrwxrwx 2 root sys 96 Jan 14 15:38 java
-rw-rw-rw- 1 root sys 587 Jan 9 15:24 make_sys_image.hwsys.a
02652
drwxr-x--- 3 root sys 8192 Jan 14 14:34 patch.29960
-rw-r--r-- 1 root sys 452 Jan 14 17:09 swagent.log
dr-xr-xr-x 3 bin bin 96 Jan 9 14:53 swmodtmp
-rw-rw-rw- 1 root sys 0 Jan 14 17:39 tmp_gone_file_list.245
9
-rw-rw-rw- 1 root sys 0 Jan 14 17:40 tmp_gone_file_list.246
5
-rw-rw-rw- 1 root sys 0 Jan 9 15:39 tmp_gone_file_list.345
3
drwxr-x--- 2 root sys 96 Jan 14 13:16 weblogic.8.11

Clean up old log files
Output from "/sbin/rc2.d/S206clean_adm start":
----------------------------

Start system message logging daemon
Output from "/sbin/rc2.d/S220syslogd start":
----------------------------
System message logger started

Start pty allocator daemon
Output from "/sbin/rc2.d/S230ptydaemon start":
----------------------------
Starting the ptydaemon

Configuring OLA/R interface
Output from "/sbin/rc2.d/S290pci_olar start":
----------------------------
Creating OLA/R device node at major 5...
Note that this system must have OLA/R capable hardware
and firmware for these features to be enabled.
OLA/R capabilities not supported on this platform
"/sbin/rc2.d/S290pci_olar start" SKIPPED

Start network tracing and logging daemon
Output from "/sbin/rc2.d/S300nettl start":
----------------------------
Initializing Network Tracing and Logging...
Done.

Configure HP igelan Gigabit Ethernet interfaces
Output from "/sbin/rc2.d/S305hpigelan start":
----------------------------

Configure HP iether 100BT/Gigabit Ethernet interfaces
Output from "/sbin/rc2.d/S306hpiether start":
----------------------------

Configure HP Gigabit Ethernet interfaces
Output from "/sbin/rc2.d/S308hpgelan start":
----------------------------

Configure HP Ethernet interfaces
Output from "/sbin/rc2.d/S320hpether start":
----------------------------

Create PCI FDDI device files.
Output from "/sbin/rc2.d/S329hppcifddi start":
----------------------------

Configure HP 100BASE-T interfaces
Output from "/sbin/rc2.d/S333hpbtlan start":
----------------------------

Configure VLAN interfaces
Output from "/sbin/rc2.d/S337vlan start":
----------------------------

Configure LAN interfaces
Output from "/sbin/rc2.d/S340net start":
----------------------------

Start name server daemon
Output from "/sbin/rc2.d/S370named start":
----------------------------
"/sbin/rc2.d/S370named start" SKIPPED

Start NFS core subsystem
Output from "/sbin/rc2.d/S400nfs.core start":
----------------------------
starting NFS CORE networking

starting up the rpcbind
/usr/sbin/rpcbind
starting up the nfskd daemon
/usr/sbin/nfskd

Start NIS+ server subsystem
Output from "/sbin/rc2.d/S406nisplus.server start":
----------------------------
NOTE: NISPLUS_SERVER is not set, exiting.

Start NIS+ client subsystem
Output from "/sbin/rc2.d/S408nisplus.client start":
----------------------------
NOTE: NISPLUS_CLIENT is not set, exiting.

Start NIS server subsystem
Output from "/sbin/rc2.d/S410nis.server start":
----------------------------
NOTE: NIS_MASTER_SERVER or NIS_SLAVE_SERVER is not set, exiting.

Start NIS client subsystem
Output from "/sbin/rc2.d/S420nis.client start":
----------------------------
NOTE: NIS_CLIENT is not set, exiting.

Start NFS client subsystem
Output from "/sbin/rc2.d/S430nfs.client start":
----------------------------
starting NFS CLIENT networking

starting up the rpcbind
rpcbind already started, using pid: 553
starting up the BIO daemons
/usr/sbin/biod 16
starting up the Status Monitor daemon
/usr/sbin/rpc.statd
starting up the Lock Manager daemon
/usr/sbin/rpc.lockd
starting up the Automount daemon
/usr/sbin/automount -f /etc/auto_master
mounting remote NFS file systems ...
mounting CacheFS file systems ...
Start the Trusted Mode with NIS+ subsystem
Output from "/sbin/rc2.d/S440comsec start":
----------------------------
starting the Trusted Table Sync Daemon (ttsyncd)
Not an NIS+ root or master server
"/sbin/rc2.d/S440comsec start" SKIPPED

Configure pseudo devices for MAC/LLA access
Output from "/sbin/rc2.d/S462maclan start":
----------------------------

Start multicast routing daemon
Output from "/sbin/rc2.d/S490mrouted start":
----------------------------
"/sbin/rc2.d/S490mrouted start" SKIPPED

Start Internet services daemon
Output from "/sbin/rc2.d/S500inetd start":
----------------------------
Internet Services started

Start dynamic routing daemon
Output from "/sbin/rc2.d/S510gated start":
----------------------------
"/sbin/rc2.d/S510gated start" SKIPPED

Start router discover protocol daemon
Output from "/sbin/rc2.d/S520rdpd start":
----------------------------
"/sbin/rc2.d/S520rdpd start" SKIPPED

Configuring PPP Interface
Output from "/sbin/rc2.d/S522ppp start":
----------------------------
Creating 16 tunnel device nodes at major 31...

Start RARP protocol daemon
Output from "/sbin/rc2.d/S525rarpd start":
----------------------------
"/sbin/rc2.d/S525rarpd start" SKIPPED

Start remote system status daemon
Output from "/sbin/rc2.d/S530rwhod start":
----------------------------
"/sbin/rc2.d/S530rwhod start" SKIPPED

Configuring man pages for Internet Services
Output from "/sbin/rc2.d/S535inetsvcs start":
----------------------------

Starting mail daemon
Output from "/sbin/rc2.d/S540sendmail start":
----------------------------
/etc/mail/aliases: 7 aliases, longest 9 bytes, 88 bytes total
sendmail

Starting outbound connection daemons for DDFA software
Output from "/sbin/rc2.d/S550ddfa start":
----------------------------
"/sbin/rc2.d/S550ddfa start" SKIPPED

Start SNMP Master Network Management daemon
Output from "/sbin/rc2.d/S560SnmpMaster start":
----------------------------

Start OSPF MIB Network Management subAgent
Output from "/sbin/rc2.d/S565OspfMib start":
----------------------------
"/sbin/rc2.d/S565OspfMib start" SKIPPED

Start SNMP HP-UNIX Network Management subAgent
Output from "/sbin/rc2.d/S565SnmpHpunix start":
----------------------------

Start SNMP MIB-2 Network Management subAgent
Output from "/sbin/rc2.d/S565SnmpMib2 start":
----------------------------

Start SNMP Trap Dest Network Management subAgent
Output from "/sbin/rc2.d/S565SnmpTrpDst start":
----------------------------

Start DCE daemons
Output from "/sbin/rc2.d/S570dce start":
----------------------------
Beginning DCE startup...
NOTE: No components of DCE are configured.

Finished DCE startup.
"/sbin/rc2.d/S570dce start" SKIPPED

Start SNMP PCI FDDI Network Management subAgent
Output from "/sbin/rc2.d/S576SnmpFddi4 start":
----------------------------

Start RPC daemon if needed
Output from "/sbin/rc2.d/S590Rpcd start":
----------------------------
starting the DCE RPC Daemon
/opt/dce/sbin/rpcd

Start Isogon License Server Daemon
Output from "/sbin/rc2.d/S600iforls start":
----------------------------
"/sbin/rc2.d/S600iforls start" SKIPPED

Start remote boot daemon
Output from "/sbin/rc2.d/S610rbootd start":
----------------------------
"/sbin/rc2.d/S610rbootd start" SKIPPED

Starting X Font Server at TCP port 7000
Output from "/sbin/rc2.d/S620xfs start":
----------------------------
"/sbin/rc2.d/S620xfs start" SKIPPED

Start vt daemon
Output from "/sbin/rc2.d/S630vt start":
----------------------------
starting the vtdaemon

Starting UDC server
Output from "/sbin/rc2.d/S648eus start":
----------------------------
"/sbin/rc2.d/S648eus start" SKIPPED

Start time synchronization
Output from "/sbin/rc2.d/S660xntpd start":
----------------------------
"/sbin/rc2.d/S660xntpd start" SKIPPED

Start accounting
Output from "/sbin/rc2.d/S700acct start":
----------------------------
"/sbin/rc2.d/S700acct start" SKIPPED

Install/Load XF86 DLKM Helper Modules
Output from "/sbin/rc2.d/S700xf86 start":
----------------------------

Starting the password/group assist subsystem
Output from "/sbin/rc2.d/S705pwgr start":
----------------------------
pwgrd started

Starting HP Disk Array Manager daemons.
Output from "/sbin/rc2.d/S710hparamgr start":
----------------------------

Starting disk array monitor daemons.
Output from "/sbin/rc2.d/S710hparray start":
----------------------------

Start print spooler
Output from "/sbin/rc2.d/S720lp start":
----------------------------
"/sbin/rc2.d/S720lp start" SKIPPED

Starting HP Distributed Print Service
Output from "/sbin/rc2.d/S722pd start":
----------------------------

Start clock daemon
Output from "/sbin/rc2.d/S730cron start":
----------------------------
cron started

Start diagnostic subsystem
Output from "/sbin/rc2.d/S742diagnostic start":
----------------------------

Start environment monitoring daemon
Output from "/sbin/rc2.d/S750envd start":
----------------------------
Environmental daemon started

Start auditing subsystem
Output from "/sbin/rc2.d/S760auditing start":
----------------------------
"/sbin/rc2.d/S760auditing start" SKIPPED

Start audio server daemon
Output from "/sbin/rc2.d/S770audio start":
----------------------------
"/sbin/rc2.d/S770audio start" SKIPPED

Start Distributed Single Logical Screen daemon
Output from "/sbin/rc2.d/S780slsd start":
----------------------------
"/sbin/rc2.d/S780slsd start" SKIPPED

Start USB hub daemon
Output from "/sbin/rc2.d/S785hub start":
----------------------------

SAM System administration configuration
Output from "/sbin/rc2.d/S800spa start":
----------------------------

Starting ObAM Application Server
Output from "/sbin/rc2.d/S860webadmin start":
----------------------------
"/sbin/rc2.d/S860webadmin start" SKIPPED

Initialize Software Distributor agent daemon
Output from "/sbin/rc2.d/S870swagentd start":
----------------------------

Starting servicecontrol manager daemons
Output from "/sbin/rc2.d/S890ServCtlMgr start":
----------------------------

Starting CIFS Client
Output from "/sbin/rc2.d/S900cifsclient start":
----------------------------
"/sbin/rc2.d/S900cifsclient start" SKIPPED

Starting Event Monitoring Service
Output from "/sbin/rc2.d/S900ems start":
----------------------------
Starting the EMS Persistence Client

Start EMS SNMP subagent
Output from "/sbin/rc2.d/S900emsa start":
----------------------------

Configure HP Fibre Channel Mass Storage interfaces
Output from "/sbin/rc2.d/S900hpfcms start":
----------------------------

Configure HP Internal RAID interfaces
Output from "/sbin/rc2.d/S900i2o_raid start":
----------------------------
No Internal RAID Adapters installed.
"/sbin/rc2.d/S900i2o_raid start" SKIPPED

Starting SAMBA Server
Output from "/sbin/rc2.d/S900samba start":
----------------------------
"/sbin/rc2.d/S900samba start" SKIPPED

Configure HP Fibre Channel Tachyon TL/TS Mass Storage interfaces
Output from "/sbin/rc2.d/S989fc_td start":
----------------------------

Configure HP Dual Port 2Gbps/1Gbps Fibre Channel Universal PCI-X Adapter
Output from "/sbin/rc2.d/S990fc_fcd start":
----------------------------

VVR - Starting Communication Daemon
Output from "/sbin/rc2.d/S994vxnm-vxnetd start":
----------------------------
No VVR license installed on the system - vxnetd not started
"/sbin/rc2.d/S994vxnm-vxnetd start" SKIPPED

VVR - Starting VRAS daemon
Output from "/sbin/rc2.d/S996vradmind start":
----------------------------

No VVR license installed on the system; vradmind not started.
"/sbin/rc2.d/S996vradmind start" SKIPPED

VVR - Starting VxRSync daemon
Output from "/sbin/rc2.d/S996vxrsyncd start":
----------------------------

No VVR license installed on the system; in.vxrsyncd not started.
"/sbin/rc2.d/S996vxrsyncd start" SKIPPED

Starting VERITAS Enterprise Administrator Service (vxsvc)
Output from "/sbin/rc2.d/S999isisd start":
----------------------------

Start NFS server subsystem
Output from "/sbin/rc3.d/S100nfs.server start":
----------------------------
starting NFS SERVER networking

starting up the rpcbind daemon
rpcbind already started, using pid: 553
starting up the mount daemon
/usr/sbin/rpc.mountd
starting up the NFS daemons
/usr/sbin/nfsd 16
starting up the Status Monitor daemon
rpc.statd already started, using pid: 600
starting up the Lock Manager daemon
rpc.lockd already started, using pid: 606

Start X print server(s)
Output from "/sbin/rc3.d/S200tps.rc start":
----------------------------
"/sbin/rc3.d/S200tps.rc start" SKIPPED

Starting the Apache subsystem
Output from "/sbin/rc3.d/S823hpws_apache start":
----------------------------
"/sbin/rc3.d/S823hpws_apache start" SKIPPED

Starting Tomcat.
Output from "/sbin/rc3.d/S823hpws_tomcat start":
----------------------------
"/sbin/rc3.d/S823hpws_tomcat start" SKIPPED

Starting the Webmin subsystem
Output from "/sbin/rc3.d/S823hpws_webmin start":
----------------------------
"/sbin/rc3.d/S823hpws_webmin start" SKIPPED

Starting the XMLTools subsystem
Output from "/sbin/rc3.d/S823hpws_xmltools start":
----------------------------

Start CDE login server
Output from "/sbin/rc3.d/S990dtlogin.rc start":
----------------------------

**************************************************
HP-UX run-level transition completed
Wed Jan 21 15:10:36 PST 2004
**************************************************

[John Payne_2]

Ken,

I agree with Sridhar, it sounds an awful lot like a networking problem of some sort.

Maybe the vlan on your port of your switch, or settings there (speed, duplex), some other setting maybe in /etc/rc.config.d/netconf (gateway? IP address?), a bad entry in /etc/hosts (problibly not here, it looks like you already looked there, but I add it for others.), etc, etc.

This is what happens to me when I install a box on one subnet, but give it the permanent address for a different subnet and forget to change the vlan during the last reboot. (These are the symptoms everytime I do that...)

Hope it helps

John

[Ken O'Neil]

Well Sridhar did get me pointed in the right direction (wish I could bump up those points, sorry about that). Thanks!

I didn't like the message in the rc.log file from the S120swconfig, so after a search on that I found this thread:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=252147

That got me focused on the DNS, and after a little testing with nsswitch and nslookup I realized that I made the simplest of errors: I fat-fingered the DNS IPs, entering 10.0.1.151 instead of 10.1.0.151. Once I update the resolv.conf file with the correct IPs all was well.

Also, after going back and checking the config files in my /var/opt/ignite/clients/hq-cfghp2 directory I found that I had indeed specified an invalid IP in the configuration.

I'm going to test the updated Ignite config now and make sure the entire process functions properly, but I expect it will.

Thanks again you two for the quick responses!

[Bill Hassell · Accepted solution]

Hangs at NFS during a cold install (or Golden Image) are very common and as you've found, networking such as DNS, subnet mask, router/gateway address, etc will cause a hang. I make it a habit to turn off every NFS feature in /etc/rc.config.d/nfsconf in Golden Images and cold installs. It's the only startup that consistently hangs on bootup with networking problems.

[Steven E. Protter]

I'd path NFS as fully, make sure all patches are in.

instl_adm is your firend hear.

-d to display current network settings

-f to load new settings from a file.

Interesting thing happened to me February of last year.

I installed a pax patch to pack files up to 8 GB

pax_iux could not unpack the files.

If the file is downloading and can not unpack, this is something to look into. I believe this flaw was however corrected in Ignite 4.4

SEP

[Ken O'Neil]

Bill,
That's good advice with NFS and I'm going to update my base image tomorow. Thanks!

Steve,
Initially I used inst_adm -T and had no errors, but the -d and -f options will come in handy in the future. Thanks!

[Sridhar Bhaskarla]

Whoa!!.. my inclination was towards DNS but I was in a hurry to go home so I just dropped few clues hiding my soapbox.

Good to hear that your issue is fixed.

-Sri