Ignite TFTP timeout issue

[thiyagu_3]

HI,
Suddenly the ignite server that we have started giving problem where the TFTP timeout happens while the clients tries to boot via the N/W..below is the error O/P of the client..Booting over the network
Loading.: Core LAN Gb A

Client MAC Address: 00 17 A4 51 81 E0 ..
Client IP Address: 10.209.157.162
Subnet Mask: 255.255.252.0
BOOTP Server IP Address: 10.209.157.141
DHCP Server IP Address: 10.209.157.141
Boot file name: /opt/ignite/boot/nbp.efi

Retrieving File Size.
Retrieving File (TFTP).

Starting: Core LAN Gb A

@(#) HP-UX IA64 Network Bootstrap Program Revision 1.0
Downloading HPUX bootloader
Starting HPUX bootloader
Obtaining size of fpswa.efi (328192 bytes)
Downloading file fpswa.efi (328192 bytes)

(C) Copyright 1999-2006 Hewlett-Packard Development Company, L.P.
All rights reserved

HP-UX Boot Loader for IPF -- Revision 2.035

Booting from Lan
Obtaining size of AUTO (323 bytes)
Downloading file AUTO (323 bytes)
Obtaining size of AUTO (323 bytes)
Downloading file AUTO (323 bytes)



Obtaining size of AUTO (323 bytes)
Downloading file AUTO (323 bytes)
1. target OS is B.11.23 IA
2. target OS is B.11.31 IA
3. target OS is B.11.31_ic148 IA
4. target OS is B.11.31_ic155 IA
5. Exit Boot Loader

Choose Operating System to Install :2

Obtaining size of AUTO (323 bytes)
Downloading file AUTO (323 bytes)
Obtaining size of Rel_B.11.31/IINSTALL (50109790 bytes)
Downloading file Rel_B.11.31/IINSTALL \
TFTP session failed. (reason:Time out)
Could not open Rel_B.11.31/IINSTALL

loading failed

HPUX>

MY ignite server configuration details...
-------------------------------------------------------------------------------------------------
bash-2.02# uname -a
HP-UX iignite B.11.23 U ia64 0758361661 unlimited-user license
------------------------------------------------------------------------------------------------
bash-2.02# netstat -in
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
lan0 1500 10.209.156.0 10.209.157.141 1839187 0 2894268 0 0
lo0 4136 127.0.0.0 127.0.0.1 116501 0 116501 0 0
bash-2.02# netstat -rn
Routing tables
Destination Gateway Flags Refs Interface Pmtu
127.0.0.1 127.0.0.1 UH 0 lo0 4136
10.209.157.141 10.209.157.141 UH 0 lan0 4136
10.209.156.0 10.209.157.141 U 2 lan0 1500
127.0.0.0 127.0.0.1 U 0 lo0 0
default 10.209.156.1 UG 0 lan0 0
bash-2.02#
------------------------------------------------------------------------------------------------
bash-2.02# swlist | grep -i ignite
# Contacting target "iignite"...
# Target: iignite:/
IGNITE C.7.4.157 HP-UX Installation Utilities (Ignite-UX)
Ignite-UX-11-31 C.7.4.157 HP-UX Installation Utilities for Installing 11.31 Systems
------------------------------------------------------------------------------------------------
bash-2.02# showmount -e
export list for iignite:
/var/opt/ignite/clients (everyone)
/ignite (everyone)
------------------------------------------------------------------------------------------------
bash-2.02# pwget -n tftp
tftp:*:107:105:Trivial FTP user:/home/tftp:/usr/bin/false
------------------------------------------------------------------------------------------------
bash-2.02# grep -e boot -e inst /etc/inetd.conf
#bootps dgram udp wait root /usr/lbin/bootpd bootpd
bootps dgram udp wait root /opt/ignite/lbin/instl_bootd bootpd
instl_boots dgram udp wait root /opt/ignite/lbin/instl_bootd instl_bootd
tftp dgram udp wait root /usr/lbin/tftpd tftpd\
/opt/ignite\
/var/opt/ignite\
/ignite/ignite_cfg\
-------------------------------------------------------------------------------------------------
bash-2.02# instl_adm -d
# instl_adm defaults:
# NOTE: Manual additions between the lines containing "instl_adm defaults"
# and "end instl_adm defaults" will not be preserved.
server="10.209.157.141"
netmask[]="255.255.252.0"
route_gateway[0]="10.209.156.1"
route_destination[0]="default"
init_hp_lanadmin_args="-X 100FD"
is_net_info_temporary=false
run_ui=false
control_from_server=true
# end instl_adm defaults.
bash-2.02#
--------------------------------------------------------------------------------------------------
bash-2.02# instl_adm -T
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.11/config
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.11/hw_patches_cfg
* Checking file: /ignite/ignite_cfg/11.11_Sep_2005_cfg
* Checking file: /ignite/ignite_cfg/config.local
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.23/config
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.23/hw_patches_cfg
* Checking file: /ignite/ignite_cfg/11.23PI_LR_debug_TCOE_cfg
* Checking file: /ignite/ignite_cfg/11.23_Sep_2006_cfg
* Checking file: /ignite/ignite_cfg/11.23_jun07_cfg
* Checking file: /ignite/ignite_cfg/11.23_DEC_07_MCOE_cfg
* Checking file: /ignite/ignite_cfg/11.23_Sept2008_cfg
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.31/config
* Checking file: /ignite/ignite_cfg/data/Rel_B.11.31/hw_patches_cfg
* Checking file: /ignite/ignite_cfg/11.31_LR_debug_ic155_cfg
* Checking file: /ignite/ignite_cfg/11.31_debug_cfg
* Checking file: /ignite/ignite_cfg/11.31_perf_cfg
* Checking file: /ignite/ignite_cfg/11.31_perf_sud_cfg
* Checking file: /ignite/ignite_cfg/11.31_Sep_07_MCOE_cfg
* Checking file: /ignite/ignite_cfg/11.31_DEC_07_MCOE_cfg
* Checking file: /ignite/ignite_cfg/11.31_March_08_MCOE_cfg
* Checking file:
/ignite/ignite_cfg/11.31_Sep_08_DCOE_Update_3_fusion_cfg
* Checking file: /ignite/ignite_cfg/11.31_March_09_IC330_cfg
* Checking file: /ignite/ignite_cfg/11.31_March_09_fusion_perf_cfg
bash-2.02#
-------------------------------------------------------------------------------------------------
bash-2.02# manage_index -l
HP-UX 11.11_Sep_2005
HP-UX 11.23PI_LR_debug_TCOE
HP-UX 11.23_Sep_2006
HP-UX 11.23_jun07
HP-UX 11.23_DEC_07_MCOE
HP-UX 11.23_Sept2008
HP-UX 11.31_LR_debug_ic155
HP-UX 11.31_debug
HP-UX 11.31_perf
HP-UX 11.31_perf_sud
HP-UX 11.31_Sep_07_MCOE
HP-UX 11.31_DEC_07_MCOE
HP-UX 11.31_March_08_MCOE
HP-UX 11.31_Sep_08_DCOE_Update_3_fusion
HP-UX 11.31_March_09_IC330
HP-UX 11.31_March_09_fusion_perf
bash-2.02#
--------------------------------------------------------------------------------------------------
bash-2.02# ll /opt/ignite/boot
total 208912
-r--r--r-- 1 bin bin 323 Feb 22 2007 AUTO
-r--r--r-- 1 bin bin 172 Aug 10 2006 AUTO.bak
drwxr-xr-x 2 bin bin 8192 Aug 11 23:11 Rel_B.11.11
drwxr-xr-x 2 bin bin 8192 Feb 1 05:38 Rel_B.11.23
drwxr-xr-x 2 bin bin 8192 Jan 19 01:16 Rel_B.11.31
-r--r--r-- 1 bin bin 88 Aug 29 2007 auto_globals
-r--r--r-- 1 bin bin 332800 Feb 1 05:38 boot_lif
-r--r--r-- 1 bin bin 332800 Aug 10 2006 boot_lif.bak
-r--r--r-- 1 bin bin 328192 Feb 15 2007 fpswa.efi
-rw-r--r-- 1 root sys 296 Feb 1 05:38 fs_cfg.def
-r--r--r-- 1 bin bin 654025 Feb 15 2007 hpux.efi
-r--r--r-- 1 bin bin 24576 Feb 15 2007 nbp.efi
-rw-r--r-- 1 bin bin 15985032 Jan 16 22:22 oldB.11.00_INSTALL
-rw-r--r-- 1 bin bin 15728640 Jul 27 2006 oldB.11.00_INSTALLFS
-rw-r--r-- 1 bin bin 19409416 Jan 16 22:22 oldB.11.00_VINSTALL
-rw-r--r-- 1 bin bin 15728640 Jul 27 2006 oldB.11.00_VINSTALLFS
-rw-r--r-- 1 bin bin 22616784 Jan 16 22:23 oldB.11.00_WINSTALL
-rw-r--r-- 1 bin bin 15728640 Jul 27 2006 oldB.11.00_WINSTALLFS
-rw-r--r-- 1 bin bin 91 Jan 16 22:21 oldB.11.00_auto_conf_pa
bash-2.02#
----------------------------------------------
Kindly help me in getting this resolved...the only error message that i see in ignite server syslog is ..
Feb 7 21:57:52 iignite tftpd[14619]: Timeout (no requests in 10 minutes)

[Stephan.]

Hi,
first ... best question i have seen since a long time, just perfect.

Back on topic, did you try to restart the ignite service or a manual tftp to obtain the files?

[thiyagu_3]

HI ,
Thx for your reply
Yes I have restarted the ignite server many times and the mannual tftp works obsolutely fine

[Stephan.]

Just to be sure ...

/opt/ignite/boot/Rel_B.11.31/IINSTALL exists?

Probably it worked before, did you check for network issues (Firewall, ...)?

Killing any tftp process on the ignite server and restarting inetd couold perhaps help but to be honest i cannot find a problem in the configuration.

[thiyagu_3]

bash-2.02# ll /opt/ignite/boot/Rel_B.11.31/IINSTALL
-r-xr-xr-x 1 bin bin 50109790 Nov 2 2008 /opt/ignite/boot/Rel_B.11.31/IINSTALL
bash-2.02# ps -aef | grep -i tft
root 6805 3686 0 23:02:44 pts/ta 0:00 grep -i tft
root 6225 1 0 22:44:23 ? 0:00 tftpd /opt/ignite /var/opt/ignite /ignite/ignite_cfg #bootps dgram udp wait root /usr/lbin/bootpd bootpd
tftp 6755 6225 0 22:58:40 ? 0:00 tftpd /opt/ignite /var/opt/ignite /ignite/ignite_cfg #bootps dgram udp wait root /usr/lbin/bootpd bootpd
bash-2.02# kill -9 6225
bash-2.02# kill -9 6755
bash-2.02# ps -aef | grep -i tft
root 6950 3686 1 23:03:03 pts/ta 0:00 grep -i tft
bash-2.02# inetd -c
bash-2.02# tail -f /var/adm/syslog/syslog.log
Feb 7 22:55:54 iignite inetd[6295]: tftp/udp: bind: Address already in use
Feb 7 22:55:54 iignite inetd[6295]: bootps/udp: bind: Address already in use
Feb 7 22:55:54 iignite inetd[6295]: Configuration complete
Feb 7 22:57:53 iignite inetd[6295]: bootps/udp: Service enabled
Feb 7 22:57:53 iignite inetd[6295]: tftp/udp: Service enabled
Feb 7 22:57:53 iignite inetd[6295]: bootps/udp: bind: Address already in use
Feb 7 23:03:25 iignite inetd[6295]: Rereading configuration
Feb 7 23:03:25 iignite inetd[6295]: bootps/udp: bind: Address already in use
Feb 7 22:57:53 iignite inetd[6295]: tftp/udp: bind: Address already in use
Feb 7 23:03:25 iignite inetd[6295]: Configuration complete
-----------------------------------------------------------------------------------------------
Did as you suggested but still no progress ...Below is the updates from the network team...
---------
As per the below tracert from source & destination, Please find my below observation.
Source Ip address: 10.209.157.112
Destination Ip address: 10.209.157.141

Both the source & destination are in the same Vlan.
There is no firewall between source & destination.

The source is connected to Gi10/47 on puna-sli-edge-b45
The destination is connected to Gi10/14 on the same switch puna-sli-edge-b45

I have checked the above interfaces & observed that no errors are found & increasing after clearing the counters as well.
Also checked the CPU utilization of the switch puna-sli-edge-b45 didnâ t observed any issues with CPU utilization.

While checking I observed that for the destination ip address 10.209.157.141 we are getting we are getting â Qâ while pinging with the datagram size of more than 1000. I have opened a TAC case with Cisco & as per Cisco the server tries to force the sender's TCP window to 1 to slow down the TCP stream at the source. Possibly there is a problem at the server side and it cannot receive traffic at the incoming rate.
------------

[Stephan.]

You force the client to 100 FD by

>>init_hp_lanadmin_args="-X 100FD"

Is the switch on the client site configured to 100 FD as well?

[thiyagu_3]

init_hp_lanadmin_args="-X 100FD

Hi,

The setup is like the iignite server and the ignite client both the servers connect to the same cisco catalyst switch which operates at 1Gbps speed and the ports are configured for the same...Though I tried with the below mentioned 3 options
init_hp_lanadmin_args="-X 100FD
init_hp_lanadmin_args="-X 1000FD
finally without the parameter "init_hp_lanadmin_args"
In all the case the result is the same...

[Stephan.]

Did you try to change the timeout settings?

You can change _hp_tftp_cmds in the INSTALLFS file, dhe default values that Ignite-UX uses are rexmt=2 timeout=25. (check tftp(1) for more details). The string assigned to this variable should contain one tftp command statement per line.

f.e.
init _hp_tftp_cmds="rexmt 5
timeout 40"

[thiyagu_3]

Did the changes as suggested ...But problem remains the same

bash-2.02# instl_adm -d
# instl_adm defaults:
# NOTE: Manual additions between the lines containing "instl_adm defaults"
# and "end instl_adm defaults" will not be preserved.
server="10.209.157.141"
netmask[]="255.255.252.0"
route_gateway[0]="10.209.156.1"
route_destination[0]="default"
init_hp_lanadmin_args="-X 1000FD"
init _hp_tftp_cmds="rexmt 5
timeout 40"
is_net_info_temporary=false
run_ui=false
control_from_server=true
# end instl_adm defaults.
bash-2.02# inetd -c
bash-2.02#

[Stephan.]

I'm running out of ideas ...

Is it possible to check with another client if the problem is general or client specific?

Last idea would be the microsoft style, rebooting the server and trying again :-x

[thiyagu_3]

Hi,
Sorry to bother you again ...In that subnet we have more than 200 HP servers and I almost tried with around 20-25 client and all the clients facing the same problem....Also not only rebooting the server I tried reinstalling the OS and reconfiguring the ignite server but still the same problem persist ...Kindly let me know is there any way to check whether any packet is getting droped at server ethernet port side or to monitor the receiving traffic from server end....

[Stephan.]

Hi again,
can you post the bootptab and dhcptab?

And back to basics:

>>Suddenly the ignite server that we have started giving problem ...

What changes have been recently made to
- the enviroment
- the ignite server
?

[thiyagu_3]

Hi ,

Below is the filtered O/P of /etc/inetd.conf
bootps dgram udp wait root /opt/ignite/lbin/instl_bootd bootpd
instl_boots dgram udp wait root /opt/ignite/lbin/instl_bootd instl_bootd

the bootps service uses the server prg /opt/ignite/lbin/instl_bootd which in turn uses the cfg file /etc/opt/ignite/instl_boottab to assign the IP addresses for PA and IA clients and the O/P for of the same is pasted below
10.209.157.162:0x0017A45181E0:20070208000606:
10.209.157.163:0x00306E4CABC8:20070206053028:
10.209.157.164:::
10.209.157.165:::
10.209.157.166:::

[Stephan.]

Hi,
i cross checked everything with our Ignite enviroment and i can not find an error. I guess that something has been changed but no idea what.

So all i can do is bump this one and hope that someone of the other guys here can help you.

[amit_ac_in]

Hi,

Can you check the Permissions for IINSTALL

something like this:

-r-xr-xr-x 1 bin bin 51685533 Feb 6 2009 IINSTALL*

[thiyagu_3]

bash-2.02# ll /opt/ignite/boot/Rel_B.11.31/IINSTALL
-rw-r--r-- 1 bin bin 139417032 Feb 15 2007 /opt/ignite/boot/Rel_B.11.31/IINSTALL
bash-2.02#

[likid0]

Yes, the perms on the IINSTALL file should be:

# ls -ltr | grep IINSTALL
-r-xr-xr-x 1 bin bin 51685533 Feb 6 2009 IINSTALL

[thiyagu_3]

tried by changing the permission to 555 and 755 for IINSTALL no progress

[amit_ac_in]

Hi,

Can you please check the following :
1)what errors are you getting in
/var/opt/ignite/logs/ignite and
/var/opt/ignite/logs/bootsys

2)What messages does the installation logs of the client tell. The directory to fine the logs /var/opt/ignite/clients/

[thiyagu_3]

The below are the logs that you required...
bash-2.02# pwd
/var/opt/ignite/logs
bash-2.02# ll
total 16
-rw-r--r-- 1 root sys 444 Feb 10 00:27 bootsys
bash-2.02# cat bootsys
bootsys start: pid:(13491) Sat Feb 10 00:27:17 MST 2007
(13491) Called with options: "-f -a -i HP-UX 11.31_March_09_fusion_perf xignite"
(13491) Called with options: "-f -a -i HP-UX 11.31_March_09_fusion_perf xignite"
(13491) === Preparing system: xignite
Primary boot path set to 0/1/1/0.0x0.0x0 (/dev/disk/disk3_p2)
Rebooting xignite now.
bootsys_prep return code = 0
bootsys exit: pid:(13491) Sat Feb 10 00:27:50 MST 2007 (exit status: 0)
bash-2.02#


Here My problem is when we boot the client via N/W and when the client downloads the boot kernel (IINSTALL) it shows the tftp timeout issue ...but if I boot the client using 1st OS CD there by connecting the ignite server for installation or by using bootsys command the installation goes fine...so under /var/opt/ignite/clients directory I have the clients details which succeeded the with installation ..I dont have any entries for the TFTP timedout clients

[SCC (Kavanagh) Support]

A shot in the dark but you could perhaps play with the tftp block size (see manpage)