network card hang and System error: [232] Connection reset by peer

[Ludovic Derlyn]

hi everybody,

I have a problem with network card and data protector (DP 5.5 with the latest patchs)
This network card has been hang 8 times this month, essentially the Saturday, when full backup is in progress ( server is installed on HPUX and some clients are installed on W2K3, others on HPUX)
Problems occurs essentially when save running on W2K3.
My DP server is in cluster , and status of this card is UP but I haven't access to the server and ping doesn't works (this network card is default gateway, so when the network card hang , server is not accessible directly)
Reset the network card by lan admin resolve temporarely this problem (switch to stand by and come back on initial network card)

On data protector save , i have this error :
IPC read error system error[232] connection reset by peer

I have implemented some checks for future problems:

- ipcs > /tmp/ipcs.txt
- kcvalue to check shmmax and shmseg
- system_perf.sh
(thanks to SEP ;-) )

I have checked :

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=629353

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=255726

http://forums2.itrc.hp.com/service/forums/questionanswer.do?threadId=642469

My questions are :

- Have I need specifics kernel parameters for DP ?
- Where i can found an equivalent document for windows 2003 and DP5.5 title :
"IPC Read Error ,System error: [232] Connection reset by peer" document id:
OV-EN002648 product: omniback II version:
3.50 os: NT updated: 2001-Jul-28

Have you any suggest ? I have checked log of DP (sm and inet nothing, syslog nothing...)

Regards

L-DERLYN

[Steven E. Protter]

Shalom,

Use mstm or xstm to test the card.

Either way, lean hard on hardware support and get them to swap it out.

Before that have your cisco switch/network admin check the switch port configurations. I've seen this issue triggered by switch port issues in the past. I shoul have said this before.

Also, make sure both ends of DP client and server are thoroughly patched.

Suspects:
Network Switch.
NIC card.

sEP

[Steven E. Protter]

Shalom,

Use mstm or xstm to test the card.

Either way, lean hard on hardware support and get them to swap it out.

Before that have your cisco switch/network admin check the switch port configurations. I've seen this issue triggered by switch port issues in the past. I shoul have said this before.

Also, make sure both ends of DP client and server are thoroughly patched.

Because its an IPC error make sure shmmax and shmseg are adequate on both ends of this transaction.

Suspects:
Network Switch.
NIC card.

sEP

[Ludovic Derlyn]

Hi Steven,

I have not specified , but i have opened an incident to HP support :

first test : check cable and change it --> same result
second test : change network card --> same result

I have checked switch, no specific errors are occured....

thanks for your answers

I 'm waiting for next incident to see if some check are relevant

regards

L-DERLYN

[rick jones]

You say the card is the default gateway - do you mean that the IP assigned to the card is configured as the default gateway IP, or that the default gateway is the IP of the router and is reached through this card?

What is the setting of the IP ndd parapeter ip_ire_gw_probe?

Is there another HP-UX system connected to the same broadcast domain as this one? You could try linkloop when ping isn't working to see if there is still link-level connectivity.

[Ludovic Derlyn]

hi Rick

ndd -get /dev/ip ip_ire_gw_probe return 1

There is another HP-UX system connected to the same broadcast domain as this one

the default gateway is the IP of the router
for all server is 10.x.x.1

I will adding loinkloop in my checklist

thanks for your answer

Regards

L-DERLYN

[rick jones]

If the router will not respond to ICMP Echo Requests, then the dead gateway detection will think the router is dead and mark the route as unusable. If you use ndd to set ip_ire_gw_probe to 0 you will disable dead gatway detection.

It would be best to try the linkloop first and/or a ping to a system in the same IP subnet (ie doesn't need the router to reach) first though. Not that disabling dead gatway detection will necessarily hurt anything of course.

[Ludovic Derlyn]

hi

Incident is occured this Night, always same time with same backup , but incremental not full
I will check all client to sure that is correctly patched for DA

In attachment, result of ipcs and kcusage,
shmseg and shmmax seems correct...
just a remark, what is the signifiant of "no entry " for result of ipcs ?

system_perf.sh has not correctly work, I have some entries but i doesn't finished...
I don't know why...

I haven't the result of linkloop, becasue it's not executed , next time...
If you have any suggest...

regards
L-DERLYN

[Ludovic Derlyn]

Hi

FYI :

The problem that you noticed with the AB465A cards is related to a bug with the 'igelan' driver.
This problem has been documented in JAGaf96873: The network interface hangs during heavy traffic.

This JAG will be corrected in a GR patch that will be available in November 2006, however a binary
fix for this problem is already available today.
The fix consist of a 'updated' libigelan.a library

regards

L-DERLYN