errno = 55 = No buffer space available

[Patrick J. Henry]

To all -- Thank you for the help on using netclose() in my TCP programs. I ran 2 of my programs over the weekend and they did just fine. However, I did slow them down. But now that I have set them to cycle every 5 seconds, I am getting a "No buffer space available" error. What buffer space is this referring to?

[Richard Whalen]

I would guess that it's referring to buffers in the networking software. If I remember right, you are using Wollongog PathWay; I no nothing about tuning that.

[Steven Schweda]

Are you offering any clues yet as to the
version of the PathWay (or WIN/TCP) software
which you are using? Or which C compiler?

My PathWay version 3.0 documentation is not
extremely informative:

55 ENOBUFS - No buffer space available

Resources available were insufficient to to perform the operation.


Could be a VMS user quota or system parameter
problem, if you were lucky. Otherwise,
there might be something in the PathWay
configuration itself which affects it, but
It's hard to guess from here, knowing nothing
substantial about your system.

It might be interesting to know which
function actually failed, but I know how to
set _my_ expectations.

[Wim Van den Wyngaert]

For TCP connections, close initiates the connection closing protocol, though it returns immediately. Thus, the internal connection structures persist until the connection has reached the CLOSED state.

In addition to close for TCP connections, there is an ioctl call, NETCLOSE, which indicates that the local connection will send no more data, but is still able to receive data from the foreign peer. In this case, subsequent writes are illegal and will terminate with errors, but subsequent reads
will work until the connection is closed by the foreign peer.

BTW : you pay the bill by assigning points (also in your pevious question)

Wim (with the help of Google)

[Steven Schweda]

> BTW : you pay the bill by assigning points

I prefer cash, but I'll take IA64 hardware.

[Patrick J. Henry]

I appologize for not assigning points. I did not understand the procedure. I have gone back and assigned points to all for the first question that I asked. I will look into the buffer space issue as a possible user quota.

By the way, I found the Pathway API manual.
It is release 1.0 dated December 1992.

I did not find any sort of sys admin manual which might help me set quotas and parameters for the Wollongong software.

Once again, thank you for your help. My contract is up on Friday, so if I don't solve this mystery, someone else will have to.

[Wim Van den Wyngaert]

The points didn't arrive. Did you press "submit points" after assigning them ?

Wim

[Wim Van den Wyngaert]

When you encounter the problem, try the equivallent of ucx show dev (try it during operations too to see how many sockets you use).

Wim

[Steven Schweda]

Probably "netstat -a", or similar, assuming
that the symbol's defined.

[Wim Van den Wyngaert]

Found this too. Need for your W... stuff.

#define NETREAD(s,b,l) ((s)>10 ? netread((s),(b),(l)) : read((s),(b),(l)))
#define NETWRITE(s,b,l) ((s)>10 ? netwrite((s),(b),(l)) : write((s),(b),(l)))
#define NETCLOSE(s) ((s)>10 ? netclose(s) : close(s))

Wim

[Steven Schweda]

> Found this too.

Apparently.

> Need for your W... stuff.

I don't think so. I can see what these
things do (UNIX file numbers v. VMS channel
numbers), but I can't see why you would need
(or want) to use them, nor can I see what
good they would do for this problem.

--------

> By the way, I found the Pathway API manual.
> It is release 1.0 dated December 1992.

Interesting, but not the same as knowing the
version of the running software. Did you
ever look under TWG$TCP:[000000...] for a
*version*.exe to run, or any other clues?

> It might be interesting to know which
> function actually failed, [...]

Still true, or did I miss something?

> [...] if I don't solve this mystery,
> someone else will have to.

I can wait.

[Patrick J. Henry]

The PCs running the simulators were powered off during the night. So, I have restarted my testing.

I was able to find this:

SO1CP1$ run PATHWAY$LIST_SOFTWARE.EXE
Installed PathWay Products:
--------------------------
PathWay Access for OpenVMS 1.0.1.0
PathWay API for OpenVMS 1.0.1.0
PathWay Runtime for OpenVMS 1.1.1.1

SO1CP1$ sho symb net*
NETADMIN == "$TWG$TCP:[NETDIST.ETC]NETADMIN"
NETECHO == "$TWG$TCP:[NETDIST.USER]NETECHO"
NETSTAT == "$TWG$TCP:[NETDIST.USER]NETSTAT"
NETTIME == "$TWG$TCP:[NETDIST.USER]NETTIME"
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1850 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1845 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1840 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1834 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1829 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1824 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1818 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1813 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1806 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1800 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1793 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2046 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2033 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2032 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2031 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2030 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2017 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2016 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.2015 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1787 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1782 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1777 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1771 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1766 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1761 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1751 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1745 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1740 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1734 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1729 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1724 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1718 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1715 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1710 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1704 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1698 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1692 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1686 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1681 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1676 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1670 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1665 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1660 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1653 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1648 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1643 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1635 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1630 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1625 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1619 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1614 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1609 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1603 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1597 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1590 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1580 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1575 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1569 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1564 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1559 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1553 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1548 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1543 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1539 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1534 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1527 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1521 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1516 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1511 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1505 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1500 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1495 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.4954 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.4822 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.4706 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.4605 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.4506 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.4485 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.4355 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.4238 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.4155 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.4070 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.4050 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3937 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3916 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3803 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3686 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3571 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3470 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3370 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3350 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3221 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3120 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.3037 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.3015 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.2900 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.2785 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.2669 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.2553 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.2436 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.2337 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.2220 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.2089 151.126.144.175.502 ESTABLISHED
tcp 52 0 so1cp1.1854 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1851 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1831 151.126.144.176.503 TIME_WAIT
tcp 0 0 so1cp1.1826 151.126.144.176.503 TIME_WAIT
tcp 0 0 so1cp1.1974 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1857 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1754 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1654 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1602 151.126.144.176.503 ESTABLISHED
tcp 0 400 so1cp1.telnet 151.126.144.158.1821 ESTABLISHED
tcp 0 0 so1cp1.1724 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1593 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1442 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1326 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1494 151.126.144.176.503 ESTABLISHED
tcp 0 0 so1cp1.1475 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1210 151.126.144.175.502 ESTABLISHED
tcp 0 0 so1cp1.1094 151.126.144.176.503 ESTABLISHED
SO1CP1$

151.126.144.175 and 151.126.144.176 are the IP addresses that my simulators are running on. And I am talking to ports 502, 503, 15216, and 15217.

SO1CP1$ netadmin
netadmin> help
Commands may be abbreviated. Commands are:

protocol protocol to administer: gated, named, ntpd, snmpd or inetserv
log turn logging on, increase logging level
nolog turn logging off
dodump dump named's or gated's internal database
reload reload named's, gated's or inetserv's database
cease terminate given protocol
status print current status
achkpt reset named's cache file with current cache information
trace dump named's status information
quit exit netadmin
exit exit netadmin
help print help information

netadmin> help cease
terminate given protocol
netadmin>

IF the "FIN_WAIT_2" indicates that the port is waiting to be closed, how do I close it? The help information on the "NETDAMIN" program is not very verbose!

[Steven Schweda]

> Installed PathWay Products:
> --------------------------
> PathWay Access for OpenVMS 1.0.1.0
> PathWay API for OpenVMS 1.0.1.0
> PathWay Runtime for OpenVMS 1.1.1.1

Ok. That's old, but it could be worse.

With all those connections, it seems likely
that your errno 55 means that you ran out of
channels, or something similar.

What I don't know about TCP/IP could fill
volumes of RFCs, but a quick Google search
for FIN_WAIT_2 suggests that SO_KEEPALIVE
might be helpful. For example:

http://h71000.www7.hp.com/wizard/wiz_3626.html

That, or more careful closing of connections
(at both ends), perhaps. (It's been a long
time since I did any of this stuff, but when
I did, I never saw this behavior, so you're
exceeding my range of experience.)

[Jim_McKinney · Accepted solution]

The FIN_WAIT_2 states indicates that your host has sent each of those connections a FINish packet (i.e. your application has performed a close() of some sort on the socket). The IP stack on the remote end has ACKed that it has received the request to finish (or close if you will), but, the remote application has yet to perform its close() which would result in the stack sending a FINish signal. The local IP stack is waiting for that FINish packet. Unless your network is experiencing problems, you should check the application code on the remote end as it appears to be misbehaving.

You might note how many of these lingering connections are present when you experience "errno = 55". Their abundance might be the source of your unavailable buffer space.

[Wim Van den Wyngaert]

If the "active closer" doesn't receive the close within 75 seconds, it will close the socket itself. Not clear for me if this is 75 seconds in total or 75 seconds after the last data received. The 75 seconds is implementation specific and could be much higher in your old version. Or be a parameter.

Wim

[Jim_McKinney]

Some stacks implement a FIN_WAIT_2 timeout and some do not. The timeout interval varies with the implementaion. The KEEPALIVE timer can be used to control connections where there is no FIN_WAIT_2 timeout. I have no idea how the TWG stack behaves.

[Patrick J. Henry]

Well, I am pretty sure that the "Mickey Mouse" simulator that I conjured up is my problem. I added the KEEPALIVE socket option and ran a test. I was still leaving behind un-finished business.

SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1388 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1338 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1336 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1334 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1333 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1331 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1329 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1327 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1392 ip21dcn.7310 SYN_SENT
tcp 0 0 so1cp1.1391 ip21dcn.7311 SYN_SENT
tcp 0 890 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

I kept running NETSTAT watching to see if a "timeout" would clear things up. But no success. I went to lunch and a co-worker shut the system down to install a disk drive. When I got back, there were no "FIN_WAIT_2" entries when I ran NETSTAT.

So, I ran another test wherein I processed only one record from the simulator. I halted my CaldonTCP.EXE program and checked NETSTAT. There was one entry with the FIN_WAIT_2 status for the simulator IP & Port, which corresponded to the one record that I processed from the simulator.

SO1CP1$ sho time
27-MAR-2007 11:34:36
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1475 151.126.144.175.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1504 ip21dcn.7310 SYN_SENT
tcp 0 0 so1cp1.1503 ip21dcn.7311 SYN_SENT
tcp 0 337 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

Then I terminated/deleted the simulator on the PC and ran NETSTAT again.

SO1CP1$ sho time
27-MAR-2007 11:35:29
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1475 151.126.144.175.15216 FIN_WAIT_2
tcp 0 183 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

In less than a minute, the entry for 151.126.144.175.15216 was gone!!

So, I going to assume that the problem is in the simulator and will not appear in the actual peice of equipment that my programs will be talking to. I hate to assume, but this software will not be installed at the plant for another 6 months or so.

All that is left is to put the code into the source code control database and update the associated documents.

You guys ahve been a really big help.

[Patrick J. Henry]

I must be blind !!! The "FIN_WAIT_2" is still there!

So, I checked again, and now it is gone.

SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1716 ip21dcn.7310 SYN_SENT
tcp 0 0 so1cp1.1714 ip21dcn.7311 SYN_SENT
tcp 0 258 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

I gotta find out if the system has been rebooted again.

[Patrick J. Henry]

WOW! I need to be more careful. I reran my test, and about 75 seconds after I terminated the simulator the entries in NETSTAT disappeared.

The entries went from a "FIN_WAIT_2" status to a "TIME_WAIT" status upon termination of the simulator. 75 seconds later, they were gone!

Terminated my program:
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1974 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1972 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1970 151.126.144.158.15216 FIN_WAIT_2
tcp 0 0 so1cp1.1980 ip21dcn.7310 SYN_SENT
tcp 0 0 so1cp1.1979 ip21dcn.7311 SYN_SENT
tcp 0 495 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

Terminated simulator:
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1974 151.126.144.158.15216 TIME_WAIT
tcp 0 0 so1cp1.1972 151.126.144.158.15216 TIME_WAIT
tcp 0 0 so1cp1.1970 151.126.144.158.15216 TIME_WAIT
tcp 0 0 so1cp1.1989 ip21dcn.7311 SYN_SENT
tcp 0 311 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

And finally, they are gone:
SO1CP1$ netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 so1cp1.1990 ip21dcn.7310 SYN_SENT
tcp 0 0 so1cp1.1989 ip21dcn.7311 SYN_SENT
tcp 0 258 so1cp1.telnet 151.126.144.158.2180 ESTABLISHED

Again, thanks for all your help. I would have never known where to look to find out what was happening.

[Patrick J. Henry]

I guess that does it for this issue. I will have to change my profile so that it uses my home e-mail after I leave here.