Getting error 25 (ENOTTY) when calling BLOPEN on a file descriptor for a PTS slave device

[Kevin McKee]

We are trying to convert from a physical HP9000, HP-UX 10.20 system, to an HP SRP container. We have noticed that when we telnet to the physical HP9000 system we connect as a TTY device, but when we connect the container we connect as a PTS pseudo-terminal. Some of the programs that we run within the system are now complaining that this is not a TTY device (ENOTTY error 25).

I have written a test C program to try to isolate which call is causing the issue, and it seems to be the "BLOPEN" command to open a block mode file descriptor.

#include <stdio.h>
#include <sys/fcntl.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <termio.h>
#include <signal.h>
#include <stropts.h>

int main()
{       int fdm;
        int fds;
        char *slavename;

        int j_bfd;
        FILE *pf_stream;

        unsigned short i_c_lflagsav;
        unsigned short i_c_oflagsav;
        struct termio s_termio;

        if ((fdm = open("/dev/ptmx",O_RDWR)) == -1){
          printf("Error getting file descriptor %d\n", errno);
          return -1;
        }
        printf("Opened /dev/ptmx\n");

        printf("before grantpt\n");
        grantpt(fdm);
        printf("after grantpt\n");
        printf("before unlockpt\n");
        unlockpt(fdm);
        printf("after unlockpt\n");

        slavename = ptsname(fdm);

        printf("Slavename\n");
        printf("%s\n",slavename);

        if ((fds = open(slavename,O_RDWR)) == -1){
          printf("Error opening slavename\n");
          return -1;
        }

        ioctl(fds, I_PUSH, "ptem");       /* push ptem */
        ioctl(fds, I_PUSH, "ldterm");    /* push ldterm */

        /* no echo.  */
        printf("before no echo\n");
        if (ioctl(fds,TCGETA,&s_termio) == -1) {
          printf("No echo error\n");
          return(-1);
        }

        i_c_lflagsav = s_termio.c_lflag;
        i_c_oflagsav = s_termio.c_oflag;

        printf("Before no tabs\n");
        if ((s_termio.c_oflag & TAB3) != 0) {

         /* set the output mode flag for no altering of tabs */
         s_termio.c_oflag &= ~TAB3;

         /* set new value */
         if (ioctl(fds,TCSETA,&s_termio) == -1) {
            printf("Error\n");
            /* try to reset driver to the way it was before we changed it */
            s_termio.c_oflag = i_c_oflagsav;
            /* set old value */
            ioctl(fds,TCSETA,&s_termio);
            return(-1);
         }
        }

        /*  get a block mode file descriptor from blopen */
        printf("Before block mode open\n");
        if ((j_bfd=blopen(fds,O_RDWR)) < 0 ) {
                printf("Error %d during blopen\n", errno);

        /* if we changed tab handling then try to reset the driver back. */
        printf("Before reset\n");
        if (((s_termio.c_oflag & TAB3) != 0))
         ioctl(fds,TCSETA,&s_termio);

        printf("Error getting block mode file descriptor %d\n", j_bfd);

        return(-1);
        }

        return 0;
}

The output of the program is

Opened /dev/ptmx
before grantpt
after grantpt
before unlockpt
after unlockpt
Slavename
/dev/pts/0 (I always get /dev/pts/0, even if I run this from multiple terminals? I would've thought I'd get /dev/pts/1, /dev/pts/2, on subsequent terminals?)
before no echo
Before no tabs
Before block mode open
Error 25 during blopen
Before reset
Error getting block mode file descriptor -1

So my question is, does anyone know what causes BLOPEN to generate an error 25? How does it determine if it's a TTY device or not?

[stevedaily]

---

I have same problem...   

---