shmat fails with EACESS error

[Johan_52]

Hello,

I have a PA-1.1 program running on HP-UX 11.0 with setuid bit set. This program gets a shared memory key from a file and attaches the shared memory (the segment is created with the IPC_PRIVATE option set). The access mask is 0600 (Read/Write for the owner). Using ipcs -a I see that the segment is created correctly with the correct owner.
Every now and then attaching this segment fails with a 'Permission denied' (EACCES) error. At that moment a new segment is created with the same owner and access mask. The only difference I see between the new and the already existing segment is that the group of the creator is different. But in my opinion that should not make any difference since the owner is correct (because of the setuid option).
If anyody has seen this before, please enlighten me, because I am running out of ideas.

Thanks in advance !!

Johan Harmsen

[Juan González]

Hi Johan,
If I were you I'd follow these steps:

1. Make sure that the executable setuid file has the rigth own (root if you want a classic root setuid executable).
2. Put geteuid() system call in the code before the shmat and check is what you want.

If the owner is the same that the geteuid() returns and the permissions are 0600 it might work.

Good Luck
Juan Gonzalez

[A. Clay Stephenson]

Hi Johan,

I haven't seen this particular error before but one thing does occur to me. Are you always passing in NULL as shmaddr or do you do something different on the first attachment?

I can also think of a possible workaround (though I admit it should not be necessary):
Do a shmctl to get the gid of the shm and then do a setgid to that group.

Food for thought, Clay

[A. Clay Stephenson · Accepted solution]

Hi Johan,

I've thought about this a bit more and I think you may be using IPC_PRIVATE in a way it was not intended. It's really supposed to be used among a parent process and its descendants although that is not enforced. The fact that a new segment is being created is the crucial clue. I would suggest that you code it this way and still have the benefits of IPC_PRIVATE.

#define MY_ID 'X'
#define MY_KEY_FILE "/usr/local/keyfile"

/* MY_KEY_FILE can be anything; its contents don't matter; all we care about is that the file exists; (we're just going to use its filesystem id, system id and inode number) */

key_t my_key;
int shm_id;
char *p;

my_key = ftok(MY_KEY_FILE,MY_ID);
/* This will generate the same key as long as MY_KEY_FILE & MY_ID are the same; all your processes will use this */
shm_id = shmget(my_key,(size_t) 10240,IPC_CREAT | 0600);

/* You will need to supply the correct size;
this will creat a shm segment if it does not exist*/
p = (char *) shmat(shm_id,(void *) NULL,0);

Since I suspect, that your are pointing to a struct, you will need to make the appropriate type casts but I think this will solve your problem.

Regards, Clay

[Johan_52]

Hello again,

Thanks for the replies. Unfortunately I'm still stuck (maybe evenmore then before ...)

My suspicion that the group had something to do with it, was wrong. I set the 'set group Id on execution bit'. This did not make any difference, other than that the group of the share memory segment indeed changed to the group of the executable (I still got the EACCES error).

I didn't know the IPC_PRIVATE was intended to share a memory segment between processes in the same process tree. Is this documented somewhere ?

I tried the suggestion with ftok but that also didn't change the behaviour.

As far as I can see now the problem occurs when the program attaching the shared memory is run as a child of another application (SAP in this case). I think this application uses brk or sbrk to change the 'shared memory space allocation'.
Although the documentation is not explicit about this, I think the new values are inherited by child processes. So when the forked process creates a shared memory segment it attaches this at an address somewhere between the default and the new upper limit. When I start the same program from the command line this value is outside the default range so I get an error.
I would have expected an EINVAL and not an EACCES however.

So my question : Is this a possible scenario ?
And if yes : can I use brk or sbrk in my program to avoid this problem, and to what value should i set the it ? What would be the repercussions of this, would my program use more memory and swap space ?

Hope somebody can enlighten me !

Best regards,

Johan

[Johan_52]

Hello,

Saw I omitted one thing in my previous reply.
It seems impossible to use any other value than 0 for shmaddr on your first call to shmat. Unlike other Unix(es) HP-UX shmat always fails with EINVAL if another value is specified.

Johan

[A. Clay Stephenson]

Hi again Johan,

Your question about the use of IPC_PRIVATE made me go back and examine the shmget man page. I was working from memory but it turns out that I was correct, IPC_PRIVATE is intended
for a parent process and its children.

I've looked at the errno values for the shm operations and it appears that EACCES can only be related to permissions. I agree you should see EINVAL (or more likely ENOMEM) if the problem is related to sbrk. I suggest that you do this, continue using ftok() and make a key but set the permissions (at least for now) to 0666.

If you still have problems, I would put a shmctl IPC_STAT call in and print all the values in the struct to a log file along with pid, and a timestamp. That way, you should be able to see when it fails and what has happened at a more detailed level than ipcs can reveal and at the moment it occurs.

Hope this gets you closer, Clay

[Johan_52]

Hi Clay,

Thanks for the eyeopener, usually I refer to the X/OPEN System Interfaces 4 manual when looking for information on system calls. But regarding shared memory this is not valid for HP-UX.
Strange thing remains that the code with IPC_PRIVATE has been running in the similar contexts for around 7 years, without any problem, until last week ...
Unfortunately there was no chance to test today, so I have no feedback on your last reply. But I can hardly imagine that the problem is related to access rights.
ipcs -a shows the correct creator and owner
the program uses setuid, and when I display the effective uid in the program it is correct.
On the other hand I wrote a small test program to see if I could prove my assumptions about (s)brk, but the test program worked as expected. Giving the correct error at the correct time.

I'll keep you informed.

Best Regards,

Johan

[A. Clay Stephenson]

Hi Johan,

I think I'm on the right track with permissions. The group seems to refer to process group and if there are problems errno is set to EACCESS not EPERM as one would expect. I could also cause problems if I used mixed 64-bit and 32-bit code and did not assert the IPC_SHARE32 flag when the shm segment was created. That results in EINVAL however. Since this started last week, I have to ask what changed? Patches?

Just trying to help, Clay

[Johan_52]

Hi Clay,

Thanks again for the response I really appreciate it ! What has changed, well the problem occurs at a customer of ours. They tell me that they installed a bunch of patches on their SAP 3.08 system to make the migration to SAP version 3.51 possible. I have not been able to contact the person who in installed this to ask for details. According to the customer no changes were made to their operating system (HP-UX 11.0. The only thing I can say is that I know SAP is very tightly coupled to the Operating System and I have heard many times that SAP upgrades at least required a number of patches and usually a certain version of the Operating System. (Installing the operating system is usually easier than upgrading Sap :-).

I'm pretty sure I don't mix 32 and 64 bit code, besides that the system on which the error occurs is a PA-RISC 1.1 system, which only runs 32 bit code AFAIK.

Could you tell me how you got the EACCES error, I must admit that I don't really understand to which group you are referring when you say 'The group seems to refer to the process group'. I tried several things with setpgid and setsid but didn't see any problem.

Thanks once again.

Johan

[A. Clay Stephenson]

Hi Johan,

Here's my simple test. If I create the shm segment with permission mask 0600 and run it as two different users, I do indeed get EACCES
on the shmat. If I creat with 0666, the attach is ok. I haven't tried it with setuid/setgid.

This program should be simple enough to help up zero in on the problem.

Hope this helps, Clay

[A. Clay Stephenson]

Hi Johan,

I had one other silly thought: I don't suppose it's possible that somehow your executable got installed on a nosuid mounted filesystem?

Clay

[Johan_52]

Hi Clay,

The test has still not been performed, in the mean time I send the same problem report to SAP. Their initial response was also that the access rights to the shared memory segment must be incorrect. But after looking for hours and hours I still fail to see how the problem could possibly occur. Maybe your suggestion by setting the mode 0666 helps.
This is the another part of the reply dealing with my suspicion about (s)brk.

Quote

SAP does not explicitly change the brk value. This also would have no impact
on
HP for data segments and shm segments are residing in different quadrants of

HP's address space.
quad1: text 00000000..3FFFFFFF
quad2: data 40000000..7FFFFFFF
quad3: shm 80000000..BFFFFFFF
quad4: shm C0000000..FCFFFFFF

Also, address overlapping cannot occur on HP, for all shm segments on HP
share the same virtual address space. All shm segments have unique and
exclusixe
ranges in quad3 or quad4.

Up to R/3 Rel 3.1x, SAP delivers an an optional SHMEM_MAGIC kernel, with
quad1: text and data 00000000..3FFFFFFF
quad2: shm 40000000..7FFFFFFF
quad3: shm 80000000..BFFFFFFF
quad4: shm C0000000..FCFFFFFF
shm memory segments residing in quad2 can only be accessed from programs
also chattr'd with SHMEM_MAGIC

Unquote

I have not yet been able to verify if the customer is using such an optional kernel. But if they do, the search will obviously go into a completely different direction.

All segments that I have seen debugging this problem have addresses in 'quad3' space.

Let's not make any futher assumptions before we find out more about the SAP kernel.

Thanks,

Johan

[Johan_52]

Hi Clay,

Just to let you know that I ran the test again with the access mask set to 0666. Using ipcs and shmctl I could see that the segment is indeed world read/writable. But I still get the EACCES error when trying the attach the segment. The program is not run from a non-setuid file system.

/home/unispool on /dev/vg01/lvunispool delaylog on Sun May 13 09:36:43 2001

This is the function I currently use to attach the memory segment.

if ((ShmKey = ftok(fref, 'S')) == -1)
{ perror("ftok"); exit(1); }
if ((ShmId = shmget(ShmKey,
sizeof(SPREQ), 0666 | IPC_CREAT)) == -1)
{ perror("shmget"); exit(1); }
if ((sp = (SPREQ *) shmat(ShmId, (void *) 0x0,
0)) == (SPREQ *) -1)
{ perror("Shmat"); exit(1); }


I have not been able to get anymore information from SAP

Maybe one day I'll understand what's going on ...

Best regards,

Johan

[Johan_52]

Hi Clay,

Thanks for your help, unfortunately I couldn't solve the problem. I give up on this one, at least until the customer installs a newer version of SAP (sometime later this year).
The good news is that thanks to your help the code improved because I now used the shared memory keys in stead of IPC_PRIVATE, that should make the code more robust.

Thanks once again, and best regards,

Johan

[Johan_52]

Hi Clay,


Finally somebody found the cause of the problem. I got the following information from our customer.

Quote

We are running one 32bit R/3 system version 3.1 on a 64bit HPUX with 4GB of main memory.

A 32bit SAP is only able to adress 1,8GB of main memory.

To be able to use the memory above 1,8GB there is a functionality from HP called "high memory window" With this you can run a 32bit application in the "HMA" High memory area (Known from MS-DOS :-)

Doing this you get two SAP instances on one physical machine.

As long as the instance in the "low memory window" calls the external program accessing the shared memory segment, everything is OK. But if the instance in the "high memory window" calls the external program, the segment errors will occur.

Unquote.

To be honest I didn't even know that these memory windows existed

Best regards,

Johan

[A. Clay Stephenson]

Hi Johan,

I'm glad to hear that you won this battle. I actually think it would have worked had the process creating the shm segment asserted the IPC_SHARE32 flag.

Clay