topic Re: Stop stack smashing attacks in Operating System - HP-UX

Stop stack smashing attacks

Mark Ferraretto — Wed, 07 Feb 2001 11:38:22 GMT

Hello all,

In sun solaris you can set a kernel parameter called noexec_user_stack which prevents an application's stack from being executable. This helps to prevent buffer overflow attacks. Is there an equivalent parameter that can be set in HP-UX? I'm running HP-UX 11.00.01 on a L2000

Thanks
Mark

Re: Stop stack smashing attacks

Cheryl Griffin — Wed, 07 Feb 2001 13:13:35 GMT

Mark,
I checked many resources on this.... everything points to a security vulnerability on Solaris, nothing to HP-UX.

I checked with someone who works with kernel stacks and they also have not heard anything of noexec_user_stack OR of anything that makes a stack nonexecutable.

This simply means that currently there is no equivalent to noexec_user_stack in HP-UX and no documented need for it, as of current versions.

Re: Stop stack smashing attacks

Mark Ferraretto — Thu, 08 Feb 2001 02:04:56 GMT

Actually, I think that the vulnerability is a cross-platform one. All applications have a heap and a stack as part of their run-time structure. A stack-smashing attack inserts pointers into the stack and then uses code that executes the code the pointer in the stack refers to. This is a common buffer overflow attack.

A solution is to tell the kernel to make the stack non-executable. This means that code cannot be executed from the stack and so prevents the problem.

On Solaris, you tell the kernel to make the stack non-executable by setting the kernel parameter I described above. I was hoping there might be a similar type of parameter under HP-UX.

If the vulnerability is with Solaris only, then this means that the stack on HP-UX systems is non-executable by default. Can you confirm this?

Thanks
Mark

Re: Stop stack smashing attacks

Len Lattanzi — Thu, 08 Feb 2001 05:21:16 GMT

Compile then run the attached GCC program. trampolines are on the stack. Even without executable stack the return address might be smashed to point to a dangerous location (say reboot!)
This example is derived from http://www.uwsg.iu.edu/hypermail/linux/kernel/0006.0/0703.html

Re: Stop stack smashing attacks

Magnus Ljung — Thu, 08 Feb 2001 07:57:28 GMT

FYI,

HP-UX 11i: New security features

executable_stack Kernel Parameter

It is now possible change the executable_stack tunable kernel parameter
to zero in order to protect program stacks against attacks that cause
buffer overflows.

You can change the executable_stack parameter as follows:

- Use the System Administration Manager (SAM) to change the parameter for
all programs.

- Use chatr(1) with its es option to change the parameter for specific programs.

Changes were made to the kernel execve() function, to virtual memory
code, and to the chatr(1) and elfdump(1) commands.