A buffer overrun can be used to put a pointer to exec() and format context so that exec() will call up a terminal or any other existing program (including mail and format command line utils). This context would need to be put where the return pointer had been.

This limits the possibilities somehow, but does not remove the threat completely.

If I’m not making sense I’d like to know abou tit.

thanks,
Alex

To be clear, this is essentially the solution that ridiculous_fish described:

What if, instead of handing qsort() a pointer to the comparison_function() code, we instead passed a pointer to some data structure containing the stack frame of sort_ints() AND the address of comparison_function()? That data structure could turn around and call comparison_function() passing it the stack frame it needs. And the data structure could live on the sort_ints() stack, so we would have one per invocation of sort_ints(), which is just what we need.

The difference is that the trampoline code (and necessity for writable-and-executable memory) is no longer necessary: because of the different calling convention, the caller (e.g., qsort, still dumb as rocks) essentially sets the stack pointer to the one stored in the closure before jumping to the code. The code is all created at compile-time and lives in your read-only section.

(A fun trick is to allocate those “stack frames” and closures on a garbage-collected heap, which lets you acutally return such inner functions back out of the dynamic context (control stack) that created them, to be called whenever you like. So functions can create and return new, parameterized functions. Or “functions are first-class values.”)

(Another fun trick is to “capture” the current “heap-allocated” stack data-structure and store it away for later, even returning it out of the dynamic context as above; called a “continuation”, this could be “invoked” (returned into, if you will) any time later (even multiple times!). For example, you could implement a sort of multi-threading system this way. Or a backtracking algorithm.)

(The above “tricks” are fundamental features of Scheme. I believe (recent) Smalltalk does the same, with more explicit stack-frame introspection.)

The only solution I can think of is for the nested function to have its own copy of the variables of the containing function, along with a bool that is set when it’s being called normally (not via a pointer) by the containing function; if it is, then it uses the containing function’s memory storage for these (the reference distance from the stack pointer will be fixed), and if it isn’t, then it uses the local space.

There is a reasonably strong correlation between languages that support lexically-scoped functions and those that run on some kind of virtual machine. A decent virtual machine is the trivial answer to this problem, since you can architect the machine to make code insertion impossible (at least in traditional senses, e.g. writing data on the stack).

I can’t see any method for doing this kind of thing if:
- you want to compile directly to hardware, and
- the pointer representation can’t be made wider, and
- all code pages must be read-only, and
- code needs to be re-entrant

http://www.rpi.edu/~pudeyo/articles/wndproc/#atl

(*function_pointer)();

which translates to, in the 32-bit case,

mtctr function_pointer
bctrl

would instead be, with descriptors,

ld r1,0(function_pointer) ; code address
mtctr r1
ld r2,8(function_pointer) ; data pointer (not sure about the register here)
bctrl

but this is precisely what is desired for function closures. Not only are there no problems with non-executable stacks, but it is much more efficient since there is no need to use expensive cache flush and barrier instructions. And the “trampoline” (which is just another descriptor) is much smaller.

It is possible to have a secondary stack just for the closure data, as other people have pointed out, and the original paper, on which GCC’s implementation is based, mentions this (http://people.debian.org/~aaronl/Usenix88-lexic.pdf). The problem is with longjmp() which must be aware of the system, or else the stacks will get out of sync.

Ironically, generating code at runtime is slightly easier on x86 than PowerPC because the former have coherent instruction and data caches, so no explicit cache flush is generally needed.

http://www.openbsd.org/cgi-bin/man.cgi?query=gcc-local&sektion=1

i would be interested in reading your comments on the following discussion regarding the differences between the GNU and NeXT objc runtime:

http://www.objc.net/blogger/

thanks.