PtP
11-08-2004, 12:22 PM
The DivX problem is going to be repeated on quite a few applications.
Anything that generates machine code into a memory buffer and then calls it will be hit, until the authors modify their software to tell Windows what they're doing. The calls for this have existed since the introduction of the Win32 API - it's just that until now, nothing went wrong if you didn't use them.
The technical details go like this. Since x86- compatible CPUs acquired memory protection, at the 386, it's been possible for operating systems to mark memory segments as readable or writable - much like filename permissions under UNIX/Linux. But until AMD introduced the NX bit, it hasn't been possible to control if segments were executable. If memory could be read, you could try to execute it.
Buffer overflow attacks work in just this way: upload a large packet, stuffed with machine code and with trickery to get the target computer to execute it. For more gory details, see Jon Erickson's Hacking: the art of exploitation, from No Starch Press, or doubtless any one of dozens of website. DEP marks stack and data memory as non-executable, so that if you try to execute it, you get an error.
DivX will be generating machine code into a buffer and calling it. This is a recognised trick for accessing screen memory extremely quickly: you generate code that implicitly knows about the screen memory layout, the logical operatings you want to do on the image, and so on. They you call it. It runs a load faster than code that's full of "if (24bitscreen) {} else {}" idioms, because all those questions were answered while it was geing generated.
To avoid DEP biting you when you do this, you need to allocate the memory that will hold the code with the correct flags. With a quick look in MSDN, it seems that VirtualAlloc with the appropriate flProtect option is what you need. Windows will set up a memory segment that's execute-enabled, and give you back a pointer to it. You do need to use VirtualAlloc: malloc() doesn't know a thing about this stuff.
As for why the DivX install is giving trouble, I don't know. I'd suspect an installer which generates machine code for its own operatings on the fly, or something like that. Other things that could easily get bitten by DEP include naive Java JITs and similar programs. Straight C or C++ code that doesn't try to get clever with the guts of the hardware doesn't seem to do this.
Like the DivX answer page says, you can turn off DEP for individual programs under XPsp2. My Computer=> Properties=>Advanced=>Performance.
Anything that generates machine code into a memory buffer and then calls it will be hit, until the authors modify their software to tell Windows what they're doing. The calls for this have existed since the introduction of the Win32 API - it's just that until now, nothing went wrong if you didn't use them.
The technical details go like this. Since x86- compatible CPUs acquired memory protection, at the 386, it's been possible for operating systems to mark memory segments as readable or writable - much like filename permissions under UNIX/Linux. But until AMD introduced the NX bit, it hasn't been possible to control if segments were executable. If memory could be read, you could try to execute it.
Buffer overflow attacks work in just this way: upload a large packet, stuffed with machine code and with trickery to get the target computer to execute it. For more gory details, see Jon Erickson's Hacking: the art of exploitation, from No Starch Press, or doubtless any one of dozens of website. DEP marks stack and data memory as non-executable, so that if you try to execute it, you get an error.
DivX will be generating machine code into a buffer and calling it. This is a recognised trick for accessing screen memory extremely quickly: you generate code that implicitly knows about the screen memory layout, the logical operatings you want to do on the image, and so on. They you call it. It runs a load faster than code that's full of "if (24bitscreen) {} else {}" idioms, because all those questions were answered while it was geing generated.
To avoid DEP biting you when you do this, you need to allocate the memory that will hold the code with the correct flags. With a quick look in MSDN, it seems that VirtualAlloc with the appropriate flProtect option is what you need. Windows will set up a memory segment that's execute-enabled, and give you back a pointer to it. You do need to use VirtualAlloc: malloc() doesn't know a thing about this stuff.
As for why the DivX install is giving trouble, I don't know. I'd suspect an installer which generates machine code for its own operatings on the fly, or something like that. Other things that could easily get bitten by DEP include naive Java JITs and similar programs. Straight C or C++ code that doesn't try to get clever with the guts of the hardware doesn't seem to do this.
Like the DivX answer page says, you can turn off DEP for individual programs under XPsp2. My Computer=> Properties=>Advanced=>Performance.