I posted a few notes about this to the Microsoft DirectShow forum, but those things have a tendency to be badly indexed and less than permanent, so I’m summarizing what I have learned in the last 24 hours here, where it might be easier for others to find.
The short version of this week’s stunning discovery is that, if you provide the Video Mixing Renderer 9 with a video stream that includes an alpha channel, the VMR9 will treat all alpha values in your stream that are below 17 (0×11) as though they were zero. I have scoured the MSDN for info on this and come up with nothing. The closest I can find is their page on setting Direct3D 9′s alpha testing state.
A book on game programming, written by two Microsoft employees, is more explicit, stating what I just said here: the VMR9 ignores pixels with an alpha value of 16 or less.
What that means in pratice is that, if you have a smooth gradient in your alpha values, starting with fully opaque at (for example) the bottom of your screen, diminishing downward from 255 (full opacity) to the middle values (partial transparency), ending at 0 (full transparency), and apply that gradient to a completely black image on Input 1 of the VMR9 while a completely white image is on Input 0, the output looks like this:
That all-white band at the top is where the alpha values dropped below 17. You should still see some gray there, with only the top-most couple of rows of pixels showing bright white. But you don’t. You see what you see. Apparently, this is to avoid needless computation for pixels so close to being fully transparent that it isn’t worth the overhead to blend them. However, since the virtue of the VMR9 is supposed to be that it uses hardware acceleration, one has to wonder why this is still the way it works. Can’t a modern graphics card do compositing without CPU overhead? Maybe it can’t, but I’d be surprised. Regardless, this ought to be a setting you can change via some API call.
But you can’t.
The only way to change it appears to be to use the SetRenderState method on the Direct3DDevice used by the VMR9. And there’s no way (I can find) to get at it. If you write a filter that processes IMediaSample objects from the VMR9, you can use a series of queries and calls to get its Direct3D surface and, from that, a Direct3D device, but it’s not the one the VMR9 is using. (Why it isn’t baffles me, since you are getting access to the VRAM on the card through that same IMediaSample.)
A custom allocator-presenter would let you have control over the Direct3D device used for presentation, but, by that time in the sequence of video processing, the composition has already been done. That suggests that the solution, if there is one, might be in writing a custom compositor (or, alternatively, a custom rendering filter).
After a long day of Googling, the only other person I can find who ever noticed this was a fellow named Ulf, who posted about it online.
Seven years ago.
He likewise suggests that a custom compositor might be the way to solve it. (I have e-mailed him to ask if he ever found a solution, but it’s an old post and I certainly won’t hold it against him if he doesn’t reply.)
So, there you have it: Another day shot to Hell because Microsoft locked up an arbitrary setting that is available in Direct3D, but not in the VMR9 which relies on Direct3D. One that messes up my picture. And one that Microsoft couldn’t be bothered to document, much less make available through an API.
If anyone knows how to beat this, by getting at the VMR9′s Direct3D device or otherwise, please let me know in comments here.






