DirectXGraphics: Accessing Texture Memory
Author: Jack Hoxley
Written: 22nd May 2001
Contact: [EMail]
Download: Graph_14.Zip (122 kb)

Contents of this lesson
1. Introduction
2. Copying from Texture to Texture
3. Gaining access to Texture memory
4. Gaining access to the back-buffer and front-buffer
5. Doing something with the memory

1. Introduction

Welcome to the last in the series of tutorials on advanced texturing. By the end of this article you'll probably have enough knowledge to do most things related to textures - if not, you'll be able to pick them up fairly quickly. Accessing texture memory is the last major thing to cover.

Up till this point all we've done with texturing is load them, render them and blend them - there will be times when you need a little bit more functionality. Maybe you want to combine lots of little textures onto one larger texture, maybe you want to create a texture using a mathematical filter (noise for example), or maybe you need to draw a copy of the world map onto a texture for use in the user interface... This is what this tutorial is about.

2. Copying from Texture to Texture

Before I launch into the main core of this tutorial I want to cover the copying of one texture to another. This was a very common thing to do in DirectDraw7 - many people migrating from version 7 to version 8 of DirectX will be scratching their heads wondering where they're Blt( ) and BltFast( ) calls had been hidden...

A typical use of this method is to load in lots of small tiles (Grass, Stone, Sand, Water etc...) and stick them all on one large palette like texture. Particularly useful if you only want to store 1 copy of each tile's texture, and you don't know during development what textures will be used...

Direct3D only allows us to copy between surfaces (Direct3DSurface8), not textures (Direct3DTexture8) - so the first step to copying between the textures is to convert them to a surface. Strictly speaking we're not going to convert them or copy them to surfaces, we're going to make a surface point at a specific level in the texture, identical in theory to pointers in C/C++ (if your familiar with them). These surfaces can then be used in copy operations, and the results will be directly mirrored onto the texture - which we can then render from. Using this method of referencing a texture through a surface can also be used to make a texture a render target (for special effects). Here's the complete Code:

'## DECLARATIONS ##
'//Our renderable textures
Dim TexSource As Direct3DTexture8
Dim TexDest As Direct3DTexture8
Dim TexComb As Direct3DTexture8
'//The surfaces that will point to them.
Dim SurfSource As Direct3DSurface8
Dim SurfDest As Direct3DSurface8
Dim SurfComb As Direct3DSurface8
'## INITIALISATION ##
'//Create Our TEXTURE objects
Set TexSource = D3DX.CreateTextureFromFileEx(D3DDevice, App.Path & "\texsource.bmp", _
                                                                        128, 128, 1, 0, D3DFMT_R5G6B5, D3DPOOL_MANAGED, _
                                                                        D3DX_FILTER_LINEAR, D3DX_FILTER_LINEAR, 0, ByVal 0, _
                                                                        ByVal 0)
Set TexDest = D3DX.CreateTextureFromFileEx(D3DDevice, App.Path & "\texdest.bmp", _
                                                                        128, 128, 1, 0, D3DFMT_R5G6B5, D3DPOOL_MANAGED, _
                                                                        D3DX_FILTER_LINEAR, D3DX_FILTER_LINEAR, 0, ByVal 0, _
                                                                        ByVal 0)
Set TexComb = D3DX.CreateTexture(D3DDevice, 128, 128, 1, 0, D3DFMT_R5G6B5, D3DPOOL_MANAGED)
'## RENDER LOOP ##
'//When Rendering, We make a pointer to the texture:
    Set SurfSource = TexSource.GetSurfaceLevel(0)
    Set SurfDest = TexDest.GetSurfaceLevel(0)
    Set SurfComb = TexComb.GetSurfaceLevel(0)
'//We then copy 1/2 of SurfSource and 1/2 of SurfDest to SurfComb
    rctSource.Top = 0: rctSource.Left = 0: rctSource.Right = 64: rctSource.bottom = 128
    ptDest.X = 0: ptDest.Y = 0
    D3DDevice.CopyRects SurfSource, rctSource, 1, SurfComb, ptDest
    rctSource.Top = 0: rctSource.Left = 64: rctSource.Right = 128: rctSource.bottom = 128
    ptDest.X = 64: ptDest.Y = 0
    D3DDevice.CopyRects SurfDest, rctSource, 1, SurfComb, ptDest

Finally there is the CopyRects( ) call. This is the basis of everything we want to do - it takes multiple Rect's (Rectangles) defined by the structure RECT, and copies them from the destination to a given point on the destination surface. Anyone familiar with DirectDraw7 will be at home straight away here - it's basically a BltFast( ) call. The only difference being that you can specify an array of Rectangles. The most important thing to note is that you must make sure that the rectangles are valid, as in, they cannot overlap the edge of a surface - D3D will not clip the rectangle to fit - it'll just refuse to do it. Also note that we're now using pixels as coordinates and measurements - not the standard texture addressing 0.0 to 1.0 scale...

3. Gaining access to Texture memory

Now we move onto the real meat of this tutorial - accessing individual pixels. BUT, this is complicated - we're going down as far as binary manipulation of memory, if you're not too sure with your ANDs, ORs, Bits, Shifting and so on then you may well get out of your depth here; I'll make it as easy as possible - but if you do get lost then you'll need to go looking through some guides elsewhere (many better articles than I could write exist).

The first stage is about getting access to the memory - how to play with it, and how it all works is going to be explained in a minute. First off, why do you you need to know about this?

Well, this sort of thing comes up in a variety of forms - traditionally it's been the realm of the advanced DirectDraw programmer looking for some special effects (Alpha Blending, Colour Blending, Particle Effects, Lighting etc...), and to a certain degree it still is going to be a toy of the advanced programmer, but it has it's uses to the simple program. Take the map that your world is based on - you could, during level creation, save a picture of it (for an in game map), but using this method you can read the data from the file and generate an appropriate texture on the fly as you need it - especially useful if you want multiple zoom levels, or the level itself is very large (and saving a .bmp is not appropriate). Alternatively, should you want to play around with your own image format, unless you're using the DXTex tool (included with the SDK) it can be a pain in the back side getting an alpha channel embedded into your texture - using this method you could make a tool that takes two images (the RGB channel and the A channel) and combines them, saves them to a custom format, then when your game loads it reads this data, creates the relevant texture type and writes the pixel data straight in. I'm sure you'll find some use for it...

Dim pData As D3DLOCKED_RECT, pxArr() As Byte
    TexDMA.LockRect 0, pData, ByVal 0, 0
        'we can now play around with the stuff in pData
        ReDim pxArr(pData.Pitch * 128) As Byte 'enough bytes, we're not using integers or longs because we have
                                                                   'to mess around with the signed bit in them (what makes it a -n)
        DXCopyMemory pxArr(0), pData.pBits, pData.Pitch * 128 'where 128 is the height of the surface
            '//At this point in time, pxArr() holds a copy of all the texture's pixel data
        DXCopyMemory ByVal pData.pBits, pxArr(0), pData.Pitch * 128 'thanks to MetalWarrior for helping sort a bug with this!
    TexDMA.UnlockRect 0

Locking - this allows your program to gain access to a portion of the textures memory; due to DirectX8's complex memory management functions it's a little difficult to know where the texture's memory will be, when you lock the resource (it's not only textures that you can lock) Direct3D will stick it in a place where it's easy to get at (usually system memory) and then tell us where we can go find it. Direct3D returns, as the pBits member, a pointer to the first bit of texture memory, we therefore need to read a certain amount of data from this point onwards; Visual Basic has no native support for pointers, so we have to copy the memory at the location pointed to by the pointer to a more permanent place that we can access; this is what the DXCopyMemory( ) call does for us (it's a wrapper for the CopyMemory API call). We then use the same function to copy the data back again when we're finished - whatever we do to the array in between these calls will be mirrored on the next frame update. Finally we unlock the texture and let Direct3D go on about it's business...

A few important things to note:
1. Invalid Data - we're using CopyMemory here, no formatting is applied - this can be a good thing, but it can also lead to bad things! Namely the fabled "Blue Screen Of Death" - on several occasions here I've managed to lose count of quite how many blue screens I provoked...
2. Speed - the actual locking and copying is pretty fast, thats of no great problem (usually); BUT as we'll see later on, to process each pixel it can take some quite complicated maths and logic - multiply the time this takes by the number of pixels in the texture (16,384 in a small 128x128 texture) and suddenly you have a lot of processing to do...
3. Clever things - dont try them during the lock. Locking messes around with intenal windows functions, effectively stopping programs accessing the memory in some cases, as well as suspending other applications; the point being that you shouldn't try doing anything major with the Win32 API or DirectX during the lock.

4. Gaining access to the back-buffer and front-buffer

This is just a quick extension of the previous section; but I thought I may as well include it as there are some situations where accessing the flipping chain directly is required. This sort of thing will be familiar to the seasoned DirectDraw7 programmer, but for those new to all this stuff the back-buffer is where the scene is rendered and composed, and the front-buffer represents the screen (and holds the final image). The present (flip in DD7) function called at the end of every frame swaps the addresses for the front and back buffer.

Dim FrontBuffer As Direct3DSurface8
Dim BackBuffer As Direct3DSurface8

D3DDevice.GetFrontBuffer FrontBuffer
Set BackBuffer = D3DDevice.GetBackBuffer(0, D3DBACKBUFFER_TYPE_MONO)

'you can now lock as per normal:
FrontBuffer.LockRect pData, rct, 0
BackBuffer.LockRect pData, rct, 0
'etc...

5. Doing something with the memory

okay, now things start to heat up. conveniently this is also where those not upto scratch with binary manipulation will keel over dead.

I'm going to show you how to manipulate two pixel formats, one extremely easy, one moderately difficult. Having said that though, as soon as you've done one worked example this all becomes extremely simple and you'll easily be able to apply it to all 30 something pixel formats that Direct3D8 supports. First off then, whats a pixel format? 8 bit, 16 bit, 24 bit and 32 bit are all basic pixel formats, they tell you that (in order) there are 8, 16, 24 or 32 bits of memory allocated to store the colour of every pixel. 8 bit is rarely used anymore with almost complete support for at least 16 bit rendering now. 24 and 32 bit modes are the easiest possible formats, 16 bit is a bit of a pain in the back side. You will so far of seen the enumeration flags "D3DFMT_X8R8G8B8" and "D3DFMT_R5G6B5" in these tutorials - these are what describe the pixel formats, normally you only specify them in the CreateTextureFromFileEx( ) calls. The former is a 32 bit mode, 8 bits are unused, 8 bits are red, 8 bits are green and 8 bits are blue. The latter is a 16 bit mode, it tells us that there are 5 bit for red, 6 bits for green and 5 bits for blue (There is more blue because our eyes are more sensitive to the green spectrum).

As you've seen already (when locking the surface) we copy all the data to an array of bytes. each byte is made up of 8 bits. We could copy them straight to 32bit or 16bit integers, but this makes things very difficult later on because of the signed bit (the part that makes the number + or - ), so we're going to stay away from them. First off, for those sharp people you'll have noticed that 8 bits = 1 byte, data stored in 1 byte increments, D3DFMT_X8R8G8B8 indicates that each colour component is stored in a byte. Well done, you've just worked out how to decode the 24 and 32 bit colour modes (24 bit is 32 bit but without the alpha, A, or unused, X, channel).

TexDMA2.LockRect 0, pData, ByVal 0, 0
    'we can now play around with the stuff in pData
    ReDim pxArr(pData.Pitch * 128) As Byte 'enough bytes
    If Not (DXCopyMemory(pxArr(0), ByVal pData.pBits, pData.Pitch * 128) = D3D_OK) Then
        'handle errors here if unable to copy data.
    End If
    
    'Should be XRGB format, instead, in BGRX format...
    For x = 0 To (pData.Pitch * 128) - 1 Step 4
        'unused = pxArr(x + 3)
        bRed = pxArr(x + 2)
        bGreen = pxArr(x + 1)
        bBlue = pxArr(x + 0)
            bRed = 0
            bGreen = 0
            bBlue = 255
        pxArr(x + 2) = bRed
        pxArr(x + 1) = bGreen
        pxArr(x + 0) = bBlue
    Next x
    If Not (DXCopyMemory(ByVal pData.pBits, pxArr(0), pData.Pitch * 128) = D3D_OK) Then
        'handle error for bad copy here...
    End If
TexDMA2.UnlockRect 0

The previous piece of code was for the 32 bit mode, to read out the colours all we need to do is read the bytes int he correct order. The original array is a single dimension, stored as BGRXBGRXBGRXBGRXBGRX so we need the "Step 4" in the main loop, we would also need to use a conversion formula should we want to get the X/Y coordinates for the current pixel we're dealing with. Also note that it's stored backwards! instead of RGBX format we find that it's actually stored as BGRX format. The reasons are a little complicated, and not really that important here - but look into big/little endian formats if you're interested. The above piece of code will make the whole texture perfect-blue, simply because we're changing the bRed, bGreen and bBlue values to (0,0,255).

A quick note on pitch, you'll have seen the pData.Pitch member being used in several parameters so far; this value represents the real width of the texture - in memory. it is extremely important. As we've just seen, it takes 4 bytes to store the 32 bit colour value; which means that we must have 4 bytes representing each pixel in memory. Which therefore means that the width of the texture may be 128x128 pixels, but in memory it's going to be more like 512x128 bytes - each row requires 4x the number of pixels in bytes. whilst the streamlines nature of DirectX8 allows you to be fairly confident that you can assume it'll be 512 bytes per row it's a good idea to check first.

Right, I'm bored with 32 bit mode - it's too easy <grin>

Whilst 32 bit modes are already the prefered mode they are by no means the most common modes - all of the new cards tend to have full 32 bit support, but many of the popular 3D cards of last year (the Voodoo3 in particular) only support 16 bit mode. This means that you'll need to support both modes for locking/writing. It's 16 bit mode that makes things fun.

I'm going to show you how to manipulate a 16 bit 565 RGB value - extract the 16 bit value, read out the values, change them, put them back into the 16 bit value. I'll admit now that this one had me stumped for a week or so - the theory that I scribbled worked perfectly on paper, and there was no logical reason why it shouldn't of worked in code, but it refused. Then I sat down and ran a couple of very simple tests, check the results, and *!!Click!!* I realised that I'd been combining the bytes in the wrong order, and suddenly my perfect piece of theory worked a treat. Expect many similiar situations...

[RRRRRGGG] [GGGBBBBB] (where [] signifies a byte)

above is what our 16 bit value will look like in memory, and in our array it will be split into 2 bytes (the square brackets). Straight away you can see the problem - the red and green are mixed together, and so is the green and blue - in fact, the green is in both bytes! On closer inspection you'll see that you cant actually read out the values directly anyway (even if they aren't mixed) - RRRRR000 is very different from 000RRRRR (248 for the former, and 31 for the latter). The process for extracting the colour channels looks like this:

1. Take the two bytes, combine them into one straight 16 bit line
2. Mask out the green and blue channels, shift right 11 bits = Red
3. Mask out the red and blue channels, shift right 5 bits = Green
4. Mask out the red and green channels = Blue
- Manipulate colours here -
5. Shift the Red left by 11 bits, shift the green across by 5 bits
6. Combine the Red, Green and Blue values into a 16 bit long
7. Mask out the lowest 8 bits, shift right 8 bits = second byte
8. Mask out the highest 8 bits = first byte.

If you understand all of that then you're on a roll - read on. For those of you that look at it as though it was chinese (assuming chinese isn't your first language!), here's a quick guide.
Bit Shifting, this either goes left or right, take the binary value 0011100, shift it left by 1 = 0111000 and shift it right by 1 = 0001110, this is the same as multiplying by 2^n (n bits) for going left, and dividing by 2^n (n bits) to go right. Be careful when shifting left that you dont multiply the value out of range (easy to do!).
Masking, you use a given value and using the AND logical operator you can remove certain parts of the chain, making it easy to extract only the part that you want.

In More detail.
Step 1: Combining
Take the array of locked data, and two bytes, bFirst and bSecond; and a Long to store the result, lRes. Why a long? and not an integer. Whilst an integer is a perfect fit (16 bit) we'll have to mess with the signed bit - which I like to avoid wherever possible. a 32 bit long gives us a 15 bit cushion between the data we're interested in and the signed bit.

bFirst = pxArr(x)
bSecond = pxArr(x + 1)
lRes = (bSecond * 2 ^ 8) Or bFirst

The above piece of code takes the two bytes out of the array and logically combines them using the OR operator. If you're not sure of how the OR operator works take the following truth table:

Basically, if either of the two bits are true (1) then the output will be true (1). We can therefore combine the two values:

1111111100000000 = bSecond shifted left 8 bits
0000000011111111 = bFirst
---------------------
1111111111111111 = lRes

Step 2: Extracting the Red component
In the 16 bit value, 1111100000000000 are the bits that the red channel occupies. We can use AND to extract these values. Take the following truth table for the AND operator:

AND only outputs a true value (1) if both of it's inputs are 1 (makes sense really). Therefore if we take the decimal value for 1111100000000000, 63488, and logically AND it with the complete 16 bit chain, the output will be only the red bits, but shifted left 11 bits (so we undo this later by shifting right by 11 bits).

1011011101100111 = 16 bit chain
1111100000000000 = Red mask, 63488
---------------------
1011000000000000 = output, red shifted left by 11 bits
0000000000010110 = Red correctly shifted right by 11 bits.

bRed = (lRes And 63488) / 2 ^ 11
bRed = (255 / 31) * bRed 'to convert to the familiar 0-255 range.

The above piece of code is what the sample uses to extract the Red component, notice that it also converts it to the familiar 0-255 range; if you've done any work with graphics in a paint package you'll probably have use these values. It's not an important step (as long as you remember to do the opposite later on).

Step 3: Extracting the Green component
This is almost identical to extracting the red component, but we use a different mask.
The green bits occupy this section: 0000011111100000, which is 2016 in decimal, which is our mask.

To extract the green component we AND the 16 bit value with 2016, and then shift it right by 5 bits to result in the correct value:

bGreen = (lRes And 2016) / 2 ^ 5
bGreen = (255 / 63) * bGreen 'to convert it to 0-255 range

Again, we convert it to the 0-255 range.

Step 4: Extracting the Blue component
Now, guess what we do here - almost exactly what we've done in the last 2 steps; with the only exception that we dont need to do any bit shifting (The blue bits are already in the correct place). Blue occupies 0000000000011111 in the 16 bit chain, which is 31 in decimal.

That wasn't too nasty was it. in theory all the bit shifting, masking and so may look horrible - but you can get your head around it pretty quickly.

Step 5 & 6: Preparing the bytes again.
We're now at the stage that we've read the colour channels, messed around with them, and we now want to stick them back into the texture/surface. The first step is to convert the channels back into valid numbers.

Note that we converted from 5 bit accuracy to 8 bit accuracy (0-255 range), so we now need to convert from 8 bit back to 5 bit. This will result in a loss of precision - see the note at the end about accuracies.

'//Convert RED
    bRed = Int((31 / 255) * bRed) 'convert it back to the 0-31 scale
    lRed = bRed * 2 ^ 11
    
'//Convert GREEN
    bGreen = Int((63 / 255) * bGreen) 'convert it back to the 0-63 scale
    lGreen = bGreen * 2 ^ 5
    
'//Convert BLUE
    bBlue = Int((31 / 255) * bBlue) 'convert back to the 0-31 scale
    lBlue = bBlue
    
'//Assemble Complete Long
    lRes = lRed Or lGreen Or lBlue

Above is the complete code to combine the bRed, bGreen and bBlue values back to the lRes value. First we convert from 8 bit to 4 bit (or 6 bit for green) and we then use the OR logical operator to put them into one long 16 bit chain.

Step 7 & 8: Splitting the 16 bit value
At this point in time we've gotten back to the 16 bit long, but to store it in our array we'll need it to be in two bytes.

To extract the second byte we mask out the highest 8 bits, then shift it right by 8 bits.
To extract the first byte we just need to mask out the lowest 8 bits.

As shown in the above example; the final two lines putting the bytes back into their correct places in the array...

Finally, some notes that you may find useful:
1. Hex values; in almost every case so far it would have been easier to use hexidecimal notation, such as &HFF&... I didn't want to use them because VB hasa tendency to change them around and muck them up; it seemed simpler just to keep them as decimals. You can store them in constants if you prefer.
2. Accuracies - an 8 bit colour has 256 possible colours (2^8 = 256); whereas 5 bit only has 32 shades and 6 bit has 64 shades. In the 16 bit mode I just demonstrated, there are 32 shades or red, 64 shades of green and 32 shades of blue. This isn't too much of a problem (beyond it not looking so pretty), but when we convert it to 8 bit and/or back again we'll lose some accuracy. 5 bit to 8 bit implies that every 8 values in 8 bit are represented by one of the 32 shades; and similiar with 6 bit (each shade represents 4 colours). This is of particular importance when you go from 8 to 5 (or 6) bit accuracy; you'll need a change of 4 or 8 for it to be reflected in the 16 bit version; anything less will not show up. The method of conversion used above is probably extremely primative compared with what the photoshops and paint-shop-pro's use (it's unlikely they're algorithms are suitable for games), but you may want to look into a slightly better conversion.
3. Other Formats - there are at least 8 texture formats that you're likely to use; ranging from alpha to no alpha, 16 bit to 32 bit... whilst you can be fairly confident how it works, 3 simple tests can be applied. 3 textures, one full red, one full green, one full blue. Load them all in and log the lRes value (the complete 16 bit chain) and scribble it down on paper in binary - if the pattern is a little strange 1110000000000111 instead of 11111100000 then you know you've got the two bytes the wrong way around. Simple things like that will make it all easy...
4. Using the calculator - the little program "Calc.exe" built into windows can sort out all your binary-decimal conversion if you're too lazy (or cant) to do it by hand; stick it in scientific mode and type in a decimal number, then change it to binary mode - and out comes the binary equivelent of your number; and vice-versa.

Well, another massive tutorial completed. I hope it's been of use to you - I've seen many 100's of posts on message boards about this sort of thing, it seems to confuse quite a lot of people! including me at times... On to Lesson 15: Billboarding for special effects.