Terrain, Texture Splatting, and Three.js?

So there comes a time in every developers life when they're unable to use the tools they know and love to get the job done.  Sometimes the tools aren't a good fit for the task and other times there are political reasons.  Maybe you work for the man and the man says use tool "x".

That's a bummer but that's reality.  On the bright side, you'll have an opportunity to learn and gain experience you might not have had before.  So here's my story:

One day I was in this situation where I had to make a terrain demo in a web page WITHOUT using any plugins.  If you haven't already seen my video about this, check it out here.  This wouldn't be an issue with Unity 5 and above (due to export to asm.js) but at this time, version 4 was the latest.

So let's back up a moment.  What is texture splatting and why would we want to use it?

Simply put, texture splatting is a technique for combining different textures.  And not only that, it allows you to change how those textures are combined (blended) at every texel.  That means you don't have to blend those textures together the same way over the whole object, the way they are blended can vary.

For instance, check out the terrain below.  It is blending a combination of 4 different textures in different ways across the terrain.

What you're seeing is a combination of the following:

You can see that the mix of textures is applied in different ways across the terrain.  For instance, some areas have more snow, more dirt, more grass, or more rock.

What you're not seeing in those images is the instructions for how to combine them at every point.  You just see the result.  So what's the secret sauce here?

Check out this image:

Immediately you can see that the crazy colors in this image are associated with the terrain you saw above.  That's because each color channel in the image (RGBA) corresponds to how much a certain texture should be used.  You might get something like this:

Red channel: dirt 

Green channel: grass

Blue channel: cliff rock

Alpha channel: snow

Note that alpha is not a color but an extra spot usually reserved for transparency.  Each color channel is multiplied by the texture its associated with and added up to get the final result. This process is done using pixel shaders (executes in code that runs directly on the GPU) which makes it very fast to compute.  Here's some shader code that actually does exactly that:

vec4 pixelColor1 = texture2D(texture1, UV1);
vec4 pixelColor2 = texture2D(texture2, UV2);
vec4 pixelColor3 = texture2D(texture3, UV3);
vec4 pixelColor4 = texture2D(texture4, UV4);
vec4 alphaMap = texture2D(tAlphaMap, alphaUV);

gl_FragColor = pixelColor1 * alphaMap.r + pixelColor2 * alphaMap.g +
               pixelColor3 * alphaMap.b + pixelColor4 * alphaMap.a;

A quick explanation: texture2D is function that looks up a color from texture using a texture coordinate and  tAlphaMap refers to the splat texture shown above.  Each color channel (.r .g .b .a) is a number between 0 and 1 which makes multiplying color by it the same as taking a percentage of that color (ex .25 = 25%).  The idea is that your percentages will all add up to one so you get an appropriate contribution for each texture based on your values.  For example, check out a couple examples with just dirt and grass:

.5 * dirt + .5 * grass

.75 * dirt + .25 * grass

So great, texture splatting is a convenient and fast technique for texturing terrain.  It's also what Unity uses along with a bunch of nice brushes and tools for painting with them.  Again, unfortunately we can't use Unity... at least for rendering in the final application.

No problem, nothing is stopping us from authoring our splat texture ("alpha map") from Unity and using it elsewhere.  Well, nothing other than the fact that Unity doesn't provide a convenient way to get data out.  So let's see what's accessible to us via script.

Great, we can access the data.  Now how do we get it out?  Like this:

void SaveSplat() {

  // Use the selected texture if it exists or else bring up a dialog to choose
  string assetPath;
  if (alphaMap) {
    assetPath = AssetDatabase.GetAssetPath(alphaMap);
  } else  {
    assetPath = EditorUtility.SaveFilePanelInProject("Save texture",
      "mySplatAlphaMap.asset", "asset", "Please enter a file name to save the texture to.");
  }

  // If a valid location was chosen
  if (assetPath.Length != 0) {

    // Get the terrain and its data, my script that's being used here is TerrainTool
    Terrain terrain = (target as TerrainTool).transform.GetComponent;();
    TerrainData data = terrain.terrainData;

    float[,,] maps = data.GetAlphamaps(0, 0, data.alphamapWidth, data.alphamapHeight);
    int numSplats = maps.GetLength(2);

    Color32[] image = new Color32[data.alphamapWidth * data.alphamapHeight];

    for (int y = 0; y < data.alphamapHeight; ++y) {

      // Flip the image if desired such as when planning to export the texture to use elsewhere
      int vertical = (invertY) ? data.alphamapHeight - y - 1: y;

      for (int x = 0; x < data.alphamapWidth; ++x) {

        int imageIndex = y * data.alphamapWidth + x;

        // The colors are in the range from 0 to 1 but an image file is expected to be from 0 to 255

        image[imageIndex].r = (byte)(maps[vertical, x, 0] * 255.0f);
        image[imageIndex].g = (numSplats > 1) ? (byte)(maps[vertical, x, 1] * 255.0f) : (byte)0;
        image[imageIndex].b = (numSplats > 2) ? (byte)(maps[vertical, x, 2] * 255.0f) : (byte)0;
        image[imageIndex].a = (numSplats > 3) ? (byte)(maps[vertical, x, 3] * 255.0f) : (byte)0;
      }
    }

    // make a texture to store our image colors in

    Texture2D finalSplatTexture = new Texture2D(data.alphamapWidth, data.alphamapHeight);
    finalSplatTexture.SetPixels32(image);
    alphaMap = finalSplatTexture;

    // Save this out as a .asset
AssetDatabase.CreateAsset(finalSplatTexture, assetPath);

  }
}

So this goes through the whole alphamap and saves it as a .asset.  Unfortunately ".asset" is a Unity format which won't help us much so we need to take one more step to convert that into something we want.
And, frequently when we need something, someone else has already done it out there and has made their work publicly available.  For this step, we are in luck.  A quick search turns up this:

import System.IO;

@MenuItem("Assets/Export Texture")
  static function Apply () {

  var texture : Texture2D = Selection.activeObject as Texture2D;
  if (texture == null)
  {
    EditorUtility.DisplayDialog("Select Texture", "You Must Select a Texture first!", "Ok");
    return;
  }

  var bytes = texture.EncodeToPNG();\
  File.WriteAllBytes(Application.dataPath + "/exported_texture.png", bytes);

}

That adds a handy menu item that we can use once we've selected our splat .asset to generate a .png file.  Mission accomplished!

So now onto the next piece, putting it into a three.js web application.  Unfortunately, that's an entirely different blog post unto itself.  If you're interested in it feel free to leave a comment for me.  I read all of them.

Oh, and you can get a Unity package with everything we've talked about  HERE.

If you wanna see this in action, make sure you have a WebGL enabled browser and check out the result HERE

To see the source for that demo, hop on over to my github repo where it's waiting for you HERE.