The TBN Matrix

Published by damix on 1/1/2022 at 4:00:00 AM

Let p 1 , p 2 and p 3 be the vertices of a triangle in a 3D space S , with texture coordinates ( u 1 , v 1 ) , ( u 2 , v 2 ) and ( u 3 , v 3 ) respectively. Vectors p 1 p 2 and p 1 p 3 lie in a plane P . Let ( t , b ) be a base of P . then, there must exist α , β , γ , δ such that:

p 1 p 2 = α t + β b

p 1 p 3 = γ t + δ b

Let's consider the application T which associates a vector p q P with the variation of the ( u , v ) texture coordinates experienced when going from the tail p to the head q :

T : P R 2 : p q ( u q u p , v q v p )

It can be proven that such application is linear. Applying T to both sides of the equations for p 1 p 2 and p 1 p 3 holds:

[ u 2 u 1 v 2 v 1 ] = α T ( t ) + β T ( b )

[ u 3 u 1 v 3 v 1 ] = γ T ( t ) + δ T ( b )

Now we impose the T ( t ) = ( 1 , 0 ) and T ( b ) = ( 0 , 1 ) ; that is, we want t to be the vector along which the u texture coordinate increments by 1 , and b to be the vector along which the v texture coordinate increments by 1 .

[ u 2 u 1 v 2 v 1 ] = α [ 1 0 ] + β [ 0 1 ]

[ u 3 u 1 v 3 v 1 ] = γ [ 1 0 ] + δ [ 0 1 ]

Which leads to:

α = u 2 u 1                 β = v 2 v 1                 γ = u 3 u 1                 δ = v 3 v 1

Substituting in our initial equations for p 1 p 2 and p 1 p 3 holds:

p 1 p 2 = ( u 2 u 1 ) t + ( v 2 v 1 ) b

p 1 p 3 = ( u 3 u 1 ) t + ( v 3 v 1 ) b

Let's adopt coordinates for all 3D vectors of S :

[ x 2 x 1 y 2 y 1 z 2 z 1 ] = ( u 2 u 1 ) [ x t y t z t ] + ( v 2 v 1 ) [ x b y b z b ]

[ x 3 x 1 y 3 y 1 z 3 z 1 ] = ( u 3 u 1 ) [ x t y t z t ] + ( v 3 v 1 ) [ x b y b z b ]

Let's rewrite the expression using a single matrix equation.

[ x 2 x 1 x 3 x 1 y 2 y 1 y 3 y 1 z 2 z 1 z 3 z 1 ] = [ x t x b y t y b z t z b ]   [ u 2 u 1 u 3 u 1 v 2 v 1 v 3 v 1 ]

Now we can solve for x t , y t , z t and x b , y b , z b :

[ x 2 x 1 x 3 x 1 y 2 y 1 y 3 y 1 z 2 z 1 z 3 z 1 ]   [ u 2 u 1 u 3 u 1 v 2 v 1 v 3 v 1 ] 1 = [ x t x b y t y b z t z b ]

Remembering the formula for the inverse of a 2 x 2 matrix we can write:

[ x 2 x 1 x 3 x 1 y 2 y 1 y 3 y 1 z 2 z 1 z 3 z 1 ]   [ v 3 v 1 u 1 u 3 v 1 v 2 u 2 u 1 ] ( u 2 u 1 ) ( v 3 v 1 ) + ( u 1 u 3 ) ( v 1 v 2 ) = [ x t x b y t y b z t z b ]

We can now write explicit equations for all of the unknown variables:

D = ( u 2 u 1 ) ( v 3 v 1 ) ( u 1 u 3 ) ( v 1 v 2 )   x t = ( x 2 x 1 ) ( v 3 v 1 ) + ( x 3 x 1 ) ( v 1 v 2 ) D   y t = ( y 2 y 1 ) ( v 3 v 1 ) + ( y 3 y 1 ) ( v 1 v 2 ) D   z t = ( z 2 z 1 ) ( v 3 v 1 ) + ( z 3 z 1 ) ( v 1 v 2 ) D   x b = ( x 2 x 1 ) ( u 1 u 3 ) + ( x 3 x 1 ) ( u 2 u 1 ) D   y b = ( y 2 y 1 ) ( u 1 u 3 ) + ( y 3 y 1 ) ( u 2 u 1 ) D   z b = ( z 2 z 1 ) ( u 1 u 3 ) + ( z 3 z 1 ) ( u 2 u 1 ) D

The vectors t and b are called tangent and binormal; their coodinates, together with the coordinates ( x n , y n , z n ) of the ordinary normal n , can be arranged in the TBN matrix.

T B N := [ x t x b x n y t y b y n z t z b z n ]

The normal n and its coordinates x n , y n , z n are computed as the cross product of t and b .

x n = y t z b z t y b y n = z t x b x t z b z n = x t y b y t x b

In code, the tangent and the binormal are typically computed on the CPU and stored as vertex attributes in model space. Here is a code snippet for ES6/TypeScript. It is fairly easy to port to your favourite language (or the one your boss wants you to work with).

const D = (u2 - u1) * (v3 - v1) - (u1 - u3) * (v1 - v2); const xt = ((x2 - x1) * (v3 - v1) + (x3 - x1) * (v1 - v2)) / D; const yt = ((y2 - y1) * (v3 - v1) + (y3 - y1) * (v1 - v2)) / D; const zt = ((z2 - z1) * (v3 - v1) + (z3 - z1) * (v1 - v2)) / D; const xb = ((x2 - x1) * (u1 - u3) + (x3 - x1) * (u2 - u1)) / D; const yb = ((y2 - y1) * (u1 - u3) + (y3 - y1) * (u2 - u1)) / D; const zb = ((z2 - z1) * (u1 - u3) + (z3 - z1) * (u2 - u1)) / D;

In the vertex shader they can be transformed to world or view space as needed. The normal can be obtained as the cross product between the tangent and the binormal. Here is a GLSL vertex shader snippet.

v_tangent = (mat3(u_view * u_model) * a_tangent).xyz; v_binormal = (mat3(u_view * u_model) * a_binormal).xyz; v_normal = cross(v_tangent, v_binormal);

The vertex shader outputs these vectors and the fragment shader receives the interpolated versions.

The availability of the tangent, binormal and normal vectors in the fragment shader is the foundation of many shading techniques that operate in tangent space; the TBN matrix represents a change of basis from the tangent space to whatever coordinate system you want to work with; typically +world_ space or view space.

Normal mapping

The most famous and elementary technique is probably normal mapping. In this technique a texture called normal map is used to encode the direction of the surface normal; this enables modeling the interaction of light with the mesostructure or an object, i.e. the little bumps and corrugations that for performance reasons we don't want to represent using vertices and triangles.

Normal mapping

The TBN matrix is constructed from three columns that are the coordinates of the tangent, binormal and normal vectors. These vectors are output by the vertex shader. The fragment shader then reads the normal map and apply a transformation to bring the value of the read texel in the [-1, +1] range. This is the surface normal of the mesostructure expressed in tangent space coordinates. The TBN matrix is used to change these coordinates to the same coordinate system in which we express light positions, usually world or view.

Here is a GLSL fragment shader snippet targeting WebGL2/OpenGL ES 3.

mat3 TBN = mat3(v_tangent, v_binormal, 0.5 * normalize(v_normal)); vec3 sampledNormal = texture(u_normal, v_texcoord).rgb * 2.0 - 1.0; vec3 normal = normalize(TBN * sampledNormal);

You will typically want to normalize and rescale the normal, to make the mesostructure more or less pronounced. In the snippet above we set the length of the normal to 0.5, which results in relatively mild bumps. A value of 0.1 would result in much more accentuated bumps, while a value of 10.0 would make the surface look perfecly smooth, as in plain texture mapping.