I had been staring at OpenGL’s tessellation shaders for hours, trying to make sense of patches, layouts, and all those mysterious gl_TessLevel variables. Here is a summary of it all.
What’s a “patch”?
A patch is the basic input to the tessellation pipeline, like a vertex for the vertex shader, or a fragment for the fragment shader.
It’s simply a group of vertices that defines a piece of a surface to be subdivided.
You tell OpenGL how many vertices belong to each patch using:
glPatchParameteri(GL_PATCH_VERTICES, 4);
That means every four vertices you draw become one patch.
glDrawArrays(GL_PATCHES, 0, 4);
One patch goes in, and the Tessellation Control Shader (TCS) runs once per vertex in that patch.
What happens in the Tessellation Control Shader?
The TCS receives that patch through the built-in array gl_in[].
Since we declared layout(vertices = 4) out;, there are four control points per patch.
Each invocation of the TCS is identified by gl_InvocationID, and writes its output to gl_out[gl_InvocationID].
In our simple setup:
gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
That’s it, we just forward the control points to the next stage.
It doesn’t look like much, but this is crucial: without it, the TES wouldn’t know where any of the control points are.
So yes, even though it seems like we’re “not using” the out array, we are.
We’re handing off those positions from TCS → TES.
What about the tessellation levels?
Inside the TCS, we also define how finely the tessellator should subdivide our patch:
gl_TessLevelOuter[0..3] = ...
gl_TessLevelInner[0..1] = ...
The number of outer and inner levels depends on the domain type:
triangles → outer = 3, inner = 1
quads → outer = 4, inner = 2
isolines → outer = 2, inner = 0
Outer levels control the edges.
Inner levels control the density inside the shape.
So for quads: four edges, two interior directions.
For triangles: three edges, one interior.
For isolines: just lines, no interior.
How does the Tessellation Evaluation Shader fit in?
This stage defines how to interpret those patches.
Here’s the line that matters:
layout(quads) in;
That tells the tessellator that our patch represents a quad domain, meaning it’ll use (u, v) coordinates between 0 and 1 to interpolate new vertices across the surface.
If I had written layout(triangles) in;, it would expect only 3 control points (a triangular patch).
That’s why your TCS vertex count and TES layout must always agree:
layout(vertices = 4) + layout(quads) → works
layout(vertices = 3) + layout(triangles) → works
If they don’t match, OpenGL just ignores the extras.
What are Tessellation Coordinates (gl_TessCoord)?
Every new vertex that the tessellator generates comes with a built-in variable called gl_TessCoord.
This tells you where that vertex lies inside the tessellated patch — kind of like UVs, but generated by the tessellator.
Depending on the domain type (layout(…) in;), it behaves differently:
| Domain | gl_TessCoord contents | Description |
|---|---|---|
| quads | vec3(x, y, z) where z = 0 | (x, y) in [0,1] across the quad |
| triangles | vec3(u, v, w) with u+v+w=1 | Barycentric coordinates inside the triangle |
| isolines | vec3(u, v, w) where u,v∈[0,1] | u = position along line, v = line index |
In your TES, you can access them like:
float u = gl_TessCoord.x;
float v = gl_TessCoord.y;
For quads, that means:
- u = 0 → left, u = 1 → right
- v = 0 → bottom, v = 1 → top
So if you interpolate between four control points (P0, P1, P2, P3):
vec3 P = mix(mix(P0, P1, u), mix(P3, P2, u), v);
you get the position of the tessellated vertex corresponding to that (u, v) point inside the patch.
gl_TessCoord is how the tessellator tells you “this is where I am on the surface.”
It’s what allows you to interpolate positions, compute normals, or drive displacement maps.
Without it, all those new tessellated vertices would have no context, you wouldn’t know where to place them between the original control points.
Why is the MVP multiplied in the Tessellation Evaluation Shader?
This one tripped me up for a while.
I initially wondered:
“If I apply the MVP in the vertex shader, won’t tessellation just interpolate between those projected positions?”
It will, but that’s wrong.
Projection (via the MVP) is not linear, and tessellation interpolates linearly.
If you tessellate in clip space, edges curve and distort.
The fix:
Keep everything in object or world space until tessellation is done.
Then apply the MVP in the TES, after new vertices are generated:
gl_Position = uMVPMatrix * vec4(P, 1.0);
That way, every new vertex gets properly transformed into clip space.
How does everything connect?
| Stage | Operates on | Coordinate space | What it does |
|---|---|---|---|
| Vertex Shader | 1 vertex | Object space | Passes raw vertex data |
| Tessellation Control | Patch (N vertices) | Object space | Sets tessellation levels |
| Tessellator | — | Object space | Subdivides patch |
| Tessellation Evaluation | Generated vertices | Object → Clip | Interpolates + applies MVP |
| Fragment Shader | Fragments | Screen space | Final color |
Quick Recap
• 4 vertices = 1 patch → glPatchParameteri(GL_PATCH_VERTICES, 4)
• TCS runs per patch → layout(vertices = 4) out;
• TES interprets patch as quad → layout(quads) in;
• Tessellation levels:
- triangles → outer=3, inner=1
- quads → outer=4, inner=2
- isolines → outer=2, inner=0
• Apply MVP in TES, not vertex shader.
• gl_out[] is the bridge — even if you only pass data through.