Post Processing Effects in OpenGL
18 Feb 2024 · 10 min read
Post processing effects are effects which are applied after the rendering of the scene. The way we achieve those by taking the last image from our frame buffer and manipulate pixels using custom fragment shaders.
Every canvas below is a live demo. The scene is a triangle (the same one we are going to draw in OpenGL) plus a thin white
ring, so the kernel effects have some edges to work on. Hold the mouse on a canvas to see the original frame.
The kernel effects at the end use a sample picture instead, because a triangle doesn't have enough detail to show them (I drew it
with the canvas API, there is no image file). The demos run on WebGL2 through three.js, and the shader inside each one is the exact code you see above it.
Only the #version line is different: three.js adds #version 300 es and a precision line for us.
Initialization
Frame Buffer Creation
In order to use these effects in OpenGL we need a frame buffer to render the scene to an image.
We are not going to read any data back from our frame buffer, so a render buffer is enough for depth and stencil. We also need a texture to write the final image to.
uint32_t fbo, rbo, texture;
glGenFramebuffers(1, &fbo);
glGenRenderbuffers(1, &rbo);
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, SCREEN_WIDTH, SCREEN_HEIGHT, 0, GL_RGB, GL_UNSIGNED_BYTE, nullptr);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0);
glBindRenderbuffer(GL_RENDERBUFFER, rbo);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, SCREEN_WIDTH, SCREEN_HEIGHT);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, rbo);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
If you want to learn more about these functions you can use docs.gl, a website which contains signature and description of OpenGL functions.
The variables SCREEN_WIDTH and SCREEN_HEIGHT is not supposed to be screen size, there could also be multiple
frame buffers that you may want to use different part of your screen, or you may have a viewport window for the
editor of your application it could be the size of it.
And that's all for creating the frame buffer. Now you can render your content by just moving your rendering code between the binding of the frame buffer.
For the context we are going to draw a simple triangle, with its vertex data hardcoded in the vertex shader
(positions[gl_VertexID]) because we don't need a vertex buffer for this tutorial.
while (is_running) {
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
// we are not doing anything with depth but its good practice
// because we will disable it later when we draw our screen quad.
glEnable(GL_DEPTH_TEST);
glClearColor(0.0, 0.0, 0.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glBindVertexArray(triangle_vao);
glUseProgram(triangle_program);
glDrawArrays(GL_TRIANGLES, 0, 3);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
// second pass, see below
}
Try to run with and without the frame buffer binding. With it you should see nothing, because the triangle went into our texture instead of the screen. Now that we have our triangle rendered to a texture we can manipulate it to look however we want. For that we need a screen quad to present the texture and a shader to manipulate it.
Screen Quad Resource Creation
The vertex shader of the quad is two triangles that cover the whole screen, and it passes the texture coordinates through.
const char* screen_vertex_source = R"(
#version 450 core
layout(location = 0) out vec2 v_tex_coord;
vec2 VERTICES[6] = vec2[](
vec2(-1.0, -1.0), vec2(-1.0, 1.0), vec2(1.0, 1.0),
vec2(1.0, 1.0), vec2(1.0, -1.0), vec2(-1.0, -1.0));
vec2 TEX_COORDS[6] = vec2[](
vec2(0.0, 0.0), vec2(0.0, 1.0), vec2(1.0, 1.0),
vec2(1.0, 1.0), vec2(1.0, 0.0), vec2(0.0, 0.0));
void main() {
v_tex_coord = TEX_COORDS[gl_VertexID];
gl_Position = vec4(VERTICES[gl_VertexID], 0.0, 1.0);
})";
The fragment shader is where all the effects will live. For now it just reads the texture, and this is our baseline.
Compile it with the vertex shader above, set u_screen_texture to texture unit 0 once, and draw the quad.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
o_color = texture(u_screen_texture, v_tex_coord);
}
// second pass, inside the loop
glDisable(GL_DEPTH_TEST);
glClearColor(0.0, 0.0, 0.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT);
glBindVertexArray(screen_vao);
glUseProgram(screen_program);
glBindTexture(GL_TEXTURE_2D, texture);
glDrawArrays(GL_TRIANGLES, 0, 6);
You should see the same triangle as before, but now it went through a texture, which means we can do anything to it.
The same thing in three.js
The demos on this page are this exact pipeline. A WebGLRenderTarget is the frame buffer (its samples also gives us
MSAA, which glTexImage2D above does not), and the screen quad is a PlaneGeometry(2, 2) with our fragment shader on it.
renderer.setRenderTarget(target); // glBindFramebuffer(fbo)
renderer.render(scene, camera); // triangle
renderer.setRenderTarget(null); // glBindFramebuffer(0)
renderer.render(quad, camera); // screen quad, material.uniforms.u_screen_texture = target.texture
Simple Effects
Gray Scale
Gray scale is the simplest one. Everything below goes to the fragment shader of the screen quad, we only change main.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
o_color = texture(u_screen_texture, v_tex_coord);
float average = (o_color.r + o_color.g + o_color.b) / 3.0;
o_color = vec4(average, average, average, o_color.a);
}
Here we are finding the average of every component of the pixel (rgb) and then using that value as our final color. This will give us a basic gray scale image. Look at the triangle, red, green and blue all end up equally bright, and you can't tell where one ends and the other starts.
But this is not a very sophisticated way to achieve this, we can play with the average value a little bit more. The human eye is more sensitive to green light, so a simple average like this might not accurately represent perceived brightness.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
o_color = texture(u_screen_texture, v_tex_coord);
float average = 0.2126 * o_color.r + 0.7152 * o_color.g + 0.0722 * o_color.b;
o_color = vec4(average, average, average, o_color.a);
}
The values we are multiplying are called weights and they represent which channel will contribute the most, and surely it would be the one with the bigger weight. Now the green corner is clearly the brightest and the blue one is almost black.
Chromatic Aberration
Chromatic aberration is another frequently used post-processing effect. Essentially, it involves shifting the color channels away from each other to create a desired visual effect.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
uniform vec3 u_offset;
void main() {
o_color = texture(u_screen_texture, v_tex_coord);
o_color.r = texture(u_screen_texture, v_tex_coord + u_offset.r).r;
o_color.g = texture(u_screen_texture, v_tex_coord + u_offset.g).g;
o_color.b = texture(u_screen_texture, v_tex_coord + u_offset.b).b;
}
This is a very basic implementation of chromatic aberration and it does not have any correction applied
on top of it. The u_offset uniform here controls how much each channel will be offset. r = 0.009, g = 0.006 and
b = -0.006 is what you see above, and the white ring is the best place to look at it: it splits into colored fringes.
You can set the offsets after the binding of the screen shader like the following.
glUseProgram(screen_program);
glUniform3f(glGetUniformLocation(screen_program, "u_offset"), 0.009f, 0.006f, -0.006f);
Kernel Effects
Kernel effects are effects which affect multiple pixels around by weights. Usually we define those kernels as 3x3 matrices.
A basic kernel effect can be achieved like below, the only thing that changes between effects is the kernel array.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
vec2 offset = 1.0 / vec2(textureSize(u_screen_texture, 0));
vec2 offsets[9] = vec2[](
vec2(-offset.x, offset.y), vec2(0.0, offset.y), vec2(offset.x, offset.y),
vec2(-offset.x, 0.0), vec2(0.0, 0.0), vec2(offset.x, 0.0),
vec2(-offset.x, -offset.y), vec2(0.0, -offset.y), vec2(offset.x, -offset.y)
);
float kernel[9] = float[](
0.0, 0.0, 0.0,
0.0, 1.0, 0.0,
0.0, 0.0, 0.0
);
vec3 color = vec3(0.0);
for (int i = 0; i < 9; i++) {
color += vec3(texture(u_screen_texture, v_tex_coord + offsets[i])) * kernel[i];
}
o_color = vec4(color, 1.0);
}
This one is the identity kernel, only the center pixel counts, so nothing changes.
NOTE: the sum of the kernel matrix should sum up to 1. If it is less than one the image will be darker and if it is greater than 1 the image will be brighter and if it is zero only the edges will be shown.
Sharpen
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
vec2 offset = 1.0 / vec2(textureSize(u_screen_texture, 0));
vec2 offsets[9] = vec2[](
vec2(-offset.x, offset.y), vec2(0.0, offset.y), vec2(offset.x, offset.y),
vec2(-offset.x, 0.0), vec2(0.0, 0.0), vec2(offset.x, 0.0),
vec2(-offset.x, -offset.y), vec2(0.0, -offset.y), vec2(offset.x, -offset.y)
);
float kernel[9] = float[](
-1.0, -1.0, -1.0,
-1.0, 9.0, -1.0,
-1.0, -1.0, -1.0
);
vec3 color = vec3(0.0);
for (int i = 0; i < 9; i++) {
color += vec3(texture(u_screen_texture, v_tex_coord + offsets[i])) * kernel[i];
}
o_color = vec4(color, 1.0);
}
The kernel does the work here: every pixel is multiplied by 9 and its eight neighbors are subtracted, so a pixel that is brighter than its surroundings gets even brighter (-8 + 9 = 1, so flat areas stay the same). It was almost invisible with our triangle, so from here on the demos use a sample picture. Hold to compare: the red box, the green circle and the letters get a thin bright/dark outline, and the hairlines become more contrasty.
Blur
Blur might be one of the most popular effects used in game and film industry. There is a lot of ways to achieve blur and our way of doing this is the easiest one. Only the kernel changes, the rest of the shader is the same as above.
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
vec2 offset = 1.0 / vec2(textureSize(u_screen_texture, 0));
vec2 offsets[9] = vec2[](
vec2(-offset.x, offset.y), vec2(0.0, offset.y), vec2(offset.x, offset.y),
vec2(-offset.x, 0.0), vec2(0.0, 0.0), vec2(offset.x, 0.0),
vec2(-offset.x, -offset.y), vec2(0.0, -offset.y), vec2(offset.x, -offset.y)
);
float kernel[9] = float[](
1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0,
2.0 / 16.0, 4.0 / 16.0, 2.0 / 16.0,
1.0 / 16.0, 2.0 / 16.0, 1.0 / 16.0
);
vec3 color = vec3(0.0);
for (int i = 0; i < 9; i++) {
color += vec3(texture(u_screen_texture, v_tex_coord + offsets[i])) * kernel[i];
}
o_color = vec4(color, 1.0);
}
In this implementation, we're creating a 3x3 kernel. Since the sum of all the kernel values equals 16, we normalize each element of the matrix by dividing it by the total sum of the matrix. This normalization process ensures that each value represents the average contribution of its neighboring pixels to the final result, effectively distributing the blur across the image.
A 3x3 kernel only reaches one pixel away, so the blur is subtle. Look at the thin lines and the small text, they melt into the background, while the big shapes barely change. Running the same pass multiple times blurs more.
Bonus: Sobel
The note above said a kernel that sums to zero shows only the edges. Sobel is the classic example, and it is how most edge detection starts. It uses two kernels, one for horizontal changes and one for vertical changes:
The factorization shows what each one does: [-1 0 1] is the difference between the right and the left pixel (a derivative),
and [1 2 1] is a small blur across it, so noise doesn't count as an edge. Both kernels sum to 0, so a flat area gives 0 (black).
We run both on the brightness of the pixel and combine them into the edge strength:
in vec2 v_tex_coord;
out vec4 o_color;
uniform sampler2D u_screen_texture;
void main() {
vec2 offset = 1.0 / vec2(textureSize(u_screen_texture, 0));
vec2 offsets[9] = vec2[](
vec2(-offset.x, offset.y), vec2(0.0, offset.y), vec2(offset.x, offset.y),
vec2(-offset.x, 0.0), vec2(0.0, 0.0), vec2(offset.x, 0.0),
vec2(-offset.x, -offset.y), vec2(0.0, -offset.y), vec2(offset.x, -offset.y)
);
float gx[9] = float[](
-1.0, 0.0, 1.0,
-2.0, 0.0, 2.0,
-1.0, 0.0, 1.0
);
float gy[9] = float[](
1.0, 2.0, 1.0,
0.0, 0.0, 0.0,
-1.0, -2.0, -1.0
);
float x = 0.0;
float y = 0.0;
for (int i = 0; i < 9; i++) {
vec3 color = texture(u_screen_texture, v_tex_coord + offsets[i]).rgb;
float brightness = dot(color, vec3(0.2126, 0.7152, 0.0722));
x += brightness * gx[i];
y += brightness * gy[i];
}
o_color = vec4(vec3(length(vec2(x, y))), 1.0);
}
The brightness is the same weighted gray scale from the beginning. The sky is almost black because it changes slowly, and every hard edge, like the checkerboard, the red box and the letters, lights up. The soft sun gives a weak outline because its edge is spread over many pixels. Soft edges are what the 3x3 neighborhood sees as a small change.
You can achieve so much with this setup and of course you can use multiple effects at once. My recommendation is to check the 3d-game-shaders-for-beginners repo from lettier, it encapsulates many other post-processing effects and how they are implemented.