Parallax Corrected Cubemaps

Like normal cubemaps but better

0005

2024/11/13 02:16

EN-US | PT-BR

1 Introduction

This article was inspired by Image-based Lighting approaches and parallax-corrected cubemap

This article assumes that you already know how to load and bake cubemaps.

2 Classic Cubemap Reflection

So, how is the default way of doing cubemap reflections? well, it's quite simple: calculate the view direction from the camera position to the fragment position, reflect it using the reflect function with the fragment normal and done!

  1. vec3 viewDirection = normalize(worldPosition - viewPosition);
  2. vec3 reflectedDirection = reflect(viewDirection, normal);
  3. vec3 reflection = texture(reflectionCubemap, reflectedDirection).rgb;

As you can see, the reflection seems to be moving with us, like if it was very far away like the sky, which would be fine if this was a sky cubemap but this is a local cubemap baked from a specific location.

Tip: When baking cubemaps disable all specular reflections from lights and other cubemaps, because those depend on the view direction and it could look very weird when looking from other angles, cubemaps should generally contain only diffuse light.

If we draw this into paint.net, this is what we get:

Cubemap Reflection

Because we are sampling the cubemap using only our reflected direction "R" without taking the cubemap position into account (denoted by the ball), the cubemap ends up sampling the wrong position denoted by "P?" instead of the correct position denoted by "P" (which we don't have) and this is why we mostly see the ceiling instead of the walls in the video.

3 Parallax Corrected Cubemap

A Parallax Corrected Cubemap solves this issue by finding the correct P using raytracing and then sampling the cubemap using the direction from the cubemap position to P (the C direction).

Parallax Corrected Cubemap

4 Raytracing

Don't let the word "raytracing" scare you, even if we could generate a BVH from our scene and use it to find P, it would be too slow and complicated to use, instead we are going to simplify our scene to a user defined box and then intersect a ray from inside of it.

4.1 In Code

We only need a small function for intersecting a ray with a box from the inside.

Using the code in the article mentioned at the introduction, I made a more generic version of it and I also added a aabb check to avoid the ray from being intersected from outside the box because I noticed it causes a lot of graphical issues in some scenarios.

  1. //P is rayOrigin + (rayDirection * t)
  2. //where t is the return value
  3. //returns -1.0 if the ray is outside the box
  4. float intersectRayInsideBox(vec3 rayOrigin, vec3 rayDirection, mat4 worldToLocal) {
  5. vec3 localOrigin = (worldToLocal * vec4(rayOrigin, 1.0)).xyz;
  6. vec3 aabbCheck = abs(localOrigin);
  7. if (max(aabbCheck.x, max(aabbCheck.y, aabbCheck.z)) > 1.0) {
  8. return -1.0;
  9. }
  10. vec3 localDirection = mat3(worldToLocal) * rayDirection;
  11. vec3 firstPlane = (vec3(-1.0) - localOrigin) / localDirection;
  12. vec3 secondPlane = (vec3(1.0) - localOrigin) / localDirection;
  13. vec3 furthestPlane = max(firstPlane, secondPlane);
  14. return min(furthestPlane.x, min(furthestPlane.y, furthestPlane.z));
  15. }

Even if the function was made for a aabb, it actually supports any rotated or scaled box, because we apply the inverse of the box matrix to the ray (the worldToLocal matrix) instead of the box which is always a 2x2x2 AABB in the perspective of the function.

On the CPU side, we only need to calculate the worldToLocal matrix and send it to the shader alongside the cubemap position.

  1. Vector3f position = new Vector3f(0f, 2f, 0f);
  2. Quaternionf rotation = new Quaternionf(0f, 0f, 0f, 1f);
  3. Vector3f halfExtents = new Vector3f(5f, 2f, 5f);
  5. Matrix4f worldToLocal = new Matrix4f()
  6. .translate(position)
  7. .rotate(rotation)
  8. .scale(halfExtents)
  9. .invert()
  10. ;
  12. Vector3f cubemapPosition = new Vector3f(0f, 1.65f, 0f);

"position" is the position of the box, "rotation" is the rotation of the box defined by a quaternion, "halfExtents" is the half of the extensions of the box (a box of width 10.0 has a half extent of 5.0 in the X axis) and "cubemapPosition" is the cubemap position where it was baked at.

  1. uniform mat4 cubemapWorldToLocal;
  2. uniform vec3 cubemapPosition;

Because the intersection only works if the fragment is inside the box, you should always make the box slightly larger than the area it represents.

And then, we only need to intersect a ray from the fragment position using the reflected direction to find P and sample the cubemap using the corrected direction.

  1. vec3 viewDirection = normalize(worldPosition - viewPosition);
  2. vec3 reflectedDirection = reflect(viewDirection, normal);
  3. vec3 reflection = vec3(0.0);
  4. float t = intersectRayInsideBox(worldPosition, reflectedDirection, cubemapWorldToLocal);
  5. if (t >= 0.0) {
  6. vec3 P = worldPosition + (reflectedDirection * t);
  7. vec3 correctedDirection = normalize(P - cubemapPosition);
  8. reflection = texture(reflectionCubemap, correctedDirection).rgb;
  9. }

5 Final Result

And this is the final result: