8.1 光线追踪¶
Ray Tracing
光线追踪可能是最知名的高质量图形技术。它背后的理念并不复杂：要找出当你朝某个方向看时你看到了什么，考虑一束从那个方向到达你位置的光线，然后沿着这束光线向后追踪，看看它来自哪里。或者，正如通常所说的，从你的位置朝某个方向投射一束光线，看看它击中了什么。那就是你朝那个方向看时所看到的。确定光线击中什么的操作被称为光线投射。它是光线追踪以及其它高级图形技术的基础。
Ray tracing is probably the best known technique for higher quality graphics. The idea behind it is not complicated: To find out what you see when you look in a given direction, consider a ray of light that arrives at your location from that direction, and follow that light ray backwards to see where it came from. Or, as it is usually phrased, cast a ray from your location in a given direction, and see what it hits. That's what you see when you look in that direction. The operation of determining what is hit by a ray is called ray casting. It is fundamental to ray tracing and to other advanced graphics techniques.
8.1.1 光线投射¶
Ray Casting
我们已经看到在 three.js API 的 5.3.2 小节 中，类型为 THREE.RayCaster 的对象使用了光线投射。一个 Raycaster 需要一个初始点和方向，以向量的形式给出。这个点和向量确定了一束光线，即从起始点开始，沿着某个方向无限延伸的半无限线。Raycaster 可以找到光线与 three.js 场景中一组给定对象的所有交点，并按距离光线起始点的顺序排序。在本章中，我们感兴趣的是第一个交点，即离起始点最近的那一个。
要将光线投射应用于渲染，假设我们有一个由三维空间中的物体组成的场景，使用点光源和定向光进行照明。我们想要从给定的视角渲染场景的图像。想象这个图像是一种矩形窗口，通过它来观察场景是很方便的。给定图像中的一个点，我们需要知道如何给这个点上色。为了计算这个点的颜色，我们首先从观察者的位置通过这个点并进入场景投射一束光线。我们想要找到那束光线与场景中物体的第一个交点：
在这个插图中，场景包含几个红色球体和两个点光源。从视点 (A) 通过图像中的一个点 (B) 投射出一束光线。这束光线与两个球体相交，但我们只对离视点最近的交点 (C) 感兴趣。那就是在图像中 B 点处可见的点。
我们需要计算观察者在 B 点处将看到的颜色。为此，就像在 OpenGL 中一样，我们需要在 C 点有一个法向量，我们需要 C 点处表面的材质属性。这些数据必须能够从场景描述中计算出来。可见颜色还取决于照亮表面的光源。考虑一个光源，设 L 是从 C 点指向光源方向的向量。如果 L 和法向量之间的角度大于 90 度，那么光源位于表面后面，因此不会增加任何照明。否则，我们可以使用光线投射：从 C 点沿着 L 方向投射一束光线。如果那束光线在到达光源之前击中了一个物体，那么那个物体将阻挡来自该光源的光线到达 C 点。插图中的 Light 2 就是这种情况：从 C 点指向 Light 2 方向的光线在到达光源之前在 E 点处与一个物体相交。另一方面，C 点被 Light 1 照亮。从表面一点指向光源方向的光线称为 阴影光线，因为它可以用来确定表面点是否处于另一个物体的阴影中。
有了所有这些信息，我们可以使用与 OpenGL 中使用的相同的照明方程 (4.1.4 小节) 来给图像中的 B 点上色。光线投射解决了隐藏表面问题，无需深度缓冲区。而且，作为额外的好处，我们得到了阴影，在 OpenGL 中很难做到！
（如果从视点通过 B 点的光线没有击中场景中的任何物体，那么 B 将被赋予背景颜色或随不同方向变化的“天空”颜色。或者也许整个场景被一个 skybox 包围，一个没有击中任何其他物体的光线将击中 skybox。）
光线投射在概念上很简单，但实现细节可能会很棘手。球体实际上相当容易。有一个公式可以找到直线与球体的交点，球体上某点的法向量与从球心到该点的线的方向相同。假设球体具有均匀的材质，我们就拥有了所需的所有数据。
然而，表面通常以三角形网格的形式给出，属性仅在三角形的顶点处指定。假设我们的光线投射器找到的交点位于其中一个三角形内。我们将不得不通过插值三角形顶点处指定的属性值来计算该交点的属性。
插值算法通常使用所谓的三角形上的 重心坐标：如果 A、B 和 C 是三角形的顶点，P 是三角形内的一点，那么 P 可以唯一地写成 aA + bB + c*C 的形式，其中 a、b 和 c 是在零到一范围内的数字，且 a + b + c = 1。系数 a、b 和 c 称为点 P 在三角形中的重心坐标。如果某个量在三角形的顶点处具有值 V(A)、V(B) 和 V(C)，那么我们可以使用 P 的重心坐标来获得 P 处的插值值：
V(P) = a*V(A) + b*V(B) + c*V(C)
这个量可能是漫反射颜色、纹理坐标或法向量。（当然，我在这里仍然省略了很多数学内容——如何测试一条线是否与一个三角形相交以及如何找到交点的重心坐标。有公式，但从概念上讲，它们不会为讨论增加太多内容。）
We have already seen ray casting used by objects of type THREE.RayCaster in the three.js API (Subsection 5.3.2). A Raycaster takes an initial point and a direction, given as a vector. The point and vector determine a ray, that is, a half-infinite line that extends from a starting point, in some direction, to infinity. The Raycaster can find all the intersections of the ray with a given set of objects in a three.js scene, sorted by order of distance from the rays's starting point. In this chapter, we are interested in the first intersection, the one that is closest to the starting point.
To apply ray casting to rendering, let's say that we have a scene consisting of objects in three-dimensional space, using point lights and directional lights for illumination. We want to render an image of the scene from a given point of view. It's convenient to imagine the image as a kind of rectangular window through which the scene is being viewed. Given a point in the image, we need to know how to color that point. To compute a color for the point, we begin by casting a ray from the position of the viewer through the point and into the scene. We want to find the first intersection of that ray with an object in the scene:
In this illustration, the scene contains several red spheres and two point lights. A ray is cast from the viewpoint (A) through a point (B) in the image. The ray intersects two of the spheres, but we are only interested in the intersection point (C) that is closest to the viewpoint. That's the point that is visible at B in the image.
We need to compute the color that the viewer will see at point B. For that, just as in OpenGL, we need a normal vector at point C, and we need the material properties of the surface at C. That data has to be computable from the scene description. The visible color also depends on the light that is illuminating the surface. Consider a light source, and let L be the vector that points from C in the direction of the light. If the angle between L and the normal vector is greater than 90 degrees, then the light source lies behind the surface and so does not add any illumination. Otherwise, we can use ray casting again: Cast a ray from C in the direction L. If that ray hits an object before it gets to the light, then that object will block light from that source from reaching C. That's the case for Light 2 in the illustration: The ray from C in the direction of Light 2 intersects an object at point E before it gets to the light. On the other hand, the point C is illuminated by Light 1. A ray from a point on a surface in the direction of a light source is called a shadow ray, because it can be used to determine whether the surface point lies in the shadow of another object.
With all this information, we can color point B in the image using the same lighting equation that is used in OpenGL (Subsection 4.1.4). Ray casting solves the hidden surface problem with no need for a depth buffer. And, as a bonus, we get shadows, which are hard to do in OpenGL!
(If the ray from the viewpoint through B doesn't hit any objects in the scene, then B would be assigned a background color or a "sky" color that varies with different directions. Or maybe the entire scene is surrounded by a skybox, and a ray that doesn't hit any other object will hit the skybox.)
Ray casting is conceptually simple, but implementation details can be tricky. Spheres are actually fairly easy. There is a formula for finding the intersections of a line with a sphere, and a normal vector at a point on a sphere has the same direction as the line from the center of the sphere to that point. Assuming that the sphere has a uniform material, we have all the data we need.
However, surfaces are often given as triangular meshes, with properties specified only at the vertices of the triangles. Suppose that the intersection point found by our ray caster lies in one of those triangles. We will have to compute the properties of that intersection point by interpolating the property values that were specified at the vertices of the triangle.
The interpolation algorithm typically uses something called barycentric coordinates on the triangle: If A, B, and C are the vertices of a triangle, and P is a point in the triangle, then P can be written uniquely in the form a*A + b*B + c*C, where a, b, and c are numbers in the range zero to one, and a + b + c = 1. The coefficients a, b, and c are called the barycentric coordinates of the point P in the triangle. If some quantity has values V(A), V(B), and V(C) at the vertices of the triangle, then we can use the barycentric coordinates of P to get an interpolated value at P:
V(P) = a*V(A) + b*V(B) + c*V(C)
The quantity might be, for example, diffuse color, texture coordinates, or a normal vector. (Of course, I'm still leaving out a lot of the math here—how to test whether a line intersects a triangle and how to find barycentric coordinates of the point of intersection. There are formulas, but conceptually, they wouldn't add much to the discussion.)
8.1.2 递归光线追踪¶
Recursive Ray Tracing
基本的光线投射可以用来计算OpenGL风格的渲染，并且通过增加阴影光线，还可以实现阴影效果。通过投射更多的光线，可以实现更多的特性。改进后的算法被称为光线追踪。
考虑镜面反射。在OpenGL中，镜面反射可以使物体看起来有光泽，因为可以在物体将光源的光线反射到观察者的方向时看到镜面高光。但现实中，具有类似镜子表面的物体不仅会反射光源；它还会反射其他物体。如果我们试图计算一个类似镜子表面的点A的颜色，我们需要考虑来自其他物体的镜面反射对该颜色的贡献。为此，我们可以从A点投射一个“反射光线”。反射光线的方向由A点处表面的法向量和从A点到观察者的方向决定。这张插图展示了一个2D版本。将其想象为3D情况的横截面：
在这里，从点A发出的反射光线击中了紫色正方形上的点B，观察者将在A点看到B点的反射。（记住，在光线追踪中，我们从观察者那里反向追踪光线的路径，以找出它们来自哪里。）
为了弄清楚B点的反射看起来如何，我们需要知道从B点到达A点的光线的颜色。但是为B点找到颜色与为A点找到颜色是同一类问题，我们应该以相同的方式解决它：通过将光线追踪算法应用于B！也就是说，我们使用B点处表面的材质属性，我们从B点向光源投射阴影光线以确定B点是如何被照亮的，如果紫色正方形具有类似镜子的表面——我们从B点投射一个反射光线以找出它反射了什么。在插图中，从B点发出的反射光线击中了五边形上的点C，所以正方形反射了一个五边形的图像，而圆盘反射了一个正方形的图像，包括它对五边形的反射。光线追踪可以处理场景中物体之间的多次镜面反射！
因为在一个点上应用光线追踪算法可能涉及到在额外的点上应用相同的算法，光线追踪是一种递归算法。为了与简单的光线投射区分开来，光线追踪通常被称为“递归光线追踪”。
光线追踪可以以类似的方式扩展以处理透明度或更准确地说，半透明度。在计算半透明物体上一个点的颜色时，我们需要考虑到通过物体到达该点的光线。为此，我们可以从该点投射另一个光线，这次是进入物体内部。当光线从一种介质，例如空气，进入另一种介质，例如玻璃时，光线的路径可能会弯曲。这种弯曲称为折射，通过半透明物体投射的光线称为“折射光线”。上面的插图显示了从点A发出的折射光线穿过物体并在D点从物体中出现。为了找到那条光线的颜色，我们需要找出它击中了什么，如果击中了任何东西，我们需要在交点递归地应用光线追踪。
（光线从一种介质穿过到另一种介质时的弯曲程度取决于这两种介质的一个属性，称为“折射率”。更准确地说，它取决于两个折射率之间的比率。在实践中，物体外部的折射率通常被认为等于一，这样弯曲就只取决于物体的折射率。折射率是半透明物体的另一种材质属性。它通常缩写为IOR。）
我们应该更详细地看看这些计算。目标是计算图像上某点的颜色。我们从视点通过图像投射一束光线进入场景，并确定光线与物体的第一个交点。该点的颜色是通过汇总来自不同来源的贡献来计算的。
首先，有来自各种光源的漫反射、镜面反射和可能的环境光反射。这些贡献基于物体的漫反射、镜面反射和环境光颜色，基于物体的法向量，以及光源的特性。注意，物体的某些颜色属性，通常是环境光和漫反射颜色，可能来自纹理。镜面贡献可以产生镜面高光，这本质上是光源的反射。阴影光线用于确定哪些定向光和点光源照亮了物体；除此之外，这部分计算与OpenGL类似。
接下来，如果表面具有类似镜子的反射，则会投射反射光线，并对那个光线递归应用光线追踪来找到其颜色。来自该光线的贡献与其他贡献结合在一起，形成表面的颜色，这取决于镜面反射的强度。对于完美的镜子，反射光线贡献了100%的颜色，但通常贡献较小。镜面反射率是一个新的材质属性。它负责一个物体在另一个物体表面上的反射，而镜面颜色负责镜面高光。
最后，如果物体是半透明的，那么会投射折射光线，并使用光线追踪来找到它的颜色。该光线对物体颜色的贡献取决于物体的透明度程度，因为一些光线可能被吸收而不是被传输。透明度的程度可能取决于光的波长——正如它在有色玻璃中那样。
当然，所有这些计算都需要对颜色的红色、绿色和蓝色分量各做三次。
光线追踪算法是递归的，正如每个程序员都知道的，递归需要一个基本情况。也就是说，必须有一个时刻，算法不是调用自己，而是简单地返回一个值。当投射的光线没有与任何物体相交时，就会出现基本情况。另一种基本情况可能发生在确定投射更多光线不能对图像的颜色做出任何显著贡献时。例如，每当反射光线时，根据反射它的物体的颜色，一些光线会被吸收而不是反射。反射多次后，光线对最终结果的贡献将非常少。光线也可能因为光的衰减而失去能量，光线追踪算法可能会考虑这一点。此外，光线追踪算法应该总是以最大递归深度运行，以对算法调用自身的次数设定绝对限制。
Basic ray casting can be used to compute OpenGL-style rendering and, with the addition of shadow rays, to implement shadows as well. More features can be implemented by casting a few more rays. The improved algorithm is called ray tracing.
Consider specular reflection. In OpenGL, specular reflection can make an object look shiny in the sense that specular highlights can be seen where the object reflects light from a light source towards the viewer. But in reality, an object that has a mirror-like surface doesn't just reflect light sources; it also reflects other objects. If we are trying to compute a color for a point, A, on a mirror-like surface, we need to consider the contribution to that color from mirror-like reflection of other objects. To do that, we can cast a "reflected ray" from A. The direction of the reflected ray is determined by the normal vector to the surface at A and by the direction from A to the viewer. This illustration shows a 2D version. Think of it as a cross-section of the situation in 3D:
Here, the reflected ray from point A hits the purple square at point B, and the viewer will see a reflection of point B at A. (Remember that in ray tracing, we follow the path of light rays backwards from the viewer, to find out where they came from.)
To find out what the reflection of B looks like, we need to know the color of the ray that arrives at A from B. But finding a color for B is the same sort of problem as finding a color for A, and we should solve it in the same way: by applying the ray-tracing algorithm to B! That is, we use the material properties of the surface at B, we cast shadow rays from B towards light sources to determine how B is illuminated, and—if the purple square has a mirror-like surface—we cast a reflected ray from B to find out what it reflects. In the illustration, the reflected ray from B hits a pentagon at point C, so the square reflects an image of the pentagon, and the disk reflects an image of the square, including its reflection of the pentagon. Ray-tracing can handle multiple mirror-like reflections between objects in a scene!
Because applying the ray-tracing algorithm at one point can involve applying the same algorithm at additional points, ray tracing is a recursive algorithm. To distinguish this from simple ray casting, ray tracing is often referred to as "recursive ray tracing."
Ray tracing can be extended in a similar way to handle transparency or, more properly, translucency. When computing a color for a point on a translucent object, we need to take into account light that arrives at that point through the object. To do that, we can cast yet another ray from that point, this time into the object. When a light ray passes from one medium, such as air, into another medium, such as glass, the path of the light ray can bend. This bending is called refraction, and the ray that is cast through a translucent object is called the "refracted ray." The above illustration shows the refracted ray from point A passing through the object and emerging from the object at D. To find a color for that ray, we would need to find out what, if anything, it hits, and we would need to apply ray tracing recursively at the point of intersection.
(The degree of bending of a light ray that passes from one medium to another depends on a property of the two media called the "index of refraction." More exactly, it depends on the ratio between the two indices of refraction. In practice, the index of refraction outside objects is often taken to be equal to one, so that the bending depends only on the index of refraction of an object. The index of refraction is another material property for translucent objects. It is often abbreviated IOR.)
We should look at the computations in a little more detail. The goal is to compute a color for a point on an image. We cast a ray from the viewpoint through the image and into the scene, and determine the first intersection point of the ray with an object. The color of that point is computed by adding up the contributions from different sources.
First, there is diffuse, specular, and possibly ambient reflection of light from various light sources. These contributions are based on the diffuse, specular, and ambient colors of the object, on the normal vector to the object, and on the properties of the light sources. Note that some color properties of the object, usually the ambient and diffuse colors, might come from a texture. The specular contribution can produce specular highlights, which are essentially the reflections of light sources. Shadow rays are used to determine which directional and point lights illuminate the object; aside from that, this part of the calculation is similar to OpenGL.
Next, if the surface has mirror-like reflection, then a reflected ray is cast and ray tracing is applied recursively to find a color for that ray. The contribution from that ray is combined with other contributions to the color of the surface, depending on the strength of the mirror reflection. For a perfect mirror, the reflected ray contributes 100% of the color, but in general the contribution is less. Mirror reflectivity is a new material property. It is responsible for reflections of one object on the surface of another, while specular color is responsible for specular highlights.
Finally, if the object is translucent, then a refracted ray is cast, and ray tracing is used to find its color. The contribution of that ray to the color of the object depends on the degree of transparency of the object, since some of the light can be absorbed rather than transmitted. The degree of transparency can depend on the wavelength of the light—as it does, for example, in colored glass.
And, of course, all of these calculations need to be done three times, for the red, the green, and the blue components of the color.
The ray tracing algorithm is recursive, and, as every programmer knows, recursion needs a base case. That is, there has to come a time when, instead of calling itself, the algorithm simply returns a value. A base case occurs whenever a casted ray does not intersect any objects. Another kind of base case can occur when it is determined that casting more rays cannot contribute any significant amount to the color of the image. For example, whenever a ray is reflected, some of that ray is absorbed rather than reflected, depending on the color of the object from which it is reflected. After reflecting many times, a ray would have very little color left to contribute to the final result. A ray can also lose energy because of attenuation of light with distance, and a ray-tracing algorithm might take that into account. In addition, a ray tracing algorithm should always be run with a maximum recursion depth, to put an absolute limit on the number of times the algorithm will call itself.
8.1.3 光线追踪的局限性¶
Limitations of Ray Tracing
尽管光线追踪可以产生非常逼真的图像，但它也有一些无法做到的事情。例如，虽然光线追踪对点光源和定向光效果良好，但它无法处理区域光。区域光是指具有面积的光源，即它是一个从整个表面积而不是从一个点发出光线的物体。当然，真正的光源是区域光。荧光灯从圆柱体的表面发出光线。一个明亮的窗户可以被视为一种区域光。即使是灯泡也不是真正从一个点辐射光线。光线追踪使用阴影光线来判断光源是否照亮了物体。但阴影光线只在一个方向上投射，并且只能击中区域光上的一个点。要准确地实现区域光，需要为光源表面上的每个点都需要一个不同的阴影光线。
光线追踪器可以通过用点光源网格替换区域光来近似处理区域光。然后它可以向网格中的每个点投射一个阴影光线。在网格中使用更多的点光源将提供更好的近似。当然，投射所有这些阴影光线可能会显著增加计算时间。
光线追踪中的另一个照明问题是它没有考虑到反射光的照明。例如，来自光源的光线应该从镜子上反射，反射的光线应该照亮其他物体。光线追踪使用阴影光线来判断光源是否照亮了物体。但这对反射光不起作用，因为算法不知道应该将阴影光线瞄准哪里——反射光可能来自任何方向。
这不仅仅是镜子的问题。任何反射光，即使是漫反射，也应该照亮附近的物体。例如，从绿色物体上漫反射的光线应该给附近的物体增添一点绿色。这种效果称为“颜色渗透”。实际上，光线可以被反射和再反射多次，为它击中的每个物体增添一点颜色。与区域光一样，可以向光线追踪中添加近似方法来模拟这种效果。例如，“光子映射”为光线追踪增加了一个预处理阶段，该阶段模拟了从光源发出的大量光线或“光子”，并跟踪它们在场景中的路径，看看它们如何为它们击中的物体增添颜色。然后在光线追踪阶段使用这个“光子图”的信息来产生更逼真的颜色。
OpenGL使用环境光作为多次反射和再反射光线的近似。假设环境光平等地照亮每个物体。然而，这是一个较差的近似。更好的近似使用环境遮挡，即环境光朝向表面时可以被附近物体阻挡或“遮挡”。像环境光一样，环境遮挡在物理上是不真实的，但在实践中，它可以使照明看起来更逼真，物体看起来更立体。一种估计环境遮挡的技术使用光线投射。我们假设环境光来自场景的背景。为了估计某点的环境遮挡，从该点向随机方向投射多条光线，并计算这些光线中有多少被场景中的几何体阻挡，有多少达到了天空。被阻挡的光线越多，该点的环境遮挡程度就越大。
尝试考虑场景中光线与物体的所有相互作用的算法被称为使用全局照明。可以向光线追踪中添加上述近似方法以增加其现实性，但代价是显著增加的复杂性和可能大量的额外计算。我们开始明白为什么非常逼真的图像需要这么长时间来计算了！在下一节中，我们将看一种尝试精确处理全局照明的方法。
Although ray tracing can produce very realistic images, there are some things that it can't do. For example, while ray tracing works well for point lights and directional lights, it can't handle area lights. An area light is one that has area. That is, it is an object that emits light from its entire surface area rather than from a single point. Of course, real lights are area lights. A fluorescent light emits light from the surface of a cylinder. A brightly lit window can be considered to be a kind of area light. Even a light bulb does not really radiate light from a single point. Ray tracing uses shadow rays to tell whether a light source illuminates an object. But a shadow ray is cast in only one direction, and can only hit one point on an area light. To implement area lights exactly, a different shadow ray would be needed for each point on the surface of the light.
A ray tracer can handle area lights in an approximate way, by replacing the area light with a grid of point lights. It can then cast a shadow ray towards each point in the grid. Using more point lights in the grid will give a better approximation. Of course, casting all those shadow rays can add significantly to the computation time.
Another problem with lighting in ray tracing is that it doesn't take into account illumination by reflected light. For example, light from a light source should reflect off a mirror, and the reflected light should illuminate other objects. Ray tracing uses shadow rays to tell whether a light source illuminates an object. But that won't work for reflected light, since the algorithm doesn't know where to aim the shadow ray—reflected light could come from any direction.
This is not just a problem with mirrors. Any reflected light, even from diffuse reflection, should illuminate nearby objects. For example, light that is reflected diffusely from a green object should add a green tint to nearby objects. This effect is called "color bleeding." In reality, light can be reflected and re-reflected many times, contributing a bit of color to each object that it hits. As with area lights, approximate methods can be added to ray tracing to simulate this effect. For example, "photon mapping" adds a pre-processing phase to ray tracing which simulates the emission of a large number of light rays, or "photons," from light sources, and tracks their paths through the scene to see how they add color to the objects that they hit. The information from this "photon map" is then used during the ray tracing phase to produce more realistic colors.
OpenGL uses ambient light as an approximation for light that has been reflected and re-reflected many times. Ambient light is assumed to illuminate every object equally. However, that is a poor approximation. A better approximation uses ambient occlusion, the idea that ambient light heading towards a surface can be blocked, or "occluded," by nearby objects. Like ambient light, ambient occlusion is not physically realistic, but in practice, it can make lighting look more realistic and objects look more three-dimensional. One technique for estimating ambient occlusion uses ray casting. We assume that the ambient light comes from the background of the scene. To estimate ambient occlusion at a point, cast a number of rays from that point in random directions, and count how many of those rays are blocked by geometry in the scene and how many reach the sky. The more rays that are blocked, the greater the degree of ambient occlusion at that point.
Algorithms that attempt to take into account all of the interactions of light with objects in a scene are said to use global illumination. Approximate methods such as those mentioned above can be added on to ray tracing to increase its realism, at the cost of significantly increased complexity and a potentially large amount of extra computing. We begin to see why very realistic images can take so long to compute! In the next section we will look at a method that attempts to handle global illumination exactly.