unity 光探头_光探头

unity 光探头

Hello everyone. I wanted to show you something I’ve been working on last couple of FAFFs. The purpose of this post is to interest some of the technical types among you. If you don’t care about a technical discussion about an unfinished feature, you can just read about the basics or jump right to the video showing some WIP pretty pixels.

大家好。 我想向您展示我在最后几个FAFF上所做的工作。 这篇文章的目的是使您感兴趣的一些技术类型。 如果您不关心关于未完成功能的技术讨论，则可以阅读基础知识或直接跳到显示一些WIP漂亮像素的视频。

基础(The Basics)

Theproblemwe are trying to solve is how tousebaked lighting on dynamic objectsand characters. After lightmapping a scene all static objects have nice, high quality lighting. If we don’t do anything about the dynamic objects though, they might look dull and detached from the environment with their direct light and ambient only.

我们正在试图解决的问题是如何使用烤照明动态物体和人物。 对场景进行灯光映射后，所有静态对象都具有良好的高质量照明。 如果我们不对动态对象做任何事情，它们可能仅在直射光线和环境下看起来暗淡无光，与环境分离。

Onesolutionis to uselight probesto sample the nice baked lighting at various points in the scene. We can then interpolate the nearby probes to guess what would the lighting be at the current location of our character (or other moving object) and use the result to light the character.

一种解决方案是使用光探针在场景中的各个点采样精美的烘焙照明。 然后，我们可以对附近的探针进行插值，以猜测角色(或其他移动物体)当前位置处的照明情况，并使用结果来照亮角色。

Let’s see how it looks like in action! (Apologies for the cheesy lighting with over the top light bounce, but it makes it easier to illustrate the effect)

让我们看看它的实际效果！ (道歉的低光照明会反射顶部的光线，但更容易说明效果)

[vimeo clip_id=”20385528″ width=”640″ height=”360″]

[vimeo clip_id =” 20385528” width =” 640” height =” 360”]

The light probes can be used to store the full incoming lighting or just the indirect lighting (plus full lighting from emissive materials and ‘baked only’ lights), i.e. the Dual Lightmaps style. Either way it’s quite flexible how we can then use the probe in the shader: do full lighting per vertex, per pixel with normal mapping or even per vertex indirect light blended with per pixel direct light with normal mapping.

可以使用光探针存储完整的入光或仅存储间接照明(以及来自发光材料和“仅烘焙”的光的完整照明)，即双重光照贴图样式。 无论哪种方式，我们在着色器中使用探针的方式都非常灵活：对每个顶点，每个像素进行法线贴图，甚至对每个顶点间接光与每个像素直接光与法线贴图混合都进行完全照明。

To use the light probes, you need to add them to your scene, bake andvoilà– all dynamic renderers with the feature enabled use light probe lighting, both in edit and play mode.

要使用光探针，您需要将它们添加到场景中，进行烘焙和贴图-启用此功能的所有动态渲染器在编辑和播放模式下均使用光探针照明。

I haven’t decided yet how will placing of the probes in the scene work, as it ties in with what interpolation technique I’ll use. Probably the probes will either be placed automatically (with some global control from the user’s side) or manually as it has been done in this demo. If I go for the latter, I’ll make sure to add some utility functions to pre-place the probes over a navigation mesh, expose probe placing to scripting and possibly allow probe locations to be part of prefabs – everything to keep the manual process painless while still giving a lot of control.

我还不确定如何将探针放置在场景中，因为它与我将使用的插值技术有关。 可能会自动放置探针(在用户方面进行一些全局控制)，或者手动放置探针，如本演示中所述。 如果我要使用后者，我将确保添加一些实用程序功能，以将探针预先放置在导航网格上，将探针放置位置暴露于脚本中，并可能允许探针位置成为预制件的一部分–保持手动过程的所有步骤无痛，同时仍能提供很多控制。

You can be sure this feature will be released as soon as it’s ready and not later. I don’t know when that might be, though ;) For now light probes remain my FAFF project while I’m focusing on a bigger feature, which should make a lot of you happy as well.

您可以确定此功能将在准备就绪后立即发布，而不是稍后发布。 不过，我不知道什么时候会出现；)现在，当我专注于更大的功能时，光探头仍然是我的FAFF项目，这也应该使很多人感到高兴。

细节 (The Details)

光探头和球谐函数 (Light Probes and Spherical Harmonics)

To bake a light probe, we need the amount of light incoming from every possible direction – and that’s a lot of directions! But – we also know that most likely the incoming light doesn’t change that quickly between different directions. In other words it doesn’t have high frequency changes, so if we compress that data in the frequency domain on a sphere by discarding all the higher frequencies – no one should notice.

要烤制光探头，我们需要从每个可能的方向入射的光量-这是很多方向！ 但是–我们也知道，入射光很可能在不同方向之间变化不大。 换句话说，它没有高频变化，因此，如果我们通过丢弃所有更高的频率在一个球体的频域中压缩该数据，则没人会注意到。

Storing the incoming light using Spherical Harmonics achieves just that. Spherical harmonics basis functions can be thought of as an equivalent of harmonics in Fourier analysis on a sphere. The more bands (groups of basis functions) we decide to take into account, the more accurately we’ll be able to reconstruct the original incoming light signal. Beast can bake light probes directly as spherical harmonics coefficients, which tell us “how much” of each of the basis functions our signal did contain. The original function describing how much light comes from a given direction can then be reconstructed as just a linear combination of the coefficients and the basis functions.

使用球形谐波存储入射光就可以实现这一点。 球谐函数基函数可以被认为是球体上的傅立叶分析中的谐波等效项。 我们决定考虑的频带(基函数组)越多，我们就能越准确地重构原始的入射光信号。 Beast可以直接将光探针作为球形谐波系数进行烘烤，从而告诉我们信号中包含的每个基本函数“多少”。 然后，可以将描述从给定方向发出多少光的原始函数重构为系数和基函数的线性组合。

Let’s look at an example. We could ask Beast to bake one light probe for us for a given location. That light probe would just be a bunch of SH coefficients – if we chose that 3 bands are enough, that would require storing a coefficient for each of the 9 basis functions for each of the 3 color channels, so a total of 27 floats. If then a dynamic object would end up in the exact same location as the light probe, we could say – hey, we know what’s the incoming light at that position. In the vertex shader (or pixel shader, if we wanted more precision and normal mapping) we could then decode the SH coefficients for a given direction dictated by the object’s normal and light our dynamic object with the light probe that way.

让我们来看一个例子。 我们可以要求野兽为我们给定位置烘烤一个光探头。 该光探头将只是一堆SH系数-如果我们选择3个波段就足够了，那将需要为3个颜色通道的每个9个基函数中的每个存储一个系数，因此总共有27个浮点。 如果这样，一个动态物体最终将与光探头位于完全相同的位置，我们可以说–嘿，我们知道那个位置的入射光是什么。 然后，在顶点着色器(或像素着色器，如果需要更高的精度和法线贴图)中，我们可以解码由对象法线指示的给定方向的SH系数，并使用光探针照亮动态对象。

真正的问题：放置和插值(The real problem: placement and interpolation)

That brings us to the real problem: how do we decide where to place the probes in the scene? And how do we interpolate the probes’ values once we have a bunch of probe locations and an object sitting somewhere in between?

这给我们带来了一个真正的问题：我们如何确定将探针放置在场景中的什么位置？ 当我们拥有一堆探针位置和一个介于两者之间的对象时，我们该如何插入探针的值？

The useful property of spherical harmonics encoded probes is that a linear interpolation between two probes would just be a linear interpolation of their coefficients. So if we had all the probes placed along a path and our object was moving along that path (racing game, anyone?), we would just linearly interpolate the two probes at the ends of the current path segment to find the lighting.

球形谐波编码探头的有用特性是，两个探头之间的线性插值将只是其系数的线性插值。 因此，如果我们将所有探针放置在一条路径上，并且我们的对象正在沿着该路径移动(竞赛游戏，有人吗？)，我们只需在当前路径段的末端线性插入两个探针以找到照明。

At the same time we have to think where do we need the light probes to be located. Surely we only want them where our dynamic objects can go – no point in baking and storing probes which will never be used. Also we want the probes to encode interesting changes in lighting, but we don’t want extra probes where the lighting changes slowly or the change is something our lighting artist doesn’t care about in this spot.

同时，我们必须考虑需要将光探头放置在何处。 当然，我们只希望它们可以放在我们的动态对象可以到达的地方–烘烤和存储永远不会使用的探针毫无意义。 另外，我们希望探针对照明中有趣的变化进行编码，但我们不希望多余的探针在照明变化缓慢的地方发生变化，或者这种变化是我们的照明艺术家不在乎的。

At this point I should probably list the properties I would like the interpolation function to have, but they’re actually quite intuitive. Let me just emphasise the most important one: if there’s a probe at the location we’re evaluating the function for, it should return a weight of 1 for that probe and weights of 0 for all the others. This is another way of saying: light probes sample the underlying lighting information, so there’s no need for guessing there – we know what’s the lighting at those exact points.

在这一点上，我可能应该列出我希望插值函数具有的属性，但是它们实际上非常直观。 让我只强调最重要的一个：如果要评估函数的位置处有一个探针，则该探针的权重应为1，所有其他探针的权重应为0。 这是另一种说法：光探头对潜在的照明信息进行采样，因此无需在那里猜测–我们知道在这些确切点上的照明是什么。

Here’s a couple of possible solutions:

以下是一些可能的解决方案：

Uniform grid.The entire scene is put into a bounding box which is then subdivided to an artist-defined density and a probe is placed at each cell’s corner. This approach doesn’t require much interaction from the user and the interpolation is simple, robust and relatively cheap (it’s just a trilinear interpolation of the probes at the cell’s corner and it’s easy to find the right cell). The downside is that you often have to dial up the density quite a bit to capture that light you care about in just one spot, which makes a lot of the probes filling the major part of the volume completely useless – and that wastes a lot of memory. Also you will get into situations when there’s a row of probes close to the wall and a row of probes just in the wall or on the other side of it (so e.g. much darker): your character approaches the wall and suddenly the darkness starts to creep in although it shouldn’t. The way this has been solved forMilo and Kateand at least one other current title I know aboutis that each probe encodes additional visibility data which limits it’s influence up to the nearest obstacle. This however adds to the memory footprint and interpolation time. It also might introduce artefacts on it’s own if the grid is not fine enough.

统一网格。将整个场景放入边界框，然后将其细分为艺术家定义的密度，并在每个单元的角落放置一个探针。 这种方法不需要用户太多的交互，并且插值简单，健壮且相对便宜(它只是在单元角上的探针的三线性插值，很容易找到合适的单元)。 不利之处在于，您通常必须调高密度才能在一个位置上捕获您关心的光线，这使得填充大部分体积的许多探头完全无用，并且浪费了很多记忆。 另外，当墙壁附近有一排探针，而墙壁或墙壁的另一侧也有一排探针时(例如，较暗)，您会陷入困境：您的角色接近墙壁，突然间，黑暗开始爬行，尽管它不应该。米洛(Milo)和凯特(Kate)以及我所知道的至少另一本当前书名的解决方法每个探针都编码其他可见性数据，从而将其影响范围限制在最近的障碍物附近。 但是，这会增加内存占用量和内插时间。 如果网格不够精细，它本身也可能会引入伪影。

Adaptively subdivided grid.It’s a concept similar to the one above, except that probe density can vary where needed. The structure could be an octree, in which we keep subdividing cells if we expect changes in lighting that need capturing. A good heuristic for doing so might be testing if current cell contains scene geometry – if it does, there’s a better chance of higher-frequency changes in the lighting. After the probes have been baked there’s also the possibility of clustering similar probes. This approach should solve the “memory monster” issue of the uniform grid, but at a cost of slightly more complex and branchy search and interpolation code. It still needs to store visibility information like the previous solution and the subdivision heuristic might be wrong.

自适应细分的网格。这是与上述概念类似的概念，不同之处在于探头的密度可以根据需要变化。 该结构可以是八叉树，如果我们期望需要捕获的照明发生变化，则可以在其中继续细分单元。 进行此操作的一种很好的启发方法可能是测试当前单元格是否包含场景几何图形-如果包含场景几何图形，则更有可能发生照明频率更高的变化。 探针烘烤后，也有可能将相似的探针聚在一起。 这种方法应该解决统一网格的“内存怪兽”问题，但代价是稍微复杂和分支搜索和内插代码。 它仍然需要像以前的解决方案一样存储可见性信息，并且细分启发法可能是错误的。

K nearest probes.With this approach we just search for K nearest probes and interpolate between them. This time probes don’t need to be placed on a grid – we can put them anywhere. The biggest issue here is that the set of probes used for interpolation can change suddenly at any time – even when the just-excluded and just-included probes had high interpolation weights, resulting in visible light popping. To minimise this effect, we can employ some damping – interpolate from the old value to the new over time. The delay will be visible in some cases, but still better than a sudden pop of lighting.

K个最近的探针。使用这种方法，我们只搜索K个最近的探针并在它们之间进行插值。 这次无需将探针放置在网格上–我们可以将它们放置在任何地方。 这里最大的问题是，用于插值的探针组可以在任何时候突然改变-即使当刚排除和刚包括的探针具有较高的插值权重时，也会导致可见光弹出。 为了使这种影响最小化，我们可以采用一些阻尼–随着时间的推移从旧值插入到新值。 在某些情况下，该延迟是可见的，但仍比突然亮起的照明要好。

Tetrahedralisation.Once the probes have been placed, we can find theDelaunay tetrahedralisation of such point set. To find the interpolated probe for our current location, we first search for the containing tetrahedron and then calculate barycentric coordinates which can be used as interpolation weights for the four vertices of the tetrahedron.Finding the containing tetrahedron can be done efficiently by always starting to test the last one we were in. If the test fails (it usually won’t), we find the tetrahedron’s normal most accurately pointing towards our location (highest dot product) and move to the tetrahedron adjacent in that direction. The first two solutions can be made fully automatic and that’s definitely an advantage. On the other hand somehow I can’t accept the fact that we won’t have more direct control over which areas are important to sample and which aren’t, which will lead to over-sampling and wasting memory in some areas and under-sampling and loosing information in other areas at the same time.

四面体化。放置探针后，我们可以找到此类点集的Delaunay四面体化。 为了找到当前位置的插值探针，我们首先搜索包含的四面体，然后计算重心坐标，该坐标可用作四面体四个顶点的插值权重。 总是开始测试我们所处的最后一个，可以有效地找到包含四面体的物体。如果测试失败(通常不会)，我们会发现四面体的法线最准确地指向我们的位置(最高点积)并移动到在该方向上相邻的四面体。 前两个解决方案可以实现全自动，这绝对是一个优势。 另一方面，我不能接受这样一个事实，即我们将无法更直接地控制哪些区域对采样很重要，哪些区域不采样，这将导致某些区域过度采样并浪费内存，并且-同时在其他区域采样和丢失信息。

The other two solutions have the advantage of giving us control over where the probes are placed. That might become a problem, though, as when the level geometry changes, manual work has to be re-done. A couple of things could be made to improve the work flow there, like automatic pre-placing of probes over the nav mesh – and these initial positions could then be modified by hand. Also, with probe positioning exposed to scripting, developers could write scripts automatically placing probes in the areas which make sense for a given game and artist workflow, but couldn’t be generalised enough to be included in Unity. Probes could also be parented to objects to move with them (only at edit time) or made part of prefabs.

其他两种解决方案的优点是使我们可以控制探针的放置位置。 但是，这可能会成为一个问题，因为当关卡几何形状发生变化时，必须重新执行手动操作。 可以采取一些措施来改善那里的工作流程，例如将探针自动预先放置在导航网格上–然后可以手动修改这些初始位置。 同样，通过将探针放置在脚本中，开发人员可以自动编写脚本，将探针放置在对给定游戏和美工工作流程有意义的区域中，但不能被广泛推广到Unity中。 探针也可以作为对象的父项，以便随它们一起移动(仅在编辑时)或成为预制件的一部分。

The K-nearest probes approach seems quite reasonable and it has been used in a couple of successful games. If K is low, the interpolated probe can be calculated efficiently and it’s easy to ensure the property of getting the probe’s value when at it’s exact position. The issues that still bother me is at times unexpected interpolation (K-nearest probes aren’t always the ones you would expect) and temporal damping trying to compensate popping, but sometimes introducing visible light changes when the interpolation tries to catch upeven if the object doesn’t move any more.

K近邻探测方法似乎很合理，并且已在一些成功的游戏中使用。 如果K低，则可以高效地计算内插探针，并且很容易确保在精确位置处获得探针值的特性。 有时令我感到困扰的问题有时是意外的插值(K近邻探头并不总是您所期望的)和暂时性的阻尼试图补偿爆裂，但有时在插值试图赶上时会引入可见光变化，即使对象不再移动。

The video above shows the first interpolation scheme I tried out: 2-nearest probes with temporal damping. I project the character’s center onto the line passing through the two nearest probes and linearly interpolate if the position falls in between or clamp if it falls outside of the segment. If one or both of the nearest probes change, I take a snapshot of the interpolated probe and lerp it out over time as the new interpolated probe lerps in. I think the results are acceptable, but that’s for you to judge. There’s a couple of moments when the character has already stopped and the light is still catching up. The speed of that interpolation can be tweaked per object and can also be controlled from script as a function of e.g. object’s current speed.

上面的视频显示了我尝试的第一个插值方案：带有时间阻尼的2近探针。 我将角色的中心投影到经过两个最近的探针的直线上，并在位置介于两者之间的位置进行线性插值，或者在位置位于线段之外的位置进行线性插值。 如果最近的一个或两个探针都发生变化，我会对插值探针进行快照，然后随着新插值探针的插入而逐渐消失。我认为结果是可以接受的，但这是您的判断。 有一会儿，角色已经停止，灯光仍然在追赶。 插值的速度可以针对每个对象进行调整，也可以根据脚本控制，例如，对象的当前速度。

The probes in the video are baked in a similar fashion to near lightmaps in Dual Lightmaps, so they exclude direct lighting from Auto lights, but they do include their indirect lighting contribution and also full contribution from Baked Only lights and emissive materials (like the green puddle of goo by the barrel). Their contribution is calculated per vertex, while real-time direct light is calculated per pixel with a normal-mapped specular material. All this is handled internally by the surface shader framework, so the shader on the character is just the built-in bump specular.

视频中的探针以类似于双光照贴图中的近光照贴图的方式进行烘焙，因此它们从自动照明中排除了直接照明，但是它们确实包括了间接照明，也包括了仅烘焙的照明和自发光材料(例如绿色)的全部照明。桶里的咕咕叫的水坑)。 它们的贡献是按每个顶点计算的，而实时直射光是使用法线贴图镜面材质的每个像素计算的。 所有这些操作都是由表面着色器框架内部处理的，因此角色上的着色器只是内置的凹凸镜面反射。

Next I will probably try out the Delaunay tetrahedralisation approach. I’m having high hopes for that one, as it seems it should result in interpolation closer to what we intuitively expect while retaining the fine-grained control. Also memory consumption and search and interpolation performance should be on par with K-nearest probes and adaptively subdivided grid, but I should probably test some actual implementations on real-world scenes before getting too attached to those claims. The biggest worry here is that Delaunay tetrahedralisation is at best tricky if the input data forms degenerated patterns (e.g. all points along a line) and it might still create long and thin tetrahedra, especially at the hull surface.

接下来，我可能会尝试Delaunay四面体化方法。 我对此抱有很高的期望，因为它看起来应该能够使插值更接近于我们直观地期望的值，同时保留细粒度的控制。 内存消耗，搜索和内插性能也应该与K近邻探针和自适应细分的网格相提并论，但是我可能应该在不完全依赖那些主张之前就在真实场景中测试一些实际实现。 这里最大的担心是，如果输入数据形成退化的模式(例如，沿一条线的所有点)，则Delaunay四面体化最多是棘手的，并且它可能仍会产生细长的四面体，尤其是在船体表面。

One option worth investigating here is that if we discover all points are roughly co-planar (or at least don’t form more than one layer), the entire problem can be brought down to Delaunay triangulation in 2D and interpolation in 2D as well.

这里值得研究的一种选择是，如果我们发现所有点都大致共面(或至少不形成多于一层)，则整个问题可以归结为2D的Delaunay三角剖分和2D的插值。

This is it for now. It would be good to hear your suggestions on the topic, so feel free to comment :)

现在就这样。 希望听到您对此主题的建议，所以请随时发表评论:)

翻译自: //03/09/light-probes/

unity 光探头

如果觉得《unity 光探头_光探头》对你有帮助，请点赞、收藏，并留下你的观点哦！