2022.07.28 Reprojection in Unreal

After many hours of troubleshooting, I discovered that the ViewProjection Matrix columns I was sending to the Material via Vector parameters (VP_1, VP_2, VP_3, VP_4) were actually being sent as Vector3s, removing the W component. This caused the position of the projector to be irrelevant which is not at all the intended consequence.

To fix this, I converted the rows of the ViewProjection Matrix into LinearColors, which have 4 components (RGBA) and can be sent to the Material as a color. I had to verify that doing this did not clamp the values between 0 and 1, which is often the case when converting to colors.

After verifying that the data was not being corrupted by the conversion to LinearColor, the result was the following Blueprint graph:

As well as the following Material graph:

With the following Custom node HLSL code:

Viewer Camera focused on a running dude

Altogether this produced the correct effect, of the projector's position and orientation relative to the cube defining how the Viewer Camera's rendered image is "projected" onto the cube.

MyProject4 - Unreal Editor 2022-07-28 15-41-41.mp4