Modify Metal fragment shading based on vertex world position Y coordinate - scenekit

Question

I am trying to use a Metal fragment shader with SCNTechnique to modify the fragment color based on the vertex Y world position.
My understanding so far
SCNTechnique can be configured with a sequence of render passes. A render pass allows for injection of a vertex and a fragment shader. These shaders are written in Metal. The Metal Shading Language Specification describes what inputs/outputs are supported for these two.
The vertex shader is called for every vertex that's being rendered. We can pass additional information from the vertex shader to the fragment shader (like position in 3D space, see MSLS section 5.2).
The fragment shader is closest to a pixel, and might be called multiples time for a single "pixel", if there are multiple triangles that "qualify" for that pixel. (Usually) after fragment shading, a fragment might be discarded if it fails the depth or stencil test.
What I attempted
This is what I attempted. (I hope it makes clear where my understanding is lacking).
struct VertexOut {
float4 position [[position]];
};
vertex VertexOut innerVertexShader(VertexIn in [[stage_in]]) {
VertexOut out;
out.position = in.position;
return out;
};
fragment half4 innerFragmentShader(VertexOut in [[stage_in]],
half4 color [[color(0)]]) {
half4 output;
output = color; // test to see if getting rendered color works
output.g = in.position.y; // test to see if getting y works
return output;
}
These shaders are referenced inside an SCNTechnique dictionary.
[
"passes": [
"innerPass: [
"draw": "DRAW_NODE",
"node": "inner",
"metalVertexShader": "innerVertexShader",
"metalFragmentShader": "innerFragmentShader"
]
],
"sequence": ["innerPass"],
"symbols": [:],
"targets": [:],
]
// ...
let technique = SCNTechnique(dictionary: techniqueDictionary)
This does the following: the technique is instantiated correctly and attached to the scene (because it affects the rendering). But it appears to not apply the camera transform or node position transform to the vertices. And instead renders each node as being viewed from (0,0,1) at position (0,0,0). The colors are wrong. If I remove the shaders from the SCNTechnique, every renders like I would expect.
How can I leverage regular SceneKit behavior (camera transform etc.), and only modify the color output based on the fragments' y world position? I'd expect that needs to happen on a fragment level, using the world position somehow obtained in the vertex shader. I have searched for things like "Metal basic vertex shader" and have come up with naught.
I have seen shaders like this but I'm convinced I should be able to rely on SceneKit rendering for stuff like lighting, PBR materials, camera transforms, etc. At this point I feel like whenever I search for some Metal topic, I end up on the same websites which haven't succeeded yet in taking my understanding to the next level. So, any new/additional resources are appreciated as well.
Background
For the past two months I have been working on my own game project, which uses SceneKit as the main graphics framework. I have turned to SCNTechnique and Metal shaders for custom effects. These last two in particular have given me solid headaches, both on the lack of sample code/documentation/runtime feedback.
I have considered moving to Unity/Unreal or even cancelling this project altogether because of this. But because I'm stubborn and also because I really don't want to port my Swift code to C#/C++, I haven't given up on SceneKit yet.

Answer 1

Having spent the past couple of days investigating this topic, my understanding of vertex and fragment shading and how SceneKit tackles these things has developed significantly.
As #mnuages pointed out in a comment, for this use case shader modifiers are the way to go. They leverage default SceneKit shading (as asked by OP) and allow for shader code injection.
Additional information
To compensate for some of the limitations of SceneKit documentation, I’ll elaborate a bit for other people looking into the subject.
For more information on how the shader modifiers tie into SceneKit default vertex/fragment shaders, see my answer to a related question or SceneKit's default shaders. The second link demonstrates the extent of SceneKit’s rendering logic that you get for free when leveraging shader modifiers instead of writing your own shader.
This page helped me build an understanding of the different stages of vector transforms from vertex to fragment (model space ➡️ world space ➡️ camera space ➡️ projection space).
Alternate approach (custom shader)
If you want to have single pass with a fully customized shader, this is a simple example. It passes the world y position from the vertex shader to the fragment shader.
// Shaders.metal file in your Xcode project
#include <metal_stdlib>
using namespace metal;
#include <SceneKit/scn_metal>
typedef struct {
float4x4 modelTransform;
float4x4 modelViewTransform;
} commonprofile_node;
struct VertexIn {
float3 position [[attribute(SCNVertexSemanticPosition)]];
};
struct VertexOut {
float4 fragmentPosition [[position]];
float height;
};
vertex VertexOut myVertex(
VertexIn in [[stage_in]],
constant SCNSceneBuffer& scn_frame [[buffer(0)]],
constant commonprofile_node& scn_node [[buffer(1)]]
) {
VertexOut out;
float4 position = float4(in.position, 1.f);
out.fragmentPosition = scn_frame.viewProjectionTransform * scn_node.modelTransform * position;
// store world position for fragment shading
out.height = (scn_node.modelTransform * position).y;
return out;
}
fragment half4 myFragment(VertexOut in [[stage_in]]) {
return half4(in.height);
}
let dictionary: [String: Any] = [
"passes" : [
"y" : [
"draw" : "DRAW_SCENE",
"inputs" : [:],
"outputs" : [
"color" : "COLOR"
],
"metalVertexShader": "myVertex",
"metalFragmentShader": "myFragment",
]
],
"sequence" : ["y"],
"symbols" : [:]
]
let technique = SCNTechnique(dictionary: dictionary)
scnView.technique = technique
You could combine this render pass with other passes (see SCNTechnique).