I'm studying app development with swift and metal kit
I want to convert an int type array to Metalbuffer, because I have to cover that array data in Shader.h
Before operating the shader.h, the array's values have not problems.
But, in and after the shader.h, the buffer's that converted from array have some problems.
for example, I send [1, 2, 3, 4] to Metalbuffer and I call the renderEncoder.setVertexBuffer(Metalbuffer).
then, In the Shader.h it appears like [1, 0, 2, 0]
what's the problem???
this is my sample code
var int_array = Array(reapeating: 1, count: 100)
init(){
Buffers = MetalBuffer<Int>(device: device, array: int_array, index: kBuffers.rawValue, options: [])
}
func updateIntArray(){
for i in 0..<100 {
int_array[i] = i % 20
}
Buffers = MetalBuffer<Int>(device: device, array: int_array, index: kBuffers.rawValue, options: [])
(other codes about rendering)
renderEncoder.setVertexBuffer(Buffers)
}
the updateIntArray is called per every frame by view controller.
I printed the sizeof(Buffers[i]) after the shader call, it was 4!
How can I preserve the data without being inserted with 0?
Thank you!
There's a lot of code missing from your question, but I will show you how I pass an array as a parameter to my fragment shader.
I have a ShaderTypes.h file to create a custom value type that can be used with both Swift and MSL (Metal Shading Language). You will need a bridging header for this if you want to use it on the Swift side.
#ifndef ShaderTypes_h
#define ShaderTypes_h
#include <simd/simd.h>
typedef struct {
float intensity;
float time;
float width;
float height;
float originX;
float originY;
}KeyPointValues;
My goal is to pass an array of KeyPointValues as a parameter to my fragment shader. On the swift side I do something like this:
var keyPoints = [KeyPointValues()]
for index in 0...10 {
keyPoints.append(KeyPointValues())
}
var count = keyPoints.count
guard let keyPointBuffer = context.device.makeBuffer(bytes: keyPoints, length: MemoryLayout<KeyPointValues>.stride * count) else { return }
commandEncoder.setFragmentBuffer(keyPointBuffer, offset: 0, index: 0)
commandEncoder.setFragmentBytes(&count, length: MemoryLayout.size(ofValue: count), index: 1)
You need to pass in count as well because there is no keyPoints.count counterpart in MSL.
Then the fragment shader looks something like this:
#include <metal_stdlib>
#include <simd/simd.h>
#import "ShaderTypes.h"
using namespace metal;
fragment half4 rosyFilter(device const KeyPointValues *keyPoints [[buffer(0)]],
constant uint &count [[ buffer(1) ]]
) {
for(uint index = 0; index < count; index++) {
float intensity = keyPoints[index].intensity;
}
}
Hopefully this can get you started in the right direction. You are using an array of Int, so it should be easier as there is probably no need to define a custom struct to use between Swift and MSL.
I am also somewhat new to Metal so I'm not sure if this is the best way of doing things. I appreciate any feedback from people with more experience.
Related
I've generated C code from MATLAB by using the codegen tool. The function can be described as follows:
function [result] = calculate_data(my_matrix)
for idx = 1:length(my_matrix)
result = result + sum(my_matrix(idx,1:3));
end
end
When using the codegen tool, I explicitly stated that my_matrix is a type of double(:inf, 3). In other words, the number of rows is unbounded, but it will have 3 columns. When the code is generated, this is the function that is generated that I am to execute:
calculate_data(my_matrix : UnsafePointer<emxArray_real_T>!, result : UnsafeMutablePointer<emxArray_real_T>!)
emxArray_real_T is defined as follows in a different c file:
struct emxArray_real_T
{
double *data;
int *size;
int allocatedSize;
int numDimensions;
boolean_T canFreeData;
};
When I see my initialization options for the above class, this one in particular makes sense:
emxArray_real_T.init(data: UnsafeMutablePointer<Double>!, size: UnsafeMutablePointer<Int32>!, allocatedSize: Int32, numDimensions: Int32, canFreeData: boolean_T)
I've tried to follow this document as a means to wrap my head around how to call the generated C code, but I think I might be missing a basic step. Here is what I am doing:
// create an 2d array with some fake data
var mySampleData = [[Double]]();
for i in 0 ..< 3 {
mySampleData.append([1.1, 2.2, 3.3]);
}
// begin fulfilling requirements for emxArray_real_T
var data_pointer = UnsafeMutablePointer<Double>.allocate(capacity: 3);
data_pointer.initialize(from: mySampleData)
However, the above code throws an error stating that:
Generic parameter 'C' could not be inferred
I take it that I am then doing something completely wrong, and am probably on an incorrect path. There is a similar post that relates to my question, How to convert float[][] type array to "emxArray_real_T *x" , however the provided solution seems to be for C, as opposed to for Swift 4. How can I effectively call a C function using Swift 4, and meet the requirements of the emxArray_real_T.init method? Using fake data is ok to demonstrate the basic principle.
In a simple Xcode project with mocked C constructs for the struct emxArray_real_T and the function calculate_data I can run the following code successfully. To create an object of type emxArray_real_T I do
var data: [Double] = (0 ..< 12).map(Double.init)
var size: [Int32] = [4, 3]
var array = emxArray_real_T(
data: &data,
size: &size,
allocatedSize: 12,
numDimensions: 2,
canFreeData: false
)
This object can be passed to the function calculate_data like calculate_data(&array, nil). In a real application nil would be another array object. For the sake of simplicity it is just used as a placeholder here.
Your second issue can be solved by using the right types ([Double] instead of Double in line 6):
var mySampleData = [[Double]]();
for i in 0 ..< 3 {
mySampleData.append([i*1, i*2, i*3].map(Double.init));
}
let pointer = UnsafeMutablePointer<[Double]>.allocate(capacity: 3)
pointer.initialize(from: mySampleData, count: 3)
print((pointer + 0).pointee)
print((pointer + 1).pointee)
print((pointer + 2).pointee)
pointer.deallocate()
The output will be
[0.0, 0.0, 0.0]
[1.0, 2.0, 3.0]
[2.0, 4.0, 6.0]
as expected.
I have to admit that I used Swift 5.0.1. This should not make significant differences, though.
I have an Objective-C class that populates a std:vector with millions of points. The structure of the vector is:
typedef std::vector<CGPoint> CGContour;
typedef std::vector<CGContour> CGContours;
So a CGContour is a vector of CGPoints and CGContours is a vector of the CGContour vector.
I need to access this data in a Swift class somehow. I don't want to use an NSArray because it has a huge overhead compared to using vector (it is like 10x as big and slow).
What would be the most efficient way to get millions of CGPoints accessible in Swift from my Objective-C class?
Edit:
I am populating my CGContours vector like this:
contourVector = CGContours(contours.size());
populatedContourNum = 0
//contours is OpenCV's contours
for( long c = 0; c < contours.size(); c++) {
if (populatedContourNum >= contourVector.size()) {
contourVector.resize(contourVector.size() + 1);
}
contourVector[populatedContourNum] = CGContour(contours[c].size());
for( long pointNum = 0; pointNum < contours[c].size(); pointNum++ )
{
contourVector[populatedContourNum][pointNum] = CGPointMake(contours[c][pointNum].x * scale,
contours[c][pointNum].y * scale);
}
populatedContourNum++;
}
Some parts are not clear enough but I will try to show you some example.
First of all, you need to prepare a class which can access your contourVector. (I cannot see if it is an instance field or a global variable, if it is an instance field, you may use the existing class.)
Create a header for the prepared class, again you may utilize the existing header, but this header needs to be compiled both in C-context and in C++ context. So, if your existing header contains some declaration which cannot be compiled in C-context, you may need separated two headers or some #ifs.
#import <Foundation/Foundation.h>
NS_ASSUME_NONNULL_BEGIN
#interface YourClass : NSObject
- (NSInteger)contoursSize;
- (NSInteger)contourSizeAtIndex:(NSInteger)index;
- (CGPoint *)contourAtIndex:(NSInteger)index;
//...
#end
NS_ASSUME_NONNULL_END
Then add 3 methods to the class specified in the header:
#import "YourClass.h"
#import <vector>
typedef std::vector<CGPoint> CGContour;
typedef std::vector<CGContour> CGContours;
static CGContours contourVector;
#implementation YourClass
- (NSInteger)contoursSize {
return contourVector.size();
}
- (NSInteger)contourSizeAtIndex:(NSInteger)index {
return contourVector[index].size();
}
- (CGPoint *)contourAtIndex:(NSInteger)index {
return contourVector[index].data();
}
#end
Please do not forget to include the header inside your Project-Bridging-Header.h:
//
// Use this file to import your target's public headers that you would like to expose to Swift.
//
#import "YourClass.h"
You need to create a Swift side wrapper class, as you cannot create UnsafeBufferPointer in Objective-C.
class YourClassWrapper {
let yourInstance = YourClass()
var count: Int {
return yourInstance.contoursSize()
}
subscript(index: Int) -> UnsafeBufferPointer<CGPoint> {
guard 0..<count ~= index else {fatalError("Index \(index) out of bounds \(0..<count)")}
let start = yourInstance.contour(at: index)
let count = yourInstance.contourSize(at: index)
return UnsafeBufferPointer(start: start, count: count)
}
}
With these preparations above, you can access each CGPoint as:
let wrapper = YourClassWrapper()
let point = wrapper[0][1]
Or you can get the pointer to the first element in CGContour as:
let ptr = wrapper[0].baseAddress!
You may need to modify some parts to fit this into your actual code. Hope you can make it.
I have a block that is passing data in that I'd like to convert to an array of array of floats -- e.g. [[0.1,0.2,0.3, 1.0], [0.3, 0.4, 0.5, 1.0], [0.5, 0.6, 0.7, 1.0]]. This data is passed to me in the form of data:UnsafeMutablePointer<UnsafeMutableRawPointer> (The inner arrays are RGBA values)
fwiw -- the block parameters are from SCNParticleEventBlock
How can I dereference data into a [[Float]]? Once I have the array containing the inner arrays, I can reference the inner array (colorArray) data with:
let rgba: UnsafeMutablePointer<Float> = UnsafeMutablePointer(mutating: colorArray)
let count = 4
for i in 0..<count {
print((rgba+i).pointee)
}
fwiw -- this is Apple's example Objective-C code for referencing the data (from SCNParticleSystem handle(_:forProperties:handler:) )
[system handleEvent:SCNParticleEventBirth
forProperties:#[SCNParticlePropertyColor]
withBlock:^(void **data, size_t *dataStride, uint32_t *indices , NSInteger count) {
for (NSInteger i = 0; i < count; ++i) {
float *color = (float *)((char *)data[0] + dataStride[0] * i);
if (rand() & 0x1) { // Switch the green and red color components.
color[0] = color[1];
color[1] = 0;
}
}
}];
You can actually subscript the typed UnsafeMutablePointer without having to create an UnsafeMutableBufferPointer, as in:
let colorsPointer:UnsafeMutableRawPointer = data[0] + dataStride[0] * i
let rgbaBuffer = colorsPointer.bindMemory(to: Float.self, capacity: dataStride[0])
if(arc4random_uniform(2) == 1) {
rgbaBuffer[0] = rgbaBuffer[1]
rgbaBuffer[1] = 0
}
Were you ever able to get your solution to work? It appears only a handful of SCNParticleProperties can be used within an SCNParticleEventBlock block.
Based on this answer, I've written the particle system handler function in swift as:
ps.handle(SCNParticleEvent.birth, forProperties [SCNParticleSystem.ParticleProperty.color]) {
(data:UnsafeMutablePointer<UnsafeMutableRawPointer>, dataStride:UnsafeMutablePointer<Int>, indicies:UnsafeMutablePointer<UInt32>?, count:Int) in
for i in 0..<count {
// get an UnsafeMutableRawPointer to the i-th rgba element in the data
let colorsPointer:UnsafeMutableRawPointer = data[0] + dataStride[0] * i
// convert the UnsafeMutableRawPointer to a typed pointer by binding it to a type:
let floatPtr = colorsPointer.bindMemory(to: Float.self, capacity: dataStride[0])
// convert that to a an UnsafeMutableBufferPointer
var rgbaBuffer = UnsafeMutableBufferPointer(start: floatPtr, count: dataStride[0])
// At this point, I could convert the buffer to an Array, but doing so copies the data into the array and any changes made in the array are not reflected in the original data. UnsafeMutableBufferPointer are subscriptable, nice.
//var rgbaArray = Array(rgbaBuffer)
// about half the time, mess with the red and green components
if(arc4random_uniform(2) == 1) {
rgbaBuffer[0] = rgbaBuffer[1]
rgbaBuffer[1] = 0
}
}
}
I'm really not certain if this is the most direct way to go about this and seems rather cumbersome compared to the objective-C code (see above question). I'm certainly open to other solutions and/or comments on this solution.
I'd like to use a Metal compute shader to calculate some positions that are then fed into a Metal shader. Sounds straight forward, but I'm having trouble getting my MTLBuffer data into the Metal based SCNProgram.
The compute kernel is as follows, in this contrived example it's taking in three 3D vectors (in both buffers).
kernel void doSimple(const device float3 *inVector [[ buffer(0) ]],
device float3 *outVector [[ buffer(1) ]],
uint id [[ thread_position_in_grid ]]) {
float yDisplacement = 0;
. . . .
outVector[id] = float3(
inVector[id].x,
inVector[id].y + yDisplacement,
inVector[id].z);
}
This kernel function is run each frame in the - renderer:willRenderScene:atTime: method of my SCNSceneRendererDelegate. There's two buffers, and they get switched after each frame.
Buffers are created as follows;
func setupBuffers() {
positions = [vector_float3(0,0,0), vector_float3(1,0,0), vector_float3(2,0,0)]
let bufferSize = sizeof(vector_float3) * positions.count
//copy same data into two different buffers for initialisation
buffer1 = device.newBufferWithBytes(&positions, length: bufferSize, options: .OptionCPUCacheModeDefault)
buffer2 = device.newBufferWithBytes(&positions, length: bufferSize, options: .OptionCPUCacheModeDefault)
}
And the compute shader is run using the following (in the willRenderScene func);
let computeCommandBuffer = commandQueue.commandBuffer()
let computeCommandEncoder = computeCommandBuffer.computeCommandEncoder()
computeCommandEncoder.setComputePipelineState(pipelineState)
computeCommandEncoder.setBuffer(buffer1, offset: 0, atIndex: 0)
computeCommandEncoder.setBuffer(buffer2, offset: 0, atIndex: 1)
computeCommandEncoder.dispatchThreadgroups(numThreadgroups, threadsPerThreadgroup: threadsPerGroup)
computeCommandEncoder.endEncoding()
computeCommandBuffer.commit()
computeCommandBuffer.waitUntilCompleted()
let bufferSize = positions.count*sizeof(vector_float3)
var data = NSData(bytesNoCopy: buffer2.contents(), length: bufferSize, freeWhenDone: false)
var resultArray = [vector_float3](count: positions.count, repeatedValue: vector_float3(0,0,0))
data.getBytes(&resultArray, length:bufferSize)
for outPos in resultArray {
print(outPos.x, ", ", outPos.y, ", ", outPos.z)
}
This works, and I can see my compute shader is updating the y coordinate for each vector in the array.
This scene consists of three spheres evenly spaced. The vertex shader simply takes the position calculated in the compute shader and adds it to each vertex position (well the y component anyway). I give each sphere an index, the vertex shader uses this index to pull the appropriate position out of my computed array.
The Metal vertex function is shown below, it's referenced by a SCNProgram and set to the material of each sphere.
vertex SimpleVertex simpleVertex(SimpleVertexInput in [[ stage_in ]],
constant SCNSceneBuffer& scn_frame [[buffer(0)]],
constant MyNodeBuffer& scn_node [[buffer(1)]],
constant MyPositions &myPos [[buffer(2)]],
constant uint &index [[buffer(3)]]
)
{
SimpleVertex vert;
float3 posOffset = myPos.positions[index];
float3 pos = float3(in.position.x,
in.position.y + posOffset.y,
in.position.z);
vert.position = scn_node.modelViewProjectionTransform * float4(pos,1.0);
return vert;
}
MyPositions is a simple struct containing an array of float3s.
struct MyPositions
{
float3 positions[3];
};
I have no problem passing data to the vertex shader using the setValue method of each sphere's material as shown below (also done in the willRenderScene method). Everything works as expected (the three spheres move upwards).
var i0:UInt32 = 0
let index0 = NSData(bytes: &i0, length: sizeof(UInt32))
sphere1Mat.setValue(index0, forKey: "index")
sphere1Mat.setValue(data, forKey: "myPos")
BUT this requires the data be copied from the GPU to CPU to GPU and is really something I'd rather avoid. So my question is... How do I pass a MTLBuffer to a SCNProgram?
Have tried the following in willRenderScene but get nothing but EXEC_BAD...
let renderCommandEncoder = renderer.currentRenderCommandEncoder!
renderCommandEncoder.setVertexBuffer(buffer2, offset: 0, atIndex: 2)
renderCommandEncoder.endEncoding()
Complete example is over on GitHub.
Thanks for reading, been struggling with this one. Workaround is to use a MTLTexture in place of a MTLBuffer as I've been able to pass these into an SCNProgram via the diffuse mat prop.
just switch the bindings of the buffers from step to step.
step1
computeCommandEncoder.setBuffer(buffer1, offset: 0, atIndex: 0)
computeCommandEncoder.setBuffer(buffer2, offset: 0, atIndex: 1)
step2
computeCommandEncoder.setBuffer(buffer1, offset: 0, atIndex: 1)
computeCommandEncoder.setBuffer(buffer2, offset: 0, atIndex: 0)
step3
computeCommandEncoder.setBuffer(buffer1, offset: 0, atIndex: 0)
computeCommandEncoder.setBuffer(buffer2, offset: 0, atIndex: 1)
and so on ...
the out buffer becomes the new in buffer and vice versa ...
For communicating with a BLE characteristic, I have a Swift struct that looks like:
struct Packet {
var control1:UInt8 = 0
var control2:UInt8 = 0
var payload:(UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8,UInt8) = (0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
init(control1:UInt8, control2:UInt8) {
self.control1 = control1
self.control2 = control2
}
}
I have payload defined as a tuple, because that seems to be the only way to have an array (of bytes in this case) of fixed size embedded in a Swift struct. Verbose, but whatever.
I have a big ol' source:[UInt8] that I want to pull swatches of into that Packet struct, so I can send them via BLE to the remote device. When I do:
var packet = Packet(control1: self.pageIndex, control2: sentenceIndex)
let offset = (Int(self.pageIndex) * self.pageSize) + (Int(sentenceIndex) * self.sentenceSize)
let limit = offset + self.sentenceSize
packet.payload = self.source[offset..<limit]
For the last line, I get the rather confusing error:
Cannot subscript a value of type '[UInt8]'
Cryptic I say, because it actually can. If I take the assignment to the packet.payload out, it has no problem subscripting the value.
What I'm really interested in at a higher level, is how one puts together a struct with a fixed size array of bytes, and then copies swatches of a large buffer into those. I would like to both understand the above, as well as know how to solve my problem.
UPDATE:
I ended up backing up a little, influenced by both answers below, and rethinking. My main driving force was that I wanted a simple/clever way to have convert a struct with an internal array to/from NSData, primary in BLE communications. What I ended up doing was:
struct Packet {
var pageIndex:UInt8 = 0
var sentenceIndex:UInt8 = 0
var payload:ArraySlice<UInt8> = []
var nsdata:NSData {
let bytes:[UInt8] = [self.pageIndex, self.sentenceIndex] + self.payload
return NSData(bytes: bytes, length: bytes.count)
}
}
Not the most efficient because I have to create the intermediate [UInt8] array, but I decided that a simple way to convert didn't exist, that I'd have to do things with as conversions or memcpy and friends.
I'm not sure which of the two below to mark as an answer, since both influenced what I ended up with.
There are two ugly/simple solutions:
To assign each member of the tuple separately:
var offset = ...
packet.payload = (source[offset++], source[offset++], ... , source[offset++])
To just copy the raw memory (recommended)
var values = Array(source[offset..<limit])
memcpy(&packet.payload, &values, sentenceSize)
Note that it's possible to create an array from a tuple:
func tupleToArray<T>(tuple: Any, t: T.Type) -> [T] {
return Mirror(reflecting: tuple).children.flatMap{ $0.value as? T }
}
tupleToArray((1, 2, 3, 4, 5), t: Int.self) // [1, 2, 3, 4, 5]
But the other way around doesn't work, as Swift's reflection is read-only.
Another much more complicated but more beautiful solution would be to use Dependent Types, which enables you to have arrays with compile-time known length. Check out this great blog post, in which he also mentions this post on the Apple Developer forums which is basically what you'd need:
let vector = 3.0 ⋮ 4.0 ⋮ 5.0 // [3.0, 4.0, 5.0]
vector[1] // 4.0
vector.count // 3
sizeofValue(vector) // 3 * 8 ( same size as a tuple with 3 elements)
First of all don't use tuples to create contiguous arrays of memory. Go ahead and use the [UInt8] type. I would recommend using a stride function to create your indices for you like this. You will have to handle the case of your data source not being a multiple of the Packet payload size.
struct Packet {
var control1: UInt8 = 0
var control2: UInt8 = 0
static let size = 16
var payload = [UInt8].init(count: Packet.size, repeatedValue: 0)
init(control1: UInt8, control2: UInt8) {
self.control1 = control1
self.control2 = control2
}
}
// random values between 0...255
let blob = (0..<(Packet.size * 3)).map{_ in UInt8(arc4random_uniform(UInt32(UInt8.max)))}
for index in 0.stride(through: blob.count - 1, by: Packet.size) {
var packet = Packet(control1: 4, control2: 5)
packet.payload[0..<Packet.size] = blob[index..<index + Packet.size]
print(packet.payload)
}
As far as the cannot subscript error, I encountered that too. I suspect that this has changed recently. I was able to eliminate the error by matching the packet indice slice with the data source slice.
UPDATE
A commenter correctly pointed out that Packet structure contained a reference to an Array and therefore did not meet the OP's need. While I was focused more on iterating through a large data source using stride, here is an alternative using an untyped [UInt8] for such a simple data structure.
// payload size in count of UInt8
let size = 16
// field offsets
let control1 = 0
let control2 = 1
let payload = 2..<(2 + size)
// random values between 0...255
let blob = (0..<size * 3).map{_ in UInt8(arc4random_uniform(UInt32(UInt8.max)))}
for index in 0.stride(through: blob.count - 1, by: size) {
var buffer = [UInt8](count: 2 + size, repeatedValue: 0)
buffer[control1] = 255
buffer[control2] = 0
buffer[payload] = blob[index..<index + size]
let data = NSData(bytesNoCopy: &buffer, length: buffer.count, freeWhenDone: false)
// send data
}