Related
I have a custom model which takes in cropped faces from BlazeFace Model then outputs a prediction of 3 classes.
Before sending them to my custom model I resize the cropped faces to be of shape [1,224,224,3]
Output at every prediction:
Float32Array [
6.522771905936864e-11,
3.698188456857654e-12,
1,
]
Code for resizing the cropped faces and making predictions:
const getPrediction = async tensor => {
if (!tensor) {
console.log("Tensor not found!");
}
// Load both models
const bfModel = await blazeFaceModel;
const returnTensors = true;
const faces = await bfModel
.estimateFaces(tensor, returnTensors)
.catch(e => console.log(e));
// Reshape tensor from rank 3 to rank 4
const tensorReshaped = tensor.reshape([1, 224, 224, 3]);
const scale = {
height: styles.camera.height / tensorDims.height,
width: styles.camera.width / tensorDims.width
};
// Faces is an array of objects
if (!isEmpty(faces)) {
// Setting faces in state
setModelFaces({ faces });
}
//Looping over array of objects in faces
faces.map((face, i) => {
const { topLeft, bottomRight } = face;
const width = Math.floor(
bottomRight.dataSync()[0] - topLeft.dataSync()[0] * scale.width
);
const height = Math.floor(
bottomRight.dataSync()[1] - topLeft.dataSync()[1] * scale.height
);
const boxes = tf
.concat([topLeft.dataSync(), bottomRight.dataSync()])
.reshape([-1, 4]);
// Cropping out faces from original tensor
const crop = tf.image.cropAndResize(
tensorReshaped,
boxes,
[0],
[height, width]
);
// Resize cropped faces to [1,224,224,3]
const alignCorners = true;
const imageResize = tf.image.resizeBilinear(
crop,
[224, 224],
alignCorners
);
makePrediction(imageResize);
});
};
// Make predictions on the tensors
const makePrediction = async image => {
if (!image) {
console.log("No input!");
}
const model = await loadedModel;
const prediction = await model.predict(image, { batchSize: 1 });
if (!prediction || isEmpty(prediction)) {
console.log("Prediction not available");
}
console.log(prediction);
console.log(prediction.dataSync());
};
EDIT
I tried changing the batch size when making predictions to 1 and still the same issue
I tried reconverting the keras model to tfjs format and still the same issue
I tried disposing of the tensor after making a prediction but still there is an error
So i printed out the tensors of the resized faces and its a lot of 0's
Tensor before prediction
Tensor
[[[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
...
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]]]
undefined
Tensor during prediction
Tensor
[[[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
...
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]]]
undefined
boxes of tf.image.cropAndResize are normalized coordinates between 0 and 1. Therefore topLeft and bottomRight should be normalized by using [imageWidth, imageHeight]
normalizedTopLeft = topLeft.div(tensor.shape.slice(-3, -2))
// slice will get [h, w] of a tensor of shape [b, h, w, ch] or [h, w, ch]
// do likewise for bottomRight
// use normalizedTopLeft instead of topLeft for cropping
I have two identical arrays of size (2,3,2)
[[[1, 7],
[2, 8],
[3, 9]],
[[4, 10],
[5, 11],
[6, 12]]]
stored in a multidimensional array
>>> a = np.array([[[[ 1, 1],
[ 7, 7]],
[[ 2, 2],
[ 8, 8]],
[[ 3, 3],
[ 9, 9]]],
[[[ 4, 4],
[10, 10]],
[[ 5, 5],
[11, 11]],
[[ 6, 6],
[12, 12]]]])
>>> a.shape
(2, 3, 2, 2)
I'm trying to mask the sub arrays in a with m:
>>> m = np.array([[[1, 0],
[0, 0],
[1, 1]],
[[1, 1],
[0, 1],
[0, 0]]])
which should result in:
[[[[ 1, 1],
[ 0, 0]
[ 0, 0],
[ 0, 0],
[ 3, 3],
[ 9, 9]]
[[ 4, 4],
[10, 10],
[ 0, 0],
[11, 11],
[ 0, 0],
[ 0, 0]]]
I tried to use np.concatenate and np.append eg, np.prod(np.concatenate([a,m],axis=0))
but none of my solutions worked.
Expand the dimensions of m so it broadcasts with a:
In [183]: a*m[...,None]
Out[183]:
array([[[[ 1, 1],
[ 0, 0]],
[[ 0, 0],
[ 0, 0]],
[[ 3, 3],
[ 9, 9]]],
[[[ 4, 4],
[10, 10]],
[[ 0, 0],
[11, 11]],
[[ 0, 0],
[ 0, 0]]]])
I want to initialise a three dimensional ruby array. For a two-dimensional array, I can do
a = Array.new(4){ Array.new(5, 0) }
so I have tried
a = Array.new(4) { Array.new(5, Array.new(6, 0)) }
but if I do a[1][2][3] = 5, it not only sets that element to 5, it sets the corresponding element to 5 in other sub arrays i.e.
[[[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]],
[[0, 0, 0, 5, 0, 0], [0, 0, 0, 5, 0, 0], [0, 0, 0, 5, 0, 0], [0, 0, 0, 5, 0, 0], [0, 0, 0, 5, 0, 0]],
[[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]]
The issue is that if you pass the value as an argument, you will get an array where each value refers to that same object. You do exactly that with:
Array.new(5, Array.new(6, 0))
You can use Array#*, i.e array multiplication instead:
[[[0] * 6] * 5] * 4
Just use the same syntax (with a block instead of an argument : Array.new(n){...}) for all the dimensions.
a = Array.new(4) { Array.new(5) { Array.new(6, 0) } }
This way, your code will create 20 distincts Array.new(6, 0) sub-sub-arrays instead of replicating the same object 5 times.
require 'pp'
a = Array.new(4) { Array.new(5) { Array.new(6, 0) } }
a[1][2][3] = 5
pp a
# [[[0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0]],
# [[0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 5, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0]],
# [[0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0]],
# [[0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0],
# [0, 0, 0, 0, 0, 0]]]
The following worked properly.
a=Array.new(4){ Array.new(5) { Array.new(6,0) }}
You can use recursion to initialize the elements of a multidimensional array of any dimension.1
Code
def init(n, *rest)
rest.empty? ? Array.new(n, 0) : Array.new(n) { init(*rest) }
end
Examples
dims = [2, 3, 2]
a = init(*dims)
#=> [[[0, 0], [0, 0], [0, 0]], [[0, 0], [0, 0], [0, 0]]]
a[0][0][0] = 1
a #=> [[[1, 0], [0, 0], [0, 0]], [[0, 0], [0, 0], [0, 0]]]
dims = [2, 3, 2, 4, 2, 3]
a = init(*dims)
a.flatten.size
#=> 288 (= 2*3*2*4*2*3)
a[0][0][0][0][0][0] = 1
a.flatten.count(1)
#=> 1
Explanation
The steps are as follows.
Suppose, as in the example,
dims = [2, 3, 2]
Then
n, *rest = dims
#=> [2, 3, 2]
n #=> 2
rest
#=> [3, 2]
As rest.empty? #=> false, the method returns
Array.new(2) { init(*[3, 2]) } # (1)
#=> [[[0, 0], [0, 0], [0, 0]], [[0, 0], [0, 0], [0, 0]]]
To obtain this result, we execute the block
{ init(*[3, 2]) }
for each of the two elements of the array being constructed. This requires us to compute init([3, 2]) (same as { init(3, 2) }), which is done as follows.
n, *rest = [3, 2]
#=> [3, 2]
n #=> 3
rest
#=> [2]
Again, as rest.empty? #=> false, we return
Array.new(3) { init(*[2]) } # (2)
#=> [[0, 0], [0, 0], [0, 0]]
to expression (1) above. For each of the 3 elements of this array we execute the block
{ init(*[2]) }
This requires us to compute init([2]), which is done as follows.
n, *rest = [2]
#=> [2]
n #=> 2
rest
#=> []
This time, since rest.empty? #=> true, we compute and return
Array.new(2, 0)
#=> [0, 0]
to expression (2) above.
1 Yes, there's no such thing as a "multidimensional array" in Ruby, but I expect readers to know what I mean. The alternative is something like "...of an array containing nested arrays of equal size with any number of levels of nesting", but even that is incomplete as it fails to exclude, for example, [[1, [2,3]], [4,5]]. It would be helpful if Ruby had a name for such objects.
I'm trying to teach simple single neuron perceptron to recognize repetitive sequences of 1.
here is data I use to teach it:
learning_signals = [
[[1, 1, 0, 0], 1],
[[1, 1, 0, 1], 1],
[[1, 1, 1, 0], 1],
[[0, 1, 1, 0], 1],
[[0, 1, 1, 1], 1],
[[0, 0, 1, 1], 1],
[[1, 0, 1, 1], 1],
[[0, 0, 0, 0], 0],
[[1, 0, 0, 0], 0],
[[0, 1, 0, 0], 0],
[[0, 0, 1, 0], 0],
[[0, 0, 0, 1], 0],
[[1, 0, 1, 0], 0],
[[1, 0, 0, 1], 0],
# [[0, 1, 0, 1], 0],
This is the array of learning templates each of them are array of data and correct result for that data.
As you see. last row commented - if I do uncomment it - perceptron will fail to learn. without it perceptron does not work right in case with "0101". So the question is:
Is this task can be solved with single neuron perceptron or should I use few layered perceptron?
How can I determine which tasks can be solved with such a simple perceptron? Are there any rule that I can apply to my task and say that it could be done with simple perceptron?
here is the code of perceprton written in coffeescript:
class window.Perceptron
weights: []
calc: (signal) ->
#neuron.calc signal
adjust: ->
foo: 0.1
calc: (signal) ->
sum = 0
for s, i in signal
sum += s*#weights[i]
if sum>0.5 then return 1 else return 0
sum
learn: (templates) ->
#weights = []
for i in [1..templates[0][0].length]
#weights.push Math.random()
li = 0
max_li = 50000
console.log #weights
while true
gerror = 0
li++
for template, i in templates
res = #calc template[0]
# console.log "result: #{res}"
error = template[1] - res
gerror += Math.abs error
for weight, i in #weights
#weights[i] += #foo*error*template[0][i]
if ((gerror == 0) || li > max_li) then break
if gerror == 0
console.log "Learned in #{li} iterations"
else
console.log "Learning failed after #{max_li} iterations"
console.log #weights
I was trying this simple line of assigning codes to a structured array in numpy, I am not quiet sure, but something wrong happens when I assign a matrix to a sub_array in a structured array I created as follows:
new_type = np.dtype('a3,(2,2)u2')
x = np.zeros(5,dtype=new_type)
x[1]['f1'] = np.array([[1,1],[1,1]])
print x
Out[143]:
array([('', [[0, 0], [0, 0]]), ('', [[1, 0], [0, 0]]),
('', [[0, 0], [0, 0]]), ('', [[0, 0], [0, 0]]),
('', [[0, 0], [0, 0]])],
dtype=[('f0', '|S3'), ('f1', '<u2', (2, 2))])
Shouldn't the second field of the subarray equals at this stage
[[1,1],[1,1]]
I think you want to set things slightly differently. Try:
x['f1'][1] = np.array([[1,1],[1,1]])
which results in:
In [43]: x = np.zeros(5,dtype=new_type)
In [44]: x['f1'][1] = np.array([[1,1],[1,1]])
In [45]: x
Out[45]:
array([('', [[0, 0], [0, 0]]), ('', [[1, 1], [1, 1]]),
('', [[0, 0], [0, 0]]), ('', [[0, 0], [0, 0]]),
('', [[0, 0], [0, 0]])],
dtype=[('f0', '|S3'), ('f1', '<u2', (2, 2))])
This is not to say that this isn't strange behavior though since both x['f1'][1] and x[1]['f1'] print the same results, but clearly are different:
In [51]: x['f1'][1]
Out[51]:
array([[1, 1],
[1, 1]], dtype=uint16)
In [52]: x[1]['f1']
Out[52]:
array([[1, 1],
[1, 1]], dtype=uint16)
In [53]: x[1]['f1'] = 2
In [54]: x
Out[54]:
array([('', [[0, 0], [0, 0]]), ('', [[2, 1], [1, 1]]),
('', [[0, 0], [0, 0]]), ('', [[0, 0], [0, 0]]),
('', [[0, 0], [0, 0]])],
dtype=[('f0', '|S3'), ('f1', '<u2', (2, 2))])
In [55]: x['f1'][1] = 3
In [56]: x
Out[56]:
array([('', [[0, 0], [0, 0]]), ('', [[3, 3], [3, 3]]),
('', [[0, 0], [0, 0]]), ('', [[0, 0], [0, 0]]),
('', [[0, 0], [0, 0]])],
dtype=[('f0', '|S3'), ('f1', '<u2', (2, 2))])
I'd have to think about it a bit more to figure out exactly what is going on.