Related
I actually work on some matrices and when I inverse it, the inverse is wrong. I use np.linalg.inv() to inverse my square matrix.
My example his:
import numpy as np
M = np.array([[9.0, 5.329070518200756e-17, -7.199999999999999, -5.3999999999999995, 0.0, 0.0], [5.329070518200756e-17, 9.0, 5.3999999999999995, -7.199999999999999, 0.0, 0.0], [-7.199999999999999, 5.3999999999999995, 11.422222222222222, 1.6653345369377363e-17, -2.4222222222222225, -7.266666666666667], [-5.3999999999999995, -7.199999999999999, 1.6653345369377363e-17, 11.422222222222222, 7.266666666666667, -2.4222222222222225], [0.0, 0.0, -2.4222222222222225, 7.266666666666667, 24.22222222222222, 0.0], [0.0, 0.0, -7.266666666666667, -2.4222222222222225, 0.0, 24.22222222222222]])
M_inv = np.linalg.inv(M)
# It's not the Identity!
print( M_inv # M)
Do you have clues ?
Thank you for your answers!
Check numpy.linalg.inv because Rank defective matrices cannot be inverted, need to supply a square or it can also be a valid numpy.matrix instance, the linalg.inv method processes M = np.array([...]) numpy.linalg.inv(M) incorrectly.
I have two sets of data (x, y) corresponding to two 1-D histograms that are meant to be plotted next to each other as subplots. Both x and y values are different and hence they would be represented in different axes. The histogram heights (first item in hists) and the corresponding sequence of bins (second items in hists) are given for each subplot as the following:
*Please note that each height correspond to the bin in the sequence; heights are already known for each bin. I just want to put data in a bar format using hist function
array_1 = np.array([ 8.20198063, 8.30645018, 8.30829034, 8.63297701, 0., 0., 10.43478942])
array_random_1 = np.array([ 8.23460584, 8.31556503, 8.3090378, 8.63147021, 0., 0., 10.41481862])
array_2 = np.array([10.4348338, 8.69943553, 8.68710347, 6.67854038])
array_random_2 = np.array([10.41597028, 8.76635268, 8.19516216, 6.68126994])
bins_1, bins_2 = [8.0, 8.6, 9.2, 9.8, 10.4, 11.0, 11.6, 12.2], [0.0, 0.25, 0.5, 0.75, 1.0]
Here is my try to plot these two subplots using hist function from python:
fig, (ax1, ax2) = plt.subplots(1, 2, sharex=False, sharey=False, figsize=(12,3))
ax1.hist(array_1, bins_1, ec='blue', fc='none', lw=1.5, histtype='step', label='1')
ax1.hist(array_random_1, bins_1, ec='red', fc='none', lw=1.5, histtype='step', label='Random_1')
ax1.set_xlabel('X1')
ax1.set_ylabel('Y1')
ax2.hist(array_2, bins_2, ec='blue', fc='none', lw=1.5, histtype='step', label='2')
ax2.hist(array_random_2, bins_2, ec='red', fc='none', lw=1.5, histtype='step', label='Random_2')
ax2.set_xlabel('X2')
plt.show()
However, as you can see bars are not drawn to the correct height (blue bars are missing entirely) in left-side panel and everything is missing from the second panel. What is the issue in making these 1d histograms? Does this mean that I cannot use hist for my purpose?
What I want is the following which is doable using bar. How to do it using hist?
By what I understood.
In your code try replacing:
histtype='step'
with
histtype='bar'
As an exercice in pytorch framework (0.4.1) , I am trying to display the gradient of X (gX or dSdX) in a simple Linear layer (Z = X.W + B). To simplify my toy example, I backward() from a sum of Z (not a loss).
To sum up, I want gX(dSdX) of S=sum(XW+B).
The problem is that the gradient of Z (dSdZ) is None. As a result, gX is wrong too of course.
import torch
X = torch.tensor([[0.5, 0.3, 2.1], [0.2, 0.1, 1.1]], requires_grad=True)
W = torch.tensor([[2.1, 1.5], [-1.4, 0.5], [0.2, 1.1]])
B = torch.tensor([1.1, -0.3])
Z = torch.nn.functional.linear(X, weight=W.t(), bias=B)
S = torch.sum(Z)
S.backward()
print("Z:\n", Z)
print("gZ:\n", Z.grad)
print("gX:\n", X.grad)
Result:
Z:
tensor([[2.1500, 2.9100],
[1.6000, 1.2600]], grad_fn=<ThAddmmBackward>)
gZ:
None
gX:
tensor([[ 3.6000, -0.9000, 1.3000],
[ 3.6000, -0.9000, 1.3000]])
I have exactly the same result if I use nn.Module as below:
class Net1Linear(torch.nn.Module):
def __init__(self, wi, wo,W,B):
super(Net1Linear, self).__init__()
self.linear1 = torch.nn.Linear(wi, wo)
self.linear1.weight = torch.nn.Parameter(W.t())
self.linear1.bias = torch.nn.Parameter(B)
def forward(self, x):
return self.linear1(x)
net = Net1Linear(3,2,W,B)
Z = net(X)
S = torch.sum(Z)
S.backward()
print("Z:\n", Z)
print("gZ:\n", Z.grad)
print("gX:\n", X.grad)
First of all you only calculate gradients for tensors where you enable the gradient by setting the requires_grad to True.
So your output is just as one would expect. You get the gradient for X.
PyTorch does not save gradients of intermediate results for performance reasons. So you will just get the gradient for those tensors you set requires_grad to True.
However you can use register_hook to extract the intermediate grad during calculation or to save it manually. Here I just save it to the grad variable of tensor Z:
import torch
# function to extract grad
def set_grad(var):
def hook(grad):
var.grad = grad
return hook
X = torch.tensor([[0.5, 0.3, 2.1], [0.2, 0.1, 1.1]], requires_grad=True)
W = torch.tensor([[2.1, 1.5], [-1.4, 0.5], [0.2, 1.1]])
B = torch.tensor([1.1, -0.3])
Z = torch.nn.functional.linear(X, weight=W.t(), bias=B)
# register_hook for Z
Z.register_hook(set_grad(Z))
S = torch.sum(Z)
S.backward()
print("Z:\n", Z)
print("gZ:\n", Z.grad)
print("gX:\n", X.grad)
This will output:
Z:
tensor([[2.1500, 2.9100],
[1.6000, 1.2600]], grad_fn=<ThAddmmBackward>)
gZ:
tensor([[1., 1.],
[1., 1.]])
gX:
tensor([[ 3.6000, -0.9000, 1.3000],
[ 3.6000, -0.9000, 1.3000]])
Hope this helps!
Btw.: Normally you would want the gradient to be activated for your parameters - so your weights and biases. Because what you would do right now when using the optimizer, is altering your inputs X and not your weights W and bias B. So usually gradient is activated for W and B in such a case.
There's a much simpler way. Simply use retain_grad():
https://pytorch.org/docs/stable/autograd.html#torch.Tensor.retain_grad
Z.retain_grad()
before calling backward()
blue-phoenox, thanks for your answer. I am pretty happy to have heard about register_hook().
What led me to think that I had a wrong gX is that it was independant of the values of X. I will have to do the math to understand it. But using CCE Loss instead of SUM makes things much more clean. So I updated the example for those who might be interested. Using SUM was a bad idea in this case.
T_dec = torch.tensor([0, 1])
X = torch.tensor([[0.5, 0.8, 2.1], [0.7, 0.1, 1.1]], requires_grad=True)
W = torch.tensor([[2.7, 0.5], [-1.4, 0.5], [0.2, 1.1]])
B = torch.tensor([1.1, -0.3])
Z = torch.nn.functional.linear(X, weight=W.t(), bias=B)
print("Z:\n", Z)
L = torch.nn.CrossEntropyLoss()(Z,T_dec)
Z.register_hook(lambda gZ: print("gZ:\n",gZ))
L.backward()
print("gX:\n", X.grad)
Result:
Z:
tensor([[1.7500, 2.6600],
[3.0700, 1.3100]], grad_fn=<ThAddmmBackward>)
gZ:
tensor([[-0.3565, 0.3565],
[ 0.4266, -0.4266]])
gX:
tensor([[-0.7843, 0.6774, 0.3209],
[ 0.9385, -0.8105, -0.3839]])
I am trying to create time stamp arrays in Swift.
So, say I want to go from 0 to 4 seconds, I can use Array(0...4), which gives [0, 1, 2, 3, 4]
But how can I get [0.0, 0.5 1.0, 2.0, 2.5, 3.0, 3.5, 4.0]?
Essentially I want a flexible delta, such as 0.5, 0.05, etc.
You can use stride(from:through:by:):
let a = Array(stride(from: 0.0, through: 4.0, by: 0.5))
An alternative for non-constant increments (even more viable in Swift 3.1)
The stride(from:through:by:) functions as covered in #Alexander's answer is the fit for purpose solution where, but for the case where readers of this Q&A wants to construct a sequence (/collection) of non-constant increments (in which case the linear-sequence constructing stride(...) falls short), I'll also include another alternative.
For such scenarios, the sequence(first:next:) is a good method of choice; used to construct a lazy sequence that can be repeatedly queried for the next element.
E.g., constructing the first 5 ticks for a log10 scale (Double array)
let log10Seq = sequence(first: 1.0, next: { 10*$0 })
let arr = Array(log10Seq.prefix(5)) // [1.0, 10.0, 100.0, 1000.0, 10000.0]
Swift 3.1 is intended to be released in the spring of 2017, and with this (among lots of other things) comes the implementation of the following accepted Swift evolution proposal:
SE-0045: Add prefix(while:) and drop(while:) to the stdlib
prefix(while:) in combination with sequence(first:next) provides a neat tool for generating sequences with everything for simple next methods (such as imitating the simple behaviour of stride(...)) to more advanced ones. The stride(...) example of this question is a good minimal (very simple) example of such usage:
/* this we can do already in Swift 3.0 */
let delta = 0.05
let seq = sequence(first: 0.0, next: { $0 + delta})
/* 'prefix(while:)' soon available in Swift 3.1 */
let arr = Array(seq.prefix(while: { $0 <= 4.0 }))
// [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0]
// ...
for elem in sequence(first: 0.0, next: { $0 + delta})
.prefix(while: { $0 <= 4.0 }) {
// ...
}
Again, not in contest with stride(...) in the simple case of this Q, but very viable as soon as the useful but simple applications of stride(...) falls short, e.g. for a constructing non-linear sequences.
How can one create an array filled with values within a range (having a begin and end value) and a step? It should support begin and end values of float type.
For floats with custom stepping you can use Numeric#step like so:
-1.25.step(by: 0.5, to: 1.25).to_a
# => [-1.25, -0.75, -0.25, 0.25, 0.75, 1.25]
If you are looking on how to do this with integer values only, see this post or that post on how to create ranges and simply call .to_a at the end. Example:
(-1..1).step(0.5).to_a
# => [-1.0, -0.5, 0.0, 0.5, 1.0]