I am actually translating a matlab script into python an i have a problem using arrays in python (I am still a beginner) numpy.
My question is this:
In matlab I am computing the fourier transform of several signals and I am storing dynamically it in a 3 by 3 array say U. A simple example of what i want to do is as follows;
l = 3 ;
c = 0 ;
for i = 1:3
for j = 1:10
c=c+1 ;
a = j + 1;
U(i,c,:)=a ;
end
end
I want to translate this to python and I am unable to create the array U that stores dynamically the value of 'a' in U.
Note : Here am computing 'a' as j+1 for simplicity but in my script 'a' is an array (the fourier transform of a signal)
Sorry for my bad english, I am french. T
I believe you will ultimately want something like this. One of the things that was confusing was what your loop variable c and j were doing. It seems like you wanted c=j, so I changed that below. The one thing you need to watch out for is that python objects are indexed from 0, whereas Matlab objects are index from 1. So below, if you actually start examining the values of i and j, you will see that they start from 0.
import numpy
L = 3;
C = 10;
N = 50; # Size of the Fourier array
U = numpy.zeros((L,C,N))
for i in range(L):
for j in range(C):
# Create a matrix of scalars, for testing
a = i*j*numpy.ones((N,));
U[i,j,:] = a;
Related
I am getting an error using repmat. My Matlab version is 2017a. "Requested 3711450x2726 (75.4GB) array exceeds maximum array size..." First, some context.
I have an adjacency matrix of social network data call it D. D is 2725x2725 with 1s denoting a link between agents i and j and 0s otherwise. I have been provided a function and sub-functions for a network formation model. There are K regressors (x variables). The model requires forming a dyad-specific regressor matrix W that is W = 0.5N(N-1) x K. In my data, this is 3711450 x K. For a start, I select only one x variable so K=1.
In the main function, there are two steps. The first step calculates the joint MLE from a logit. I have a problem in the second step computation of the variance covariance matrix with array size. Inside this step, there is a calculation that creates a 3711450 x n (2725) matrix using repmat.
INFO = ((repmat((exp_Xbeta ./ (1+exp_Xbeta).^2),1,K) .* X)'*X);
exp_Xbeta is 3711450 x K and X is a sparse 3711450 x 2725 matrix with Bytes = 178171416 of class double. The error occurs at INFO.
I've tried converting X to a tall matrix but thus far no joy. I've tried adding sparse to the INFO line but again no joy. Anyone have any ideas short of going to a cluster or getting more ram? Could I somehow convert X from a sparse matrix to a full matrix inside a datastore and then call the datastore using tall? I have not been able to figure out how to do that if it is possible.
Once INFO is constructed as an array it will be used later in one of the sub-functions. So, it needs to be callable. In case you're curious, INFO is the second derivative matrix.
I have found that producing the INFO matrix all at once was too much for my memory constraints. I split up the steps, but still, repmat and subsequent steps were a problem. Now, I've turned to building up the INFO matrix one step at a time, while never holding more than exp_Xbeta, X, and two vectors in memory. Replacing the construction of INFO with
for i = 1:d
s1_i = step1(:,1).*X(:,i);
s1_i = s1_i';
for j = 1:d;
INFO(i,j) = s1_i*X(:,j);
end
clear s1_i;
end
has dropped the memory requirement, though its slow, and things seem to be working. For anyone interested, below is a little example illustrating the point.
clear all
N = 20
n = 0.5*N*(N-1)
exp_Xbeta = rand(n,1);
X = rand(n,N);
step1 = (exp_Xbeta ./ (1+exp_Xbeta).^2);
[c,d] = size(X);
INFO = zeros(d,d);
for i = 1:d
s1_i = step1(:,1).*X(:,i)
s1_i = s1_i'
for j = 1:d
INFO(i,j) = s1_i*X(:,j)
end
clear s1_i
end
K = 1
INFO2 = ((repmat((exp_Xbeta ./ (1+exp_Xbeta).^2),1,K) .* X)'*X);
% Methods produce equivalent matrices
INFO
INFO2
I want to create 3d arrays that are functions of 2d arrays and apply matrix operations on each of the 2D arrays. Right now I am using for loop to create a series of 2d arrays, as in the code below:
for i=1:50
F = [1 0 0; 0 i/10 0; 0 0 1];
B=F*F';
end
Is there a way to do this without the for loop? I tried things such as:
F(2,2) = 0:0.1:5;
and:
f=1:0.1:5;
F=[1 0 0; 0 f 0; 0 0 1];
to create them without the loop, but both give errors of dimension inconsistency.
I also want to perform matrix operations on F in my code, such as
B=F*F';
and want to plot certain components of F as a function of something else. Is it possible to completely eliminate the for loop in such a case?
If I understand what you want correctly, you want 50 2D matrices stacked into a 3D matrix where the middle entry varies from 1/10 to 50/10 = 5 in steps of 1/10. You almost have it correct. What you would need to do is first create a 3D matrix stack, then assign a 3D vector to the middle entry.
Something like this would do:
N = 50;
F = repmat(eye(3,3), [1 1 N]);
F(2,2,:) = (1:N)/10; %// This is 1/10 to 5 in steps of 1/10... or 0.1:0.1:5
First pre-allocate a matrix F that is the identity matrix for all slices, then replace the middle row and middle column of each slice with i/10 for i = 1, 2, ..., 50.
Therefore, to get the ith slice, simply do:
out = F(:,:,i);
Minor Note
I noticed that what you want to do in the end is a matrix multiplication of the 3D matrices. That operation is not defined in MATLAB nor anywhere in a linear algebra context. If you want to multiply each 2D slice independently, you'd be better off using a for loop. Doing this vectorized with native operations isn't supported in this context.
To do it in a loop, you'd do something like this for each slice:
B = zeros(size(F));
for ii = 1 : size(B,3)
B(:,:,ii) = F(:,:,ii)*F(:,:,ii).';
end
... however, examining the properties of your matrix, the only thing that varies is the middle entry. If you perform a matrix multiplication, all of the entries per slice are going to be the same... except for the middle, where the entry is simply itself squared. It doesn't matter if you multiple one slice by the transpose of the other. The transpose of the identity is still the identity.
If your matrices are going to be like this, you can just perform an element-wise multiplication with itself:
B = F.*F;
This will not work if F is anything else but what you have above.
Creating the matrix would be easy:
N = 50;
S = cell(1,N);
S(:) = {eye(3,3)};
F = cat(3, S{:});
F(2,2,:) = (1:N)/10;
Another (faster) way would be:
N = 50;
F = zeros(3,3,N);
F(1,1,:) = 1;
F(2,2,:) = (1:N)/10;
F(3,3,:) = 1;
You then can get the 3rd matrix (for example) by:
F(:,:,3)
I have trouble translating the following code into C, as the format for array indexing in C slightly differs from MATLAB. If I could get some pointers (get it?) as how to translate the following code, I'd greatly appreciate it. I am only stuck at the following lines, as the indexing in C won't allow me to do it as follows:-
b = a(x:x+1, y:y+1);
a(x:x+1, y:y+1) = b;
My code:-
a = [1 2 3 4; 2 3 4 5; 3 4 5 6; 4 5 6 7];
c = dctmtx(2);
q_mtx = [16 11; 12 12];
for x = 1:2:M
for y = 1:2:N
b = a(x:x+1, y:y+1); %<----HOW DO I DO THIS IN C???
b = c*b*c';
b = b./q_mtx ;
a(x:x+1, y:y+1) = b;
end
end
Your help is greatly appreciated! Thanks!
As far as I know, there is no build-in operators for subscripting the arrays.
So, you can either look up for suitable libraries (I think, for 2D arrays some Image processing libraries can work, like OpenCV), or write your one code.
The line you are interested in is easy to translate into C.
If in Matlab you have b = a(x:x+1, y:y+1);, in C it would be four lines:
b[0][0]=a[x][y];
b[1][0]=a[x+1][y];
b[0][1]=a[x][y+1];
b[1][1]=a[x+1][y+1];
Here I assume that you defined arrays statically (as you wrote in comments).
This is an advantage of the Matlab - it is really convenient to work with matrices.
Edit: regarding your question in comments, for bigger size you can use for loops:
Matlab code b = a(x:x+32, y:y+55) would correspond to
for(int i=0;i<32;i++)
for(int j=0;j<55;j++)
b[i][j]=a[x+i][y+j];
I'm trying to understand and learn the C language, and since I used to work in Matlab, I'm interested in knowing how this code would be converted into C.
for j=1:n
v=A(:,j);
for i=1:j-1
R(i,j)=Q(:,i)'*A(:,j);
v=v-R(i,j)*Q(:,i);
end
R(j,j)=norm(v);
Q(:,j)=v/R(j,j);
end
Do you know about the Matlab Coder? Matlab can automatically generate c/c++ code for you. It has its limitations, but if are trying to learn c from Matlab, using the coder should be the best way for you to populate many examples.
Arrays are declared and accessed like so:
const int N = 10; // needs to be a constant
double v[N]; // 1-d
double A[N][N]; // 2-d
v[0] = A[1][2]; // indexing starts at 0, not 1
C doesn't do automatic vectorization like matlab, so you have to do it in for-loops manually. Instead of R(i,j)=Q(:,i)'*A(:,j),
for (int k = 0; k < N; ++k) {
R[i][j] += Q[k][i] * A[k][j];
}
That last piece also demonstrates what a for-loop looks like - the first "argument" of the "for" is the initialization of the indexing variable k, the second sets the condition under which the for loop continues, and the third increments k. The code to be executed in the loop is enclosed in braces {}.
The main logical difference is that you have to do everything element-by-element in C.
I need to sort these values randomly in an array.
int [] d = new int[26];
d[0]=1;
d[1]=5;
d[2]=10;
d[3]=25;
d[4]=50;
d[5]=75;
d[6]=100;
d[7]=200;
d[8]=300;
d[9]=400;
d[10]=500;
d[11]=750;
d[12]=1000;
d[13]=2000;
d[14]=3000;
d[15]=4000;
d[16]=5000;
d[17]=7500;
d[18]=10000;
d[19]=25000;
d[20]=50000;
d[21]=100000;
d[22]=250000;
d[23]=500000;
d[24]=750000;
d[25]=1000000;
Assuming you are writing this in C++ you can use random_shuffle function template from the Standard library. http://www.cppreference.com/wiki/algorithm/random_shuffle
If you want to write your own function, simply take two random indices and swap their values. Put that in a loop and do it as many times as think necessary for the array to be well shuffled (I would say a number of times equal to two times the number of elements in the array).
In psudo-code (since you haven't indicated the language)
NUMBER_OF_SHUFFLES = 2;
for(ix = 0; ix < NUMBER_OF_SHUFFLES * myArray.length; ix++)
index1 = random(myArray.length)
index2 = random(myArray.length)
temp = index1
myArray[index1] = myArray[index2]
myArray[index2] = temp
There are more sophisticated ways of doing it as well. Check out this discussion: An Efficient way of randomizing an array - Shuffle code
If it's java you may use
Arrays.shuffle(d);