I have a dictionary with 1000 keys and each key has 23 entries, each is an xarray.DataArray.
Each entry looks like this:
<xarray.DataArray 'time' (time: 23)>
array(['1861-01-16T12:00:00.000000000', '1861-02-15T00:00:00.000000000',
'1861-03-16T12:00:00.000000000', '1861-04-16T00:00:00.000000000',
'1861-05-16T12:00:00.000000000', '1861-06-16T00:00:00.000000000',
'1861-07-16T12:00:00.000000000', '1861-08-16T12:00:00.000000000',
'1861-09-16T00:00:00.000000000', '1861-10-16T12:00:00.000000000',
'1861-11-16T00:00:00.000000000', '1861-12-16T12:00:00.000000000',
'1862-01-16T12:00:00.000000000', '1862-02-15T00:00:00.000000000',
'1862-03-16T12:00:00.000000000', '1862-04-16T00:00:00.000000000',
'1862-05-16T12:00:00.000000000', '1862-06-16T00:00:00.000000000',
'1862-07-16T12:00:00.000000000', '1862-08-16T12:00:00.000000000',
'1862-09-16T00:00:00.000000000', '1862-10-16T12:00:00.000000000',
'1862-11-16T00:00:00.000000000'], dtype='datetime64[ns]')
Coordinates:
* time (time) datetime64[ns] 1861-02-15 ... 1862-12-16T12:00:00
month (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
I am trying to concatenate all these entries and get a new variable with dimensions ( 1000,23)
which would look like
<xarray.DataArray 'entries','time' (entries:1000,time: 23)>
and then I want to be able to write this into a Netcdf file so to use .to_netcdf.
If I do
tt=xr.concat(entry[0],entry[1])
I get the following:
<xarray.DataArray 'time' (time: 23)>
array(['1861-01-16T12:00:00.000000000', '1861-02-15T00:00:00.000000000',
'1861-03-16T12:00:00.000000000', '1861-04-16T00:00:00.000000000',
'1861-05-16T12:00:00.000000000', '1861-06-16T00:00:00.000000000',
'1861-07-16T12:00:00.000000000', '1861-08-16T12:00:00.000000000',
'1861-09-16T00:00:00.000000000', '1861-10-16T12:00:00.000000000',
'1861-11-16T00:00:00.000000000', '1861-12-16T12:00:00.000000000',
'1862-01-16T12:00:00.000000000', '1862-02-15T00:00:00.000000000',
'1862-03-16T12:00:00.000000000', '1862-04-16T00:00:00.000000000',
'1862-05-16T12:00:00.000000000', '1862-06-16T00:00:00.000000000',
'1862-07-16T12:00:00.000000000', '1862-08-16T12:00:00.000000000',
'1862-09-16T00:00:00.000000000', '1862-10-16T12:00:00.000000000',
'1862-11-16T00:00:00.000000000'], dtype='datetime64[ns]')
Coordinates:
* time (time) datetime64[ns] 1861-01-16T12:00:00 1861-02-15 ... 1862-11-16
month (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
<xarray.DataArray 'time' (time: 23)>
array(['1861-01-16T12:00:00.000000000', '1861-02-15T00:00:00.000000000',
'1861-03-16T12:00:00.000000000', '1861-04-16T00:00:00.000000000',
'1861-05-16T12:00:00.000000000', '1861-06-16T00:00:00.000000000',
'1861-07-16T12:00:00.000000000', '1861-08-16T12:00:00.000000000',
'1861-09-16T00:00:00.000000000', '1861-10-16T12:00:00.000000000',
'1861-11-16T00:00:00.000000000', '1861-12-16T12:00:00.000000000',
'1862-01-16T12:00:00.000000000', '1862-02-15T00:00:00.000000000',
'1862-03-16T12:00:00.000000000', '1862-04-16T00:00:00.000000000',
'1862-05-16T12:00:00.000000000', '1862-06-16T00:00:00.000000000',
'1862-07-16T12:00:00.000000000', '1862-08-16T12:00:00.000000000',
'1862-09-16T00:00:00.000000000', '1862-10-16T12:00:00.000000000',
'1862-11-16T00:00:00.000000000'], dtype='datetime64[ns]')
Coordinates:
* time (time) datetime64[ns] 1861-02-15 ... 1862-12-16T12:00:00
month (time) int64 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
So how I loop over this and transform the entries of the dictionary into a new dimension for the xarran.DataArray and get the (1000,23) xarray.DataArray ?
Thanks !
Some ideas:
ds = xr.Dataset({'Entry_0': entry[0],
'Entry_1': entry[1],
'Entry_2': entry[2]
#... and so on
})
Or another way to do the same thing is:
# create empty xarray dataset
ds = xr.Dataset({})
# add data arrays to xarray dataset
ds['Entry_0'] = entry[0]
ds['Entry_1'] = entry[1]
ds['Entry_2'] = entry[2]
# add so on
Or use a for loop for more concise code:
# create empty list to store xarray data arrays
data_arrays = []
# for loop to add data arrays to list
for idx in range(len(entry)):
da = entry[idx]
data_arrays.append(da)
# concatenate list of data arrays along new dimension into xarray dataset
ds = xarray.concat(data_arrays, dim='new_dim')
Related
I am trying to create two vectors
first:
[ 10 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 10 ]
second:
[ 1 2 3 4 5 6 7 8 9 10 9 8 7 6 5 4 3 2 1 ]
I had almost no problems with the first one:
i = 1:10;
for t = 1: 2*length(i)-1
y1(t) = abs(length(x)-t)+1;
end
But there are some problems with the second...
Does anyone have any idea how I can create it using the same for loop?Thanks in advance
If you want to do it with a loop:
N = 10;
for t = 1:2*N-1
y2(t) = -abs(t-N)+N;
end
But its probably easier to use the following, first make an array 1:N, and then concatenate the array N-1:-1:1:
y2 = [1:N, N-1:-1:1]
This question already has an answer here:
Pandas: how to create a simple counter that increases each n rows?
(1 answer)
Closed 1 year ago.
I have a dataframe which looks like this:
import pandas as pd
df = pd.DataFrame({
'SENDER_ID': [1,2,3,4,5,6,7,8,9,10,11,12] })
df =
SENDER_ID
0 1
1 2
2 3
3 4
4 5
5 6
6 7
7 8
8 9
9 10
10 11
11 12
Now I want to add a column which has the the element multiple times.
SENDER_ID counter
0 1 0
1 2 0
2 3 0
3 4 1
4 5 1
5 6 1
6 7 2
7 8 2
8 9 2
9 10 3
10 11 3
11 12 3
The dataframe always has a length of multiple of 3 and is much larger then in this simple example.
What is the easiest and most generic way to add this new column?
Another way using pd.RangeIndex:
df['count'] = pd.RangeIndex(0, len(df)//3).repeat(3)
print(df)
# Output:
SENDER_ID count
0 1 0
1 2 0
2 3 0
3 4 1
4 5 1
5 6 1
6 7 2
7 8 2
8 9 2
9 10 3
10 11 3
11 12 3
I think I found a solution which works:
max_list_length = int(len(df) / 3)
liste = [[n]*3 for n in range(0,max_list_length)]
value = sum(liste, [])
>>>> value
>>>> [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3]
for n in range (0, len(df)):
df.at[n, 'counter'] = value[n]
I have a matrix like following,
A =
1 2 3
4 5 6
7 8 9
10 11 12
4 5 6
7 8 9
4 5 6
1 2 3
I could extract unique rows in this matrix using command A_unique = unique(A,'rows') and result as follows
A_unique =
1 2 3
4 5 6
7 8 9
10 11 12
I need to find number of times each rows exists in the main matrix A
Some thing like following
A_unique_count =
2
3
2
1
How can I find count of unique rows? Can anyone help? Thanks in Advance
Manu
The third output of unique gives you the index of the unique row in the original array. You can use this with accumarray to count the number of occurrences.
For example:
A = [1 2 3
4 5 6
7 8 9
10 11 12
4 5 6
7 8 9
4 5 6
1 2 3];
[uniquerow, ~, rowidx] = unique(A, 'rows');
noccurrences = accumarray(rowidx, 1)
Returns:
noccurrences =
2
3
2
1
As expected
I would recommend #excaza's approach. But just for variety:
A_unique_count = diff([0; find([any(diff(sortrows(A), [], 1), 2); 1])]);
I have a variable like this that is all one row:
1 2 3 4 5 6 7 8 9 2 4 5 6 5
I want to write a for loop that will find where a number is less than the previous one and put the rest of the numbers in a new row, like this
1 2 3 4 5 6 7 8 9
2 4 5 6
5
I have tried this:
test = [1 2 3 4 5 6 7 8 9 2 4 5 6 5];
m = zeros(size(test));
for i=1:numel(test)-1;
for rows=1:size(m,1)
if test(i) > test(i+1);
m(i+1, rows+1) = test(i+1:end)
end % for rows
end % for
But it's clearly not right and just hangs.
Let x be your data vector. What you want can be done quite simply as follows:
ind = [find(diff(x)<0) numel(x)]; %// find ends of increasing subsequences
ind(2:end) = diff(ind); %// compute lengths of those subsequences
y = mat2cell(x, 1, ind); %// split data vector according to those lenghts
This produces the desired result in cell array y. A cell array is used so that each "row" can have a different number of columns.
Example:
x = [1 2 3 4 5 6 7 8 9 2 4 5 6 5];
gives
y{1} =
1 2 3 4 5 6 7 8 9
y{2} =
2 4 5 6
y{3} =
5
If you are looking for a numeric array output, you would need to fill the "gaps" with something and filling with zeros seem like a good option as you seem to be doing in your code as well.
So, here's a bsxfun based approach to achieve the same -
test = [1 2 3 4 5 6 7 8 9 2 4 5 6 5] %// Input
idx = [find(diff(test)<0) numel(test)] %// positions of row shifts
lens = [idx(1) diff(idx)] %// lengths of each row in the proposed output
m = zeros(max(lens),numel(lens)) %// setup output matrix
m(bsxfun(#le,[1:max(lens)]',lens)) = test; %//'# put values from input array
m = m.' %//'# Output that is a transposed version after putting the values
Output -
m =
1 2 3 4 5 6 7 8 9
2 4 5 6 0 0 0 0 0
5 0 0 0 0 0 0 0 0
I have a 1 x 15 array of values:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
I need to rearrange them into a 3 x 5 matrix using a for loop:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
How would I do that?
I'm going to show you three methods. One where you need to have a for loop, and two others when you don't:
Method #1 - for loop
First, create a matrix that is 3 x 5, then keep track of an index that will go through your array. After, create a double for loop that will help you populate the array.
index = 1;
array = 1 : 15; %// Array we wish to access
matrix = zeros(3,5); %// Initialize
for m = 1 : 3
for n = 1 : 5
matrix(m,n) = array(index);
index = index + 1;
end
end
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
Method #2 - Without a for loop
Simply put, use reshape:
matrix = reshape(1:15, 5, 3).';
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
reshape will take a vector and restructure it into a matrix so that you populate the matrix by columns first. As such, we want to put 1 to 5 in the first column, 6 to 10 in the second and 11 to 15 in the third column. Therefore, our output matrix is in fact 5 x 3. When you see this, this is actually the transposed version of the matrix we want, which is why you do .' to transpose the matrix back.
Method #3 - Another method without a for loop (tip of the hat goes to Luis Mendo)
You can use vec2mat, and specify that you need to have 5 columns worth for your matrix:
matrix = vec2mat(1:15, 5);
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
vec2mat takes a vector and reshapes it into a matrix of as many columns as you specify in the second parameter. In this case, we need 5 columns.
For the sake of (bsx)fun, here is another option...
bsxfun(#plus,1:5,[0:5:10]')
ans =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
less readable, maybe faster, but who cares if it is such a small of an array...
A = [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ] ;
A = reshape( A' , 3 , 5 ) ;
A' = 1 2 3 4 5
6 7 8 9 10
11 12 13 14 15