Why is my numpy array 2d when there's only 1d? - arrays

I'm learning how to do multiple regression. The functions I'm using to predict return results as an array. When I go to append the data to my pandas data frame it appends, but when I try to use the data I get an error:
ValueError: setting an array element with a sequence.
So I go to check the array and see that it has 2 dimensions. I have no clue what to do, I've printed it out and it looks like it has 1 dimension to me. This is what the array looks like when printed:
array([[1217964. ],
[1215076. ],
[1205507. ],
[1200445. ],
[1175359. ],
[1172445.56110176]])
I thought about trying to drop a dimension, but I wouldn't know what to drop since I only see one dimension. Any help is much appreciated, thanks for helping a beginner out.

It occured to me that my array may have had an empty dimension. It seems like an oversight on the package dev for the regression functions to create 2d arrays from single dimension results, but I don't know that much.
I used the method squeeze to drop my empty dimension and now it works!

Related

Deleting consecutive and non-consecutive columns from a cell array in Matlab

I'm trying to delete multiple consecutive and non-consecutive columns from a 80-column, 1-row cell array mycells. My question is: what's the correct indexing of a vector of columns in Matlab?
What I tried to do is: mycells(1,[4:6,8,9]) = [] in an attempt to remove columns 4 to 6, column 8 and 9. But I get the error: A null assignment can have only one non-colon index.
Use a colon for the first index. That way only the 2nd index is "non-colon". E.g.,
mycells(:,[4:6,8,9]) = []
MATLAB could have been smart enough to recognize that when there is only one row the 1 and : amount to the same thing and you will still get a rectangular array result, but it isn't.
Before getting the above VERY VERY HELPFUL AND MUCH SIMPLER answers, I ended up doing something more convoluted. As it worked in my case, I'll post it here for anyone in future:
So, I had a cell array vector, of which I wanted to drop specific cells. I created another cell array of the ones I wanted to remove:
remcols = mycells(1,[4:6,8,9])
Then I used the bellow function to overwrite onto mycells only those cells which are different between remcols and mycells (these were actually the cells I wanted to keep from mycells):
mycells = setdiff(mycells,remcols)
This is not neat at all but hopefully serves the purpose of someone somewhere in the world.

Extracting arrays from array

I am currently working on machine learning, and as so I have an array in which the first column is the data and the second column is the label. Data was originally a cell array from Matlab (Not sure if that is important).
[My Array of arrays] https://i.stack.imgur.com/JFpWO.png
To make sure that everything is as it should be I would like to extract one of the arrays in index 0 and check its dimensions with the numpy.shape function. Currently, if I try that I just get the shape of the bigger array IE. (394,2)
Any ideas?
There are different ways.: Try..
arr[:,1,:]
or
arr[:,1]
or
[a[1] for a in arr]

Excel Array formulas details and guidance needed (INDEX,MATCH and SUM/COUNTIF)

So I'm having difficulties understanding fully how arrays works and when they are used by excel and specifically what happens in the background.
From reading the past few hours I understand that one of the reasons my Index Match doesn't work without array is simply because its a multicriteria Match that I use as below:
{=INDEX(D30:E36,MATCH(F33&G33,B30:B36&C30:C36),2)}
From what I understand the reason is that Match returns a {x,y} result which classifies it as an array formula. But considering the point is to get a row number, if the row I'm looking for is 5 then Match will return a {5,5} for row number for Index. And then Index interprets this as just 5? or what exactly happens in the background here?
Then I found an article which showed how to circumvent the array formula and not need ctrl+shift+enter as shown below. How does the below change things and what happens in the background?
=INDEX(D30:E36,MATCH(F33&G33,INDEX(B30:B36&C30:C36)),2)
The below is a an array SUM/COUNTIF formula which counts unique cells only which does not work without array brackets. Why is that and how does it work? It involves maths so I'm not sure.
{=SUM(1/(COUNTIF(A1:A5,A1:A5)))}
Thank you!

Reading a file into a 1d array in Julia

I have a pretty simple text file of just numbers that looks like
0
1.57
3.14
This example has 3 numbers, but amount is fine. I'm trying to read these into a 1d array of Float 64's, so I tried the following.
function read_x_data(fname)
f=open(fname)
xarr=readdlm(f, Float64)
print(xarr)
xarr=sortperm(xarr)
end
However, I get the error that sortperm has no matching method sortperm(::Array{Float64, 2}). I don't understand why this is happening- how can I read my data into a 1d array instead? I saw a similar question at Reading line by line in Julia, but I believe that using push n times like that is very inefficient, right? Any help with my problem or suggestions are much appreciated. Thanks!
To answer your immediate question: vec will reshape any array into a 1d vector.
sortperm returns the permutation but not the original data; hence your example, even if you added vec(xarr), would throw away the data. You probably want sort.
Finally, in Julia push! is not inefficient. You may be expecting it to be inefficient from experience with another language (Matlab?), but in Julia you can efficiently grow 1d arrays.
If your intention is to sort the data, this might work for you.
read_x_data(fname) = sort!(vec(readdlm(fname,Float64)))

Populating Multi dimensional array in octave/matlab to create waterfall plot in octave / matlab

I'm trying to create a Multi dimensional array so I can create a waterfall plot in octave, a matlab type of program. (Please note octave does not have the waterfall plot option so I have to make a work around.
I have multiple arrays that have frequency in the first column and amplitude in the second column.
example: When looped through the arrays, which are called sort_array they each need to be placed into a multidimensional array on a separate page.
4000, .5
3002, .1234
1093, .7
I was trying to have each of these arrays (sort_array) added to a single multidimensional array by using a for loop to have everything added to one array to make it easier to plot and export as a text file. I also thought using k as the page option for the multiple dimension array. But I keep getting dimensions mismatch. Any ideas how to fix this?
Please note I left out the sort_array code and just included the example array called sort_array above as an example of what the array would look like. There will be about 9000 of them.
md=[];
for k=1:9000
md_tmp=[sort_array(:,1),sort_array(:,2)]
ma(:,:,k)=[ma;ma_tmp];
end
So when I type in ma(:,:,1) I would get
4000, .5
3002, .1234
1093, .7
and if I type in ma(:,:,2) I would get the next one.
thanks
Your error is here: ma(:,:,k)=[ma;ma_tmp];
You are mixing up two concepts.
Either concatenate the new matrix onto the current one:
ma = [ma; ma_tmp];
OR
Assign the new matrix directly to the correct index (this is the cleaner, more efficient solution):
ma(:,:,k) = ma_tmp;
But you can't do both.
thanks I found a work around
sort_array=sortrows(arraytmp,-1); %sort by freq
k_tmp=repmat(k,length(sort_array(:,2)),1); %to create page for freq and amp in multidimensional array
ma_tmp=[sort_array(:,1),sort_array(:,2),k_tmp];
ma=[ma;ma_tmp];

Resources