I have the following question:
I am building a model when I first test for stationarity. Then I have an if loop, saying:
if p>0.05:
x=y['boxcox']
else:
x=y['Normal']
If the pvalue is bigger than 0.05, then I do the boxcox transformation, if not, then I use my original values. This works.
I then have a large code, that is working.
However, in the end, I want to transform my values back.
Again with the if loop.
But how do I get the if loop started?
I first wanted to do:
if any (x==y['BoxCox']):
.....transform back
This works if I orginially have transformed my values, but not if I didn't, which makes sense, because the code does not know y['BoxCox'].
But how do I get the if loop initiated?
Thanks a lot!
If I understand your question correctly, you do not transform anything back, rather you remember the initial state. "Transforming back" sounds like a potential source of bugs. What if you alter your transformation algorithm and forget to update the transforming back part?
here is a simplified example, to illustrate my understanding of your problem:
x = 4
if x > 2:
x = x + 1
else:
x = x + 100
print("Result = ", x)
print("Initial value was ???") // you cannot tell what was the initial x
You can simply do not touch the initial values, to be accessible at any time:
x = 4
if x > 2:
result = x + 1
else:
result = x + 100
print("Result = ", result)
print("Initial value = , x)
Related
I'm trying to code an Adaline neurone in Go an I'm having a problem with the scoop of an array, I update the values of it inside a for loop and it looks, like they are being updated, but when I try to access the new values from outside the loop they are always the same, they were just updated in the first iteration. Here is the code:
//This functions creates a [][]float64 and fill it with random numbers
weights := initWeights(inputLength)
// data is a [][]float 64 and expectedY a []float64
for i := 0; i < 10; i++ {
for j := range data {
//Calculate estimate
var estimate float64 = 0
for x := range data[j]{
estimate += data[j][x] * weights[x]
}
// Update weights (range passes values as a copy)
for x := 0; x < len(weights); x++ {
weights[x] = learningRate * (expectedY[j] - estimate) * data[j][x]
}
//PRINT #1
}
//PRINT #2
//
//Some more stuff
//
}
If I print weights before the loop it looks like this:
[-0.6046602879796196 0.6645600532184904 -0.4246374970712657 0.06563701921747622 0.09696951891448456 -0.5152126285020654 0.21426387258237492 0.31805817433032985 0.28303415118044517]
So it was created correctly. After I start the loops to adjust the neurone weights. Here is where the weird thing happens.
If I print in #1 I can see that the array is being updated in each iteration, but when I print in #2 the value of the array is always the same, it's the one was calculated on the first iteration of the weights loop.
PRINT #1
[0.06725611377611064 0 0 0.03490734755724929 0.014819026508554914 0.023919277971577904 0.021858582731470875 0.0051309928461725374 0.06915084698345737]
[0.030417970260300468 0.0274737201080031 0 0.02479712906046004 0.01662460439529523 0.014007493148808682 0.029246218179487176 0.004413401238393224 0.05947980105651245]
[0.008861875440076036 0 0.01792998206766924 0.017854161778140868 0.004333887749441702 0.020137868898735412 0.0125224790185058 0.008249247500686795 0.030328115811348512].
PRINT #2
[0.007796061340871362 0 0.011035383661848988 0.01289960904315235 0.003797667051516503 0.009918694200771232 0.015234505189042204 0.0008236738380263619 0.023072096303259435]
[0.007796061340871362 0 0.011035383661848988 0.01289960904315235 0.003797667051516503 0.009918694200771232 0.015234505189042204 0.0008236738380263619 0.023072096303259435]
[0.007796061340871362 0 0.011035383661848988 0.01289960904315235 0.003797667051516503 0.009918694200771232 0.015234505189042204 0.0008236738380263619 0.023072096303259435]
I've been struggling with this for the last two days and I couldn't figure out what's happening, I hope you guys can help me.
-- UPDATE --
Here is a more complete and runnable version https://play.golang.org/p/qyZGSJSKcs
In play, looks that the code is working fine... the exact same code in my computer outputs the exact same slice every iteration.
The only difference is that instead of fixed slices I'm creating them from two csv files with several hundreds of rows, so I'm guessing the problem comes from there, I'll continue investigating.
Here you have the raw data if it's helpfull:
Train data: https://pastebin.com/H3YgFF0a
Validate data: https://pastebin.com/aeK6krxD
Don't provide partial code with bits removed, just provide a runnable example - the process of doing this may well help you find the problem. Your initWeights function isn't really required for this purpose - better to use known start data.
Here is the incomplete code you have with data added. The algorithm seems to tend towards a certain set of results (presumably, with different data it might just get there quicker within 10 runs, I've upped the runs to 100).
https://play.golang.org/p/IqfCjNtd8a
Are you sure this is not working as intended? Do you have test data with expected results to test it with? I'd expect print 2 to always match print the last print 1, given the code you posted, but it was obviously incomplete.
[EDIT]
It's not clear this is a go code problem as opposed to results which surprise you from your algorithm/data.
You need to:
Provide code (you've now done this, but not code that shows the problem)
Reduce the code/data to the minimum which shows the error
Produce a test which demonstrates the surprising result with minimum data
If you can't reproduce with static data, show us how you load the data into the variables, because that is probably where your problem lies. Are you sure you are loading the data you expect and not loading lots of copies of one row for example? Are you sure the algorithm isn't working as intended (if so how)? Your descriptions of results don't match what you have shown us so far.
FOUND IT! It's such a silly thing.. the weights update process is accumulative
w(i+1) = w(i) + learningRate * (expected - estimated) * data[j][i]
so I just forgot to add the + to the weights assignment
weights[x] += learningRate * (expectedY[j] - estimate) * data[j][x]
Here is the complete snippet working properly:
for i := 0; i < cylces; i++ {
for j := range data {
//Calculate estimate
estimate = 0
for x := range data[j]{
estimate += data[j][x] * weights[x]
}
// Update weights (range passes values as a copy)
for x := 0; x < len(weights); x++ {
weights[x] += learningRate * (expectedY[j] - estimate) * data[j][x]
}
}
errorData = 0
for j := range data {
estimate = 0
for x := range data[j] {
estimate += data[j][x] * weights[x]
}
errorData += (expectedY[j] - estimate) * (expectedY[j] - estimate)
}
errorsCyles = append(errorsCyles, errorData / float64(len(data)))
}
I need to iterate Newton-Raphson in MATLAB. It seems easy but I cannot figure out where I am wrong. The problem is:
For mmm=1:
1) If m=1 take c1=c1b and c2=1-c1 and do the loop for u1,2(i) and p1,2(i)
2)If m=2 take c1=c1+dc and c2=1-c1, and this time do the loop with new c1 and c2 for u1,2(i) and p1,2(i)
3) If m=3 take c1=(c1*st(1)-(c1-dc)*st(2))/(st(1)-st(2)) and do the loop for new c1 and c2.
Then increase the iteration number: mmm=2 ;
mmm keeps count of the number of N-R iterations. The first iteration has mmm=1, the second mmm=2, etc. (This particular run only do 2 iterations).
sumint are inside of the integrals.
I need to plot these figures in the code but MATLAB gives errors below. Please help me.
Relevant part of the code:
ii=101;
u = cell(2, 1);
ini_cond = [0,0];
for i = 1:2;
u{i} = zeros(1,ii);
u{i}(:, ii) = ini_cond(i) * rand(1, 1);
end
for i=1:ii;
fikness=fik*sin(pi.*x);
u{1}(i)=(c1-H1D*(x-0.5)+AD/2.*(x-0.5).^2)./(H1-0.5*fikness-A*(x-0.5));
u{2}(i)=(c2+H1D*(x-0.5)-AD/2.*(x-0.5).^2)./(1.-H1+0.5*fikness+A*(x-0.5));
end
p = cell(2, 1);
q = cell(2, 1);
for i = 1:2;
p{i} = zeros(1,ii);
q{i} = zeros(1,ii);
end
p{1}(1)=0.5*(1.-u{1}(1).^2);
q{1}(1)=0;
p{2}(1)=0.5*(1.-u{2}(1).^2);
q{2}(1)=0;
for i=2:101
q{1}(i)=q{1}(i-1)-dx*(u{1}(i-1)-ub{1}(i-1))./dt;
p{1}(i)=0.5*(1.-u{1}(i).^2)+q{1}(i);
q{2}(i)=q{2}(i-1)-dx*(u{2}(i-1)-ub{2}(i-1))./dt;
p{2}(i)=0.5*(1.-u{2}(i).^2)+q{2}(i);
end
st = zeros(2, length(t));
st(1,:)=p{1}(100)-p{2}(100);
m=m+1;
if m==3;
c1=(c1*st(1)-(c1-dc)*st(2))/(st(1)-st(2));
c2=1-c1;
end
for i = 1:2;
sumint{i} = zeros(1,length(t));
end
sumint = cell(2, 1);
sumint{1}(1)=0.5*(p{2}(1)-p{1}(1));
sumint{2}(1)=0.5*(p{2}(1)-p{1}(1)).*(-1/2);
for i=2:ii-1;
x=(i-1)*dx;
sumint{1}(i)=sumint{1}(i-1)+(p{2}(i)-p{1}(i));
sumint{2}(i)=sumint{2}(i-1)+(p{2}(i)-p{1}(i))*(x-1/2);
end
H1DDOT=-sumint{1}.*dx./rmass;
H1D=H1D+dt*H1DDOT;
H1=H1+dt*H1D;
ADDOT=sumint{2}*dx./rmomi;
AD=AD+dt*ADDOT;
A=A+dt*AD;
H1L=H1+A.*0.5;
H1R=H1-A.*0.5;
H2=1.-H1;
rat1=AD./ADinit;
rat2=ADDOT./AD;
u are the velocities p are the pressures c1,c2 are the camber effects H1DDOT and ADDOT are the second derivation of H1 and A. sum1 and sum2 are the inside of the integrals to define the values of H1DDOT and ADDOT. H1DDOT and ADDOT are functions of time.
As you can see from the message, the error is with this line:
sumint{2}(i)=sumint{2}(i-1)+(p{2}(i)-p{1}(i)).*(x-1/2);
Now, let's find out why:
sumint{2}(i) = ...
This part means you want to insert whatever is on the right side to the ith position of the array in cell sumint{2}. That means, the right side must be a scalar.
Is it?
Well, sumint{2}(i-1)+(p{2}(i)-p{1}(i)) is certainly a scalar, since you use a single value as index to all the vectors/arrays. The problem is the multiplication .*(x-1/2);.
From the code above, it is clear that x is a vector / array, (since you use length(x) etc). Multiplying the scalar sumint{2}(i-1)+(p{2}(i)-p{1}(i)), by the vector x, will give you back a vector, which as mentioned, will not work.
Maybe you want the ith value of x?
There are several other strange things going on in your code, for instance:
for i=1:101;
fikness=fik*sin(pi.*x);
u{1}=(c1-H1D*(x-0.5)+AD/2.*(x-0.5).^2)./(H1-0.5*fikness-A*(x-0.5));
u{2}=(c2+H1D*(x-0.5)-AD/2.*(x-0.5).^2)./(1.-H1+0.5*fikness+A*(x-0.5));
end
Why do you have a loop here? You are not looping anything, you are doing the same calculations 100 times. Again, I guess it should be x(i).
Update:
Your new error is introduced because you changed x into a scalar. Then u = zeros(1,length(x)) will only be a scalar, meaning u{1}(i) will fail for i ~= 1.
I have a large data set with two arrays, say x and y. The arrays have over 1 million data points in size. Is there a simple way to do a scatter plot of only 2000 of these points but have it be representative of the entire set?
I'm thinking along the lines of creating another array r ; r = max(x)*rand(2000,1) to get a random sample of the x array. Is there a way to then find where a value in r is equal to, or close to a value in x ? They wouldn't have to be in the same indexed location but just throughout the whole matrix. We could then plot the y values associated with those found x values against r
I'm just not sure how to code this. Is there a better way than doing this?
I'm not sure how representative this procedure will be of your data, because it depends on what your data looks like, but you can certainly code up something like that. The easiest way to find the closest value is to take the min of the abs of the difference between your test vector and your desired value.
r = max(x)*rand(2000,1);
for i = 1:length(r)
[~,z(i)] = min(abs(x-r(i)));
end
plot(x(z),y(z),'.')
Note that the [~,z(i)] in the min line means we want to store the index of the minimum value in vector z.
You might also try something like a moving average, see this video: http://blogs.mathworks.com/videos/2012/04/17/using-convolution-to-smooth-data-with-a-moving-average-in-matlab/
Or you can plot every n points, something like (I haven't tested this, so no guarantees):
n = 1000;
plot(x(1:n:end),y(1:n:end))
Or, if you know the number of points you want (again, untested):
npoints = 2000;
interval = round(length(x)/npoints);
plot(x(1:interval:end),y(1:interval:end))
Perhaps the easiest way is to use round function and convert things to integers, then they can be compared. For example, if you want to find points that are within 0.1 of the values of r, multiply the values by 10 first, then round:
r = max(x) * round(2000,1);
rr = round(r / 0.1);
xx = round(x / 0.1);
inRR = ismember(xx, rr)
plot(x(inRR), y(inRR));
By dividing by 0.1, any values that have the same integer value are within 0.1 of each other.
ismember returns a 1 for each value of xx if that value is in rr, otherwise a 0. These can be used to select entries to plot.
Consider the following code snippet
for i = 1:100
Yi= x(i:i + 3); % i in Yi is not an index but subscript,
% x is some array having sufficient values
i = i + 3
end
Basically I want that each time the for loop runs the subscript changes from 1 to 2, 3, ..., 100. SO in effect after 100 iterations I will be having 100 arrays, starting with Y1 to Y100.
What could be the simplest way to implement this in MATLAB?
UPDATE
This is to be run 15 times
Y1 = 64;
fft_x = 2 * abs(Y1(5));
For simplicity I have taken constant inputs.
Now I am trying to use cell based on Marc's answer:
Y1 = cell(15,1);
fft_x = cell(15,1);
for i = 1:15
Y1{i,1} = 64;
fft_x{i,1} = 2 * abs(Y1(5));
end
I think I need to do some changes in abs(). Please suggest.
It is impossible to make variably-named variables in matlab. The common solution is to use a cell array for Y:
Y=cell(100,1);
for i =1:100
Y{i,1}= x(i:i+3);
i=i+3;
end
Note that the line i=i+3 inside the for-loop has no effect. You can just remove it.
Y=cell(100,1);
for i =1:100
Y{i,1}= x(i:i+3);
end
It is possible to make variably-named variables in matlab. If you really want this do something like this:
for i = 1:4:100
eval(['Y', num2str((i+3)/4), '=x(i:i+3);']);
end
How you organize your indexing depends on what you plan to do with x of course...
Yes, you can dynamically name variables. However, it's almost never a good idea and there are much better/safer/faster alternatives, e.g. cell arrays as demonstrated by #Marc Claesen.
Look at the assignin function (and the related eval). You could do what asked for with:
for i = 1:100
assignin('caller',['Y' int2str(i)],rand(1,i))
end
Another related function is genvarname. Don't use these unless you really need them.
%Examples:
%days([saturday,sunday,monday,tuesday,wednesday,thursday]).
%slots([1,2,3,4,5]).
%course_meetings(csen402,tutorial,t07,nehal,'tutorial for t07').
%course_meetings(comm401,lecture,all_group_4,dr_amr_talaat,'lecture 1')
%tutorialrooms([c6301,b4108,c2201,c2301,c2202,c2203]).
day_tut(Day,Slot,Place,Course,Group,Instructor,Descr):-
days(X),member(Day,X),
tutorialrooms(X1),member(Place,X1),
course_meetings(Course,tutorial,Group,Instructor,Descr),
slots(X2),member(Slot,X2),
assert(day(Day,Slot,tutorial,Place,Course,Group,Instructor,Descr)).
I would like to find a way to remove certain facts after asserting for example
every (day) fact has to have only one room for each day and slot
example: we can have day(sat,1,_,c6301,_,_,_,_) and
day(sat,1,_,c6302,_,_,_,_) but we can't have
another occurrence of day(sat,1,_,c6301,_,_,_,_).
If you simply want to remove redundant solutions of a Goal – this is what you probably mean with removing repetitions – simply replace Goal by setof(t,Goal,_). This works as long as there are only ground solutions for Goal and as long as Goal terminates universally. Thus, there is no need for any data base manipulation to remove redundant solutions.
?- member(X, [a,b,a,c]).
X = a
; X = b
; X = a % redundant!
; X = c.
?- setof(t,member(X, [a,b,a,c]),_).
X = a
; X = b
; X = c.