I have a small code snippet where I have defined a local macro i to be used in a forvalues loop:
matrix I=I(4)
scalar x1=b[1,1]
scalar sum1=0
local i = 2
forvalues i = 2/4 {
scalar chk=i-1
forvalues j = 1/`=chk' {
scalar sum1=I[`i',`j']*x`j'+sum1
}
scalar x`i'=(b[`i',1]-sum1)/I[`i',`i']
}
However, Stata complains:
error 111
i not found
Note that this error is appearing only if I use the macro in a loop, not otherwise.
The Stata problem you raise is with the line
scalar chk = i - 1
Stata sees a reference there to i, which evidently cannot be interpreted as a variable or scalar name, hence the error message you got. That's Stata's point of view.
From your point of view, the error is not using single quotation marks to extract the value or contents of the local macro i, or at least that appears to be your intention, as you nowhere else explain your aim. So, nothing is disappearing; you just referred to a non-existent entity. That's only a detail, however, and there is a larger story here.
Here is a first rewriting of your code.
matrix I = I(4)
scalar x1 = b[1,1]
scalar sum1 = 0
forvalues i = 2/4 {
local chk = `i' - 1
forvalues j = 1/`chk' {
scalar sum1 = I[`i',`j'] * x`j' + sum1
}
scalar x`i' = (b[`i',1] - sum1) / I[`i',`i']
}
Notes.
The line local i = 2 appears redundant. The forvalues loop initialises the macro.
As a matter of style, experienced Stata programmers would typically use a local macro, not a permanent scalar, for a transient loop limit.
But wait: I(4) is just an identity matrix with 4 rows and columns, 1s on the principal diagonal and 0s elsewhere. So
I[`i',`i']
is necessarily 1 and
I[`i',`j']
for earlier entries on the same row of the matrix is 0. So sum1 is never anything but 0. So you don't need the identity matrix for any obvious purpose and your code seems reducible to extracting four scalars from a vector:
forvalues i = 1/4 {
scalar x`i' = b[`i',1]
}
EDIT. Double loops like
forvalues i = 2/4 {
local chk = `i' - 1
forvalues j = 1/`chk' {
...
}
...
}
can also be written more concisely
forvalues i = 2/4 {
forvalues j = 1/`= `i' - 1' {
...
}
...
}
Related
When I try to compile the following program, the compiler complains that j and row are undeclared, which surprised me because a similar construction in Chapel - Ranges defined using bounds of type 'range(int(64),bounded,false)' are not currently supported was legal. When I declare row, j it prints 0 for both while the loop starts at 1. Declarations auto-initialise to zero, so is the row from forall row in 1..mat.m somehow different from the declaration above?
use BlockDist;
record CSRMatrix {
/* The matrix is m x n */
var m: int;
var n: int;
/* Number of non-zeroes */
var nz: int;
/* Stores the non-zeroes */
var val: [1..nz] real;
/* The first non-zero of row k is the row_ptr(k)th non-zero overall.
* row_ptr(m + 1) is the total number of non-zeroes. */
var row_ptr: [1..m + 1] int;
/* The kth non-zero has column-index col_ind(k) */
var col_ind: [1..nz] int;
}
/* We assume some global sparse matrix A has been distributed blockwise along the first
dimension and that mat is the local part of the global matrix A(s). Is not
distributed from Chapel's POV!
vec is distributed blockwise, is distributed from Chapel's POV!
Returns matmul(A(s), vec), the local part of matmul(A, vec). */
proc spmv(mat: CSRMatrix, vec: [1..mat.n] real)
{
var result: [1..mat.m] real = 0.0;
// var row: int;
// var j: int;
forall row in 1..mat.m do
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 do
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
return result;
}
#StrugglingProgrammer's response in the notes is correctly diagnosing the problem here, but to capture the explanation with more rationale and commentary:
In Chapel, the index variables in for-, foreach-, forall-, and coforall-loops are new variables defined for the scope of the loop's body, as you expected. Thus for i in myIterand declares a new variable i that takes on the values yielded by myIterand—e.g., an integer in the common case of iterating over a range like for i in 1..n and your loops.
The problem here is that Chapel is not a whitespace-sensitive language (as Python is, say), and its loops have two syntactic forms:
A keyword-based form in which the body must be a single statement following do
And a form in which the body is a compound statement using curly brackets
Thus, when your code says:
forall row in 1..mat.m do
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 do
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
to get the effect desired, you would need to write this as:
forall row in 1..mat.m do
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 {
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
}
Note that the first forall loop does not require curly brackets because its body is a single statement—the for-loop, which happens to have its own multi-statement body.
Because of the possibility of this class of errors, particularly as code is evolving, defensive Chapel programmers tend to prefer always using curly brackets to avoid the possibility of this mistake:
forall row in 1..mat.m {
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 {
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
}
}
And to be pedantic, your original code, indented to reflect the actual control flow would have been as follows (and a good Chapel-aware editor mode would ideally help you by indenting it in this way):
forall row in 1..mat.m do
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 do
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
Seeing the code written this way, you can see why the compiler complains about j and row in the final two statements, but not about row on the first one, since it is within the nested loop as expected.
It's also why, when you introduce explicit variables for j and row they evaluate to zero in those lines. Essentially, your loops are introducing new variables that temporarily shadow the explicitly-defined ones, and the two statements that were outside the loop nest refer to those original variables rather than the loops' index variables as expected. As you note, since Chapel variables are default initialized, they have the value 0.
As one final piece of trivia, since compound statements { ... } are themselves single statements in Chapel, it is legal to write loops with both do and { ... }:
forall row in 1..mat.m do {
for j in mat.row_ptr(row)..mat.row_ptr(row + 1) - 1 do {
writeln(row);
writeln(j);
result(row) += mat.val(j) * vec(mat.col_ind(j));
}
}
However, this is not at all necessary, so I personally tend to discourage its use, considering it overkill and likely to lead to further misunderstandings.
var amazon = activeSpreadsheet.getSheetByName('Amazon');
var lastRow1 = amazon.getLastRow();
var array1 = amazon.getRange('A2:A' + lastRow1).getValues();
var source = activeSpreadsheet.getSheetByName('ProductDetails');
var lastRow2 = source.getLastRow();
var array2 = source.getRange('B2:B' + lastRow2).getValues();
n = 2;
x = 0; // match
z = 0; // non-match
for (var i = 0; i < array2.length; i++){
for (var j = 0; j < array1.length; j++){
if (array2[i] !== array1[j]){
z = z + 1;
}
else {
x = 9999
}
}
newsheet.getRange([n],[5]).setValue(z);
newsheet.getRange([n],[6]).setValue(x);
if (z > x) {
newsheet.getRange([n],[1]).setValue(array2[i]);
n == n++;
z = 0;
x = 0;
}
else if (z < x) {
z = 0;
x = 0;
}
}
My project is written in GAS (google app scripts) which is essentially, for all intents and purposes JS with variation in libraries.
Basically I am grabbing an element in the array2 and passing it through a loop to match to array1. For every time it does not match it adds 1, and when it matches (should only match once if it has any matches) it stores an arbitrary large number (larger than length of array1) and compares them.
As you can see I've written out to display these values and I always get z = 5183 (length of array1) and x = 0 (meaning no non-matches found). Therefore even if something exists in array 2 and 1 it will always write it to the cell.
What should happen is if there is a match, z= 5182 and x= 9999 (or arbitrary large number) and since 5182 < 9999 it doesn't do anything.
Is my scope wrong? Or am I not writing the If/Else correctly? Or is it something else?
Your code performs a strict comparison between the elements of two Arrays. That's fine, in general. However, for those specific Arrays those elements are also Arrays, which means strict (in)equality is checking to see if those are the exact same array object in memory. See this question for more information.
You probably wanted to do a value-based comparison, which means you need to compare the specific element of that inner array (i.e., index again). if (array2[i][0] !== array1[j][0]) {...} will check the 1st element of the inner array.
Looking at the instantiation of array1 and array2, we see that these are indeed 2D arrays from a single-column Ranges, so there will be only 1 element in each inner array. You can reduce the level of indexing necessary by flattening these arrays when you read them:
const array1 = sheet.getRange(...).getValues().map(function (row) { return row[0]; });
const array2 = ...;
I'm also not sure why you are passing in arrays to Sheet#getRange - you should pass in 1-4 arguments in manners consistent with the method signatures detailed in the Apps Script documentation.
Note that there are much better algorithms for checking if a value exists in a given array - you re-scan all of the 2nd array for every value of the first array. You should conduct thorough research into this topic before asking a new question on how to improve your algorithm.
Lastly, you should implement the best practice of using batch methods - you currently call setValue in a loop. Consider storing the results to write in an array, and then writing with Range#setValues once your loop has completed. There are many questions available for you to review on this topic.
In a data step in which I have defined an array I can use the sum function, but the count function doesn't work. How can I count the number of values that are not zero within an array?
SUM_ARRAY = sum(of A1-A20); - Works
COUNT_ARRAY = count(of A1-A20); Yields the following error: "The COUNT function call has too many arguments"
The correct function instead of COUNT is either N, DIM, or HBOUND.
Unfortunately none will count specific values, only exclude missing values.
Looping through results is one way to count non 0s.
Array _a(*) a1-a20;
Count0 = 0;
Do I = 1 to dim(_a);
If _a (I) ne 0 then count0 = count0 + 1;
End;
COUNT can be coerced to do this, if your data is agreeable. I'm not sure it's better than the loop operation timewise or structurally, but it's at least an interesting solution.
Basically we delimit the data by ; including starting and ending it with the delimiter, then count the number of ;0;, and subtract from the total.
data _null_;
call streaminit(7);
array a[20];
do _i = 1 to 20;
a[_i] = max(0,rand('Uniform')-0.3);
end;
put a[*]=;
nonzero = dim(a) - count(';'||catx(';',of a[*])||';',';0;');
put nonzero=;
run;
I have an array of variables and an array of flags, both of length 77. For every observation, the array of flags is made up of consecutive 0's, followed by consecutive 1's (ie, after a flag is a 1, all flags at a later index is a one). I am trying to calculate the mean/std/min/max of the array of variables where its corresponding flag is a 0. This is my macro:
%macro meanof_precancel(input, meanstat);
j = 77;
do i = 1 to 77;
if cancelled_{i} = 1 then do;
j = i - 1;
call symputx('lastactive', j);
leave;
end;
end;
if j = 0 then &meanstat = 0;
else &meanstat = mean(of &input.1-&input.&lastactive);
%mend;
I am having difficulty finding out how to resolve the line:
else &meanstat = mean(of &input.1-&input.&lastactive);
Does anybody have a strategy to resolve it to something like the following, for j = 33:
else mean_stats = mean(of total_1-total_33);
Thanks in advance.
I used another approach in the end, although it requires a creation of 77 new variables. I created a new array which sets each value to missing whenever its corresponding flag is one, and took the mean of this new array. For those interested :
%macro meanof_precancel(input, meanstat);
array &input.temp{77};
do i = 1 to 77;
if not cancelled_{i} then
&input.temp{i} = &input{i};
else &input.temp{i} = .;
end;
&meanstat = mean(of &input.temp{*});
%mend;
as you figured out you can only sum the values of the entire array, and the reason you were having issues with:
else &meanstat = mean(of &input.1-&input.&lastactive);
is because the call symputx earlier in the macro isn't executed until the datastep has finished.
here is the corresponding SAS documentation:
Problem Trying to Reference a SYMPUT-Assigned Value Before It Is
Available
One of the most common problems in using SYMPUT is trying to reference
a macro variable value assigned by SYMPUT before that variable is
created. The failure generally occurs because the statement
referencing the macro variable compiles before execution of the CALL
SYMPUT statement that assigns the variable's value. The most important
fact to remember in using SYMPUT is that it assigns the value of the
macro variable during program execution, but macro variable references
resolve during the compilation of a step, a global statement used
outside a step, or an SCL program.
As a result: • You cannot use a
macro variable reference to retrieve the value of a macro variable in
the same program (or step) in which SYMPUT creates that macro variable
and assigns it a value.
http://support.sas.com/documentation/cdl/en/mcrolref/61885/HTML/default/viewer.htm#a000210266.htm
this would be destructive of the original data so i would be careful, but it will allow for calculation of std/mean/min/max etc from the original array.
%macro precancel_stat(input, statvar, stat);
j = 77;
do i = 1 to 77;
if cancelled_{i} = 1 then do;
j = i - 1;
do k=i to 77;
&input.{k}=.;
end;
i=77;
end;
end;
if j = 0 then &statvar = 0;
else &statvar = &stat.(of &input.{*});
%mend;
/* test datasets*/
data test;
array sum_me{77} sum1 - sum77;
array cancelled_{77} cancelled1 - cancelled77;
do k=1 to 10;
do i =1 to 77;
sum_me{i}=i;
if i lt 33+k then cancelled_{i}=0; else cancelled_{i}=1;
end;
output;
end;
run;
/* test the macro call*/
data testit ;
set test (drop= i k );
array sum_me{77} sum1 - sum77;
array cancelled_{77} cancelled1 - cancelled77;
%precancel_stat(sum_me,meanstat,mean);
%precancel_stat(sum_me,StDev,STD);
%precancel_stat(sum_me,MinVal,Min);
%precancel_stat(sum_me,MarVal,Max);
%precancel_stat(sum_me,SumVal,sum);
run;
proc print data=testit;
run;
You can't use call symput that way, because the timing is wrong; you need to know the value of &lastactive. during compilation, but you don't actually know it until the data has been looked at.
You can certainly do this with a helper array. I would use a temporary array for this purpose, if you're going to do it that way (array &input.temp[77] _temporary_;) as it won't be written out uselessly to the final dataset and resides only in memory.
Honestly, you might be best off just having two variables, the mean-variable and a counter (your j is that already, basically). Instead of putting it in the temporary array, just
meanvar=meanvar+input[i];
And then at the end of the loop
meanvar=meanvar/j;
That seems more efficient.
I want to test a function func(par1,par2,par3) with all combinations of the parameters par1, par2 and par3 and store the output in a .mat file. My code looks like this at the moment:
n1 = 3;
n2 = 1;
n3 = 2;
parList1 = rand(1,n1); % n1,n2,n3 is just some integer
parList2 = rand(1,n2); % the lists are edited by hand in the actual script
parList3 = rand(1,n3);
saveFile = matfile('file.mat','Writable',true);
% allocate memory
saveFile.output = NaN(numel(parList1),numel(parList2),numel(parList3));
counter1 = 0;
for par1 = parList1
counter1 = counter1 + 1;
counter2 = 0; % reset inner counter
for par2 = parList2
counter2 = counter2 + 1;
counter3 = 0; % reset inner counter
for par3 = parList3
counter3 = counter3 + 1;
saveFile.output(counter1,counter2,counter3) = sum([par1,par2,par3]);
end
end
end
This works except if parList3 has only one item, i.e. if n3 = 1. Then the saveFile.output has singleton dimensions and I get the error
Variable 'output' has 2 dimensions in the file, this does not match the 3 dimensions in the indexing subscripts.
Is there a elegant way to fix this?
The expression in the for statement needs to be a row array, not a column array as in your example. The loops will exit after the first value with your code. Set a breakpoint on the saveFile.output command to see what I mean. With a column array, par1 will not be a scalar as desired, but the whole parList1 column. With a row array, par1 will iterate through each value of parList1 as intended
Another thing is that you need to reset your inner counters (counter2 and counter2) or your second and third dimensions will blow up larger than you expected.
The n3=1 problem is expected behavior because matfile defines the variables with fixed number of dimensions and it will treat saveFile.output as 2D. Once you have fixed those issues, you can solve the n3=1 problem by changing the line,
saveFile.output(counter1,counter2,counter3) = sum([par1,par2,par3]);
to
if n3==1, saveFile.output(counter1,counter2) = sum([par1,par2,par3]);
else saveFile.output(counter1,counter2,counter3) = sum([par1,par2,par3]);
end
By now I realized that actually in matfiles all the singleton dimensions, except for the first two are removed.
In my actual programm I decided to save the data in the file linearly and circumvent matfile's lack of linear indexing capability by using the functions sub2ind and ind2sub.