I am looking for some help with creating a macro utilizing an array as well as DO and IF statements for subsetting. Within my macro statement I am trying to look across columns for variables, and if the variable has a specific diagnosis code to then create a new variable and label as 1, and if not label all others as 0 to create one new data set based on that new variable, sort that data set, and append it to other data sets (because the input data sets are broken down quarterly), thus creating one final data set that I can then export (preferably as a newly created ZIP file to keep storage space down). I am using SAS 9.4/ Enterprise Guide 7.1.
Code:
OPTIONS MERROR SERROR SOURCE MLOGIC SYMBOLGEN MINOPERATOR OBS=MAX;
%MACRO DIAGXX(a,b);
DATA NEW;
SET x.&a(KEEP= PATID DIAG1-DIAG5);
ARRAY &b{5} $ DIAG1-DIAG5;
DO I = 1 TO 5;
IF &b{I} IN ("1630" "1631" "1638" "1639") THEN MESO = 1;
ELSE MESO = 0;
END;
DROP I;
RUN;
PROC SORT DATA=NEW NODUPKEY;
BY PATID;
WHERE MESO=1;
RUN;
PROC APPEND BASE=ALLDATA1 DATA=NEW FORCE;
RUN;
PROC EXPORT
DATA=ALLDATA1
OUTFILE= "C:\x\x\DIAGNOSIS EXPORT\MACRO DIAGXX MESO.CSV"
REPLACE
DBMS=CSV;
RUN;
%MEND DIAGXX;
%DIAGXX(Q1,MESTH);
%DIAGXX(Q2,MESTH);
You probably want to create your MESO flag this way so that the presence of any of the codes in any of the variables in the array will set MESO to true and it will be false when the codes never appear in any of the variables.
MESO = 0;
DO I = 1 TO 5;
IF &b{I} IN ("1630" "1631" "1638" "1639") THEN MESO = 1;
END;
If you want to get fancy you might save a little time by stopping the loop once the code is found.
DO I = 1 TO 5 WHILE (MESO=0);
Related
Thanks in advance for any and all suggestions.
I am working in SAS for the first time to complete a (theoretically) simple task. I have a parent folder in a Windows directory which contains several sub-folders. The sub-folders are not systematically named. For example, if the parent folder is called "W:/Documents/ParentFolder/", then the sub-folders might be "W:/Documents/ParentFolder/ABC1D26/" and "W:/Documents/ParentFolder/HG34A/".
Each sub-folder contains several SAS datasets. In any particular sub-folder, some of the SAS datasets have the .sas7bdat extension and others have the .sd2 extension. Furthermore, no two sub-folders necessarily have the same number of datasets, and the datasets are not systematically named either.
I would like to write a program in SAS which looks inside each sub-folder, loads any .sas7bdat or .sd2 datasets it finds, and exports the dataset into a different folder as a .dta file.
There are too many SAS datasets in each sub-folder to do this task manually for each dataset, but there are not so many sub-folders that I cannot feed the sub-folder names to SAS manually. Below is a commented version of my attempt at a program which completes this task. Unfortunately, I encounter many errors, no doubt due to my inexperience with SAS.
For example, SAS gives the following errors: "ERROR: Invalid logical name;" "ERROR: Error in the FILENAME statement;" and "ERROR: Invalid DO loop control information;" among others.
Can anyone offer any advice?
%macro sas_file_converter();
/* List the sub-folders containing SAS files in the parent folder */
%let folder1 = W:\Documents\ParentFolder\ABC1D26;
%let folder2 = W:\Documents\ParentFolder\HG34A;
/* Start loop over the sub-folders. In each sub-folder, identify all the files, extract the file names, import the files, and export the files. */
%do folder_iter = 1 %to 2;
/* Define the sub-folder that is the focus of this iteration of the loop */
filename workingFolder "&&folder&folder_iter..";
/* Extract a list of datasets in this sub-folder */
data datasetlist;
length Line 8 dataset_name $300;
List = dopen('workingFolder');
do Line = 1 to dnum(List);
dataset_name = tranwrd(tranwrd(lowcase(trim(dread(List,Line))),".sas7bdat",""),".sd2","");
output;
end;
drop List Line;
run;
/* Get number of datasets in this sub-folder */
proc sql nprint;
select count(*)
into :datasetCount
from WORK.datasetlist;
quit;
/* Loop over datasets in the sub-folder. In each iteration of the loop, load the dataset and export the dataset. */
%do dataset_iter = 1 %to &datasetCount.;
/* Get the name of the dataset which is the focus of this iteration */
data _NULL_;
set WORK.DATASETLIST (firstobs=&dataset_iter. obs=&dataset_iter.);
call symput("inMember",strip(dataset_name));
end;
/* Set the libname */
LIBNAME library '&folder&folder_iter..';
/* Load the dataset */
data new;
set library.&inMember.;
run;
/* Export the dataset */
proc export data=library.&inMember.
file = "W:\Documents\OutputFolder\&inMember..dta"
dbms = stata replace;
run;
%end;
%end;
%mend;
Thanks very much for your helpful suggestions. I used the following program to perform this task. It is largely based on Richard's example. I'm posting it here for the benefit of future readers; Richard's example includes additional code that may help you understand what this program does.
Additional files/folders can be accommodated by adding them to the "%let folders" line. (I write many file/folder names here.)
Note that I separate the sub-folders with three dashes ("---") because some of the files and sub-sub-folders have spaces in their names. Note also that for the .sd2 files, I was able to simply replace the instances of "sas7bdat" with "sd2" and the program worked fine.
Thanks again.
%let inputfolder = W:\Documents\ParentFolder;
%let folders = ABC1D26---HG34A---Sub Folder\ZH323;
%let exportfolder = W:\Documents\ExportFolder;
data _null_;
do findex = 1 to countw("&folders.","---");
folder = scan("&folders", findex, "---");
path = catx("/", "&dataroot.", folder);
call execute ('libname user ' || quote(trim(path)) || ';');
length fileref $8;
call missing(fileref);
rc = filename(fileref, path);
did = dopen(fileref);
do dindex = 1 to dnum(did);
filename = lowcase(dread(did,dindex));
if scan(filename,-1) ne 'sas7bdat' then continue;
xptfilename = tranwrd(filename, '.sas7bdat', '.dta')
xptfilepath = catx("\", "&exportpath", folder, xptfilename);
datasetname = tranwrd(filename, '.sas7bdat', '');
sascode = 'PROC EXPORT data=' || trim(datasetname) || " replace file=" || quote(trim(xptfilepath)) || " dbms=stata; run;";
call execute (trim(sascode));
end;
did = dclose(did);
call execute ('libname user clear;');
rc = filename(fileref);
end;
run;
You can perform all the code generation in a DATA Step and submit via CALL EXECUTE. The only part of the program that would be macro related is specifying the sas data root folder, the names of the sub-folders to search and the export path.
The program could be very similarly macro coded, but could be tougher to debug, and would require %sysfunc wrappers around the function calls.
Example:
/* Create some sample data in some example folders */
%let workpath = %sysfunc(pathname(WORK));
%let name = %sysfunc(dcreate(ABC, &workpath));
%let name = %sysfunc(dcreate(DEF, &workpath));
libname user "&workpath./ABC";
data one two three four five;
set sashelp.class;
run;
libname user "&workpath./DEF";
data six seven eight nine ten;
set sashelp.class;
run;
libname user clear;
/* export all data sets in folders to liked named export files */
%let dataroot = &workpath;
%let folders = ABC DEF;
%let exportpath = c:\temp;
data _null_;
do findex = 1 to countw("&folders");
folder = scan("&folders", findex);
path = catx("/", "&dataroot.", folder);
call execute ('libname user ' || quote(trim(path)) || ';');
length fileref $8;
call missing(fileref);
rc = filename(fileref, path);
did = dopen(fileref);
do dindex = 1 to dnum(did);
filename = dread(did,dindex);
if scan(filename,-1) ne 'sas7bdat' then continue;
xptfilename = tranwrd(filename, '.sas7bdat', '.dta');
xptfilepath = catx('/', "&exportpath", xptfilename);
datasetname = tranwrd(filename, '.sas7bdat', '');
sascode = 'PROC EXPORT data=' || trim(datasetname)
|| " replace file=" || quote(trim(xptfilepath))
|| " dbms=stata;"
;
call execute (trim(sascode));
end;
did = dclose(did);
call execute ('run; libname user clear;');
rc = filename(fileref);
end;
run;
Currently my code have simple tables containing the data needed for each object like this:
infantry = {class = "army", type = "human", power = 2}
cavalry = {class = "panzer", type = "motorized", power = 12}
battleship = {class = "navy", type = "motorized", power = 256}
I use the tables names as identifiers in various functions to have their values processed one by one as a function that is simply called to have access to the values.
Now I want to have this data stored in a spreadsheet (csv file) instead that looks something like this:
Name class type power
Infantry army human 2
Cavalry panzer motorized 12
Battleship navy motorized 256
The spreadsheet will not have more than 50 lines and I want to be able to increase columns in the future.
Tried a couple approaches from similar situation I found here but due to lacking skills I failed to access any values from the nested table. I think this is because I don't fully understand how the tables structure are after reading each line from the csv file to the table and therefore fail to print any values at all.
If there is a way to get the name,class,type,power from the table and use that line just as my old simple tables, I would appreciate having a educational example presented. Another approach could be to declare new tables from the csv that behaves exactly like my old simple tables, line by line from the csv file. I don't know if this is doable.
Using Lua 5.1
You can read the csv file in as a string . i will use a multi line string here to represent the csv.
gmatch with pattern [^\n]+ will return each row of the csv.
gmatch with pattern [^,]+ will return the value of each column from our given row.
if more rows or columns are added or if the columns are moved around we will still reliably convert then information as long as the first row has the header information.
The only column that can not move is the first one the Name column if that is moved it will change the key used to store the row in to the table.
Using gmatch and 2 patterns, [^,]+ and [^\n]+, you can separate the string into each row and column of the csv. Comments in the following code:
local csv = [[
Name,class,type,power
Infantry,army,human,2
Cavalry,panzer,motorized,12
Battleship,navy,motorized,256
]]
local items = {} -- Store our values here
local headers = {} --
local first = true
for line in csv:gmatch("[^\n]+") do
if first then -- this is to handle the first line and capture our headers.
local count = 1
for header in line:gmatch("[^,]+") do
headers[count] = header
count = count + 1
end
first = false -- set first to false to switch off the header block
else
local name
local i = 2 -- We start at 2 because we wont be increment for the header
for field in line:gmatch("[^,]+") do
name = name or field -- check if we know the name of our row
if items[name] then -- if the name is already in the items table then this is a field
items[name][headers[i]] = field -- assign our value at the header in the table with the given name.
i = i + 1
else -- if the name is not in the table we create a new index for it
items[name] = {}
end
end
end
end
Here is how you can load a csv using the I/O library:
-- Example of how to load the csv.
path = "some\\path\\to\\file.csv"
local f = assert(io.open(path))
local csv = f:read("*all")
f:close()
Alternative you can use io.lines(path) which would take the place of csv:gmatch("[^\n]+") in the for loop sections as well.
Here is an example of using the resulting table:
-- print table out
print("items = {")
for name, item in pairs(items) do
print(" " .. name .. " = { ")
for field, value in pairs(item) do
print(" " .. field .. " = ".. value .. ",")
end
print(" },")
end
print("}")
The output:
items = {
Infantry = {
type = human,
class = army,
power = 2,
},
Battleship = {
type = motorized,
class = navy,
power = 256,
},
Cavalry = {
type = motorized,
class = panzer,
power = 12,
},
}
proc iml;
use rdata3;
read all var _all_ into pp;
close rdata3;
do i = 1 to 1050;
perms = allperm(pp[i, ]);
create pp&i from perms[colname= {"Best" "NA1" "NA2" "Worst"}];
append from perms;
close pp&i;
end;
I will like to create multiple datasets in SAS using the above code through a do loop. However, i cant seem to change the name of each dataset using the &i indicator. Can anyone help me change my code to allow me to create multiple datasets? Or are there any other alternatives on how to create multiple datasets from matrix through loops? Thanks in advance.
You don't want to use macro variables you want to use the features of IML. However you will be creating an awful lot of data sets.
data rdata3;
x = 1;
y = 2;
a = 4;
b = 5;
output;
output;
run;
proc iml;
use rdata3;
read all var _all_ into pp;
close rdata3;
do i = 1 to nrow(pp);
outname = cats('pp',putn(i,'z5.'));
perms = allperm(pp[i, ]);
create (outname) from perms[colname= {"Best" "NA1" "NA2" "Worst"}];
append from perms;
close (outname);
end;
quit;
You can add an ID variable to PERMS and append all versions of PERMS into one data set. I'm not sure I used the best IML technique, I know just enough IML to be dangerous.
proc iml;
use rdata3;
read all var _all_ into pp;
close rdata3;
perms = j(1,5,0);
create PP_out from perms[colname= {'ID' "Best" "NA1" "NA2" "Worst"}];
do i = 1 to nrow(pp);
perms = allperm(pp[i, ]);
perms = j(nrow(perms),1,i)||perms;
append from perms;
end;
close PP_out;
quit;
How can I update the below code so that I don't get the 'log window full' message? Most of the lines generated in the log window are due to proc corr. I tried adding the proc printto line at the beginning of the code but log window still gets filled up for some reason. Thanks.
PROC PRINTTO PRINT='C:\Users\test\auto.lst' NEW;
RUN;
%MACRO RunProgram(month, year, n);
data sourceh.group2;
set sourceh.group_&month.&year.;
int1=int;
int2 = ceil(int/2);
int3 = ceil(int/3);
int4 = ceil(int/4);
int5 = ceil(int/5);
int6 = ceil(int/15);
int7 = ceil(int/30);
proc sort data=sourceh.group2;
by symbol day month year int&n.;
run;
proc corr data=sourceh.group2; by symbol day;
var zone ztwo;
ods output pearsoncorr=sourceh.zcorr;
run;
%MEND ;
%macro l;
%do n=1 %to 7;
%RunProgram(Dec, 2014, &n);
%RunProgram(Nov, 2014, &n);
%end;
%mend;
%l;
Redirect your log using proc
Printto.
Proc printto log='templog.log' new;
Run;
You can reset if afterwards using
Proc printto log=log; run;
Alternatively you can set the option nonotes on so that the log doesn't get output unless there's an error. This can make it hard to debug.
Option nonotes;
Turn notes option back on:
Option notes;
Generally, I think you should output your log so that you can check to see if something's gone wrong; Reeza's answer addresses this approach. However, you can also just clear the log by using the command
dm 'clear log';
If you insert this as the first or last line of your RunProgram macro, your log will clear on each iteration of the macro. This will get rid of your problem so long as one iteration of the macro doesn't fill up your log.
i am trying to retrieve the difference of qty and qts shipped for a sales order.
i am doing this through code.
setting range on sales line table fields and using findset does loop through all lines properly but while printing it gives the difference frm the last line.
clearing the variable is also not working.
i am new to NAV 2013 so not able to find out how to loop through all this lines so that it displays result properly.i tried using findfirst inside the if loop but no success.
You need to add to "Value", not overwrite it. (use +=, not :=)
CLEAR(Value);
SalesLine.RESET;
SalesLine.SETRANGE(SalesLine."Document No.","No.");
IF SalesLine.FINDSET THEN REPEAT
Value += SalesLine.Quantity - SalesLine."Quantity Shipped";
//MESSAGE('%1',Value);
UNTIL SalesLine.NEXT =0;
Try this code
local procedure DailyCalc(DailyLotNumber: Query DailyLotNumber): Decimal
var
SalesLine: Record "Sales Line";
NewRec: Decimal;
begin
SalesLine.SetRange("Document No.", DailyLotNumber.No_);
SalesLine.SetRange("Document Type", DailyLotNumber.Document_Type);
if SalesLine.FindSet() then begin
repeat
repeat
NewRec += (SalesLine.Quantity - SalesLine."Quantity Shipped") * SalesLine."Unit Price";
until SalesLine.Next() = 0;
exit(NewRec);
until SalesLine.Next() = 0;
end;
end;