I have a SAS macro program to test opening a file and outputting the total number of observations in the log. When I run it through SAS, I am able to output the number of observations in the SAS log (as intended). However, when I run the SAS program through a batch script, the file is not detected/can not be opened. What am I doing incorrectly?
See code for SAS program:
%macro syserrClass;
options symbolgen;
/* Define macro variables */
%let usernam = %sysget(pUserNam);
%let evSuffix = %sysget(pSuffix); /* Suffix of program name: eg. year 2018 */
%let evSeq = %sysget(pSeq);
%let evLib01 = %sysget(pLib01); /* SAS lib name for SAS data set */
*%let usernam = myusername;
*%let evSuffix = 2014;
*%let evSeq = 02;
*%let evLib01 = mor2014; /* SAS lib name for SAS data set */
%let filen03 = ije_mortality_&evSuffix.&evSeq._err; /* Input file */
LIBNAME &evLib01. v9 "C:\Users\&usernam.\Documents\My SAS Files\9.4\&evLib01.";
/* Open data set */
%let dsId1 = %sysfunc(open(&evLib01..&filen03.));
/* If the data set exists, get the number of observations */
%if &dsId1. %then %do;
%let nobs1 = %sysfunc(attrn(&dsId1.,nobs));
%let rc = %sysfunc(close(&dsId1.));
%put &evLib01..&filen03. has &nobs1. observation(s).;
%end;
%else
%put Open for data set &evLib01..&filen03. failed - %sysfunc(sysmsg());
%mend;
/* Call macro */
%syserrClass;
See code for batch script (.cmd):
#echo off
setlocal
set "pUserNam=%username%"
set "pSuffix=%1"
set "pSeq=%2"
set "pLib01=%3"
set inPath01="C:\Users\%pUserNam%\Documents"
set inPath02="My SAS Files\9.4\"
set pInPath03=C:\Users\%pUserNam%\Documents\log
set pInPath05="C:\Program Files\SASHome\SASFoundation\9.4\sas.exe"
set pInPath06="C:\Program Files\SASHome\x86\SASFoundation\9.4\sas.exe"
rem set SAS exe path depending on existence of path
if exist %pInPath05% (set programname=%pInPath05%) else (set programname=%pInPath06%)
rem folder with output files - sas
set pLib02=mor%pSuffix%
rem Print summary of exceptions grouped by file name
set sasprogram="C:\Users\%username%\Documents\sascode\test_exceptrpt_macro01.sas"
%programname% -sysin %sasprogram% -log %pInPath03%\ije_mortality_exceptrpt01_%pSuffix%%pSeq%.txt -set pUserNam %pUserNam% -set pSuffix %pSuffix% -set pSeq %pSeq% -set pLib01 %pLib02%
echo SAS Program: %sasprogram%
if %ERRORLEVEL% GTR 1 (
echo ERROR: Return code = %ERRORLEVEL%
goto endproc
) else (echo NOTE: Return code = %ERRORLEVEL%)
:endproc
The main output that stands out in the log when I run through batch is:
SYMBOLGEN: Macro variable DSID1 resolves to 0
SYMBOLGEN: Macro variable EVLIB01 resolves to mor2014
SYMBOLGEN: Macro variable FILEN03 resolves to ije_mortality_201402_err
Open for data set mor2014.ije_mortality_201402_err failed - ERROR: Libref MOR2014 is not assigned.
However, when I switch up the SAS code to use the SAS interactive variables (see how I commented the second group of %let of usernam, evSuffix, evSeq, and evLib01), I get this in the log:
SYMBOLGEN: Macro variable DSID1 resolves to 2
SYMBOLGEN: Macro variable EVLIB01 resolves to mor2014
SYMBOLGEN: Macro variable FILEN03 resolves to ije_mortality_201402_err
SYMBOLGEN: Macro variable NOBS1 resolves to 5
mor2014.ije_mortality_201402_err has 5 observation(s).
Turn on MPRINT so you can see the full text of the LIBNAME statement. That will help you resolve the error.
Open for data set mor2014.ije_mortality_201402_err failed - ERROR: Libref MOR2014 is not assigned.
Related
Snowlake allows to define variables:
SET var = 1;
SHOW VARIABLES;
/*
name value type comment
VAR 1 fixed
*/
We could define second "case-sensitive" version:
SET "var" = '9';
SHOW VARIABLES;
/*
name value type comment
VAR 1 fixed
var 9 text
*/
Now trying to access both variables:
SELECT $var;
--1
SELECT $VAR;
--1
SELECT $"var";
-- SQL compilation error: syntax error line 1 at position 7 unexpected '$'.
The only way I found to access "var" is by using:
SELECT GETVARIABLE('VAR'), GETVARIABLE('var');
/*
GETVARIABLE('VAR') GETVARIABLE('VAR')
1 9
*/
Is it possible to get 9 using $var_name syntax?
Thanks in advance for any and all suggestions.
I am working in SAS for the first time to complete a (theoretically) simple task. I have a parent folder in a Windows directory which contains several sub-folders. The sub-folders are not systematically named. For example, if the parent folder is called "W:/Documents/ParentFolder/", then the sub-folders might be "W:/Documents/ParentFolder/ABC1D26/" and "W:/Documents/ParentFolder/HG34A/".
Each sub-folder contains several SAS datasets. In any particular sub-folder, some of the SAS datasets have the .sas7bdat extension and others have the .sd2 extension. Furthermore, no two sub-folders necessarily have the same number of datasets, and the datasets are not systematically named either.
I would like to write a program in SAS which looks inside each sub-folder, loads any .sas7bdat or .sd2 datasets it finds, and exports the dataset into a different folder as a .dta file.
There are too many SAS datasets in each sub-folder to do this task manually for each dataset, but there are not so many sub-folders that I cannot feed the sub-folder names to SAS manually. Below is a commented version of my attempt at a program which completes this task. Unfortunately, I encounter many errors, no doubt due to my inexperience with SAS.
For example, SAS gives the following errors: "ERROR: Invalid logical name;" "ERROR: Error in the FILENAME statement;" and "ERROR: Invalid DO loop control information;" among others.
Can anyone offer any advice?
%macro sas_file_converter();
/* List the sub-folders containing SAS files in the parent folder */
%let folder1 = W:\Documents\ParentFolder\ABC1D26;
%let folder2 = W:\Documents\ParentFolder\HG34A;
/* Start loop over the sub-folders. In each sub-folder, identify all the files, extract the file names, import the files, and export the files. */
%do folder_iter = 1 %to 2;
/* Define the sub-folder that is the focus of this iteration of the loop */
filename workingFolder "&&folder&folder_iter..";
/* Extract a list of datasets in this sub-folder */
data datasetlist;
length Line 8 dataset_name $300;
List = dopen('workingFolder');
do Line = 1 to dnum(List);
dataset_name = tranwrd(tranwrd(lowcase(trim(dread(List,Line))),".sas7bdat",""),".sd2","");
output;
end;
drop List Line;
run;
/* Get number of datasets in this sub-folder */
proc sql nprint;
select count(*)
into :datasetCount
from WORK.datasetlist;
quit;
/* Loop over datasets in the sub-folder. In each iteration of the loop, load the dataset and export the dataset. */
%do dataset_iter = 1 %to &datasetCount.;
/* Get the name of the dataset which is the focus of this iteration */
data _NULL_;
set WORK.DATASETLIST (firstobs=&dataset_iter. obs=&dataset_iter.);
call symput("inMember",strip(dataset_name));
end;
/* Set the libname */
LIBNAME library '&folder&folder_iter..';
/* Load the dataset */
data new;
set library.&inMember.;
run;
/* Export the dataset */
proc export data=library.&inMember.
file = "W:\Documents\OutputFolder\&inMember..dta"
dbms = stata replace;
run;
%end;
%end;
%mend;
Thanks very much for your helpful suggestions. I used the following program to perform this task. It is largely based on Richard's example. I'm posting it here for the benefit of future readers; Richard's example includes additional code that may help you understand what this program does.
Additional files/folders can be accommodated by adding them to the "%let folders" line. (I write many file/folder names here.)
Note that I separate the sub-folders with three dashes ("---") because some of the files and sub-sub-folders have spaces in their names. Note also that for the .sd2 files, I was able to simply replace the instances of "sas7bdat" with "sd2" and the program worked fine.
Thanks again.
%let inputfolder = W:\Documents\ParentFolder;
%let folders = ABC1D26---HG34A---Sub Folder\ZH323;
%let exportfolder = W:\Documents\ExportFolder;
data _null_;
do findex = 1 to countw("&folders.","---");
folder = scan("&folders", findex, "---");
path = catx("/", "&dataroot.", folder);
call execute ('libname user ' || quote(trim(path)) || ';');
length fileref $8;
call missing(fileref);
rc = filename(fileref, path);
did = dopen(fileref);
do dindex = 1 to dnum(did);
filename = lowcase(dread(did,dindex));
if scan(filename,-1) ne 'sas7bdat' then continue;
xptfilename = tranwrd(filename, '.sas7bdat', '.dta')
xptfilepath = catx("\", "&exportpath", folder, xptfilename);
datasetname = tranwrd(filename, '.sas7bdat', '');
sascode = 'PROC EXPORT data=' || trim(datasetname) || " replace file=" || quote(trim(xptfilepath)) || " dbms=stata; run;";
call execute (trim(sascode));
end;
did = dclose(did);
call execute ('libname user clear;');
rc = filename(fileref);
end;
run;
You can perform all the code generation in a DATA Step and submit via CALL EXECUTE. The only part of the program that would be macro related is specifying the sas data root folder, the names of the sub-folders to search and the export path.
The program could be very similarly macro coded, but could be tougher to debug, and would require %sysfunc wrappers around the function calls.
Example:
/* Create some sample data in some example folders */
%let workpath = %sysfunc(pathname(WORK));
%let name = %sysfunc(dcreate(ABC, &workpath));
%let name = %sysfunc(dcreate(DEF, &workpath));
libname user "&workpath./ABC";
data one two three four five;
set sashelp.class;
run;
libname user "&workpath./DEF";
data six seven eight nine ten;
set sashelp.class;
run;
libname user clear;
/* export all data sets in folders to liked named export files */
%let dataroot = &workpath;
%let folders = ABC DEF;
%let exportpath = c:\temp;
data _null_;
do findex = 1 to countw("&folders");
folder = scan("&folders", findex);
path = catx("/", "&dataroot.", folder);
call execute ('libname user ' || quote(trim(path)) || ';');
length fileref $8;
call missing(fileref);
rc = filename(fileref, path);
did = dopen(fileref);
do dindex = 1 to dnum(did);
filename = dread(did,dindex);
if scan(filename,-1) ne 'sas7bdat' then continue;
xptfilename = tranwrd(filename, '.sas7bdat', '.dta');
xptfilepath = catx('/', "&exportpath", xptfilename);
datasetname = tranwrd(filename, '.sas7bdat', '');
sascode = 'PROC EXPORT data=' || trim(datasetname)
|| " replace file=" || quote(trim(xptfilepath))
|| " dbms=stata;"
;
call execute (trim(sascode));
end;
did = dclose(did);
call execute ('run; libname user clear;');
rc = filename(fileref);
end;
run;
How can I update the below code so that I don't get the 'log window full' message? Most of the lines generated in the log window are due to proc corr. I tried adding the proc printto line at the beginning of the code but log window still gets filled up for some reason. Thanks.
PROC PRINTTO PRINT='C:\Users\test\auto.lst' NEW;
RUN;
%MACRO RunProgram(month, year, n);
data sourceh.group2;
set sourceh.group_&month.&year.;
int1=int;
int2 = ceil(int/2);
int3 = ceil(int/3);
int4 = ceil(int/4);
int5 = ceil(int/5);
int6 = ceil(int/15);
int7 = ceil(int/30);
proc sort data=sourceh.group2;
by symbol day month year int&n.;
run;
proc corr data=sourceh.group2; by symbol day;
var zone ztwo;
ods output pearsoncorr=sourceh.zcorr;
run;
%MEND ;
%macro l;
%do n=1 %to 7;
%RunProgram(Dec, 2014, &n);
%RunProgram(Nov, 2014, &n);
%end;
%mend;
%l;
Redirect your log using proc
Printto.
Proc printto log='templog.log' new;
Run;
You can reset if afterwards using
Proc printto log=log; run;
Alternatively you can set the option nonotes on so that the log doesn't get output unless there's an error. This can make it hard to debug.
Option nonotes;
Turn notes option back on:
Option notes;
Generally, I think you should output your log so that you can check to see if something's gone wrong; Reeza's answer addresses this approach. However, you can also just clear the log by using the command
dm 'clear log';
If you insert this as the first or last line of your RunProgram macro, your log will clear on each iteration of the macro. This will get rid of your problem so long as one iteration of the macro doesn't fill up your log.
I am looking for some help with creating a macro utilizing an array as well as DO and IF statements for subsetting. Within my macro statement I am trying to look across columns for variables, and if the variable has a specific diagnosis code to then create a new variable and label as 1, and if not label all others as 0 to create one new data set based on that new variable, sort that data set, and append it to other data sets (because the input data sets are broken down quarterly), thus creating one final data set that I can then export (preferably as a newly created ZIP file to keep storage space down). I am using SAS 9.4/ Enterprise Guide 7.1.
Code:
OPTIONS MERROR SERROR SOURCE MLOGIC SYMBOLGEN MINOPERATOR OBS=MAX;
%MACRO DIAGXX(a,b);
DATA NEW;
SET x.&a(KEEP= PATID DIAG1-DIAG5);
ARRAY &b{5} $ DIAG1-DIAG5;
DO I = 1 TO 5;
IF &b{I} IN ("1630" "1631" "1638" "1639") THEN MESO = 1;
ELSE MESO = 0;
END;
DROP I;
RUN;
PROC SORT DATA=NEW NODUPKEY;
BY PATID;
WHERE MESO=1;
RUN;
PROC APPEND BASE=ALLDATA1 DATA=NEW FORCE;
RUN;
PROC EXPORT
DATA=ALLDATA1
OUTFILE= "C:\x\x\DIAGNOSIS EXPORT\MACRO DIAGXX MESO.CSV"
REPLACE
DBMS=CSV;
RUN;
%MEND DIAGXX;
%DIAGXX(Q1,MESTH);
%DIAGXX(Q2,MESTH);
You probably want to create your MESO flag this way so that the presence of any of the codes in any of the variables in the array will set MESO to true and it will be false when the codes never appear in any of the variables.
MESO = 0;
DO I = 1 TO 5;
IF &b{I} IN ("1630" "1631" "1638" "1639") THEN MESO = 1;
END;
If you want to get fancy you might save a little time by stopping the loop once the code is found.
DO I = 1 TO 5 WHILE (MESO=0);
I have the following macro:
rsubmit;
data indexsecid;
input secid 1-6;
datalines;
108105
109764
102456
102480
101499
102434
107880
run;
%let endyear = 2014;
%macro getvols1;
* First I extract the secids for all the options given a date and
an expiry date;
%do yearnum = 1996 %to &endyear;
proc sql;
create table volsurface1&yearnum as
select a.secid, a.date, a.days, a.delta, a.impl_volatility,
a.impl_strike, a.cp_flag
from optionm.vsurfd&yearnum as a, indexsecid as b
where a.secid=b
and a.impl_strike NE -99.99
order by a.date, a.secid, a.impl_strike;
quit;
%if &yearnum > 1996 %then %do;
proc append base= volsurface11996 data=volsurface1&yearnum;
run;
%end;
%end;
%mend;
%getvols1;
proc download data=volsurface11996;
run;
endrsubmit;
data _null_;
set work.volsurface11996;
length fv $ 200;
fv = "C:\Users\user\Desktop\" || TRIM(put(indexsecid,4.)) || ".csv";
file write filevar=fv dsd dlm=',' lrecl=32000 ;
put (_all_) (:);
run;
On the code above I have: where a.secid=108105. Now I have a list with several secid and I need to run the macro once for each secid. I am looking to run it once and generate a new dataset for each secid.
How can I do that? Thanks
Here is an approach that uses
A single data step set statement to combine all the input datasets
A data set list so you don't have to call each input by name
A hash table to limit the output to your list of secids
proc sort to order the output
Rezza/DWal's approach to output separate csvs with file filevar =
%let startyear = 1996;
%let endyear = 2014;
data volsurface1;
/* Read in all the input tables */
set optionm.vsurfd&startyear.-optionm.vsurfd&endyear.;
where impl_strike ~= -99.99;
/* Set up a hash table containing all the wanted secids */
if _N_ = 1 then do;
declare hash h(dataset: "indexsecid");
_rc = h.defineKey("secid");
_rc = h.defineDone();
end;
/* Only keep observations where secid is matched in the hash table */
if not h.find();
/* Select which variables to output */
keep secid date days delta impl_volatility impl_strike cp_flag;
run;
/* Sort the data */
proc sort data = volsurface1;
by secid date secid impl_strike;
run;
/* Write out a CSV for each secid */
data _null_;
set volsurface1;
length fv $200;
fv = "\path\to\output\" || trim(put(secid, 6.)) || ".csv";
file write filevar = fv dsd dlm = ',' lrecl = 32000;
put (_all_) (:);
run;
As I don't have your data this is untested. The only constraint I can see is that the contents of indexsecid must fit in memory. If you were not concerned with the order this could be all done in one data step.
SRSwift thank you for your comprehensive answer. It run smoothly with no errors. The only issue is that I am running it on a remote server (wharton) using:
%let wrds=wrds.wharton.upenn.edu 4016;
options comamid=TCP remote=wrds;
signon username=_prompt_;
rsubmit;
and on the log it says it wrote the file to my folder on the server but I can t see any file on the server. The log says:
NOTE: The file WRITE is:
Filename=/home/uni/user/108505.csv,
Owner Name=user,Group Name=uni,
Access Permission=rw-r--r--,
Last Modified=Wed Apr 1 20:11:20 2015