I'm relatively new to stata and I need to append several .dta files together into one dataset.
I have a folder called 2015 and the files look like the following...
jan_2015.dta
feb_2015.dta
mar_2015.dta
... and so on till dec_2015.
I tried the following code:
cd C:\Users\TOSHIBA\Desktop\Lender_List\Compiled\2015
local mylist jan_2015 feb_2015 mar_2015 apr_2015 may_2015 jun_2015 jul_2015 aug_2015 sep_2015 oct_2015 nov_2015 dec_2015
foreach filename of local mylist {
use `var'_2015
append using "jan_2015.dta"
}
but the output from stata said that file _2015.dta not found.
There's no need to loop: append can take a list of files that you can get with fs:
ssc install fs
cd C:\Users\TOSHIBA\Desktop\Lender_List\Compiled\2015
clear
fs *_2015.dta
append using `r(files)'
If you are still eager to loop over files by name for some reason:
cd C:\Users\TOSHIBA\Desktop\Lender_List\Compiled\2015
clear
foreach filename in `=lower("`c(Mons)'")' {
append using "`filename'_2015.dta"
}
The main problem with your code is that the local macro lname is filename, but you dereference something called var, which evaluates to nothing, so Stata cannot find the file named _2015.dta and complains. The second problem is that your loop seems to try to open each month's file and append January's data to it. That does not sound like what you have in mind.
Related
I'm importing dta files in a folder and exporting each to a csv file. I'm not sure why but the loop doesn't save the file. Here's the code:
global path "file path"
cd "${path}"
* Geocodes for edd
clear
local files: dir "${path}Data\MeasureData\" files "*_edd.dta"
*do loop to bridge file for EDD and
foreach file in `files' {
use "${path}Data/MeasureData/`file'", clear
rename beafips edd_id
merge m:1 edd_id using Data\TempData\bridge_edd.dta
keep if _merge==3
drop _merge
export delimited "${path}Data\OutgoingData\`file'.csv", replace
}
I keep getting error like this:
file filepath\Data\OutgoingData.csv saved
I was expecting this to be saved as filepath\Data`file'.csv. What did I do wrong?
This is happening because Windows defaults to using backslashes in path directories, but in this context your computer is reading the backslash as an escape character and so isn't interpreting the ` as indicating the beginning of your local files.
This problem won't typically appear on a Mac/Linux machine as they default to using forward slashes in directory paths.
So the solution is to change all the \s to /s in your code. See here for a more detailed write-up of the problem: https://journals.sagepub.com/doi/pdf/10.1177/1536867X0800800310
I have multiple .dbf files that I would like to import one at a time, change the name of a variable, and save as a .dta file. The folder that contains the .dbf files contains other files as well that I would like Stata to ignore.
Each of the dbf files is named one of the options listed in the local macro mylist, followed by _ward_CTS.dbf. So for example, B_ward_CTS.dbf is one of the files.
My code is the following:
program drop _all
macro drop _all
set more off
cd "/Users/slums-india/cleaning/maps processing/Ward Point
Maps/Output"
clear
local files : dir "/Users/slums-india/cleaning/maps processing/WardPoint
Maps/Output" files "*.dbf"
local mylist B C D E FN FS GS HE HW KE
foreach file of local mylist {
use 'file'_ward_CTS.dbf
/*import database*/
import dbase "'file'_ward_CTS.dbf", clear
/*rename CTS number variable*/
rename cts$V4 cts_number
save "/Users/slums-india/cleaning/sra/temp/'file'_ward_CTS.dta", replace
}
I cannot seem to get this loop to run. The error I get is invalid 'file'.
What am I doing wrong?
You need to delete the first line in the loop, change ' in file and add quotes:
foreach file of local mylist {
/*import database*/
import dbase "`file'_ward_CTS.dbf", clear
/*rename CTS number variable*/
rename cts$V4 cts_number
save "/Users/slums-india/cleaning/sra/temp/`file'_ward_CTS.dta", replace
}
I am new to lua to trying to understand and put pieces to together and looking out for some help.
I have gone through the existing articles on lua file looping but unable to get the desired output.
Question - I have a folder with files, Folder path - "D:\Test_Files\Outbound\Client\final"
Files in the folder with extension - .txt
Trying to :
Get the count of files in the folder(in this case "final" folder).
Read every file, building a loop something similar to this:
list = {}
for i=0,(#Totalfilecount) do
local fr = io.open('D:\Test_Files\Outbound\Client\final\'..filename.,'rb')
local f = fr.read('*.txt')
Customfunction(f) -- Passing file content to customfunction to apply business logic.
end
Questions :
How to get file count from a directory?
How to read the directory to check if the files with "*.txt" exist?
How to use table list to store each file name and read through the loop?
How to read each file via loop and pass the value to function "Customfunction(f)"?
Code is expected to run on windows. Please share suggestions in pure lua without using external file system functions such as 'lfs' as we do not like to import external functions.
Any Suggestions/help will be greatly appreciated!
You can't (at least shouldn't) do this without extensions to Lua. To accomplish this, you have to download LuaFileSystem library. You can do it using LuaRocks:
$ luarocks install luafilesystem
Use library as such:
require "lfs"
function dirtree(dir)
assert(dir and dir ~= "", "Please pass directory parameter")
if string.sub(dir, -1) == "/" then
dir=string.sub(dir, 1, -2)
end
local function yieldtree(dir)
for entry in lfs.dir(dir) do
if entry ~= "." and entry ~= ".." then
entry=dir.."/"..entry
local attr=lfs.attributes(entry)
coroutine.yield(entry,attr)
if attr.mode == "directory" then
yieldtree(entry)
end
end
end
end
return coroutine.wrap(function() yieldtree(dir) end)
end
An example use of code above:
for filename, attr in dirtree("D:\Test_Files\Outbound\Client\final") do
print(attr.mode, filename)
end
You have to check does extension equal to txt. To read file extension use this snippet:
function GetFileExtension(path)
return path:match("^.+(%..+)$")
end
So, to answer your question(s), you can get amount of files in directory just by counting elements in array returned in dirtree. To answer second question, just use code from the post. Table that you want is returned by dirtree(), but you may want to extract only .txt files from it. To read a file, just check other SO answers. You've got given name (in array), so use it.
EDIT: You can parse result of dir and ls command to get directory listing, but you shouldnt. Althrough this way you wouldn't need to install any libraries, your code is going to be heavily OS-depedent.
Adding libraries to your code isn't so bad. Hacking things is worse.
(Not sure file extension extracting function is going to work. I didn't make dirtree code used in this post, it belongs to David Kastrup)
I am new to Powershell but have had experience with various programming languages including dBase III, basic, Fortran (shows my age). I am a musician and launch MP3 files as an accompanyment when I perform live, somewhat like karaoke. I have about 45 different MP3s that I use. I like to change the sequence of the songs that I perform, from time to time and this requires renaming the MP3 file names with a preceeding sequence number. The file names take on the form: "01 songnameA.mp3", "02 songnameB.mp3", "03 songnameC.mp3", etc. I also repeat songs so there might be an "09 songnameA.mp3" in the folder that I play back from. To do this, I typically create a text file that lists the songs in the order that I want to perform them. Each line in the text file has the form "01 Songname X" but no MP3 extension. I then manually copy the mp3 files into a folder and then edit the names applying a sequence number according to the text file. This is time-consuming.
I have created a Powershell script (version 2) that creates an array of the text file content and an array of the unnumbered MP3 song files. The script creates a 3rd array containing numbered MP3 filenames according to the sequence in the text file. This array does work and I can easily display the list of items in it which have the form "01 SongnameX.mp3". However, I have been unable to copy this array of MP3 filenames with a preceeding sequence number into another folder. I don't know how many variations of the "Copy-Item" statements I have tried but nothing works. The name of the array that contains the filenames is $nsfarray (new song file array). The command I am presently using:
$nsfarray | copy-item -Destination C:\temp
Returns the following error message:
Copy-item : Cannot find path 'C:\Users\My HP\Documents\My Scripts\01 A Good Time.mp3' because it does not exist.
The path is the default path that I use to run Powershell but somehow the MP3 file names get appended to it. The "01 A Good Time.mp3" is the first item in the $nsfarray. I know I am missing something here. Any suggestions would be appreciated.
This is really crude but the issue I see with your input file is that it obviously does not exactly match the real file name that you are looking to copy. So with that in mind lets try this on for size.
$musicPath = "c:\music"
$destinationPath = "c:\songset"
$nsfarray = "01 Age of Aquarius"
$nsfarray | ForEach-Object{
If($_ -match '^\d+ *(?<BaseName>.*)'){
Copy-Item "$musicPath\$($Matches.BaseName).mp3" -Destination "$_.mp3"
}
}
Work with explicit paths: $musicPath and $destinationPath so that we do not have to rely on our current location in PowerShell. Then we navigate each value in the array, which I have populated with one example. Now we need to extract the real file name away from your set list number. Using regex we return the part of the string that is not the beginning numbers and spaces and take everything after that.
After we simply copy the file that exists in the $musicPath folder that has the name "Age of Aquarius.mp3", which should be the real file, and copy it to the $destinationPath as "01 Age of Aquarius.mp3", which is how you want the file for your gig.
Clarification
After answering this and looking back at the question I think there is a chance I didn't understand it right. Please update the question if this is not the right way to address this. The core of your problem, if nothing else, is that you are not specifying a proper file path for the mp3s. You need to use the proper source folder and append the mp3 to the string.
I'm aware of the package touch, which does exactly this. However after installing and using the package, Stata 12 SE (Mac 10.9.5) says:
The touch command has been deprecated.
Is there an alternative? I need this to initialize a blank result file, so that I can append my regression result in a loop without caring whether I'm in the first iteration or not.
NEW ANSWER
Use one of the following in your do file to create the blank xls file "newfile.xls":
shell echo /dev/null > newfile.xls
which will overwrite existing files of the same name. To create a new blank file of that name:
shell touch newfile.xls
would also work. However if a file of the same name exists, touch would only change the modification and access times.
OLD ANSWER
The following will save an empty Stata data file "newdata.dta". Substitute your own name.
save newdata, emptyok
With Mac, you can just use the native touch: !touch filename.xls,
or
!> empty_results.txt, if you want to over-write an existing file or create it if it does not exist.