with this line i will get the data2016 but what if i have a multiple years data (2017, 2018, 2019 ...) how can write the statment
COPY INTO "DATA_HUB_DEV"."STAGE"."mytable" ("/BIC/ZDBTVABIS") FROM (SELECT $1:"/BIC/ZDBTVABIS" FROM #STAGE_DATA_LAKE_RAW/data2016);
file format = parquet
Snowflake
beginner
thanks
Assuming they are all in a different folder under your stage, you could simply remove the folder from your stage path and use a PATTERN to grab the files underneath it.
There are some good examples in the documentation link here:
https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#loading-using-pattern-matching
Related
below copy command is not working , please correct me if something wrong.
copy into mytable from #mystage pattern='20.*csv.gz'
here i am trying to load the files which are starts with 20, there are mix of files which are having the name as 2021myfile.csv.gz, myfile202109.csv.gz, above command is not loading any files though there are files which starts with 20.
if i use pattern as pattern='.*20.*csv.gz'`` it is taking all the files which is wrong, i need to load only the files that are starts with 20`.
Thanks!
This is because the pattern clause is a Regex expression.
Try this:
copy into mytable from #mystage pattern = '[20]*.*csv.gz'
Reference: Loading Using Pattern Matching
I've created an S3 [external] stage and uploaded csv files into \stage*.csv folder.
I can see stage content by doing list #my_stage.
if I query the stage
select $1,$2,$3,$4,$5,$6 from #my_s3_stage it looks like I'm randomly picking up files.
So I'm trying to select from specific file by adding a pattern
PATTERN => job.csv
This returns no results.
Note: I've used snowflake for all of 5 hours so pretty new to syntax
For a pattern you can use
select t.$1, t.$2 from #mystage1 (file_format => 'myformat', pattern=>'.*data.*[.]csv.gz') t;
The pattern is a regex expression.
For a certain file you have to add the file name to the query like this:
select t.$1, t.$2 from #mystage/data1.csv.gz;
If your file format is set in your stage definition, you don't need the file format-parameter.
More info can be found here: https://docs.snowflake.com/en/user-guide/querying-stage.html
I am working on a procedure to read files from PL / SQL I have managed to read a specific file but now I want to do it dynamic because I will have multiple files starting with the date for example:
20190218 111833_Worker.dat
20190218 114249_Job.dat
20190218 114353_Organizations
What I need is to read the most recent file of that example presented.
Any opinion is appreciated.
Regards.
"How do I list files in a directory using PL/SQL" has been asked before (e.g. one, two). You have several options:
Use external tables with a preprocessor script - example 1, example 2
Create a java class - example 1, example 2
Use SYS.DBMS_BACKUP_RESTORE.searchFiles with the undocumented X$KRBMSFT - example 1, example 2 (I couldn't get this to work on 12c, it's possible that it only works on 11g and earlier)
Once you have a way to select a list of your filenames, you can just do an order by and filter it to return only the first row, e.g.
select filename
from my_file_list
order by substr(filename, 1, 15) desc
fetch first 1 row only;
I am not expert about AS400, just know some commands and i exporti some files from AS400 (iSeries) into SQL Server 2005.
Actually i need to know which RPG Program created a file in a library. This because that file contains statistic data from other files stored in other AS400 libraries.
This screenshot show the file STTMVF in the library DAT_4DWH (by DSPLIB DAT_4DWH)
So there are a command that let me know which RPG program created the file STTMVF ?
If yes i need to open the source RPG or CL and try to understand which phisical files are used to compose this statistic file.
Thanks in advance!
You can use journal management or program references to determine what is writing to the file.
Journal management
Starting the journal
To create a basic journal you need to create a journal receiver, a journal, and activate journalling for the file. Replace RECEIVER-LIB, RECEIVER-FILE, JOURNAL-LIB, JOURNAL-FILE, FILE-LIB and FILE with values appropriate for your system.
CRTJRNRCV JRNRCV(RECEIVER-LIB/RECEIVER-FILE)
CRTJRN JRN(JOURNAL-LIB/JOURNAL-FILE) JRNRCV(RECEIVER-LIB/RECEIVER-FILE)
STRJRNPF FILE(FILE-LIB/FILE) JRN(JOURNAL-LIB/JOURNAL-FILE) OMTJRNE(*OPNCLO)
Dumping the journal
DSPJRN JRN(JOURNAL-LIB/JOURNAL-FILE) FILE(FILE-LIB/FILE) RCVRNG(*CURCHAIN) JRNCDE(R) ENTTYP(PT PX DL UP) OUTPUT(*OUTFILE) OUTFILFMT(*TYPE1) OUTFILE(QTEMP/QADSPJRN)
Querying the journal
The field JOPGM will contain the program name that inserted, updated, or deleted records from the file.
Removing the journal
ENDJRNPF FILE(FILE-LIB/FILE)
DLTJRN JRN(JOURNAL-LIB/JOURNAL-FILE)
Program references
Dumping the references
DSPPGMREF PGM(*ALLUSR/*ALL) OUTPUT(*OUTFILE) OUTFILE(QTEMP/QADSPPGM)
Querying the references
Search the file for all references where the field WHFNAM equals FILE. The field WHPNAM will contain the program name. Due to file overrides, etc this method is not as accurate as using a journal.
I just want to know if there could be any way by which we can read a value from an .xls file using a .bat file.
For eg:If i have an .xls named test.xls which is having two columns
namely 'EID' and then 'mail ID'.Now when we give the input to the .xls the EID name.it should extract the mail id which corresponds to the EID and echo the result out.
**EID** **MailID**
E22222 MynameisA#company.com
E33333 MynameisB#company.com
...
...
So by the above table,when i give the input to the xls file using my .bat file as E22222,it should read the corresponding mail ID as MynameisA#company.com and it should echo the value.
So i hope i am able to present my doubt.Please get back to me for more clarifications.
Thanks and regards
Maddy
There is no facility to do this directly with traditional .bat files. However, you might investigate PowerShell, which is designed to be able to do this sort of thing. PowerShell integrates well with existing Windows applications (such as Excel) and may provide the tools you need to do this easily.
A quick search turned up this example of reading Excel files from PowerShell.
You can't do this directly from a batch file. Furthermore, to manipulate use Excel files in scripting you need Excel to be installed.
What you can do is wrap the Excel-specific stuff in a VBScript and call that from your batch.
You can do it with Alacon - command-line utility for Alasql database.
It works with Node.js, so you need to install Node.js and then Alasql package:
To take data from Excel file you can use the following command:
> node alacon "SELECT VALUE [mail ID] FROM XLS('mydata.xls', {headers:true})
WHERE EID = ?" "E2222"
Fist parameter is a SQL-expresion, which read data from XLSX file with header and search data
for second parameter value: "E22222". The command returns mail ID value.
This will be hard (very close to impossible) in BAT, especially when using the original XLS file, but even after an export to CSV it will be much easier to use a script/programming language (Perl, C, whatever) to do this.