Importing text files or compressed files in QlikView using a batch file - batch-file

I have multiple text files in a folder. I need those files to be imported into QlikView on a daily basis. Is there any way to import those files using batch/command file?
Moreover, can I import compressed files into QlikView?

I am not sure how your load script is set up, but if you wish to refresh your QlikView document, and you don't have QlikView Server, then you can use a batch file as follows:
"<Path To QlikView>\QV.exe" /r "ReportToReload.qvw"
The /r command parameter tells QlikView to open the document, reload it and then save and close the document. However, you must make sure that the QlikView User Preference option "Keep Progress Open after Reload" is not enabled, otherwise the progress dialogue will wait for you to close it after the document has been reloaded.
You can then schedule this batch file to run via Windows' Task Scheduler, or your favourite scheduling tool.
QlikView cannot import compressed files (e.g. Zip/RAR etc.), so you would need to extract these first using a batch script.

You can loop over your directory structure and read the existing files in your load script.
LET vCustCount = NoOfRows('Kunde');
TRACE Anzahl Kunden: $(vCustCount);
FOR i=1 TO $(vCustCount)
LET vNameKunde = FieldValue('name_kunde',$(i));
FOR each vFile in filelist ('$(vNameKunde)/umsatz.qvd')
TRACE $(vFile) hat eine umsatz.qvd;
LOAD ....
FROM [$(vFile)] (qvd);
NEXT vFile
NEXT
In this case I load pre-calculated qvd files but you could do the same with txt, csv ...
And as i_saw_drones mentioned QlikView cannot import compressed files. If you need to read compressed files you can batch operate them with a unzip tool.
You should have a look at
21.1 Loading Data from Files
in the Reference Manual.
HTH

Following script checks whether qvd exists or not. If, yes then it update it otherwise create a new qvd
IF NOT isNull(qvdCreateTime('G:\TestQvd\Data.qvd')) THEN
data2:
load * from G:\TestQvd\Data.qvd(qvd);
FOR each vFille in filelist ('G:\Test\*')
LOAD * FROM
[$(vFille)]
(txt, codepage is 1252, explicit labels, delimiter is spaces, msq);
NEXT vFille
ELSE
FOR each vFille in filelist ('G:\Test\*')
data2:
LOAD * FROM
[$(vFille)]
(txt, codepage is 1252, explicit labels, delimiter is spaces, msq);
NEXT vFille
ENDIF
STORE data2 into G:\TestQvd\Data.qvd;
exit Script;

Related

Batch export in SAS for spk file

I need to export/import my spk file via windows Batch script.
For which I have been referring this document
But this document does not mention how to save the file (I mean with which extension .bat or .sas)
My command:
ExportPackage -profile "SAS_MW_TEST" -package "F:\mypath\Package4.spk" -objects "/_Applications/_05_MW/_01_SAS_MW/_20_Processes/savedesk(Folder)" -subprop -types "Condition,BusinessRuleFlow,ExternalFile,Cube,SearchFolder,Table,GeneratedTransform,OLAPSchema,InformationMap.OLAP,Measure,Column,Job.CubeBuild,Action,Library,MiningResults,DeployedJob,CalculatedMeasure,Hierarchy,InformationMap.Relational,RootFolder,Prompt,Document,ConditionActionSet,DecisionLogic,Dimension,Note,StoredProcess,PromptGroup,Job,OrchestrationJob,MessageQueue,Service.SoapGenerated,Level,SharedDimension,DeployedFlow"
ExportPackage is an executable program that you can run out of a batch file.
Use Notepad or any other text editor (which includes any SAS code editor), place the ExportPackage program command in the editor and use the File/Save As feature to save the file as something like myPackageExporter.bat
If the items in the package are say stored processes whose metadata says the source code is in an file system folder (aka source code repository), you will probably also want to zip up the folder.
This macro can help you prepare the batch script, ready for export: https://core.sasjs.io/mm__spkexport_8sas.html

How to create separate transformed file instead of overwrite ssis output flat file?

I have a dataflow that is used to do transformation of multiple flat files from given folder using for each loop container. I have a flat file again as output file. The problem is that every time I execute the the job only the last file that got transformed will be stored in destination file.
Is there a way in SSIS I can create individual transformed output file instead on overwriting on same one over and over again?
For. eg. I have 5 flat files ,test_1.txt,test_2.txt,test_3.txt ,test4_.txt
and test_5.txt in a folder.
After the job ran I can only see the data from last file test_5.txt being
transformed in my destination file.
Here's steps on a working example I tested.
Variables
I have 3 variables defined:
FileName - To be used in the foreach loop
DestinationDir - where are the files going
SourceDir - where are the files I want to process
Foreach Loop Setup
I have a foreach loop configured as:
Expression for "Directory" set to #[User::SourceDir]
Retrieve file name set to "Name and extension"
Then under the "Variable Mappings":
That means as the foreach loop is iterating over the files in the directory it will be setting the "Name and extension" of the file its on to the variable #[User:FileName]
Data Flow Task
The I add a Data Flow Task inside the foreach loop:
Then inside the DFT I have a simple Flat File Source to Flat File Destination. We'll just pass the contents of each file to new files:
During initial development I'll manually pick one file to walk through setting each of the source and destinations. Then come back and change the connection managers and set an expression on the ConnectionString.
Connection Manager Expressions
SourceFile Connection Manager:
ConnectionString gets an expression as: #[User::SourceDir] + #[User::FileName]
DestinationFile Connection Manager:
ConnectionString gets an expression as: #[User::DestinationDir] + #[User::FileName]
Testing
I have 2 test files in my source directory and no files in my destination:
After I execute my package I get success and also get new files in my destination:
There are ways to do what you are asking in SSIS with variables and expressions but there is an easier way to accomplish it using command line.
Since you are just consolidating a text files into 1 you can use a command prompt to better handle your issue:
copy *.txt output.txt

How would I store different types of data in one file

I need to store data in a file in this format
word, audio, jpeg
How would I store that all in one file? Is it even possible do would I need to store links to other data files in place of the audio and jpeg. Would I need a custom file format?
1. Your own filetype
As mentioned by #Ken White you would need to be creating your own custom file format for this sort of thing, which would then mean creating your own parser type. This could be achieved in almost any language you wanted but since you are planning on using word format, then maybe C# would be best for you. However, this technique could be quite complicated and take a relatively large amount of time to thoroughly test your file compresser / decompressor, but may be best depending on your needs.
2. Command line utilities
Another way to go about this would be to use a bash script to combine all of the files into one file, and then decompress it at the other end. For example the steps could involve:
Combine files using windows copy / linux cat command on command line
Create a metdata file of your own that says how many files are in this custom file, and how much memory each one takes up (could be a short XML or JSON file for example...)
Use the linux split command or install a Windows command line file splitter program (here's just one example) to split the file back into whatever components have made it up.
This way you only have to create a really small file type, and let the OS utilities handle the combining of them for you.
Example on Windows:
Copy all of the files in your current directory into one output file called 'file.custom'
copy /b * file.custom
Generate custom file format describing metadata (i.e. get the file size on disk in C# example here). This is just maybe what I would do in JSON. SO formatting was being annoying so here's a link (Copy paste it into an editor or online JSON viewer).
Use a decompress windows / linux command line tool to decompress each files to the exact length (and export it back to the exact name) specified in the JSON (metadata) file. (More info on splitting files on this post).
3. ZIP files
You could always store all of the files in a compressed zip file, and then just use a zip compressor, expander as and when you like to retreive any number of file formats stored within.
I found a couple of examples of :
Combining multiple files into one ZIP file in only C# .net,
Unzipping ZIP files in C#
Zipping & Unzipping with only windows built-in utilities
Zipping & Unzipping in Linux command line
Good Zipping/Unzipping library in Java
Zipping/Unzipping in Python

Cannot save GuiXT script from SAP

I cannot save GUIscript from recorder. I am using two systems F6P and Q06, and in F6P I can record a script and save the file, but in Q06 I cannot save a vbs file.
I press record and then stop and a file should be created in the destination path, but no file is created. The destination path is valid.
First change the path to desktop or other destination where you have have access to , then record the script after you stop it, It will be saved.

Cannot load files from a concatenated path

I am working on a script that will run daily. The script will compare individual configuration files from multiple host on one day with individual configuration files from a previous day. I am working on a CentOS host I have limited access to - meaning I can't make major changes.
Details
My hosts are running a cron job that uploads their configuration file to an sftp server in a generic stable directory (/var/log/Backups) by hostname (hostname.txt).
A cron job on the server creates a date stamped directory and moves the files from /var/log/Backups to /var/log/Backups/ddmmyyyy.
Later, well after all sftp and file move operations I want to load an array with the file names in a current directory and I load an array with matching file names from the previous days directory.
I want a script to diff the matching file names and output the information to a single text file.
I can't get the array to load the current days files and echo them to the terminal. I get a file operation error.
Script:
#!/bin/bash
# Set current date
now=`date +%d-%m-%Y`
echo $now
base=/var/log/GmonBackups/
loc=$base$now
echo $loc
# Load files from /var/log/GmonBackups/$now into an array
t_files=`ls loc`
echo $t_files
Something along these lines might help you get further:
today=$(date +%d%m%Y)
yesterday=$(date --date=yesterday +%d%m%Y)
base=/var/log/GmonBackups
today_dir=$base/$today
yesterday_dir=$base/$yesterday
today_files=( $today_dir/* )
yesterday_files=( $yesterday_dir/* )
A few points:
prefer $() to ``
don't use ls to get your list of files because it's not robust
I didn't put quotes around the variables because there are no spaces in your directory names.

Resources