how to write a batch script which will take a file as an input, then it will perform a sql query on that file and give a file as an output.
input will be a textfile which has 4 query in it. Now a batch file is to be written which will take 1 query at a time and execute it and output will be stored in a file. So there will be 4 seperate output file for 4 query
You don't specify which sql server you are using, in this example I will use firebird. If you use a different sql server, you have to use the correct sql commandline tool and syntax. firebird uses isql.exe.
Asuming I have the following text file "input.sql" containing 4 sql commands:
select * from CUSTOMER;
select * from DEPARTMENT;
select * from EMPLOYEE;
select * from SALES;
Then this batchfile will execute each command using isql.exe, and creates a seperate output file for each command:
#echo off
set sql_exe="C:\Program Files\Firebird\Firebird_2_5\bin\isql.exe"
set sql_options=-u sysdba -p masterkey
set sql_db="C:\Program Files\Firebird\Firebird_2_5\examples\empbuild\EMPLOYEE.FDB"
set count=1
for /f "delims=" %%a in (input.sql) do (
echo %%a > temp.sql
call :processtemp_sql
)
goto :eof
:processtemp_sql
%sql_exe% %sql_options% -i temp.sql -o output%count%.txt %sql_db%
set /A count=%count%+1
goto :eof
:eof
at the end output1.txt..output4.txt are created. Each file contains the output of one sql command.
You don't often perform SQL queries on a normal file, it generally requires a DBMS (database management system) at the other end for interpreting the SQL and extracting the relevant data.
Writing a batch file is relatively easy, be it a UNIX shell like bash or the Windows cmd.exe.
But, if you're thinking about implementing a full blown SQL query language that operates on text files (or any non-database file, really), that's probably going to take more then one question on Stack Overflow :-)
Perhaps you could flesh out your question with a little more detail, given that we may have misunderstood your requirements.
I think what you're looking for is Microsoft Log Parser 2.2. It allows you to execute SQL queries on a a number of file types, including logs, CSV and XML files.
Related
My global task to create config file for WinSCP process, and it should be multi lines (no options). I try to use Expression builder for CMD/Echo to do this. And still file, tried all option with and without /C flag which is critical for SSIS cmd task. I have option to have task for each line and it's by back plan, but trying to see if I can come up and learn how to do this in single box. I example below I can create multi line Expression using + " " combo where second quotes on new line. But it's still not good for whole task, Looks like CMD /C need single line ??
All info displayed in pic below:
This is arguments I'm using now with WinSCP executable in System task.
open ud12345d264d7b8#magdaflyn.sharefileftp.com
option transfer binary
put c:\SFTP\Magda_Members_20190901.csv "/Business Intelligence Share/Zbignev/"
close
exit
Creating multi-line expression
In order to add new lines in SSIS expression you must use the carriage return and line feed characters \r\n:
"/C Echo line11 \r\n /C Echo " + #[User::Var3] + "Line222 \r\n /C Echo Line3 > Config.dat"
Executing multiple commands
If adding new lines didn't work, you can use && operator to run multiple command using CMD:
SSIS: Execute Multiple Commands with 1 line in Execute Process Task (Arguments)
Workaround
If all what I mentioned above didn't work, then try saving the whole command into a batch file, and execute this batch file from the Execute Process Task.
You can write two lines using this syntax:
/c "echo line1 && echo line2" > config.dat
Though, you do not need to create a config file for WinSCP. All WinSCP options can be configured on its command-line. So this may be XY problem.
And while it's not clear, you may actually be creating a script file, not a config file. You can instead specify all WinSCP commands on WinSCP command-line using /command switch:
winscp.com /command "open ud12345d264d7b8#magdaflyn.sharefileftp.com" "option transfer binary" "put c:\SFTP\Magda_Members_20190901.csv ""/Business Intelligence Share/Zbignev/""" "close" "exit"
WinSCP GUI can generate a command-line sample for you.
See also WinSCP article SFTP Task for SSIS/SSDT.
Though you actually better use WinSCP .NET assembly instead of scripting in SSIS. See Using WinSCP .NET Assembly from SQL Server Integration Services (SSIS).
A project I'm working on at work involves modifying one of the subsystems to store/pull data that is currently stored in files into the database. Each of the files is a single, sometimes-large, chunk of custom (xml-based) script generated by another custom tool.
Conceptually, I'm looking for an easy way to do something like:
For Each file in folder_and_subfolders
INSERT INTO table
(script_name, version_num, script )
VALUES
({file_name}, 1, {file_contents})
;
Next
Preferably on an entire directory tree at once.
If there's no easy way to do this via T-SQL, I can write a utility to do the job, but I'd prefer something that didn't require having to write another custom tool that will only be used once.
So, I don't have SQL Server installed and therefore can't test this, but if you are looking for a simple batch file that could do what you're after, I'd suggest something like the following might well help;
#echo off
SET xmldir=./myxmlfiles/live/here/
echo --- Processing files
for %%f in ("%xmldir%*.xml") do (echo Running %%f.... && #sqlcmd -I -U %1 -P %2 -S %3 -d %4 -v filename="%xmldir%%%f" -i ProcessFile.sql)
I'm not sure how much you know about sqlcmd, but it is a command line tool that is generally provided by SQL Server. It will allow you to run SQL commands, or in the case above, run a script which is indicated by the -i parameter. I am assuming that you'd place your SQL statement in there to perform your additions to the table.
The other parameters to sqlcmd are described below;
-I sets QUOTED_IDENTIFIER on (you may or may not need this. I did for an earlier issue I faced with sqlcmd and QUOTED_IDENTIFIER)
-U sets the database username
-P sets the database password
-S sets the database server
-d sets the database to connect to
-v is the interesting one here as it lets you pass parameters to your script. Note that on the MSDN page describing this, it states that if your path or filename contains spaces, then you'll need to enclose it in quotes, so check that out. Basically though, you'd be able to refer to the parameter inside your sql script (ProcessFile.sql) like INSERT INTO mytable (file_name) VALUES ('$(filename)')
You'd have to use the logic described in the answer from my previous comment to ensure
I'm trying to set up a batch script that basically runs a SQL statement against a database, and if the script returns results it will follow some logic.
Is there a way to have SQLCMD actually return the number of rows it found, or something similar?
I see that I can have the output displayed on the screen or a file, but is there a way to have it put it into a variable so I can have the script evaluate the variable? For example:
SQLCMD -q "select count(*) from active_connections" -r #varactive
IF #varactive > 0 THEN
<do things>
ELSE END
Or would I need to switch to Powershell to handle this sort of logic?
While #Gary is technically correct that the only thing returned is the ERRORLEVEL, sqlcmd does also display its results to STDOUT. Armed with that, you could do something like this in a batch file:
set SERVERNAME=yoursqlserver
for /f "skip=2" %%x in ('sqlcmd -S %SERVERNAME% -Q "select count(*) from active_connections" ^| findstr /v /c:"rows affected"') do set COUNT=%%x
echo There are %COUNT% records in the active_connections table.
See Docs for sqlcmd and you will see quite a few options you probably never paid attention to.
The only thing an executatble "returns" to the batch script environment is the ERRORLEVEL. For SqlCmd you need the -b option to set this (based on the sql server error level)
If you use the -m option, you can control the error messages send to stdout -- I can't test at the moment, but I think this include the rows affected message (a level 0 error perhaps). You would then have to parse this too (ugly in batch scripts)
This sounds like a real kludge at best to be, you are likely better off to use a better scripting environment. PowerShell, Perl, Python, etc. all more powerful and you can find plenty of examples on-line.
Batch is best when you have a "no-deployment" requirement or you needs are simple. Easy to hit the wall as needs change.
I'm trying to use various WMI queries to gather data needed for daily reports on some of my servers. The only piece of my script that fails is the final step, which queries for the status of certain services. The query works fine when I type it manually into the command line, but it fails when I dump it into a batch script. I'm limited to using plain old Windows Command Line, so no Power Shell or vb. Here's the command:
wmic /OUTPUT:"%HOMEDRIVE%%HOMEPATH%\Desktop\AutoDMR\DMRAuto_3.txt" service where "name like 'MPCX%' or name like 'nm%' or name like 'nb%' or name like 'ssacpha%' or name like 'EMC%'" get name,state,status /FORMAT:htable
Thanks!
All percents in a batch script must be doubled. For example 'EMC%' becomes 'EMC%%'
Also, if you attempt to run command within a FOR /F IN() clause, then the commas in the GET clause must be escaped as ^,.
for /f ... in ('wmic ... get name^,state^,status ...') do ...
How to get the relative path in t sql? Take for example a .sql file is located in the folder D:\temp, I want to get path of the file hello.txt in the folder D:\temp\App_Data. How to use the relative path reference?
Let's say I am executing the sql file inside the SQL server management studio.
I had a similiar problem, and solved it using sqlcmd variables in conjunction with the %CD% pseudo-variable. Took a bit of trial and error to combine all the pieces. But eventually got it all working. This example expects the script.sql file to be in the same directory as the runscript.bat.
runscript.bat
sqlcmd -S .\SQLINSTANCE -v FullScriptDir="%CD%" -i script.sql -b
script.sql
BULK INSERT [dbo].[ValuesFromCSV]
FROM '$(FullScriptDir)\values.csv'
with
(
fieldterminator = ',',
rowterminator = '\n'
)
go
The .sql file is just.... a file. It doesn't have any sense of its own location. It's the thing that excutes it (which you didn't specify) that would have a sense of its location, the file's location.
I notice that you mentioned an App_Data folder, so I guess that ASP.NET is involved. If you want to use relative paths in your web app, see MapPath
http://msdn.microsoft.com/en-us/library/system.web.httpserverutility.mappath.aspx
The server is executing the t-sql. It doesn't know where the client loaded the file from. You'll have to have the path embedded within the script.
DECLARE #RelDir varchar(1000)
SET #RelDir = 'D:\temp\'
...
Perhaps you can programmatically place the path into the SET command within the .sql script file, or perhaps you can use sqlcmd and pass the relative directory in as a variable.
When T-SQL is executing, it is running in a batch on the server, not on the client machine running Management Studio (or any other SQL client). The client just sends the text contents of the .sql file to the server to be executed. So, unless that file is located on the database server, I highly doubt you're going to be able to interact with it from a SQL script.
The t-sql script is first preprocessed by QueryAnalyzer, SSMS or sqlcmd on the client side. These programs are aware of the file localcation and could easily handle relative pathes similar To Oeacle sqlplus.
Obviously this is just a design decision from Microsoft and I dare say a rather stupid one.
I tried method from mateuscb's comments.
I found it can not work ,i do not know why,then I managed after several test.
It can work with the script below:
runscript.bat
#set FullScriptDir=%CD%
sqlcmd -S .\SQLINSTANCE -i script.sql
script.sql
BULK INSERT [dbo].[ValuesFromCSV]
FROM '$(FullScriptDir)\values.csv'
with
(
fieldterminator = ',',
rowterminator = '\n'
)
go
Just for your information for further discussion.
well it's not a Microsoft thing first off... it's an industry standard thing.
second your solution for running T-SQL with a relative path is to use a batch script or something to inject your path statement IE:
#echo OFF
SETLOCAL DisableDelayedExpansion
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ t-SQL.SQL"`) do (
set "var=%%a"
SETLOCAL EnableDelayedExpansion
set "var=!var:*:=!"
set RunLocation=%~dp0
echo(%~dp0!var! > newsql.sql
ENDLOCAL
)
sqlcmd newsql.sql
or something like that anyway