I have a manual process that needs to be automated. Below are the steps outlined. Is SSIS the correct way to achieve all the below mentioned steps? Especially the result to CSV, zip and email steps? Can this be done using inbuilt sql server scheduler?
Connect to SQL Server: DARVIN,51401
Open SQL Query: o:\Status Report.sql
Chose database: AdventureWorks
In the menu bar above choose ‘Tools’, then ‘Options’.
When the pop-up appears, choose the tab named ‘Results’ and choose Results output format of Comma Delimited (CSV). Clikd the ‘Apply’ then the ‘OK’ button.
Execute the query
Instruct where to save the file, You can save these in O:\Reports File name format is: day^_^Report TSP MM-DD-YY
Let the query run 15-25 minutes.
When the query is complete, open the folder that you save the report in, right click on the report title and compress to a zip file. (right click, Send To Compressed (zipped) Folder) When this has been completed.
Copy the saved file and put into: O:\zippedFiles\
Email: support#adventureworks.com to let them know that you
have placed a zip file at: O:\zippedFiles\
This is exacly what SSIS is for : automating data transfer processes.
The only stp that might cause you problems is the zipping part. You can use a third party library, and do a custom script task that will achieve what you want. You will have to do some Vb.Net (or C#) to achieve that part.
The rest of what you want is pretty straightforward in SSIS.
Related
Nothing important or fancy, I am creating a demo jpeg in SQL Server and don't want to take the extra step of having to launch mspaint directly after running my script in order to see it. It would be nice if the script could just do the launching for me.
I have seen a number of posts on this subject and it looks like xp_cmdshell won't work for a UI-based app. Does anybody know of any way within a simple T-SQL script to launch notepad or mspaint or any other UI-based app without having to resort to SSIS or ServiceBroker? TIA
I am demo-ing SQL Server R services for my users group and outputting a plot of a machine learning algorithm
If I understand you correctly you want to show result from SQL Server R Services (during demo). I would recommend to install SSMS Boost for SSMS and use SSMS ResultGrid Visualizers:
Image from: http://www.ssmsboost.com/Content/images/Feature/ssms-add-in-results-grid-visualizers.png
Sample:
present TSQL/R script
execute it
visualize varbinary data(jpeg)
I'm learning to develop ETL using Pentaho Spoon, I'm pretty noob yet.
Instead of storing SQL operations inside its file, I'd like to have them on their own .sql files. It makes easier to track changes on Subversion, and in case of need I can just open the sql file on DB manager and execute it directly.
How could I do that? I suppose I could use some component to read a txt file into a variable, and another component to take that variable and execute it on DB.
How's the simplest way to achieve that?
In the standard SQL Table input, you can define the query to be a parameter ${my_query} and this parameter has to be defined (without ${...} decoration) in the transformation properties: right-click anywhere, select Properties on the popup menu, the Parameter tab.
Each time you run the transformation, you'll presented the list of parameters, among which my_query which you can overwrite.
To automatize, follow the example which was shipped with the installation zip. In the same directory as you spoon.bat/spoon.sh, there is a folder named sample, in which you will find a job to read_all_files or read all_tables. Basically this job list the files in a directory, and for each one puts it in a variable and use it as a parameter to run the transformation. Much more easier to do than to explain.
I have numerous scripts to run in SQL Server.
Currently I've made many scripts which have been saved as file and can be loaded in SSMS when needed.
I want to know if we have any shortcut(bookmark) in SQL server to script files.
So far I've found nothing on internet. (note that creating stored procedures for these ad-hoc scripts is not an option)
By pressing Cntrl+K, Cntrl+W combination you can see the Bookmark manager.
Here you can see all bookmarks in locally saved sql queries.
Like in the below screenshot I have a bookmark for getLocationBasedMemberVisitsData.Sqlfile.
On how to add a bookmark
goto Edit>Bookmarks>toggle bookmark when you are inside your file with your edit cursor at the line you want to bookmark
P.S.: You can also create a folder hierarchy much like in browsers like Chrome
Store all needed files inside a single solution. Call it Utilities. Much like VS you can browse through the solution files using Solution explorer.
Caveat: The bookmarks on loose sql files are not persisted. So on a restart(close/open) for SSMS tool these are lost.
If you do need to persist bookmarks, create bound files i.e. store the files in a solution first.
Also the bookmarks in Management Studio are consist with Visual Studio.
Addendum:
Based on asker's comment
This just bookmark a line on a script. Is these any way to give these bookmarks a name?
You can rename both the folders as well as bookmark name by double clicking on it or by right click>rename option
On another comment
What do you mean by saving as as solution. I cannot find such option in SSMS. Are you referring to VS?
SQL server management studio like VS supports creation of Solutions(project) which are essentially collection of related queries/SP with connection information and other needed stuff.
To create a new solution press cntrl+shift+N. see screen shot below
Any bookmarks on solution files are sticky. Next time you open the solutions, bookmarks are presented back to you.
The SQL solution can be opened in VS too. The bookmark information along with other user information is stored in a *.ssms_suo file much like a *.suo file of VS.
It is a good idea to store all helpful files in a utilities solution which can be open all the time so that you can easily access it.
I think a Snippet is what you are asking for:
A Transact-SQL.code snippet is a template containing the basic
structure of a Transact-SQL statement or block. You can use snippets
as a starting point when adding statements in the Database Engine
Query Editor. You can insert the pre-defined snippets supplied with
SQL Server, or create your own.
more info here
In SQL Server Management Studio you could use menu -> Tools -> External Tools.
Do you use SQL Prompt? If so, this has a Tab History feature that lets you search for query windows, regardless of whether they have been saved or not.
If not, I'd use the snipppet feature that VV5198722 has referred to, or the Template Browser (invoked from the view menu).
I'm new in c# and i feel i m in trouble.
in my project i want to import excel file to sql server 2008 but i want to user choose the excel file.i search lots of source but all of them define the way of excel file in the code,but i want to user browse and user select excel file.Then other things row -row or colomn will transfer to sql server.
On the other hand i would give information on my project,it will be good for me to explain problem.First user selects excel file and it will be imported database again user selects other excel file one by one.The importent thing is how i can give way of excel file dynamically.
if you will help,i will be grateful.
Thank you...
hi it looks like you need to use Open File Dialog box. Here is the example code how to use http://msdn.microsoft.com/en-us/library/aa969773.aspx
However, remember that if you are going to deploy this application to any server you have to bear in mind that you cant use traditional ways of initiating Excel objects and reading the data in sheets. As you will not have MS office installed on server. And if you install them you will be violating the licence. Here is an example how you can do the actual import
http://www.codeproject.com/Articles/115578/Excel-to-SQL-without-JET-or-OLE-Version-2
A while back I needed to parse a bunch of Serve-U FTP log files and store them in a database so people could report on them. I ended up developing a small C# app to do the following:
Look for all files in a dir that have not been loaded into the db (there is a table of previously loaded files).
Open a file and load all the lines into a list.
Loop through that list and use RegEx to identify the kind of row (CONNECT, LOGIN, DISCONNECT, UPLOAD, DOWNLOAD, etc), parse it into a specific kind of object corresponding to the kind of row and add that obj to another List.
Loop through each of the different object lists and write each one to the associated database table.
Record that the file was successfully imported.
Wash, rinse, repeat.
It's ugly but it got the job done for the deadline we had.
The problem is that I'm in a DBA role and I'm not happy with running a compiled app as the solution to this problem. I'd prefer something more open and more DBA-oriented.
I could rewrite this in PowerShell but I'd prefer to develop an SSIS package. I couldn't find a good way to split input based on RegEx within SSIS the first time around and I wasn't familiar enough with SSIS. I'm digging into SSIS more now but still not finding what I need.
Does anybody have any suggestions about how I might approach a rewrite in SSIS?
I have to do something similar with Exchange logs. I have yet to find an easier solution utilizing an all SSIS solution. Having said that, here is what I do:
First I use logparser from Microsoft and the bulk copy functionality of sql2005
I copy the log files to a directory that I can work with them in.
I created a sql file that will parse the logs. It looks similar to this:
SELECT TO_Timestamp(REPLACE_STR(STRCAT(STRCAT(date,' '), time),' GMT',''),'yyyy-M-d h:m:s') as DateTime, [client-ip], [Client-hostname], [Partner-name], [Server-hostname], [server-IP], [Recipient-Address], [Event-ID], [MSGID], [Priority], [Recipient-Report-Status], [total-bytes], [Number-Recipients], TO_Timestamp(REPLACE_STR([Origination-time], ' GMT',''),'yyyy-M-d h:m:s') as [Origination Time], Encryption, [service-Version], [Linked-MSGID], [Message-Subject], [Sender-Address] INTO '%outfile%' FROM '%infile%' WHERE [Event-ID] IN (1027;1028)
I then run the previous sql with logparser:
logparser.exe file:c:\exchange\info\name_of_file_goes_here.sql?infile=c:\exchange\info\logs\*.log+outfile=c:\exchange\info\logs\name_of_file_goes_here.bcp -i:W3C -o:TSV
Which outputs a bcp file.
Then I bulk copy that bcp file into a premade database table in SQL server with this command:
bcp databasename.dbo.table in c:\exchange\info\logs\name_of_file_goes_here.bcp -c -t"\t" -T -F 2 -S server\instance -U userid -P password
Then I run queries against the table. If you can figure out how to automate this with SSIS, I'd be glad to hear what you did.