I have a complicated query that marshals data to a temporary table, which I then marshal into a further output temporary table before finally selecting on it to display to screen. This gets saved out from Grid view, to text and I get the file I need for processing off site.
What I want to do is have this query be run-able and create that file on the local disk without any need for the operator to change the "Results to" option, or fiddle with anything.
What command or functionality might be available to me to do this?
I can not install any stored procedures or similar to the server involved.
Since you can't do anything on the server I would suggest writing an SSIS package. Create a data flow, and in your source object put your script. Your destination object will then point to the file you want. You have a fair number of options for output.
The SSIS package can then be run by
A SQL Job (assuming you are allowed even that)
A non SQL job running a bat file with a DTEXEC command
The DTEXECUI GUI.
Also you can store your SSIS package in the instance or on any fileshare you choose.
Related
I want to copy all data from all tables from one SQL server database to another existing SQL server database of the same structure. I have a script to initially delete all contents of all tables in my output database before proceeding so that it is 'fresh' for the copy.
I understand that 'select into' statements can get this done but I want to be able to do it in bulk. I want to emulate the behavior that works very well in Management Studio of:
Right-click a DB
Select 'Tasks'
Select 'Export Data...'
In here, I can select an output DB and then select all tables. The transfer goes straight through without issue. I cannot find a command line way to achieve this.
The reason I am after this is that we want a daily copy of the prod database in a testing environment, so need to task schedule this process to run each night.
Due to some contstraints, I can't use a bacpac in this case.
Using the import/export task in SSMS, the last step has 2 options. Run immediately or save as SSIS package. So - save it as a SSIS package. You can then run this package whenever you want. And yes - you will need to do this twice. Once for export, once for import. You can also do exactly the same thing using SSIS btw.
So how do you execute a package from the command line? Like you do for any question, you should search first. Some suggestions/examples are here.
And if needed, you can schedule this using the agent.
I have an SSIS package (SQL Server 2016) that includes an Execute DDL Task. The task is to backup a SSAS tabular cube (so it can be replicated on a 2nd server that's on a different domain).
Generally when I write SSIS packages, I parameterize everything that could change over time. In this case, I want to parameterize where I back the file up to as well as the password to use.
In a regular SSIS Script task, I can reference project parameters directly. I don't see any way to do this with an Execute DDL Task. The best I can do is source the entire script from a variable, which then references the parameters in question.
The challenge I have is that I'm working with a password, so I want to mark the password project parameter as sensitive. As soon as I do this, I cannot reference the parameter in a variable.
My options appear to be:
Leave the password project parameter not marked as sensitive (meaning it is visible to anyone who opens the package and/or has access to the SSISDB environment variables)
Hard-code the script inside the DDL task and not parameterize the password or file name at all. Further, encrypt the entire package (rather than just the sensitive properties) so that anyone who opens the package cannot see it.
The second option sounds the best, since it doesn't involve a password visible in plain text. However, it doesn't allow for any parameterization of the password. (Which means every password change would require a redeployment of the package.)
Have I missed something? Is there another way to reference a parameterized password inside an Execute DDL Task in SSIS that I don't know about? Any other recommendations for this scenario?
This is the syntax for referencing a sensitive parameter in an ssis script task:
I understand this may be a little far-fetched, but is there a way to take an existing SSIS package and get an output of the job it's doing as T-SQL? I mean, that's basically what it is right? Transfering data from one database to another can be done with T-SQL as well.
I'm wondering this because I'm trying to get away from using SSIS packages for data transfer and instead using EF/linq to do this on the fly in my application. My thought process is that currently I have an SSIS package that transfers and formats data from one database to another in preparation to be spit out to an excel. This SSIS package runs nightly and helps speed up the generation of the excel as once the data is transferred to the second db, it's already nice and formatted correctly.
However, if I could leverage EF and maybe some linq to sql in order to format the data from the first database on the fly and spit it out to excel quickly without having to use this second db, that would be great. So can my original question be done, can I extract the t-sql representation of an SSIS package some how?
SSIS packages are not exclusively T-SQL. They can consist of custom back-end code, file system changes, Office document creation steps, etc, to name only a few. As a result, generating the entirety of an SSIS package's work into T-SQL isn't possible, because the full breadth of it's work isn't limited to SQL Server.
I've created the structure of my database first in PhpMyAdmin and exported it to a .sql file.
Now I'm looking everywhere in SQL Server Management Studio where I can import/add the data in a new database.
Does anybody where to look or what to click?
I'm using the 2014 version (CTP2)
If you have a .sql file which contains SQL statements, you can just copy and paste the contents (or open the file in a query window) and run it. This assumes it has all of the create table etc. statements to create the schema/structure and not just insert statements for the data.
Check the top of the file to make sure that it is first selecting the correct database, if not add a USE statement to select the correct database.
You didn't say how big the file was, but if it is quite large and has the insert statements (data as well as schema), then you'll probably want to run by CLI using sqlcmd command. Much faster and SSMS won't freak out.
Another alternative option to running the .sql file/code is to set up a data source for mysql and just use odbc to access the database itself.
Bear in mind that there are real and very annoying differences between mysql and t-sql that can make migration a pain. If you're just creating a few tables, it may not be an issue, but if there are a ton of tables with lots of fields of different data types, you may run into issues.
If you are looking to import table structure, you can copy-paste the content and run inside SSMS in a query window. Beware of syntax differences with MySQL and SQL Server. You will most likely get errors. You need to convert your SQL script from MySQL dialect to SQL Server dialect (or just add them manually if they are not too many). If you set the databases to a SQL standard-compatibility mode at the very beginning, you will have much less trouble.
If you are ONLY looking just to import the data into existing tables inside the SQL Server only, you can do the same (i.e. copy-paste and run in query window). You will have less trouble with that.
Open the server, open "Databases" and right click the database, go to "Tasks" and then Import Data...
I have had the most 'trouble free' success importing to SQL via a flat file method (comma delimited .txt file), the only stipulation when creating a flat file (i.e from Access) make sure the text identifier is set to {none} and not "".
To import the file: in the SQL Server Management Studio right click on Databases and create a new database. Then right click on the new database -> Tasks -> Import Data... The import window opens: in the DATA SOURCE option select Flat File Source and select the .txt file...click NEXT. In the DESTINATION field select SQL Server Native Client 11.0 and go through the import process. This worked very well for me.
I have four script components in SSIS (set up as sources) that generate four output tables for further procession.
I need some component to allow me to run an SQL command against input tables as if there were tabled in SQL database.
Which component (or a set of components) I should use to query my input and generate output in an aforementioned scenario?
Thank you.
you cant!
There are a few components that allow you to "check" your data like the row sampling or even the count to count the number of rows, but if you need to run a SQL query on a data set you just read from a text file, you will need to persist your data in a temporary table, do whatever you need to do with it, and then send it to the destination.
If you are on SQL Server 2012, and the query you want to run on the data wont update it, you can use "data taps". What it basically does is send the data passing through a data flow path to a file on the file disk. It is handy but its more of a "debug" mechanism, you dont want to have it on production and rely on it for an important task. They are also only available when you are executing a package deployed to the SSISDB catalog.