I need to change the collation for quite a lot tables including the columns. I already wrote a sql-statement which generate these for me. I wanted to create a stored procedure to automate this process. I created a table to store these generated statements and want to execute row after row. How can I do that?
Thanks in advance
Related
I have a Pentaho transformation with three steps:
Table input. It works perfectly.
Execute SQL script, which I exec a sql server procedure. This procedure returns 4 tables, but I am not able to catch this three tables.
Text file output. Here I would like to save the contains of the before returned tables.
My main problem is that my partners have developed a procedure that returns some tables, when clearly this is a misunderstanding of the procedure concept. It should be a table valued function.
Despite this, I have to fix this someway, so, I am asking for advise. Anyone knows how to catch the tables?
Greetings.
I have tried this in the past and I believe it is not possible (although I can't seem to find the docs at the moment). Even if you could achieve this, how would you tell Pentaho what to do with each of the separate tables?
My solution at the time was to alter the proc so that only a single table was returned. Either by splitting the proc into several procs or combining the output into a single larger table with a parameter to split out in Pentaho.
I am trying to completely automate this process, and I'm wondering if its viable or efficient to do in VBA.
Report process involves 2 files: one sql file and one excel file.
SQL file has the algorithm, and the final step is a query who's result is then pasted into the excel file.
The algorithm is simpler(than what the audience might be used to) but has two "into" commands and several "update" commands.
Two "into" commands, the first grabs a small portion(constrained on first and last day of previous month) of a 500m+ record table. The second joins the first table with an eligibility type table.
After the second table is created, there is a series of UPDATE commands that change existing data of existing columns.
Then a series of ALTER & UPDATE commands that add new columns to the [second] table and UPDATES them with desired data.
the final step is a query who's results are copy-pasted into excel (as is, no formatting changes necessary).
I'm not too well-versed in VBA/VBNET nor TSQL stored procedures and dynamic sql, if the sql algorithm was a simple pull query with no table creation, I can build something to automate that. But the SQL has 2 table creations, and about a dozen ALTER & UPDATE commands.
Am I stirring up the wrong nest? Should I run it manually as is?
You can definitely do automate this. I created a report that ran two stored procedures and created numerous queries with temp tables including both update and alter commands then used VBA to run execute these and aggregate the data in the final summary sheet.
There is a ton of documentation out there. You can even pass your values to the stored procedure after the user inputs them.
I would add this as a comment but I do not have enough reputation to comment yet (need 50).
I am currently working with a stored procedure that performs some background processes and then returns one of two results tables.
If it works ok I get a one column table that says success, if it doesn't then I get a four column table with various error data.
While this is fine if you just execute the code from .net, I now need to execute this from within another stored procedure. While I don't need the output, I do need the background processes to take place. I'd usually insert the output into a table, but can't in this case as the columns in the output varies dependent on the result, and as such cannot define a table that it can insert into.
Easiest answer would be to rewrite the outputs of the background SP to be consistent but this isn't an option. I've even tried wrapping this inside a UDF but the stored procedure can't be called from with a function.
Whatever solution I finally use it must work on versions from SQL Server 2008 R2 up to 2016.
Does anybody have any suggestions?
Many thanks,
Mat.
I would image you could create a SP that inserts the result of the inner SP into a temporary table using the hack below.
Insert results of a stored procedure into a temporary table
If that blocks the ouput then you can return no data.
We have a large production MSSQL database (mdf appx. 400gb) and i have a test database. All the tables,indexes,views etc. are same eachother. I need to make sure that tha datas in the tables of this two database consistent. so i need to insert all the new rows and update all the updated rows into test db from production every night.
I came up with idea of using SSIS packages to make the data consistent by checking updated rows and new rows in all the tables. My SSIS Flow is ;
I have packages in SSIS for each tables seperately because;
Orderly;
Im getting the timestamp value in the table in order to get last 1 day rows instead of getting whole table.
I get the rows of the table in the production
Then im using 'Lookup' tool to compare this data with the test database table data.
Then im using conditional sprit to get a clue whether the data is new or updated.
If the data is new, i insert this data to the destination
5_2. If the data is updated, then i update the data in the destination table.
Data flow is in the MTRule and STBranch package in the picture
The problem is, im repeating creating all this single flow for each table and i have more than 300 table like this. It takes hours and hours :(
What im asking is;
Is there any way in SSIS to do this dynamically ?
PS: Every single table has its own columns and PK values but my data flow schema is always same. . (Below)
You can look into BiMLScript, which lets you create packages dynamically based on metadata.
I believe the best way to achieve this is to use Expressions. They empower you to dynamically set the source and Destination.
One possible solution might be as follows:
create a table which stores all your table names and PK columns
define a package which Loops through this table and which parses a SQL Statement
Call your main package and pass the stmt to it
Use the stmt as Data Source for your Data Flow
if applicable, pass the Destination Table as Parameter as well (another column in your config table)
This is how I processed several really huge tables: the data had to be fetched from 20 tables and moved to one single table.
You are better off writing a stored procedure that takes the tablename as parameter and doing your CRUD there.
Then call the stored procedure in a FOR EACH component in SSIS.
Why do you need to use SSIS?
You are better off writing a stored procedure that takes the tablename as parameter and doing your CRUD there. Then call the stored procedure in a FOR EACH component in SSIS.
In fact you might be able to do everything using a Stored Procedure and scheduling it in a SQL Agent Job.
My problem is as follows. I have a CSV file (~100k rows) containting history information with the column format of:
ID1,History1,ID2,History2...ID110,History110
Each row may have anywhere between 0 and 110 history entries. Each separate entry requires a stored procedure to be called.
If there were a small number of possible entries per row, I imagine the way to do this would be to transform the data using a script, and send it to a unique path. Creating 110 paths would probably work, but isn't very elegant (and quite time consuming).
What would the best way to approach this be?
Just load the data (raw csv unchanged, one row per file line) into a staging table. Then, call a stored procedure that will use a string splitter to break up and loop over the staging table rows and call your other procedure for each history entry.
see: Arrays and Lists in SQL Server 2005 and Beyond
also see this previous answer: SQL comma delimted column => to rows then sum totals?
If you want to solve this in SSIS without the staging tables, you could create a destination script component. You could use switch statement or hashtable to lookup the right sproc to execute for the data row.
It is unclear whether this is a better solution then the staging table approach above; but it is an alternative.
I know you already accepted an answer, but couldn't you use an Unpivot task to achieve what you wanted to do here?