I want to have a ssis package that would check the folder (and all of its subfolders) and import only files that match the predefined criteria. Depends on what the file name is, it will import into a certain database (all reside in the same sql server).
The process works like this:
1. check the matching files
2. for each matching file, parse the file's name to get information that determines where the file will be imported to
3. truncate the table that the file in the previous step will be loaded to
4. import the file (in Data Flow Task).
So far i got step 1 and 2 working, and the sql statement to truncate table populate as expected. I run this sql statement manually in ssms and it works.
But i can't pass step 3, got an error below:
Error: 0xC002F210 at Truncate table, Execute SQL Task: Executing the
query "set nocount on; truncate table [db1].[DBO].[TBL_B..." failed
with the following error: "Parameter name is unrecognized.". Possible
failure reasons: Problems with the query, "ResultSet" property not set
correctly, parameters not set correctly, or connection not established
correctly. Task failed: Truncate table
Attached are the setting of the package. What am i missing?
You can try to specify parameter by giving ParameterName as "0".
And since there will be no resultset to the truncate query, specify ResultSet as None.
Related
I am trying to use the import and export wizard to move a small data set from a CSV file to an existing (empty) table. I did Script Table As > Create To, to get all DML for this table. I know the field type of the two fields which are causing problems is varchar(50). I'm getting this error message:
Error 0xc020902a: Data Flow Task 1: The "Source - Reconciliation_dbo_agg_boc_consolidated_csv.Outputs[Flat File Source Output].Columns["ReportScope"]" failed because truncation occurred, and the truncation row disposition on "Source - Reconciliation_dbo_agg_boc_consolidated_csv.Outputs[Flat File Source Output].Columns["ReportScope"]" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
(SQL Server Import and Export Wizard)
The max length of all characters is 49, so I'm not sure why SQL Server is complaining about truncation. Is there any way to disable this error check and just force it to work? It should work as-is! Thanks everyone.
Is there any way to disable this error check and just force it to
work? It should work as-is! Thanks everyone.
Yes. If you're using the wizard, you can view the table schema before running it, and check the option to ignore truncation.
The max length of all characters is 49, so I'm not sure why SQL Server
is complaining about truncation.
The default datatype of source column may be Text while using import wizard, so change it to varchar(50) using advanced tab of source. Check this link for more details.
For the safe side can you please check column data type in both Source and Destination. If both are not same just declare all your columns as varchar inside table with some maximum length say for example varchar(max) or varchar(500) and see what would be the result.
Change max length of Varchar column:
ALTER TABLE YourTable ALTER COLUMN YourColumn VARCHAR (500);
Then the column will default to allowing nulls even if it was originally defined as NOT NULL. i.e. omitting the specification in an ALTER TABLE ... ALTER COLUMN is always treated as.
ALTER TABLE YourTable ALTER COLUMN YourColumn VARCHAR (500) NULL;
check column nullable or Not nullable based on requirement just change it.
Use below steps for better understanding How to import a CSV file into a database using SQL Server Management Studio:
While bulk copy and other bulk import options are not available on the SQL servers, you can import a CSV formatted file into your database using SQL Server Management Studio.
First, create a table in your database into which you will import the CSV file. After the table is created:
Log in to your database using SQL Server Management Studio.
Right click the database and select Tasks -> Import Data...
Click the Next > button.
For Data Source, select Flat File Source. Then use the Browse button to select the CSV file. Spend some time configuring the data import before clicking the Next > button.
For Destination, select the correct database provider (e.g. for SQL Server 2012, you can use SQL Server Native Client 11.0). Enter the Server name; check Use SQL Server Authentication, enter the User name, Password, and Database before clicking the Next > button.
In the Select Source Tables and Views window, you can Edit Mappings before clicking the Next > button.
Check Run immediately and click the Next > button.
Click the Finish button to run the package.
I have an execute sql task ( Task 1 ) which runs sql to returns a column called Note from Table A and stores it as a String SSIS variable type. In Table A, Note is defined as varchar(2000).
I then have an execute sql task ( Task 2) to run a stored procedure. The input parameter is Note varchar(max).
I run these 2 task in SSIS and get the following error:
DECLARE #..." failed with the following error: "The text, ntext, and image data types are invalid for local variables.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I have tried several solutions with no success. How can you get round this error and get SSIS to store the variable and feed it to the SP ?
On sql server 2012 SSIS and hitting an old 2008 database where the SP resides.
You are somehow mapping a text type column in your tsql code. Before you return from your tsql proc convert the text column to a varchar column with select cast (textval as varchar(max)) and make sure your output variables are defined as varchars.
You need to make sure the Task 1 should only return one row.
example capture
Otherwise you need to use an Object variable. And use a For each loop container to loop thru that object.
I followed the post from bilinkc as mentioned here
how to load extracted file name into sql server table in SSIS
I use this extracted file name in a variable called FileName in a data task within a For Each container. But when I want to use the same variable in a consecutive Execute SQL task it is throwing up the error
[Execute SQL Task] Error: Executing the query "
INSERT INTO HOURLY_DATA
..." failed with the following error: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Parameter settings : Data Type is VARCHAR Parameter name is 0 and Parameter length is 1000.
Is there anyway to get around this error. WHen i checked on other posts they say it is related to a TEXT length error but I am not able to get my head around this solution for my scenario.
The full query is as given below:
This is the full query :
INSERT INTO HOURLY_DATA
(UNIQUE_NUM, NAME, FILE_NAME, USER_NAME)
SELECT DISTINCT UNIQUE_NUM, 'MANUS',?, 'MENON'
FROM MY_TABLE
WHERE UNIQUE_NUM IS NOT NULL
For the parameter variable, I have a package level variable defined which depends upon this expression:
REPLACE(REPLACE(REPLACE(REPLACE(#[User::varFilePath],#[User::varSourceFolder],""),"\",""),"My File",""),".xlsx","")
But when I evaluate this expression nothing happens. Could this variable be the issue?
Thank you in advance for your time and help.
I am using a GUID for a batch identifier in SSIS. My final output goes to SQL Server.
I know how I can generate one using Select NewId() MyUniqueIdentifier in Sql Server - I can generate one using a query and an Execute SQL task.
I am however looking to do this within a SSIS package if possible without SQL Server available.
Can I generate a GUID within SSIS?
I had a similar problem. To fix it, I created an SSIS "Composant Script" in which I created a "guid" output. The script VS C# code was the following :
Row.guid = Guid.NewGuid();
Finally, I routed the output as a derived column into my database "OLE DB Destination" to generate a guid for every new entry.
Simply do it in an Execute SQL Task.
Open the task
Under General -> SQL Statement, enter your query Select NewID() MyID in the "SQLStatement" field
Under General -> Result Set, choose "Single row"
Under Parameter Mapping, Enter your User::myID in Variable Name, "Input" as direction, 0 as Parameter Name, and -1 as Parameter Size
Under Result Set, enter "MyID" for your Result Name and type the variable in Variable Name
-Click OK
Done. Note that "MyID" is a value you can choose. EDIT: "User::myID" corresponds to the SSIS variable that you create.
So I have a stored procedure in SQL Server. I've simplified its code (for this question) to just this:
CREATE PROCEDURE dbo.DimensionLookup as
BEGIN
select DimensionID, DimensionField from DimensionTable
inner join Reference on Reference.ID = DimensionTable.ReferenceID
END
In SSIS on SQL Server 2012, I have a Lookup component with the following source command:
EXECUTE dbo.DimensionLookup WITH RESULT SETS (
(DimensionID int, DimensionField nvarchar(700) )
)
When I run this procedure in Preview mode in BIDS, it returns the two columns correctly. When I run the package in BIDS, it runs correctly.
But when I deploy it out to the SSIS catalog (the same server the database is on), point it to the same data sources, etc. - it fails with the message:
EXECUTE statement failed because its WITH RESULT SETS clause specified 2 column(s) for result set number 1, but the statement sent
3 column(s) at run time.
Steps Tried So Far:
Adding a third column to the result set - I get a different error, VS_NEEDSNEWMETADATA - which makes sense, kind of proof there's no third column.
SQL Profiler - I see this:
exec sp_prepare #p1 output,NULL,N'EXECUTE dbo.DimensionLookup WITH RESULT SETS ((
DimensionID int, DimensionField nvarchar(700)))',1
SET FMTONLY ON exec sp_execute 1 SET FMTONLY OFF
So it's trying to use FMTONLY to get the result set data ... needless to say, running SET FMTONLY ON and then running the command in SSMS myself yields .. just the two columns.
SET NOTCOUNT ON - Nothing changed.
So, two other interesting things:
I deployed it out to my local SQL 2012 install and it worked fine, same connections, etc. So it may be a server / database configuration. Not sure what if anything it is, I didn't install the dev server and my own install was pretty much click through vanilla.
Perhaps the most interesting thing. If I remove the join from the procedure's statement so it just becomes
select DimensionID, DimensionField from DimensionTable
It goes back to just sending 2 columns in the result set! So adding a join, without adding any additional output columns, ups the result set to 3 columns. Even if I add 6 more joins, just 3 columns. So one guess is its some sort of metadata column that only gets activated when there's a join.
Anyway, as you can imagine, it's driving me kind of mad. I have a workaround to load the data into a temp table and just return that, but why won't this work? What extra column is being sent back? Why only when I add a join?
Gah!
So all credit to billinkc: The reason is because of a patch.
In Version 11.0.2100.60, SSIS Lookup SQL command metadata is gathered using the old SET FMTONLY method. Unfortunately, this doesn't work in 2012, as the Books Online entry on SET FMTONLY helpfully notes:
Do not use this feature. This feature has been replaced by sp_describe_first_result_set.
Too bad they didn't follow their own advice!
This has been patched as of version 11.0.2218.0. Metadata is correctly gathered using the sp_describe_first_result_set system stored procedure.
This can happen if the specified WITH results set in SSIS identifies that there are more columns than being returned by the stored proc being called. Check your stored proc and ensure that you have the correct number of output columns as the WITH results set.