I'm trying to insert array data into sql server table.
The table already defined and the connection works fine.
My current code is :
assets=[]
for asset in array:
if "some condition":
assets.append(asset)
cursor = conn.cursor()
cursor.executemany('insert into [Traffilog_Data].[dbo].[getData] values
(%d,%d,%d,%s,%s,%f,%f,%d,%d,%d,%d,%s,%s,%f,%d,%s)', assets)
all %var are according to the sql table definition.
I' get result
None
same as simple select query
what am I doing wrong?
Related
I am updating an on-premises SQL Server database table with data from a csv file using a Copy Data activity. There is an int identity Id column on the sink table that gets generated when I do the Upsert. I would like to retrieve the Id value generated in that table to use later in the pipeline.
Is there a way to do this?
I can't use a data flow as I am using a self-hosted Integration Runtime.
Hi #Nick.McDermaid, I am loading about 7,000 rows from the file to the database. I want to store the identities in the database the file comes from.
Edit:
I have 2 databases (source/target). I want to upsert (using MERGE SQL below, with the OUTPUT clause) into the target db from the source db and then return the Ids (via the OUTPUT resultset) to the source db. The problem I have is that the upsert (MERGE) SQL gets it's SELECT statement from the same target db that the target table is in (when using a Copy Data activity), but I need to get the SELECT from the source db. Is there a way to do this, maybe using the Script activity?
Edit 2: To clarify, the 2 databases are on different servers.
Edit 3 (MERGE Update):
MERGE Product AS target
USING (SELECT [epProductDescription]
,[epProductPrimaryReference]
FROM [epProduct]
WHERE [epEndpointId] = '438E5150-8B7C-493C-9E79-AF4E990DEA04') AS source
ON target.[Sku] = source.[epProductPrimaryReference]
WHEN MATCHED THEN
UPDATE SET [Name] = source.[epProductDescription]
,[Sku] = source.[epProductPrimaryReference]
WHEN NOT MATCHED THEN
INSERT ([Name]
,[Sku]
VALUES (source.[epProductDescription]
,source.[epProductPrimaryReference]
OUTPUT $action, inserted.*, updated.*;
Edit 3 (sample data):
source sample:
target output
Is there a way to do this, maybe using the Script activity?
Yes, you can execute this script using Script activity in ADF
As your tables are on different SQL servers first you have to create Linked server with source database on target Database.
go to >> Server Objects >> Linked Server >> New Linked server and create linked server with source database on target Database as below.
While creating linked server make sure same user must exist on both databases.
then I wrote Merge Query using this linked sever source.
My Sample Query:
MERGE INTO PersonsTarget as trg
USING (SELECT [LastName],[FirstName],[State]
FROM [OP3].[sample1].[dbo].[Personssource]) AS src
ON trg.[State] = src.[State]
WHEN MATCHED THEN
UPDATE SET [LastName] = src.[LastName]
,[FirstName] = src.[FirstName]
WHEN NOT MATCHED THEN
INSERT ([LastName],[FirstName],[State])
VALUES (src.[FirstName],src.[LastName],src.[State])
OUTPUT $action, inserted.*;
Then In Script activity I provided the script
Note: In linked service for on premises target table use same user which you used in linked service
Executed successfully and returning Ids:
I have created the following temp table in SQL Server Management Studio:
CREATE TABLE ##LoginMap
(
ObjectId NVARCHAR(50),
UserPrincipleName NVARCHAR(500),
Username NVARCHAR(250),
Email NVARCHAR(500),
Name NVARCHAR(250)
)
These are the same names as the column headers in the excel sheet I am trying to get the data from.
The file name is called Test Loadsheet and the sheet name is AgilityExport_04Aug2022_164839.
I am trying to insert the data into the temp table like so:
INSERT INTO ##LoginMap
SELECT *
FROM OPENROWSET('Microsoft.ACE.OLEDB.12.0',
'Excel 12.0; Database=C:\temp\Test LoadSheet.xlsx', [AgilityExport_04Aug2022_164839]);
GO
However, I am getting the error:
The OLE DB provider "Microsoft.ACE.OLEDB.12.0" for linked server "(null)" does not contain the table "AgilityExport_04Aug2022_164839". The table either does not exist or the current user does not have permissions on that table.
Where have I gone wrong with this? And what do I need to do in order to successfully get the data from each column into my table?
You have file name as Test Loadsheet in one spot, but then in your query you have it as Test LoadSheet.xlsx. Try and see if this is holding it up.
Found a link on importing data from excel to SQL if you are interested:
https://learn.microsoft.com/en-us/sql/relational-databases/import-export/import-data-from-excel-to-sql?view=sql-server-ver16#openrowset
I went about this a different way. In the excel sheet I am using a made a formula like so:
="INSERT INTO ##LoginMap(ObjectId, DisplayName, AzureEmail, AzureUsername) VALUES('"&A2&"', '"&B2&"', '"&C2&"', '"&D2&"');"
I then had this repeated for each row. This then gave me an insert statement for each that I could simply copy and paste into SSMS and allow to run the query
I am using the below query in SQL Server Management Studio 2017.
SELECT ENCRYPTBYPASSPHRASE('xxyy','test#123')
When I run the above statement, it returned an encrypted password like 0x01000000EA686E7D1AED8C501B193A2F655368FC3EABA009082C90F58987DD0487833C62
I wanted to store this in a table which contains a NVARCHAR(MAX) field using a stored procedure, but insertion happens with a blank value instead of the encrypted password.
I used a print statement to acquire the value emitted by this function with in the SP. It returned some unreadable characters as below.
됏㬷病譽快
How to use ENCRYPTBYPASSPHRASE() properly in order to insert the return value to a table field.
You can try the following Sql statement
CREATE TABLE #TempPassword (encPassword varbinary(250))
declare #encPwd varbinary(250)
set #encPwd = (SELECT ENCRYPTBYPASSPHRASE('xxyy','test#123'))
insert into #TempPassword values (#encPwd)
select * from #TempPassword
Now the output after insert it is as shown below
encPassword
0x0100000023C4E9BFBC0F4319735ED2F0C76C2D857C114D96867711F1EE290AB2E7511961
I have created a table called DimInternationalFunction.
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[DimInternationalFunction]') AND type in (N'U'))
DROP TABLE [DimInternationalFunction]
Go
Create Table DimInternationalFunction
(IntFunctionKey int NOT NULL identity primary key,
SubSubFunctionString char(10),
FunctionCode char(3),
SubFunctionCode char(6),
SubSubFunctionCode char(10),
SubSubFunctionName nvarchar(60),
SubFunctionName nvarchar(60),
FunctionName nvarchar(60))
I have initially inserted records in this table in SSMS.
After inserting the initial records manually in SSMS, now my manager wants me to insert "new records only" using SSIS.
I have tried using this in SSMS and it worked. Either it gives me 0 records inserted or sometimes it gives me 5 records inserted as a result. My manager wants me to do this in SSIS.
I tried using this script inside the OLE DB Source under Data Access Mode: SQL Command and SQL Command text:
insert into DWResourceTask.dbo.DimInternationalFunction
select f.SubSubFunctionString,
f.FunctionCode,
f.SubFunctionCode,
f.SubSubFunctionCode,
f.SubSubFunctionName,
f.SubFunctionName,
f.FunctionName
from ODS_Function F
where FunctionViewCode = 'INT'
and not exists (select * from DWResourceTask.dbo.DimInternationalFunction I
where (f.SubSubFunctionString=i.SubSubFunctionString
and f.FunctionCode=i.FunctionCode
and f.SubFunctionCode=i.SubFunctionCode
and f.SubSubFunctionCode=i.SubSubFunctionCode
and f.SubSubFunctionName=i.SubSubFunctionName
and f.SubFunctionName=i.SubFunctionName
and f.FunctionName=i.FunctionName)
)
The error message that I got after clicking preview is
The component reported the following warnings:
Error at Int Function [International Function Table [33]]: No column information was returned by the SQL command.
Choose OK if you want to continue with the operation.
Choose Cancel if you want to stop the operation.
Is there another component in SSIS that can do this? or can I just use either exec sql task component or ole db source?
I am thinking of using exec sql task connected to a data flow task, inside the data flow task I will put ole db source containing a staging table and do a delete on that or is there any other way to do it. Please help. Thanks in advance.
You could do it with an Execute SQL task.
If you want to do it "the pure SSIS way", you could use a lookup component. Set the "rows with no matching" handler to "Redirect to no match output", and configure the target table as connection. Then use the "No Match Output" only, ignoring the "Match Output". And send the records from the "No Match Output" to the target.
In spite of its name, the "Lookup" component can be used to filter data in many cases.
But I would assume the Execute SQL task would be more efficient for large data sets, keeping all data within the database engine.
I have 2 databases in SQL Server 2005.
I want a functionality that i have same table structure in 2 database for example i have a same table named as testData in 2 database named as dbTest1 and dbTest2.
Now i want a single query through which i can add all the records from table testData of database dbTest2 into table testData of database dbTest1.
I tried to use following query
insert into dbTest1.testData values select * from dbTest2.testData
but this query is not running and giving error.
I also tried
insert into dbTest1.testData(col1,col2,col3) values select * from dbTest2.testData
but this also gives error that "Invalid object name dbTest2.testData"
Could any one help in this
Thanks
Replace dbTest2.testData with dbTest2..testData - you have to specify 3 things (or optionally leave the middle blank for dbo).
i.e.
insert into dbTest1..testData
select * from dbTest2..testData
If the table doesn't already exist in dbTest1, you can do this:
select *
into dbTest1..testData
from dbTest2..testData
You need to specify all column names in query.
insert into dbTest1.dbo.testData(col1,col2,col3) select * from dbTest2.dbo.testData