SQL Server 2017 (In Azure) - when I need to create a new client in our clients database, I have to run three separate queries, and in between each query, do a lookup to be able to populate a part of the next query. I'd like to see if there is a way to combine all this into one query, or, parameterized stored procedure:
All of this takes place in the same database called Clients:
Step 1 - Create the client record in dbo.clients:
INSERT INTO dbo.clients
(ClientGuid, Name, Permissions)
VALUES
(NEWID(), 'Contoso', 1)
Step 2 - Get the Primary Key which was auto-created in Step 1:
SELECT ClientKey from dbo.clients
WHERE Name = 'Contoso'
Now write down the primary key (ClientKey) from that record, we'll say 12345678
Step 3 - Create a new billing code in the dbo.billingcodes table:
INSERT INTO dbo.billingcodes
(BillingCodeGuid, ClientKey, Name, ScoreId)
VALUES
(NEWID(), 12345678, 'Contoso Production Billing Code', 1)
How can I combine all this into one query or parameterized stored procedure where all I have to enter in are the two names from step 1 and 3 (assume the Permissions and ScoreId integers are always going to be 1) and also get an output at the end of the process of the created values for dbo.clients.ClientKey and dbo.billingcodes.BillingCodeGuid?
You could create a procedure that consists of both inserts with a line in between to get the ID of the inserted client. Assign the ID to a variable and pass it in to the second part.
See this post about some different ways about getting the inserted record’s ID Best way to get identity of inserted row?
You could do it by using procedure. You may find this link for creating procedure in SQL Server Link.
In case of Procedure , need to insert your data into first table. Then using IDENT_CURRENT (Ident_Current) you'll get your last inserted id from table, which will further use to insert it into next table.
Related
I queried some data from table A(Source) based on certain condition and insert into temp table(Destination) before upsert into Crm.
If data already exist in Crm I dont want to query the data from table A and insert into temp table(I want this table to be empty) unless there is an update in that data or new data was created. So basically I want to query only new data or if there any modified data from table A which already existed in Crm. At the moment my data flow is like this.
clear temp table - delete sql statement
Query from source table A and insert into temp table.
From temp table insert into CRM using script component.
In source table A I have audit columns: createdOn and modifiedOn.
I found one way to do this. SSIS DataFlow - copy only changed and new records but no really clear on how to do so.
What is the best and simple way to achieve this.
The link you posted is basically saying to stage everything and use a MERGE to update your table (essentially an UPDATE/INSERT).
The only way I can really think of to make your process quicker (to a significant degree) by partially selecting from table A would be to add a "last updated" timestamp to table A and enforcing that it will always be up to date.
One way to do this is with a trigger; see here for an example.
You could then select based on that timestamp, perhaps keeping a record of the last timestamp used each time you run the SSIS package, and then adding a margin of safety to that.
Edit: I just saw that you already have a modifiedOn column, so you could use that as described above.
Examples:
There are a few different ways you could do it:
ONE
Include the modifiedOn column on in your final destination table.
You can then build a dynamic query for your data flow source in a SSIS string variable, something like:
"SELECT * FROM [table A] WHERE modifiedOn >= DATEADD(DAY, -1, '" + #[User::MaxModifiedOnDate] + "')"
#[User::MaxModifiedOnDate] (string variable) would come from an Execute SQL Task, where you would write the result of the following query to it:
SELECT FORMAT(CAST(MAX(modifiedOn) AS date), 'yyyy-MM-dd') MaxModifiedOnDate FROM DestinationTable
The DATEADD part, as well as the CAST to a certain degree, represent your margin of safety.
TWO
If this isn't an option, you could keep a data load history table that would tell you when you need to load from, e.g.:
CREATE TABLE DataLoadHistory
(
DataLoadID int PRIMARY KEY IDENTITY
, DataLoadStart datetime NOT NULL
, DataLoadEnd datetime
, Success bit NOT NULL
)
You would begin each data load with this (Execute SQL Task):
CREATE PROCEDURE BeginDataLoad
#DataLoadID int OUTPUT
AS
INSERT INTO DataLoadHistory
(
DataLoadStart
, Success
)
VALUES
(
GETDATE()
, 0
)
SELECT #DataLoadID = SCOPE_IDENTITY()
You would store the returned DataLoadID in a SSIS integer variable, and use it when the data load is complete as follows:
CREATE PROCEDURE DataLoadComplete
#DataLoadID int
AS
UPDATE DataLoadHistory
SET
DataLoadEnd = GETDATE()
, Success = 1
WHERE DataLoadID = #DataLoadID
When it comes to building your query for table A, you would do it the same way as before (with the dynamically generated SQL query), except MaxModifiedOnDate would come from the following query:
SELECT FORMAT(CAST(MAX(DataLoadStart) AS date), 'yyyy-MM-dd') MaxModifiedOnDate FROM DataLoadHistory WHERE Success = 1
So the DataLoadHistory table, rather than your destination table.
Note that this would fail on the first run, as there'd be no successful entries on the history table, so you'd need you insert a dummy record, or find some other way around it.
THREE
I've seen it done a lot where, say your data load is running every day, you would just stage the last 7 days, or something like that, some margin of safety that you're pretty sure will never be passed (because the process is being monitored for failures).
It's not my preferred option, but it is simple, and can work if you're confident in how well the process is being monitored.
In SQL Server, i am inserting multiple records into table using batch update. How do i get back the ID's (unique primary key) which is being created after batch update?
If I insert one record, I can get the last inserted using IDENT(tableName). I am not sure how to get if I do batch update. Please help.
For example, I have student table, with ROLE NO and NAME. ROLE NO is auto incremented by 1, as soon I insert the names into DB using java program. I will add 3 rows at a time using batch update from my java code. In DB, it gets added with ROLE NO 2, 3 and 4. How do I get these newly generated ID in my java program, please help
I tried getting ids using getgeneratedkeys method after I do executebatch. I get exception. Is batch update + get generated keys supported.?
In SQL Server when you do an insert there is an extra option your query; OUTPUT. This will let you capture back the data you inserted into the table - including your id's. You have to insert them into a temporary table; so something like this (with your table/ column names will get you there.
declare #MyNewRoles Table (Name, RoleNo)
insert into tblMyTable
(Name)
Select
Name
Output
inserted.Name, Inserted.RoleNo
into #MyNewRoles
From tblMyTableOfNames
select * from #MyNewRoles
If you don't mind adding a field to your table, you could generate a unique ID for each batch transaction (for example, a random UUID), and store that in the table as well. Then, to find the IDs associated with a given transaction you would just need something like
select my_id from my_table where batch_id = ?
I have sqlite local database. I want to insert only fresh data from remote server to local database.Since there is no time field,it is difficult to insert only new records.How can i acheive this? I require this for my hybrid mobile app. Any helps apperciated..Thanks in advance.
Two tables:
my local db table is
tbl_orders
id name age
1 yyy 30
2 xxx 20
my remote db table is
tbl_orders
id name age
1 yyy 36
2 xxx 20
3 vvv 40
4 zzz 37
In the above the remote table contains additionally two records and also the value in first record(age column) get changed.now i want to insert and update this(i.e 1st,3rd,4th) to my local sqlite table without deleting and reinserting the whole table.
You should add a UNIQUE constrain in id.
Redefining your local table:
CREATE TABLE tbl_orders(UNIQUE id, name, age);
or without redefining table
CREATE UNIQUE INDEX tbl_orders_id ON tbl_orders(id);
With constrain, your updates become a single statement:
INSERT OR REPLACE INTO local.tbl_orders SELECT * FROM remote.tbl_orders;
Your table is replicated in two DB?
If yes, you can do an INSERT with NOT EXISTS clause in WHERE condition. Alternatively, add datetime field in your source table.
Another way:
Add a boolean field in your source DB (fl_sent), populated by a trigger when create a new row in your DB or update them. Default value is false, and when you want to syncronize your DBs your select is based on this field
SELECT * FROM myTable WHERE fl_sent = 0
For a more complete answer please post your tables.
EDIT AFTER COMMENT:
Solution 1:
Add field date in your remote table (source) (if you want, you write a trigger about toggle this field) and then execute your sendable query based on this date.
Solution 2:
Add field flag (fl_sent) set by zero (if you want, you write a trigger about toggle this field) and then execute your sendable query based on this flag.
I am very new to SQL and SQL server, would appreciate any help with the following problem.
I am trying to update a share price table with new prices.
The table has three columns: share code, date, price.
The share code + date = PK
As you can imagine, if you have thousands of share codes and 10 years' data for each, the table can get very big. So I have created a separate table called a share ID table, and use a share ID instead in the first table (I was reliably informed this would speed up the query, as searching by integer is faster than string).
So, to summarise, I have two tables as follows:
Table 1 = Share_code_ID (int), Date, Price
Table 2 = Share_code_ID (int), Share_name (string)
So let's say I want to update the table/s with today's price for share ZZZ. I need to:
Look for the Share_code_ID corresponding to 'ZZZ' in table 2
If it is found, update table 1 with the new price for that date, using the Share_code_ID I just found
If the Share_code_ID is not found, update both tables
Let's ignore for now how the Share_code_ID is generated for a new code, I'll worry about that later.
I'm trying to use a merge query loosely based on the following structure, but have no idea what I am doing:
MERGE INTO [Table 1]
USING (VALUES (1,23-May-2013,1000)) AS SOURCE (Share_code_ID,Date,Price)
{ SEEMS LIKE THERE SHOULD BE AN INNER JOIN HERE OR SOMETHING }
ON Table 2 = 'ZZZ'
WHEN MATCHED THEN UPDATE SET Table 1.Price = 1000
WHEN NOT MATCHED THEN INSERT { TO BOTH TABLES }
Any help would be appreciated.
http://msdn.microsoft.com/library/bb510625(v=sql.100).aspx
You use Table1 for target table and Table2 for source table
You want to do action, when given ID is not found in Table2 - in the source table
In the documentation, that you had read already, that corresponds to the clause
WHEN NOT MATCHED BY SOURCE ... THEN <merge_matched>
and the latter corresponds to
<merge_matched>::=
{ UPDATE SET <set_clause> | DELETE }
Ergo, you cannot insert into source-table there.
You could use triggers for auto-insertion, when you insert something in Table1, but that will not be able to insert proper Shared_Name - trigger just won't know it.
So you have two options i guess.
1) make T-SQL code block - look for Stored Procedures. I think there also is a construct to execute anonymous code block in MS SQ, like EXECUTE BLOCK command in Firebird SQL Server, but i don't know it for sure.
2) create updatable SQL VIEW, joining Table1 and Table2 to show last most current date, so that when you insert a row in this view the view's on-insert trigger would actually insert rows to both tables. And when you would update the data in the view, the on-update trigger would modify the data.
Here is the following situation:
I have a table of StudentsA which needs to be synchronized with another table, on a different server, StudentsB. It's a one-way sync from A to B.
Since the table StudentsA can hold a large number of rows, we have a table called StudentsSync (on the input server) containing the ID of StudentsA which have been modified since the last copy from StudentsA to StudentsB.
I made the following SSIS Data Flow task:
The only problem is that I need to DELETE the row from StudentsSync after a successful copy or update. Something like this:
Any idea how this can be achieved?
It can be achieved using 3 methods
1.If your target table in OutputDB has TimeStamp columns such as Create and modified TimeStamp then rows which have got updated or inserted can be obtained by writing a simple query. You need to write the below query in the execte sql task in Control Flow to delete those rows in Sync Table .
Delete from SyncTable
where keyColumn in (Select primary_key from target
where ModifiedTimeStamp >= GETDATE() or (ModifiedTimeStamp is null
and CreateTimeStamp>=GETDATE()))
I assume StudentsA's primary key is present in Sync table along with primary key of Target table. The above condition basically checks, if a new row is added then CreateTimeStamp column will have current date and modifiedTimeStamp will be null else if the values are updated then the modifiedTimeStamp will have current date
The above query will work if you have TimeStamp columns in your target table which i feel should be there if your loading data into Data Warehouse
2.You can use MERGE syntax to perform the update and insert in Control Flow with Execute SQL Task.No need to use Data Flow Task .The below query can be used even if you don't have any TimeStamp columns
DECLARE #Output TABLE ( ActionType VARCHAR(20), SourcePrimaryKey INT)
MERGE StudentsB AS TARGET
USING StudentsA AS SOURCE
ON (TARGET.CommonColumn = SOURCE.CommonColumn)
WHEN MATCHED
THEN
UPDATE SET TARGET.column = SOURCE.Column,TARGET.ModifiedTimeStamp=GETDATE()
WHEN NOT MATCHED BY TARGET THEN
INSERT (col1,col2,Col3)
VALUES (SOURCE.col1, SOURCE.col2, SOURCE.Col3)
OUTPUT $action,
INSERTED.PrimaryKey AS SourcePrimaryKey INTO #Output
Delete from SyncTable
where PrimaryKey in (Select SourcePrimaryKey from #Output
where ActionType ='INSERT' or ActionType='UPDATE')
The code is not tested as i'm running out of time .but at-least it should give you a fair idea how to proceed . .For furthur detail on MERGE syntax read this and this
3.Use Multicast Component to duplicate the dataset for Insert and Update .Connect a MULTICAST to lookmatch output and another multicast to Lookup No match output
Add a task after "Update existing entry" and after "Insert new entry" to add the student ID to a variable which will contain the list of IDs to delete.
Enclose all of the tasks in a sequence container.
After the sequence container executes add a task to delete all the records from the sync table that are in that variable you've been populating.