I have Table A and Table B.
Table A contains data from another source.
Table B contains data that is inserted from Table A along with data from other tables. I have done the initial insert of data from A to B but now what I am trying to do is insert the records that do not exist already in Table B from Table A on a daily basis. Unfortunately, there is no primary key or unique identifier in Table A which is making this difficult.
Table A contains a field called file_name which has values that looks like this:
this_is_a_file_name_01011980.txt
There can be duplicate values in this column (multiple files from the same date).
In Table B I created a column data_date which extracts the date from the table a.file_name field. There is also a load_date field which just uses GETDATE() at the time the data is inserted.
I am thinking I can somehow compare the dates in these tables to decide what needs to be inserted. For example:
If the file date from Table A (would need to extract again) is greater than the load_date of Table B, then insert these records into Table B.
Let me know if any clarification is needed.
You could use exists or except. With the explanation here it seems like except would make short work of this. Something like this.
insert tableB
select * from tableA
except
select * from tableB
Related
I have two tables with the same field names and a stored procedure that updates table B with Table A's data by doing a delete from current table and insert into current table from another table that has update values in it:
delete from ac.Table1
insert into ac.Table1
select *
from dbo.OriginalTable
where dtcreate <getdate()-1
I had to recreate Table1 through GIS software which adds GlobalIDs and an Object ID field. The original order had Object ID at the end and the new table has it at the front. Will this impact executing the SQL statement above?
Yes it will. The order of the columns should match for each value to go in desired column
You can try
Insert into ac.Table1 (column1....columnN)
I have a task to Generate a Derived Column (RestrictionID) from a table and add it to a source table if 6 columns on both tables match. The 6 columns include (Country, Department, Account .etc) I decided to go with the SSIS Lookup and generate the Derived column when there's a match. This ID also has a time and amount Limit. Once all the records have ID's, I'm supposed to calculate a running total based on the ID to enforce the limits which is the easy part.
The only problem is this Lookup table changes almost daily and any or all of the 6 columns can have NULLS. Even the completely null rows have an Id. Nulls mean the Restriction is open. eg. If the Country column on one record on the lookup table is null, then the ID of that record can be assigned to records with any country on the source. If one row on the lookup has all null columns, then this is completely open and all records on the source qualify for that ID. The Source table doesn't have NULLS.
Please assist if possible
Thanks
If NULL means any and ignore column in lookup then add this to your where:
use a stored proc and pass your values in and return:
select lookup.ID
from lookup
where #Country = isnull(lookup.Country,#Country) //If lookup inull then it refers to itself creating a 1=1 scenario
and #Department = isnull(lookup.Department,#Department)
and ...
In SQL, I've to delete a data from table A which is dependent on table B.
The data to be deleted should satisfy two conditions WorkArea='123' and FileNo='45'.
Table B has WorkArea but it does not contain data for FileNo.
And Table A contains the record satisfying both the conditions.
There isn't any reference key. For more clarity, adding a query here:
Select * from table A where WorkArea='123' and FileNo='45';
This will generate the resulting record. But as it is dependent on Table B, I cannot delete it directly. Also, to delete it from table B isn't possible because data in WorkArea is a whole and it contains many files and I have to delete a specific File.
So how can I delete data from table A?
This is Table A with col1 and col2 as primary key.
This is Table B with col1 as a primary key.
If you have no Foreign Keys, the following sentence will work.
DELETE FROM [A] WHERE [WorkArea] = '123' AND [FileNo] = '45';
Then you can programmaticaly check if there are "orphans" on table B with the following request :
SELECT DISTINCT [B].[WorkArea]
FROM [B]
LEFT JOIN [A]
ON [A].[WorkArea] = [B].[WorkArea]
WHERE [A].[WorkArea] IS NULL
To enhance this last part and produce a DELETE sentence from it, just store the result of this request into a temporary table then use it as a WHERE statement with the IN keyword.
I'm attempting to use SSIS to do a data migration from an old system to a new system. I have the tables TableA, TableB and OldTable. The tables are as follows:
TableA
ID
BusinessTypeEnum
Other
TableB
ID
TableB_ID (FK)
PermitNumber
OldTable
ID
BusinessType
PermitNumber
As you can see, TableA and TableB (which are in the new system) are really OldTable separated out into two tables.
My problem is that in order to insert a record in TableB I need to know that PrimaryKey that was assigned at the time the corresponding records was inserted in TableA.
I had originally planned on doing a LookUp when attempting to insert records into table TableB however, there isn't any data stored in TableA that would allow me to return.
Any ideas?
It sounds like you're using a single data flow: source -> split -> two destinations (TableA, TableB).
What if you divided the process into two data flows?
The first simply pulls from OldTable and loads into TableA.
The second--loading into TableB--again pulls from OldTable but also gets the relevant identity value from TableA by doing a Merge Join to, or a Lookup from, that table.
(Note: the second flow needs to be set to run only after the first completes via a precedence constraint as it relies on OldTable's data being inserted into TableA before its run.)
You can use the OUTPUT Clause in our insert statement to return the value of the identity column.
INSERT TableA (BusinessTypeEnum, Other)
OUTPUT INSERTED.ID
VALUES ('abc', 'def')
in addition to inserting a record, the above will act like a select.
I should mention that there is an INTO clause in the OUTPUT clause that can be used with a batch insert. The above example is for a single record insert.
I need to merge data from 2 tables into a third (all having the same schema) and generate a mapping of old identity values to new ones. The obvious approach is to loop through the source tables using a cursor, inserting the old and new identity values along the way. Is there a better (possibly set-oriented) way to do this?
UPDATE: One additional bit of info: the destination table already has data.
Create your mapping table with an IDENTITY column for the new ID. Insert from your source tables into this table, creating your mapping.
SET IDENTITY_INSERT ON for your target table.
Insert into the target table from your source tables joined to the mapping table, then SET IDENTITY_INSERT OFF.
I created a mapping table based on the OUTPUT clause of the MERGE statement. No IDENTITY_INSERT required.
In the example below, there is RecordImportQueue and RecordDataImportQueue, and RecordDataImportQueue.RecordID is a FK to RecordImportQueue.RecordID. The data in these staging tables needs to go to Record and RecordData, and FK must be preserved.
RecordImportQueue to Record is done using a MERGE statement, producing a mapping table from its OUTPUT, and RecordDataImportQueue goes to RecordData using an INSERT from a SELECT of the source table joined to the mapping table.
DECLARE #MappingTable table ([NewRecordID] [bigint],[OldRecordID] [bigint])
MERGE [dbo].[Record] AS target
USING (SELECT [InstanceID]
,RecordID AS RecordID_Original
,[Status]
FROM [RecordImportQueue]
) AS source
ON (target.RecordID = NULL) -- can never match as RecordID is IDENTITY NOT NULL.
WHEN NOT MATCHED THEN
INSERT ([InstanceID],[Status])
VALUES (source.[InstanceID],source.[Status])
OUTPUT inserted.RecordID, source.RecordID_Original INTO #MappingTable;
After that, you can insert the records in a referencing table as folows:
INSERT INTO [dbo].[RecordData]
([InstanceID]
,[RecordID]
,[Status])
SELECT [InstanceID]
,mt.NewRecordID -- the new RecordID from the mappingtable
,[Status]
FROM [dbo].[RecordDataImportQueue] AS rdiq
JOIN #MappingTable AS mt
ON rdiq.RecordID = mt.OldRecordID
Although long after the original post, I hope this can help other people, and I'm curious for any feedback.
I think I would temporarily add an extra column to the new table to hold the old ID. Once your inserts are complete, you can extract the mapping into another table and drop the column.