How can I use insert statement with on duplicate key update like MySQL in Snowflake?
MySQL code :
INSERT INTO table (column_list)
VALUES (value_list)
ON DUPLICATE KEY UPDATE
c1 = v1,
c2 = v2,
...;
Since snowflake does not support ON DUPLICATE KEY UPDATE, how can I implement the same in snowflake?
What you are looking for is the MERGE command
https://docs.snowflake.com/en/sql-reference/sql/merge.html
Related
Consider a table like this one (postgresql):
create table t1 (
id serial not null primary key,
code1 int not null,
company_id int not null
);
create unique index idx1 on t1(code1, company_id);
I have to a new column code2 that will be not null and with a unique index like idx1, so the migration could be like:
alter table t1 add column code2 int not null;
create unique index idx2 on t1(code2, company_id);
The problem is: how do you manage the migration with the data that are already in the table?
You cannot apply this migration neither I can define a default value for the column.
What I was thinking to do is:
create and execute migration that just add the column without any constraint
insert the data manually in the column not with a migration
create and execute another migration that adds the constraints to the column
Is this a good practice or there are other ways?
As migration tool I am using flyway but I think it's the same for every database migration tool.
I have a temp table that has two rows.
Their Id is 999359143, 999365081
I have a table that doesn't have a primary key but has a unique index based off of the id and date.
This 999359143 already exists in the table. So when I use my query it still is trying to insert the row from the temp table into the normal table and it errors. This is the query below
INSERT INTO [XferTable]
([DataDate]
,[LoanNum]
)
SELECT Distinct t1.[DataDate]
,t1.[LoanNum]
FROM #AllXfers t1 WITH(HOLDLOCK)
WHERE NOT EXISTS(SELECT t2.LoanNum, t2.DataDate
FROM XferTable t2 WITH(HOLDLOCK)
WHERE t2.LoanNum = t1.LoanNum AND t2.DataDate = t1.DataDate
)
Is there a better way to do this?
You should use the MERGE statement, which acts atomically so you shouldn't need to do your own locking (also, isolation query hints on temporary tables doesn't achieve anything).
MERGE XferTable AS SOURCE
USING #AllXfers AS TARGET
ON
SOURCE.[DataDate] = TARGET.[DataDate]
AND SOURCE.[LoanNum] = TARGET.[LoanNum]
WHEN NOT MATCHED BY TARGET--record in SOURCE but not in TARGET
THEN INSERT
(
[DataDate]
,[LoanNum]
)
VALUES
(
SOURCE.[DataDate]
,TARGET.[LoanNum]
);
Your primary key violation is probably because you are using (Date, Loan#) as the uniqueness criteria and your primary key is probably only on Loan#.
I created a table into an external server
CREATE FOREIGN TABLE external_table (
field_1 varchar(15) NULL,
field_2 int4 NULL
)
SERVER server_name
OPTIONS(compression 'pglz', stripe_row_count '500000');
Now I want to insert into external_table, but if I run this query
INSERT INTO external_table (field_1, field_2) VALUES ('test',1);
It return this error
ERROR: operation is not supported
How can I add record into a foreign table?
I've tried with the following insert
INSERT INTO external_table (field_1, field_2) select 'test',1;
It works, but I can't use a INSERT INTO with SELECT statment.
Looks like the extension you are using supports "insert into ... select .." but not direct inserts.
you can use you should probably ask this question while specifying the extension.
PS: It looks like the extension you use is cstore_fdw. It does not support direct inserts, because it completely cancels benefits of using columnar storage and create some extra overhead. If you are using cstore_fdw, try to use bulk inserts instead of single row ones. Inserting into a regular table and moving data into cstore_fdw table when data reaches certain size (i.e. stripe_row_count number of rows) is much better option.
I am creating a database table and want to make sure that data in one of the column is always bounded by data in a column of another table. for example:
Table_1 has Column_1
Column_1 can have values:
v1
v2
v3
v4
v2
v3
Now I am trying to create Table_2 with Column_3
and want to make sure that values in this column is always a subset of Table_1:Column_1
Is there a constraint I can apply to achieve this?
In Oracle and PostgreSQL, use a check constraint
eg, in Oracle:
ALTER TABLE Table_3
ADD CONSTRAINT my_name
CHECK
(column_3 in
(SELECT Column_1 FROM Table_1))
This also works with PostgreSQL
In SQL Server and DB2, I believe, you have to create a function that does the actual test, but otherwise it's the same. The function would have a single argument (column_3's value) and return EXISTS (SELECT 1 FROM Table_1 WHERE Column_1 = argument).
Unfortunately, in MySQL you will need to use on insert and update triggers
Is Table_1.Column_1 unique?
If yes, just make it a PRIMARY KEY or UNIQUE, then reference it from FOREIGN KEY on the Table_2.Column_3.
If no (as apparently implied by your example), make a FK from Table_2 to Table_1's primary key. The Table_2.Column_3 won't even exist, instead you'll get the values of Table_1.Column_1 by JOIN-ing the two tables. You can put that JOIN in a VIEW to make it appear (to the client applications) as if the Table_2.Column_3 actually exists.
I need to merge data from 2 tables into a third (all having the same schema) and generate a mapping of old identity values to new ones. The obvious approach is to loop through the source tables using a cursor, inserting the old and new identity values along the way. Is there a better (possibly set-oriented) way to do this?
UPDATE: One additional bit of info: the destination table already has data.
Create your mapping table with an IDENTITY column for the new ID. Insert from your source tables into this table, creating your mapping.
SET IDENTITY_INSERT ON for your target table.
Insert into the target table from your source tables joined to the mapping table, then SET IDENTITY_INSERT OFF.
I created a mapping table based on the OUTPUT clause of the MERGE statement. No IDENTITY_INSERT required.
In the example below, there is RecordImportQueue and RecordDataImportQueue, and RecordDataImportQueue.RecordID is a FK to RecordImportQueue.RecordID. The data in these staging tables needs to go to Record and RecordData, and FK must be preserved.
RecordImportQueue to Record is done using a MERGE statement, producing a mapping table from its OUTPUT, and RecordDataImportQueue goes to RecordData using an INSERT from a SELECT of the source table joined to the mapping table.
DECLARE #MappingTable table ([NewRecordID] [bigint],[OldRecordID] [bigint])
MERGE [dbo].[Record] AS target
USING (SELECT [InstanceID]
,RecordID AS RecordID_Original
,[Status]
FROM [RecordImportQueue]
) AS source
ON (target.RecordID = NULL) -- can never match as RecordID is IDENTITY NOT NULL.
WHEN NOT MATCHED THEN
INSERT ([InstanceID],[Status])
VALUES (source.[InstanceID],source.[Status])
OUTPUT inserted.RecordID, source.RecordID_Original INTO #MappingTable;
After that, you can insert the records in a referencing table as folows:
INSERT INTO [dbo].[RecordData]
([InstanceID]
,[RecordID]
,[Status])
SELECT [InstanceID]
,mt.NewRecordID -- the new RecordID from the mappingtable
,[Status]
FROM [dbo].[RecordDataImportQueue] AS rdiq
JOIN #MappingTable AS mt
ON rdiq.RecordID = mt.OldRecordID
Although long after the original post, I hope this can help other people, and I'm curious for any feedback.
I think I would temporarily add an extra column to the new table to hold the old ID. Once your inserts are complete, you can extract the mapping into another table and drop the column.