I have a complex sql requirement where in I need to insert data to target table(set of select statements from a source table) based on values in control table.
The control table has got list of products with attribute1, attribute2 and action columns(INCLUDE/EXCLUDE)
This insert is based on value of action column which is done at product level(For each product).
For action INCLUDE we have two cases:
1.when attribute2 is not null, For every product insert that attribute1/attribute2 combination alone and exclude all other atrribute1 combination(for that product)
when attribute2 is null, For that product include all attribute1 combinations
For action EXCLUDE in control table we have two cases:
1.For every product exclude the mentioned attribute1/attribute2 combination and include all other atrribute1 combination (for that product)
when attribute2 is null, For that product exclude the mentioned attribute1 combinations and include all other attribute 1 combinations
Below is the source table /control table and the values in it.
create table #source(
Product varchar(100),
attribute12Value varchar(100),--(its a combination of attribute1,attribute2, some value)
colx varchar(100),
coly varchar(100),
)
create table #control(
attribute1 varchar(100),
attribute2 varchar(100),
value varchar(100),
Product varchar(100),
action varchar(100)
)
insert #source(Product ,attribute12Value ,colx,coly)
select 'HP', 'hw1-i3proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'HP', 'hw2-i3proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'HP', 'hw2-i5proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'HP', 'hw2-i7proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'HP', 'hw3-i3proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'Dell', 'hw1-i3proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'Dell', 'hw1-i5proc-ver1-sale','normal','Y'
insert #source(Product ,attribute12Value ,colx,coly)
select 'Dell', 'hw1-i7proc-ver1-sale','normal','Y'
Based on the action value in the control table the output from source table should be as below:
--case1 when attribute2 not null
--For every product insert that attribute1/attribute2 combination alone and exclude all other atrribute1 combination(for that product)
insert #control(attribute1 ,attribute2,value ,Product,action)
select 'hw2','i3proc','ver1','HP','INCLUDE'
--For every product exclude the mentioned attribute1/attribute2 combination and include all other atrribute1 combination (for that product)
insert #control(attribute1 ,attribute2,value ,Product,action)
select 'hw2','i5proc','ver1','HP','EXCLUDE'
--case2 when attribute2 is null
-- For that product include all attribute1 combinations
insert #control(attribute1 ,attribute2,value ,Product,action)
select 'hw2',NULL,'ver1','HP','INCLUDE'
-- For that product exclude the mentioned attribute1 combinations and include all other attribute 1 combinations
insert #control(attribute1 ,attribute2,value ,Product,action)
select 'hw3',NULL,'ver1','HP','EXCLUDE'
Similarly this logic applies to all other products such as DELL which is then inserted into the target table.
I am able to find a INCLUDE logic like this:
select T1.Product,T1.attribute12Value,T1.colx,T1.coly from #control T
inner join #source T1 on T.action='INCLUDE'
and T.Product=T1.Product and PATINDEX(T.attribute1+'-'+isnull(T.attribute2,'')+'%',T1.attribute12Value)!=0
But I am stuck at the EXCLUDE logic where I need to exclude the mentioned attribute1/2 combination and include all other attribute1 combinations.
Sample Source Table
Can a general TSQL logic be applied for such a requirement to EXCLUDE records based on a control table?
Thanks.
T-SQL (for SQL-Server version 2008+) has an EXCEPT set based operator.
Given query A, to exclude a set of rows defined by query B from the result set :
-- result set
SELECT * FROM A
EXCEPT
-- rows to exclude/remove from result set
SELECT * FROM B
so for your exclude case 1 (for each product include all rows having specified attribute1, then remove all rows having specified attribute1/attribute2 combination) :
select T1.Product,T1.attribute12Value,T1.colx,T1.coly from #control T
inner join #source T1 on T.action='EXCLUDE' and T.Product=T1.Product AND [... Your predicate for selecting all attribute1 combinations ... ] AND attribute2 is not null
EXCEPT
select T1.Product,T1.attribute12Value,T1.colx,T1.coly from #control T
inner join #source T1 on T.action='EXCLUDE' and T.Product=T1.Product AND [... Your predicate for selecting the combination attribute1/attribute2 ... ] AND attribute2 is not null
Related
I am trying to split a column('categories') of a Table 'movies_titles' which has string separated data values in it.
e.g:
ID title categories
1 Movie A Comedy, Drama, Romance
2 Movie B Animation
3 Movie C Documentary, Life changing
I want to split the comma delimited string and place each values in a separate rows and update the table
-- this query shows the splitted strings as I want it
SELECT *
FROM dbo.movies_titles
CROSS APPLY
string_split(categories, ',')
O/P:
ID title categories value
1 Movie A Comedy, Drama, Romance Comedy
1 Movie A Comedy, Drama, Romance Drama
1 Movie A Comedy, Drama, Romance Romance
2 Movie B Animation Animation
3 Movie C Documentary, Life changing Documentary
3 Movie C Documentary, Life changing Life changing
I want to use UPDATE query to set the result obtained from value column. I just don't want to use SELECT query to view the result but permanently update the changes to the table. How do I achieve this in sql server?
You can do something similar to your intention creating new rows, because the update statement won't create the additional rows made by the split.
There can be issues if the ID column is unique, like a primary key, and there is the need to keep the title associated with that column.
I've created two scenarios on DB Fiddle, showing how you can do this using only one table as the question instructed, but a better alternative would be to save this information on another table.
This code on DB Fiddle: link
--Assuming your table is something like this
create table movies_id_as_pk (
ID int identity(1,1) primary key,
title varchar(200),
categories varchar(200),
category varchar(200)
)
--Or this
create table movies_other_pk (
another_id int identity(1,1) primary key,
ID int,
title varchar(200),
categories varchar(200),
category varchar(200)
)
--The example data
set identity_insert movies_id_as_pk on
insert into movies_id_as_pk (ID, title, categories) values
(1, 'Movie A', 'Comedy, Drama, Romance'),
(2, 'Movie B', 'Animation'),
(3, 'Movie C', 'Documentary, Life changing')
set identity_insert movies_id_as_pk off
insert into movies_other_pk (ID, title, categories)
select ID, title, categories from movies_id_as_pk
--You can't update directly any of the tables, because as the result of the split
--have more rows than the table, it would just leave the first value found:
update m set category = rtrim(ltrim(s.value))
from movies_id_as_pk m
cross apply string_split(m.categories, ',') as s
update m set category = rtrim(ltrim(s.value))
from movies_other_pk m
cross apply string_split(m.categories, ',') as s
select * from movies_id_as_pk
select * from movies_other_pk
--What you can do is create the aditional rows, inserting them:
--First, let's undo what the last instructions have changed
update movies_id_as_pk set category=NULL
update movies_other_pk set category=NULL
--Then use inserts to create the rows with the categories split
insert into movies_id_as_pk (title, category)
select m.title, rtrim(ltrim(s.value))
from movies_id_as_pk m
cross apply string_split(m.categories, ',') as s
insert into movies_other_pk (ID, title, category)
select m.ID, m.title, rtrim(ltrim(s.value))
from movies_other_pk m
cross apply string_split(m.categories, ',') as s
select * from movies_id_as_pk
select * from movies_other_pk
It actually is possible to insert or update at the same time. That is to say: we can update each row with a single category, then create new rows for the extra ones.
We can use MERGE for this. We can use the same table as source and target. We just need to split the source, then add a row-number partitioned per each original row. We then filter the ON clause to match only the first row.
WITH Source AS (
SELECT
m.ID,
m.title,
category = TRIM(cat.value),
rn = ROW_NUMBER() OVER (PARTITION BY ID ORDER BY (SELECT NULL))
FROM movies m
CROSS APPLY STRING_SPLIT(m.categories, ',') cat
)
MERGE movies t
USING Source s
ON s.ID = t.ID AND s.rn = 1
WHEN MATCHED THEN
UPDATE
SET categories = s.category
WHEN NOT MATCHED THEN
INSERT (ID, title, categories)
VALUES (s.ID, s.title, s.category)
;
db<>fiddle
I wouldn't necessarily recommend this as a general solution though, because it appears you actually have other normalization problems to sort out first. You should really have separate tables for all this information:
Movie
Category
MovieCategory
I am trying to run an insert query on a SQL wide table (Sparse Table), but I am getting an error:
"Column name or number of supplied values does not match table definition"
Query:
Insert [dbo].[Table1] (Table1 is a sparse table)
select id, [A],[B], [C], [D], [E] from (
Select ID,
CategoryId,
1 as Flag
From dbo.table2
) a Pivot(Avg(Flag) For CategoryID In (
[A],
[B],
[C],
[D],
[E]
)) As PivotTable
I am able to run the query for a normal sql table but it fails for a sparse table. I would really appreciate any help on this
Thanks in Advance
You should do:
Create an INSERT statement that explicitly lists the columns it will insert into - assuming that ID might be an IDENTITY column
that you don't want / can't insert into
Define the exact number of values to fill into these columns
your INSERT statement should be something like:
insert into table_1 (cola, colb, colc)
select cola, colb, colc from table_2
insert into tb1 values('1', '2','3') - this works fine as long you only have 3 columns
if you have 4 columns but only want to insert into 3 of them.
You have to include column names in INSERT INTO
insert into tb1 (Col1,col2,col3) select col1, col2, col3 from tb_2
Note: Always explicitly define the list of columns that an INSERT statement should fill data into
I have 2 tables called Customer and ChangeLog. having the following structure
Customer table
ChangeLog Table
My Requirement is that
I need an SSIS Package that will read the record from another table with the same structure as CustomerTable and then compare the rows on both tables. If a change in any record is found it updates the records in the customer table as well as put an entry in the ChangeLog saying which column was updated.
So when a change is found in any of the columns I need to do the following
Update the Coresposing record in the Customer Table
Insert a new row into the ChangeLog
There won't be an Insert to the Customer Table. There will be only updates
Is there any single Task in SSIS that I can use to do both the update as well as an insert to these different tables ? or else what is the quickest and efficient way to achieve this in SSIS?
Any help is much appreciated
No there is no single SSIS task made to do this. I wouldn't use SSIS for this at all. Put the logic in either a stored procedure or trigger. If you have to use SSIS for some reason, then have SSIS call the stored procedure, or UPDATE the table and let the trigger fire.
This here is better than a SSIS packages since you can use a trigger to detect your row changes, and even the values.
Try my example you can just C/P into management studio. When you update on Sample_Table you will have changes rows and which column in your table.
So what you can do is. Keep your lookup logic in SSIS (if you want something in SSIS) - Updated the Table based on matches in lookup
When these updates happend your trigger will be fired and update the rows that have changed.
Alternative you can create your lookup in a t-sql script and do an ordinary update when custid=custid instead its just as easy. But thats up to you.
EDITED
-- -------------------- Setup tables and some initial data --------------------
CREATE TABLE dbo.Sample_Table (ContactID int, Forename varchar(100), Surname varchar(100), Extn varchar(16), Email varchar(100), Age int );
INSERT INTO Sample_Table VALUES (1,'Bob','Smith','2295','bs#example.com',24);
INSERT INTO Sample_Table VALUES (2,'Alice','Brown','2255','ab#example.com',32);
INSERT INTO Sample_Table VALUES (3,'Reg','Jones','2280','rj#example.com',19);
INSERT INTO Sample_Table VALUES (4,'Mary','Doe','2216','md#example.com',28);
INSERT INTO Sample_Table VALUES (5,'Peter','Nash','2214','pn#example.com',25);
CREATE TABLE dbo.Sample_Table_Changes (ContactID int, FieldName sysname, FieldValueWas sql_variant, FieldValueIs sql_variant, modified datetime default (GETDATE()));
GO
-- -------------------- Create trigger --------------------
CREATE TRIGGER TriggerName ON dbo.Sample_Table FOR DELETE, INSERT, UPDATE AS
BEGIN
SET NOCOUNT ON;
--Unpivot deleted
WITH deleted_unpvt AS (
SELECT ContactID, FieldName, FieldValue
FROM
(SELECT ContactID
, cast(Forename as sql_variant) Forename
, cast(Surname as sql_variant) Surname
, cast(Extn as sql_variant) Extn
, cast(Email as sql_variant) Email
, cast(Age as sql_variant) Age
FROM deleted) p
UNPIVOT
(FieldValue FOR FieldName IN
(Forename, Surname, Extn, Email, Age)
) AS deleted_unpvt
),
--Unpivot inserted
inserted_unpvt AS (
SELECT ContactID, FieldName, FieldValue
FROM
(SELECT ContactID
, cast(Forename as sql_variant) Forename
, cast(Surname as sql_variant) Surname
, cast(Extn as sql_variant) Extn
, cast(Email as sql_variant) Email
, cast(Age as sql_variant) Age
FROM inserted) p
UNPIVOT
(FieldValue FOR FieldName IN
(Forename, Surname, Extn, Email, Age)
) AS inserted_unpvt
)
--Join them together and show what's changed
INSERT INTO Sample_Table_Changes (ContactID, FieldName, FieldValueWas, FieldValueIs)
SELECT Coalesce (D.ContactID, I.ContactID) ContactID
, Coalesce (D.FieldName, I.FieldName) FieldName
, D.FieldValue as FieldValueWas
, I.FieldValue AS FieldValueIs
FROM
deleted_unpvt d
FULL OUTER JOIN
inserted_unpvt i
on D.ContactID = I.ContactID
AND D.FieldName = I.FieldName
WHERE
D.FieldValue <> I.FieldValue --Changes
OR (D.FieldValue IS NOT NULL AND I.FieldValue IS NULL) -- Deletions
OR (D.FieldValue IS NULL AND I.FieldValue IS NOT NULL) -- Insertions
END
GO
-- -------------------- Try some changes --------------------
UPDATE Sample_Table SET age = age+1;
/*UPDATE Sample_Table SET Extn = '5'+Extn where Extn Like '221_';
DELETE FROM Sample_Table WHERE ContactID = 3;
INSERT INTO Sample_Table VALUES (6,'Stephen','Turner','2299','st#example.com',25);
UPDATE Sample_Table SET ContactID = 7 where ContactID = 4; --this will be shown as a delete and an insert
-- -------------------- See the results --------------------
SELECT *, SQL_VARIANT_PROPERTY(FieldValueWas, 'BaseType') FieldBaseType, SQL_VARIANT_PROPERTY(FieldValueWas, 'MaxLength') FieldMaxLength from Sample_Table_Changes;
-- -------------------- Cleanup --------------------
DROP TABLE dbo.Sample_Table; DROP TABLE dbo.Sample_Table_Changes;*/
select * from dbo.sample_table_changes
I have two tables: Table1 is all the companies, Table2 is companies whose name start with A.
Table1 company (companyId int, companyName varchar(50), companySize int)
Table2 companyStartWithA (companyId int, companyName varchar(50), companySize int)
What I want to do is to create a trigger so that when I insert/update/delete something in Table1, it will automatically do the same in Table2
My code:
CREATE TRIGGER A_TRG_InsertSyncEmp
ON company
AFTER INSERT
AS
BEGIN
INSERT INTO companyStartWithA
SELECT *
FROM INSERTED
WHERE inserted.companyName LIKE 'A%'
END
And I get an error:
An explicit value for the identity column in table 'companyStartWithA' can only be specified when a column list is used and IDENTITY_INSERT is ON.
What can I do?
Thanks
The problem is the fact that you're not explicitly specifying the column in the INSERT statement, and using a SELECT * to fill the data. Both are big no-no's - you should always explicitly specify the column that you want to insert into, and you should always explicitly specify the columns that you want to select. Doing so will fix this problem:
CREATE TRIGGER A_TRG_InsertSyncEmp
ON company
AFTER INSERT
AS
BEGIN
INSERT INTO companyStartWithA (companyName, companySize)
SELECT companyName, companySize
FROM INSERTED
WHERE inserted.companyName LIKE 'A%'
END
But as Sean Lange absolutely correctly commented - this should really be just a view rather than a separate table.....
CREATE VIEW dbo.CompanyStartsWithA
AS
SELECT companyId, companyName, companySize
FROM dbo.Company
WHERE Name LIKE 'A%'
and then you don't need any messy triggers or anything - just insert into dbo.Company and all companies with a name that starts with an A will be visible in this view....
I currently have a table like this:
Stuff
----------
StuffId identity int not null
Description nvarchar(4000) null
...
I want to store the Description in a separate table that I have set aside specifically for user-generated content:
Content
----------
ContentId identity int not null
Content nvarchar(max) not null
...
(this table already exists, and other tables already reference entries in it.)
So I need to:
Create a DescriptionContentId field on the Stuff table with a foreign key constraint.
Copy the current Description content into the Content table.
Set each DescriptionContentId to have the ContentId value that was automatically generated when inserting values in step 2.
Drop the Description column.
I know how to do steps 1 and 4, but steps 2 and 3 are eluding me, because they need to be done pretty much simultaneously. This seems like it would be a fairly common schema change. What's the best way to do it?
Update
I'm a step closer thanks to the Output keyword, but I'm still missing something. Here's what I'd like to do:
create table #tmp (StuffId int, ContentId int)
insert into Content(Content)
output s.StuffId, inserted.ContentId
into #tmp(StuffId, ContentId)
select Description
from Stuff s
where Description IS NOT NULL
But I can't reference s.StuffId because it isn't one of the fields inserted into the Content table. How can I correlate the ID of the Stuff with the ID of the Content as I'm inserting a new Content item for each Stuff entry?
The output clause will come to your rescue.
It will output the description and the identity column from the insert into a table varaible and then you can use that data to update the other table.
If description is not unique, you may have to do the following:
add a column for the stuffID column to the content table. Then output the stuffid and content id from the insert, update the table using the stuffid to ensure uniqueness, Drop the stuffid column from the content table.
an example from Books Online as to how to use the OUTPUT
DECLARE #MyTableVar table(
LastName nvarchar(20) NOT NULL,
FirstName nvarchar(20) NOT NULL,
CurrentSales money NOT NULL
);
INSERT INTO dbo.EmployeeSales (LastName, FirstName, CurrentSales)
OUTPUT INSERTED.LastName,
INSERTED.FirstName,
INSERTED.CurrentSales
INTO #MyTableVar
SELECT c.LastName, c.FirstName, sp.SalesYTD
FROM HumanResources.Employee AS e
INNER JOIN Sales.SalesPerson AS sp
ON e.EmployeeID = sp.SalesPersonID
INNER JOIN Person.Contact AS c
ON e.ContactID = c.ContactID
WHERE e.EmployeeID LIKE '2%'
ORDER BY c.LastName, c.FirstName;