The results of a union select statement in SQL Server 2008 is including records that are not found in either source table. Example:
theid is an integer ID value within each source table. Neither table includes theid value of 277741.
Select *
From DataTable1
WHere theid = 277741
-- 0 records returned
Select *
From DataTable2
Where theid = 277741
-- 0 records returned
However, when you run the Union Select statement below, a record is generated for theid.
Select *
Into ConjoinedData
From DataTable1
Union
Select *
From DataTable2
Where theid Not In (Select theid From DataTable1)
Select
From ConjoinedData
Where theid = 27741
-- 1 record returned
The theid field is not an identity field in either source table. Ultimately, the data for DataTable1 and DataTable2 came from the same parent, whose content includes an unrelated record with ID 277741. However, there are no foreign key relationships to it or any other table on either of the source tables. I have tried changing to a Union All query with no success. I have created an index on theid in both source tables and the 'created' record appears. I have dropped and recreated the source tables numerous times to no avail. Why is SQL Server getting the unrelated record from the disconnected parent table (source talbes were both cereated from the same parent using Select..Into statements and no foreign key relationship to either table back to the parent)?
277741 is different from 27741. The following script reproduces exactly what you describe:
create table Table1 (id int);
create table Table2 (id int);
go
insert into Table1(id) values (277742);
insert into Table2(id) values (27741);
go
Select * From Table1 WHere id = 277741;
Select * From Table2 Where id = 277741;
go
Select *
Into ConjoinedData
From Table1
Union
Select *
From Table2
Where id Not In (Select id From Table1);
Select *
From ConjoinedData
Where id = 27741;
Related
I'm trying to insert a new row into a table which is an exact copy of another row except for the identifier. Previously I hadn't had the issue because there was an ID-column which didn't fill automatically. So I could just copy a row like this
INSERT INTO table1 SELECT * FROM table1 WHERE Id = 5
And then manually change the ID like this
WITH tbl AS (SELECT ROW_NUMBER() OVER (PARTITION BY Id ORDER BY Id) AS RNr, Id
FROM table1 WHERE Id = 5
UPDATE tbl SET Id = (SELECT MAX(Id) FROM table1) + 1
WHERE RNr = 2
Recently the column Id has changed to not be filled manually (which I also like better). But this leads to the error that I obviously can't fill that column while IDENTITY_INSERT is false. Unfortunately, I don't have the right to use SET IDENTITY_INSERT IdentityTable ON/OFF before and after the statement, which I also would like to avoid anyways.
I'm looking for a way to insert the whole row excluding the Id column without naming each other column in the INSERT and SELECT statement (which are quite a lot).
In the code below the maximum value of the ID gets added by one, so your integrity is not violated.
insert into table1
select a.Top_ID, Column2, Column3, ColumnX
from table1 t1
outer apply (select max(id_column) + 1as Top_ID from table1) as a
where t1.Id = 1
Okay, I found a way by using a temporary table
SELECT * INTO #TmpTbl
FROM table1 WHERE Id = 5;
ALTER TABLE #TmpTbl
DROP COLUMN Id;
INSERT INTO table1 SELECT * FROM #TmpTbl;
DROP TABLE #TmpTbl
I'm now using the sqldiff tool to calculate the difference between my database before a transaction was executed and after it was executed. I'm doing this as follows:
Copy the entire sqlite file to a tmp location (old-database.sqlite)
Perform the transaction (for ex. INSERT, SCHEMA CHANGE, ...)
Than sqldiff --primary-key old-database.sqlite database.sqlite
This outputs the changes that the transaction has done. Although this works perfectly because my database isn't that large, a few megabytes maximum. I think this could be done more efficiently.
Is there any mechanism that I can use to get the same output without copying the database. Maybe something with the journal?
Kind regards,
Daan
Perhaps a little awkward to implement and perhaps at some stage (size wise) inefficient, but you could use a combination of TRIGGERS (4 per monitored table is the awkward part) and a before after snapshot of sqlite_master.
That is you could, per table to be monitored, have a copy of that table (structure wise) appended with two columns. 1 to indicate the action (inserted, before update, after update or deletion), the 2nd (optional) to record the timestamp.
Then for each table to be monitored have AFTER INSERT, BEFORE DELETE and AFTER and BEFORE UPDATE TRIGGERS to copy the add an entry to the table's log table, which would be emptied before the transaction (if wanted, if not then timestamp would probably be a required column).
For schema changes again you could have a copy of sqlite_master before and then compare it against sqlite_master after the transaction (you can't have TRIGGERS applied to system tables).
Example, consider the following example/demo:-
CREATE TABLE IF NOT EXISTS mytable (id INTEGER PRIMARY KEY, mycolumn TEXT);
CREATE TABLE IF NOT EXISTS mytablelog AS SELECT * FROM mytable WHERE 0 = 1;
ALTER TABLE mytablelog ADD COLUMN logtype INTEGER;
ALTER TABLE mytablelog ADD COLUMN timestamp INTEGER TEXT;
CREATE TABLE IF NOT EXISTS schema_log AS SELECT * FROM sqlite_master WHERE 0=1;
CREATE TABLE IF NOT EXISTS pretrans_schema AS SELECT * FROM sqlite_master WHERE 0=1;
ALTER TABLE schema_log ADD COLUMN logtype INTEGER;
ALTER TABLE schema_log ADD COLUMN timestamp INTEGER TEXT;
CREATE TRIGGER IF NOT EXISTS mytable_inserts AFTER INSERT ON mytable
BEGIN
INSERT INTO mytablelog SELECT *,0,strftime('%s','now') FROM mytable WHERE id = new.id;
END
;
CREATE TRIGGER IF NOT EXISTS mytable_deletions BEFORE DELETE ON mytable
BEGIN
INSERT INTO mytablelog SELECT *,1,strftime('%s','now') FROM mytable WHERE id = old.id;
END
;
CREATE TRIGGER IF NOT EXISTS mytable_preupdates BEFORE UPDATE ON mytable
BEGIN
INSERT INTO mytablelog SELECT *,2,strftime('%s','now') FROM mytable WHERE id = old.id;
END
;
CREATE TRIGGER IF NOT EXISTS mytable_postupdates AFTER UPDATE ON mytable
BEGIN
INSERT INTO mytablelog SELECT *,3,strftime('%s','now') FROM mytable WHERE id = new.id;
END
;
-- SELECT * FROM sqlite_master WHERE name LIKE 'sqlite_%';
/* BEFORE TRANSACTION PREPATION */
DELETE FROM mytablelog;
DELETE FROM schema_log;
DELETE FROM pretrans_schema;
INSERT INTO pretrans_schema SELECT * FROM sqlite_master;
/* DO SOMETHING AKA THE TRANSACTIONS */
CREATE TABLE IF NOT EXISTS newtable (id INTEGER PRIMARY KEY, acolumn TEXT);
INSERT INTO mytable (mycolumn) VALUES ('Mary')
-- ,('Fred'),('Jane'),('Anne'),('Alfred'),('George'),('Alan')
,('Susan'),('Betty'),('Catherine'),('John')
,(100),(200)
;
UPDATE mytable SET mycolumn = mycolumn||' has the detected letter.' WHERE mycolumn LIKE '%n%';
DELETE FROM mytable WHERE CAST(mycolumn AS INTEGER) > 0;
/* AFTER TRANSACTION */
SELECT rowid,'sm',* FROM sqlite_master UNION ALL SELECT rowid,'pt',* FROM pretrans_schema ORDER BY type,name; /* FOR DEMO/TESTING */
/* Get items added to the schema */
INSERT INTO schema_log SELECT *,4,strftime('%s','now') FROM sqlite_master WHERE name NOT IN (SELECT name FROM pretrans_schema);
/* Get items deleted from the schema */
INSERT INTO schema_log SELECT *,5,strftime('%s','now') FROM pretrans_schema WHERE name NOT IN (SELECT name FROM sqlite_master);
/* get original schema if schema updates */
INSERT INTO schema_log SELECT *,6,strftime('%s','now') FROM sqlite_master AS s
WHERE (sql <> (SELECT sql FROM pretrans_schema WHERE type = s.type AND name = s.name )) AND type IS NOT NULL /* AND NAME <> 'pretrans_schema' optional */
;
/* get current schema if schema updates */
INSERT INTO schema_log SELECT *,7,strftime('%s','now') FROM pretrans_schema AS s
WHERE (sql <> (SELECT sql FROM sqlite_master WHERE type = s.type AND name = s.name)) AND type IS NOT NULL /* AND NAME <> 'pretrans_schema' */
;
SELECT * FROM schema_log;
SELECT * FROM mytablelog;
Results
1 pre and post schema (part of)
Note the highlighted line, there is no pt entry, showing that the table has been added (no sm entry then it would have been deleted).
2 - The schema changes log (schema_log table)
4 indicates new item in the schema (5 deleted, 6 before update, 7 after update)
So the table newtable has been added as part of the transaction.
3 - The mytable transactions;
O's are insertions, 1's are deleted 2 and 3 (paired) pre and post update respectively.
Thank you for thinking along and especially #MikeT for the long and thourough answer. I have tried your answer and while it works it seemed not elegant to me and have a lot of overhead.
After a lot of search I have found that the solution is baked into SQLite but it must be enabled at compile time. It is called 'The Session Extension' and is exacly what I needed:
https://www.sqlite.org/sessionintro.html
I have a linked server that is created to pull user details from a specific Organisation Unit with a scheduled sql job agent.
The table is created to hold user details has a column for ObjectGUID number and the type is defined as varbinary(50) (I am not sure why..).
The process checks if there is a new user by comparing the ObjectGUID number the saved Users table and if there is a new number then insert the new user in the table.
However I have noticed that the comparisons actually not really working properly.
SELECT
tbl.objectGUID AS UserGUID
FROM [dbo].[ActiveDirectoryUsers] tbl
WHERE tbl.objectGUID NOT IN (SELECT UserGUID FROM dbo.Users)
When I create a new user the new user is appearing in the ActiveDirectoryUsers view.
but when the where clause added to compare results with Users table then result is always empty. It looks like I need to cast or convert the varbinary to varchar then do the comparisons. I tried to cast the varbinary into varchar and uniqueidentifier but still it does not work.
Any idea how would I do the comparisons?
Update
CREATE VIEW [dbo].[ActiveDirectoryUsers] AS
SELECT "SAMAccountName" AS sAMAccountName, "mail" AS Email,
"objectGUID" AS objectGUID
FROM OpenQuery(ADSI, 'SELECT SAMAccountName, mail, objectGUID
FROM ''ldapconnectionstring.com''')
An example of objectGUID in the Users table
0x1DBCC071C69C8242B4895D42750969B1
You should not cast varbinary to smth particular to be able to use it in WHERE clause.
Your problem is that you use NOT IN where NULL values are present.
Try to execute my code first as it is (it will return 1 row) and then uncomment NULL value insert and execute it again.
This time you'll get 0 rows:
declare #t1 table (guid varbinary(50))
insert into #t1
values(0x1DBCC071C69C8242B4895D42750969B1)--, (null);
declare #t2 table (guid varbinary(50))
insert into #t2
values(0x1DBCC071C69C8242B4895D42750969B1), (0x1DBCC071C69C8242B4895D42750969B2);
select *
from #t2 t2
where t2.guid not in (select guid from #t1);
To fix your problem, try to use NOT EXISTS instead of NOT IN like this:
select *
from #t2 t2
where not exists (select *
from #t1 t1
where t1.guid = t2.guid);
In your case the code should be like this:
SELECT tbl.objectGUID AS UserGUID
FROM [dbo].[ActiveDirectoryUsers] tbl
WHERE not exists (SELECT *
FROM dbo.Users u
where u.UserGUID = tbl.objectGUID );
I have a SQL table TABLE1 which has columns ID and LastModifiedDate.Now I have one oracle query SELECT * From Table where NEWID =? where parameter will be the value from ID column from TABLE1 and I need to insert these records into a destination table on a SQL Server.Please advise the best approach for this.I am using SQL 2008.
Where #Id will be parameter
INSERT INTO DESTINATIONTABLE
SELECT *
FROM TABLE
WHERE NEWID IN(SELECT ID From Table1 WHERE ID=#ID)
or if you want to specify the columns you could use
INSERT INTO DESTINATIONTABLE(Col1,Col2)
SELECT Col1,Col2
FROM TABLE
WHERE NEWID IN(SELECT ID From Table1 WHERE ID=#ID)
I've tried searching for the answer to this one to no avail. There is no good logic behind the way this was setup. The guy does not know what he's doing, but it's what I have to work with (long story).
I'm using SQL Server 2008R2 I need to take records from one table and transfer the data to 4 separate tables all with a one to one relationship (I know - not smart). I need to get the value from the Identity field in the first table the data is inserted into, then populate the other 3 tables with the same ID and disperse the data accordingly. for example:
OldTable: Field1, Field2, Field3, Field4
NewTable1: Identity field, Field1
NewTable2: ID, Field2
NewTable3: ID, Field3
NewTable4: ID, Field4
I'd like to handle this in a stored procedure. I'd like to do a loop, but I read that loops in SQL are inadvisable.
Loop moving through each record in OldTable... (??)
INSERT INTO NewTable1
(Field1)
Select Field1 from OldTable
INSERT INTO NewTable2
(ID, Field2)
Select SCOPE_IDENTITY?, Field2 From OldTable Where OldTable.ID = ??
etc for other 2 tables
Loop to next record in OldTable
I am not sure how to use SCOPE_IDENTITY, but I have a feeling this will be involved in how I accomplish this.
Also, I'm probably going to need to setup a trigger for whenever a new record is created in NewTable1. I know, it's insanity, but I can't do anything about it, just have to work around it.
So, I need to know
1: the best way to initially populate the tables
2: how to make triggers for new records
The solution to 1 might involve 2.
Please help!
You can use the output clause of the merge statement to get a mapping between the existing primary key in OldTable and the newly generated identity ID in NewTable1.
-- Temp table to hold the mapping between OldID and ID
create table #ID
(
OldID int primary key,
ID int
);
-- Add rows to NewTable1 and capture the ID's in #ID
merge NewTable1 as T
using OldTable as S
on 1 = 0
when not matched by target then
insert(Field1) values(S.Field1)
output S.ID, inserted.ID into #ID(OldID, ID);
-- Add rows to NewTable2 using #ID to get the correct value for each row
insert into NewTable2(ID, Field2)
select I.ID, O.Field2
from #ID as I
inner join OldTable as O
on I.OldID = O.ID
insert into NewTable3(ID, Field3)
select I.ID, O.Field3
from #ID as I
inner join OldTable as O
on I.OldID = O.ID
insert into NewTable4(ID, Field4)
select I.ID, O.Field4
from #ID as I
inner join OldTable as O
on I.OldID = O.ID
drop table #ID;
SQL Fiddle
See also Using merge..output to get mapping between source.id and target.id
How about using the OUTPUT clause of the insert statement? Assuming that Field1 is a unique key on the OldTable...
Declare #IDinserted table(ID int, Field1 varchar(255));
Insert Into NewTable1(Field1)
Output inserted.ID, inserted.Field1 into #IDinserted
Select OldID, Field1 from OldTable;
Insert Into NewTable2(RowID, Field2)
Select i.ID, o.#Field2
from #IDinserted i Inner Join OldTable o
on i.Field1=o.Field1;