I Have a table below and I will create a new column say named 'Amount'. The existing column 'Id' is foreign key and link the information in a table say 'Loan'('Id' is the primary key of table 'Loan' ). The simple thing I wanna do is assign each new created Amount cell with the right amount obtain from table 'Loan', mapping with 'Id'. I currently use local variable and self-created table type to find the amount value in 'Loan' case by case. Is there any other more efficient way to execute the same operation? Many thanks.
MyTable was as below:
My code was as followed:
ALTER TABLE MyTable
ADD Amount MONEY
CREATE TYPE ListofID AS TABLE (idx INT IDENTITY(1,1), ID VARCHAR(255))
DECLARE #Table_ID_List ListofID
INSERT #Table_ID_List (
ID )
SELECT Id FROM MyTable
DECLARE #i INT
DECLARE #cnt INT
SELECT #i = min(idx) - 1, #cnt = max(idx) FROM #Table_ID_List
DECLARE #app VARCHAR(255)
WHILE #i<#cnt
BEGIN
SELECT #i = #i + 1
SELECT #app = (SELECT ID FROM #Table_ID_List WHERE idx = #i)
UPDATE MyTable
SET Amount =(SELECT Amount FROM Loan WHERE Id = #app)
WHERE Id = #app
END
Depending on the size of your table, you may want to use the SWITCH statement to transfer the whole table at once to the new schema.
https://sqlperformance.com/2012/08/t-sql-queries/t-sql-tuesday-schema-switch-a-roo
I may have missed your question, but I believe you can do this directly via an Update statement. Something along the lines of:
ALTER TABLE MyTable ADD Amount MONEY
UPDATE
t1
SET
t1.Amount = t2.Amount
FROM
MyTable t1
JOIN Loan t2
ON t1.Id = t2.Id
Wouldn't that do the trick?
Related
I have a list of about 200,000 records with EntityID column which I load into a temp table variable.
I want to insert any records from the Temp table variable if EntityID from the Temp table does not exist in the dbo.EntityRows table. The dbo.EntityRows table contains about 800,000 records.
The process is very slow compared to when the dbo.EntityRows table had about 500,000 records.
My first guess is because of the NOT EXISTS clause, each row from the Temp variable must scan the entire 800k rows of the dbo.EntityRows table to determine if it exists or not.
QUESTION: Are there alternative ways to run this comparison check without using the NOT EXISTS, which incurs a hefty cost and will only get worse as dbo.EntityRows continues to grow?
EDIT: Appreciate the comments. Here is the query (I left out the part after the IF NOT EXISTS check. After that, if NOT EXISTS, I insert into 4 tables).
declare #EntityCount int, #Counter int, #ExistsCounter int, #AddedCounter int
declare #LogID int
declare #YdataInsertedEntityID int, #YdataSearchParametersID int
declare #CurrentEntityID int
declare #CurrentName nvarchar(80)
declare #CurrentSearchParametersID int, #CurrentSearchParametersIDAlreadyDone int
declare #Entities table
(
Id int identity,
EntityID int,
NameID nvarchar(80),
SearchParametersID int
)
insert into #Entities
select EntityID, NameID, SearchParametersID from YdataArvixe.dbo.Entity order by entityid;
set #EntityCount = (select count(*) from #Entities);
set #Counter = 1;
set #LogID = null;
set #ExistsCounter = 0;
set #AddedCounter = 0;
set #CurrentSearchParametersIDAlreadyDone = -1;
While (#EntityCount >= #Counter)
begin
set #CurrentEntityID = (select EntityID from #Entities
where id = #Counter)
set #CurrentName = (select nameid from #Entities
where id = #Counter);
set #CurrentSearchParametersID = (select SearchParametersID from #Entities
where id = #Counter)
if not exists (select 1 from ydata.dbo.entity
where NameID = #CurrentName)
begin
-- I insert into 4 tables IF NOT EXISTS = true
end
I am not sure but there are following ways how we can check
(SELECT COUNT(er.EntityID) FROM dbo.EntityRows er WHERE er.EntityID = EntityID) <> 0
(SELECT er.EntityID FROM dbo.EntityRows er WHERE er.EntityID = EntityID) IS NOT NULL
EntityID NOT EXISTS (SELECT er.EntityID FROM dbo.EntityRows er)
EntityID NOT IN (SELECT er.EntityID FROM dbo.EntityRows er)
But as per my belief getting count will give good performance.
Also index will help to improve performance as 'Felix Pamittan' said
As #gotqn said, start by using a temporary table. Create an index on EntityID after the table is filled. If you don't have an index on EntityID in EntityRows, create one.
I do things like this a lot, and I generally use the following pattern:
INSERT INTO EntityRows (
EntityId, ...
)
SELECT T.EntityId, ...
FROM #tempTable T
LEFT JOIN EntityRows E
ON T.EntityID = E.EntityID
WHERE E.EntityID IS NULL
Please comment if you'd like further info.
Well, the answer was pretty basic. #Felix and #TT had the right suggestion. Thanks!
I put a non-clustered index on the NameID field in ydata.dbo.entity.
if not exists (select 1 from ydata.dbo.entity
where NameID = #CurrentName)
So it can now process the NOT EXISTS part quickly using the index instead of scanning the entire dbo.entity table. It is moving fast again.
Create table data with column StudentId (varchar type), Marks (Double). Create table data1 with column StudentId (varchar type), OldMarks (Double),NewMarks,Date.
Create trigger on data table.If mark is changed,create entry in data1 table for student with old marks,new marks & current date.
Here is the code I've tried:
CREATE TRIGGER marksss ON [dbo].[data] after UPDATE
AS declare #studentid int;
declare #marks int;
declare #xyz int;
declare #newmarks int;
declare #oldmarks int;
select #studentid=i.student_id from inserted i;
--to fetch inserted values
select #marks=i.marks from inserted i;
begin if update(marks) --set #oldmarks=#mark set #newmarks=#marks
insert into data1(student_id,new_marks,old_marks,date)
values (#studentid,#newmarks,#oldmarks,getdate()enter code here);
end
go
the problem is that it does not display old marks
I've managed to get what you want. First of all, you want to use an instead of trigger instead. Oracle has a before trigger which is what you ideally need however MSSQL doesn't have this feature so we have to do the passed in update manually too...
Here is the code with the table setup that I used, just changed to suit your needs.
CREATE TABLE A (ID INT IDENTITY PRIMARY KEY, SCORE INT)
CREATE TABLE B (ID INT FOREIGN KEY REFERENCES A(ID), SCORE INT, OLDSCORE INT, [date] DATETIME)
GO
CREATE TRIGGER marksss ON A INSTEAD OF UPDATE
AS
BEGIN
IF (SELECT A.SCORE FROM A INNER JOIN INSERTED I ON I.ID = A.ID) != (SELECT I.SCORE FROM INSERTED I)
BEGIN
INSERT INTO B(ID,SCORE,OLDSCORE,[date])
SELECT I.ID, I.SCORE, A.SCORE, GETDATE()
FROM INSERTED I
INNER JOIN A ON I.ID = A.ID
END
BEGIN
UPDATE A
SET SCORE = (SELECT I.SCORE FROM INSERTED I)
END
END
I have the following table structure:
create table table1( ID int,
assettype varchar(50));
create table t1( poolId int,
day_rate float);
create table t2( poolId int,
day_rate float);
create table lookuptable( tablename varchar(50),
assettype varchar(50));
insert into table1 values (1,'abs'), (2,'card');
insert into t1 values ( 1,5), ( 2,10);
insert into t2 values ( 1,15), ( 2,20);
insert into lookuptable values ('t1','abs'), ('t2','card');
SqlFiddle
For a given id based on the assetType field in table1 I need to perform a lookup in the lookup table so that I display if the assettype of the id is abs
poolId day_rate
1 5
2 10
else if the assettype of the id is card
poolId day_rate
1 15
2 20
The reason I have t1 and t2 tables because they have their own set of calculation and based on the asset type of the id i want to use t1 and t2
Would you be able to guide me with some query or steps to go in the right direction
I can think of the case when structure to this but in my case I have 100 entries in the lookuptable and that would mean a case when structure written for 100 times. Is there a better way of handling this?
Try this..
declare #table varchar(20)
select #table=tablename from lookuptable where assettype = 'card'
print #table
declare #query nvarchar(1000)
set #query = N'select * from [' + #table +']';
print #query
EXECUTE sp_executesql #query
Change the assettype in first query..
Are you just trying to get all the rows in table1 and then their day_rates depending on what type of asset they are? Couldn't you do something like this.
select
,table1.ID
,table1.assettype,
,(case when table1.assettype = 'abs' then t1.day_rate else t2.day_rate end) as
day_rate
from table1
inner join t1 on table1.id = t1.poolId
inner join t2 on table1.id = t2.poolId
group by table1.ID, table1.assettype
Let me identify the perspective first.
First, you created this table1 as? well whatever, but this seems a lookup table as well to me.
create table table1(
ID int,
assettype varchar(50)
);
Next, this one is a lookup table as well because this one is your reference for calculation.
create table t1(
poolId int,
day_rate float
);
create table t2(
poolId int,
day_rate float
);
Lastly, is your direct lookup table depending on the assettype defines which table you will get the rate.
create table lookuptable(
tablename varchar(50),
assettype varchar(50)
);
So, i'll try to simplify this.
One is your table1 with an additional column
create table table1(
ID int,
assettype varchar(50),
day_rate float
);
insert into table1 values
(1,'abs',5)
,(2,'card',10)
I have no further reasons yet to split the table as One-To-Many table relationship as I don't see the lookuptable as "child" as well as t1 and t2. You could instead directly change the rates in table1 rate as desired.
Please put some comments so we can arrive at the same page. thanks
I have a table that stores two values; 'total' and 'owing' for each customer. Data is uploaded to the table using two files, one that brings in 'total' and the other brings in 'owing'. This means I have two records for each customerID:
customerID:--------Total:--------- Owing:
1234---------------- 1000----------NULL
1234-----------------NULL-----------200
I want to write a stored procedure that merges the two records together:
customerID:--------Total:--------- Owing:
1234---------------- 1000----------200
I have seen examples using COALESCE so put together something like this:
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
--Variable declarations
DECLARE #customer_id varchar(20)
DECLARE #total decimal(15,8)
DECLARE #owing decimal(15,8)
DECLARE #customer_name_date varchar(255)
DECLARE #organisation varchar(4)
DECLARE #country_code varchar(2)
DECLARE #created_date datetime
--Other Variables
DECLARE #totals_staging_id int
--Get the id of the first row in the staging table
SELECT #totals_staging_id = MIN(totals_staging_id)
from TOTALS_STAGING
--iterate through the staging table
WHILE #totals_staging_id is not null
BEGIN
update TOTALS_STAGING
SET
total = coalesce(#total, total),
owing = coalesce(#owing, owing)
where totals_staging_id = #totals_staging_id
END
END
Any Ideas?
SELECT t1.customerId, t1.total, t2.owing FROM test t1 JOIN test t2 ON ( t1.customerId = t2.customerId) WHERE t1.total IS NOT NULL AND t2.owing IS NOT NULL
Wondering why aren't you just using UPDATE on a second file execution?
Except for COUNT, aggregate functions ignore null values. Aggregate
functions are frequently used with the GROUP BY clause of the SELECT
statement. MSDN
So you don't need to worry about null values with summing. Following will give your merging records together. Fiddle-demo
select customerId,
sum(Total) Total,
sum(Owing) Owing
from T
Group by customerId
Try this :
CREATE TABLE #Temp
(
CustomerId int,
Total int,
Owing int
)
insert into #Temp
values (1024,100,null),(1024,null,200),(1025,10,null)
Create Table #Final
(
CustomerId int,
Total int,
Owing int
)
insert into #Final
values (1025,100,50)
MERGE #Final AS F
USING
(SELECT customerid,sum(Total) Total,sum(owing) owing FROM #Temp
group by #Temp.customerid
) AS a
ON (F.customerid = a.customerid)
WHEN MATCHED THEN UPDATE SET F.Total = F.Total + isnull(a.Total,0)
,F.Owing = F.Owing + isnull(a.Owing,0)
WHEN NOT MATCHED THEN
INSERT (CustomerId,Total,Owing)
VALUES (a.customerid,a.Total,a.owing);
select * from #Final
drop table #Temp
drop table #Final
This should work:
SELECT CustomerID,
COALESCE(total1, total2) AS Total,
COALESCE(owing1, owing2) AS Owing
FROM
(SELECT row1.CustomerID AS CustomerID,
row1.Total AS total1,
row2.Total AS total2,
row1.Owing AS owing1,
row2.Owing AS owing2
FROM YourTable row1 INNER JOIN YourTable row2 ON row1.CustomerID = row2.CustomerID
WHERE row1.Total IS NULL AND row2.Total IS NOT NULL) temp
--Note: Alter the WHERE clause as necessary to ensure row1 and row2 are unique.
...but note that you'll need some mechanism to ensure row1 and row2 are unique. My WHERE clause is an example based on the data you provided. You'll need to tweak this to add something more specific to your business rules.
I am using SQL Server 2005. I have heard that we can use a table variable to use instead of LEFT OUTER JOIN.
What I understand is that, we have to put all the values from the left table to the table variable, first. Then we have to UPDATE the table variable with the right table values. Then select from the table variable.
Has anyone come across this kind of approach? Could you please suggest a real time example (with query)?
I have not written any query for this. My question is - if someone has used a similar approach, I would like to know the scenario and how it is handled. I understand that in some cases it may be slower than the LEFT OUTER JOIN.
Please assume that we are dealing with tables that have less than 5000 records.
Thanks
It can be done, but I have no idea why you would ever want to do it.
This realy does seem like it is being done backwards. But if you are trying this for your own learning only, here goes:
DECLARE #MainTable TABLE(
ID INT,
Val FLOAT
)
INSERT INTO #MainTable SELECT 1, 1
INSERT INTO #MainTable SELECT 2, 2
INSERT INTO #MainTable SELECT 3, 3
INSERT INTO #MainTable SELECT 4, 4
DECLARE #LeftTable TABLE(
ID INT,
MainID INT,
Val FLOAT
)
INSERT INTO #LeftTable SELECT 1, 1, 11
INSERT INTO #LeftTable SELECT 3, 3, 33
SELECT *,
mt.Val + ISNULL(lt.Val, 0)
FROM #MainTable mt LEFT JOIN
#LeftTable lt ON mt.ID = lt.MainID
DECLARE #Table TABLE(
ID INT,
Val FLOAT
)
INSERT INTO #Table
SELECT ID,
Val
FROM #MainTable
UPDATE #Table
SET Val = t.Val + lt.Val
FROM #Table t INNER JOIN
#LeftTable lt ON t.ID = lt.ID
SELECT *
FROM #Table
I don't think it's very clear from your question what you want to achieve? (What your tables look like, and what result you want). But you can certainly select data into a variable of a table datatype, and tamper with it. It's quite convenient:
DECLARE #tbl TABLE (id INT IDENTITY(1,1), userId int, foreignId int)
INSERT INTO #tbl (userId)
SELECT id FROM users
WHERE name LIKE 'a%'
UPDATE #tbl t
SET
foreignId = (SELECT id FROM foreignTable f WHERE f.userId = t.userId)
In that example I gave the table variable an identity column of its own, distinct from the one in the source table. I often find that useful. Adjust as you like... Again, it's not very clear what the question is, but I hope this might guide you in the right direction...?
Every scenario is different, and without full details on a specific case it's difficult to say whether it would be a good approach for you.
Having said that, I would not be looking to use the table variable approach unless I had a specific functional reason to - if the query can be fulfilled with a standard SELECT query using an OUTER JOIN, then I'd use that as I'd expect that to be most efficient.
The times where you may want to use a temp table/table variable instead, are more when you want to get an intermediary resultset and then do some processing on it before then returning it out - i.e. the kind of processing that cannot be done with a straight forward query.
Note the table variables are very handy, but take into account that they are not guaranteed to reside in-memory - they can get persisted to tempdb like standard temp tables.
Thank you, astander.
I tried with an example given below. Both of the approaches took 19 seconds. However, I guess some tuning will help the Table varaible update approach to become faster than LEFT JOIN.
AS I am not a master in tuning I request your help. Any SQL expert ready to prove it?
---- PLease replace "" with '' below. I am not familiar with how to put code in this forum... It causes some troubles....
CREATE TABLE #MainTable (
CustomerID INT PRIMARY KEY,
FirstName VARCHAR(100)
)
DECLARE #Count INT
SET #Count = 0
DECLARE #Iterator INT
SET #Iterator = 0
WHILE #Count <8000
BEGIN
INSERT INTO #MainTable SELECT #Count, "Cust"+CONVERT(VARCHAR(10),#Count)
SET #Count = #Count+1
END
CREATE TABLE #RightTable
(
OrderID INT PRIMARY KEY,
CustomerID INT,
Product VARCHAR(100)
)
CREATE INDEX [IDX_CustomerID] ON #RightTable (CustomerID)
WHILE #Iterator <400000
BEGIN
IF #Iterator % 2 = 0
BEGIN
INSERT INTO #RightTable SELECT #Iterator,2, "Prod"+CONVERT(VARCHAR(10),#Iterator)
END
ELSE
BEGIN
INSERT INTO #RightTable SELECT #Iterator,1, "Prod"+CONVERT(VARCHAR(10),#Iterator)
END
SET #Iterator = #Iterator+1
END
-- Using LEFT JOIN
SELECT mt.CustomerID,mt.FirstName,COUNT(rt.Product) [CountResult]
FROM #MainTable mt
LEFT JOIN #RightTable rt ON mt.CustomerID = rt.CustomerID
GROUP BY mt.CustomerID,mt.FirstName
---------------------------
-- Using Table variable Update
DECLARE #WorkingTableVariable TABLE
(
CustomerID INT,
FirstName VARCHAR(100),
ProductCount INT
)
INSERT
INTO #WorkingTableVariable (CustomerID,FirstName)
SELECT CustomerID, FirstName FROM #MainTable
UPDATE #WorkingTableVariable
SET ProductCount = [Count]
FROM #WorkingTableVariable wt
INNER JOIN
(SELECT CustomerID,COUNT(rt.Product) AS [Count]
FROM #RightTable rt
GROUP BY CustomerID) IV ON wt.CustomerID = IV.CustomerID
SELECT CustomerID,FirstName, ISNULL(ProductCount,0) [CountResult] FROM #WorkingTableVariable
ORDER BY CustomerID
--------
DROP TABLE #MainTable
DROP TABLE #RightTable
Thanks
Lijo
In my opinion there is one reason to do this:
If you have a complicated query with lots of inner joins and one left join you sometimes get in trouble because this query is hundreds of times less fast than using the same query without the left join.
If you query lots of records with a result of very few records to be joined to the left join you could get faster results if you materialize the intermediate result into a table variable or temp table.
But usually there is no need to really update the data in the table variable - you could query the table variable using the left join to return the result.
... just my two cents.