I want to copy the value of one column to another column in SQL Server. This operation needs to be carried out across the whole DB which has 200M rows. My query syntax is:
UPDATE [Net].[dbo].[LINK]
SET [LINK_ERRORS] = [OLD_LINK_ERRORS]
However, very soon I exhaust the transaction log and the query aborts. What's the best way to initiate this in batches?
Thanks,
Updating 200M rows is not a good idea.
You could either select all of the data into a new table and copy the LINK_ERRORS field in the SELECT,
select *, OLD_LINK_ERRORS as LINK_ERRORS into LINK_tmp from LINK
GO
exec sp_rename LINK, LINK_bkp
GO
exec sp_rename LINK_tmp, LINK
GO
drop table LINK_bkp
or if the next thing you're going to do is null out the original OLD_LINK_ERRORS column, you could do something like this:
sp_rename 'LINK.OLD_LINK_ERRORS', 'LINK_ERRORS', 'COLUMN'
GO
ALTER TABLE LINK ADD OLD_LINK_ERRORS <data type>
GO
multiple updates might work.
update dbo.LINK
set LINK_ERRORS=OLD_LINK_ERRORS
where ID between 1 and 1000000
update dbo.LINK
set LINK_ERRORS=OLD_LINK_ERRORS
where ID between 1000001 and 2000000
etc...
I would consider doing this in SSIS where you can easily control the batch (transaction) size and take advantage of bulk operations SSIS provides. Of course, this may not work if you need a programmatic solution. This would be a very trivial SSIS operation.
Related
Is this the right way to make sure that a column of my table is only renamed after another column is dropped?
ALTER TABLE mytbl DROP COLUMN tmpcol
EXEC sp_rename 'mytbl.tmpcol2', 'tmpcol', 'COLUMN'
I'm not allowed to use the "GO"-separator.
I've tested the above two lines for a bunch of different table-sizes.
It worked as I expected, i.e. the 2nd Line is only executed after the 1st one.
But how can I make sure, that this will be the preferred execution plan for any table?
Is this guaranteed by EXEC ?
SQL Server run query line-by-line, there is no chance that EXEC can run before ALTER TABLE
I've created a stored procedure to add data to a table. In mock fashion the steps are:
truncate original table
Select data into the original table
The query that selects data into the original table is quite long (it can take almost a minute to complete), which means that the table is then empty of data for over a minute.
To fix this empty table I changed the stored procedure to:
select data into #temp table
truncate Original table
insert * from #temp into Original
While the stored procedure was running, I did a select * on the original table and it was empty (refreshing, it stayed empty until the stored procedure completed).
Does the truncate happen at the beginning of the procedure no matter where it actually is in the code? If so is there something else I can do to control when the data is deleted?
A very interesting method to move data into a table very quickly is to use partition switching.
Create two staging tables, myStaging1 and myStaging2, with the new data in myStaging2. They must be in the same DB and the same filegroup (so not temp tables or table variables), with the EXACT same columns, PKs, FKs and indexes.
Then run this:
SET XACT_ABORT, NOCOUNT ON; -- force immediate rollback if session is killed
BEGIN TRAN;
ALTER TABLE myTargetTable SWITCH TO myStaging1
WITH ( WAIT_AT_LOW_PRIORITY ( MAX_DURATION = 1 MINUTES, ABORT_AFTER_WAIT = BLOCKERS ));
-- not strictly necessary to use WAIT_AT_LOW_PRIORITY but better for blocking
-- use SELF instead of BLOCKERS to kill your own session
ALTER TABLE myStaging2 SWITCH TO myTargetTable
WITH (WAIT_AT_LOW_PRIORITY (MAX_DURATION = 0 MINUTES, ABORT_AFTER_WAIT = BLOCKERS));
-- force blockers off immediately
COMMIT TRAN;
TRUNCATE TABLE myStaging1;
This is extremely fast, as it's just a metadata change.
You will ask: partitions are only supported on Enterprise Edition (or Developer), how does that help?
Switching non-partitioned tables between each other is still allowed even in Standard or Express Editions.
See this article by Kendra Little for further info on this technique.
The sp is being called by code in an HTTP Get, so I didn't want the table to be empty for over a minute during refresh. When I asked the question I was using a select * from the table to test, but just now I tested by hitting the endpoint in postman and I never received an empty response. So it appears that putting the truncate later in the sp did work.
I want to replace a SalesResults table with a new version containing latest calculated results.
I guess the following would only take a few milliseconds for SQL Server to do but is it safe for any users accessing the SalesResults table at that time?
If not, it should I enclose the following in BEGIN TRANSACTION, COMMIT in order for it to be?
DROP TABLE dbo.SalesResults;
EXEC sp_rename 'SalesResultsNew', 'SalesResults'
I would do something like this during off hours or a maintenance period just to be safe
Begin Transaction
Drop Table dbo.SalesResults
Exec sp_rename 'SalesResultsNew', 'SalesResults'
Commit Transaction
This assumes that the SalesResultsNew table already exists.
If you're doing this on a consistent basis (and it sounds like you are), I'd use a synonym instead. So your actual tables would be called something like dbo.SalesResults_20170108 and you'd do something like:
create synonym dbo.SalesResults for dbo.SalesResults_20170108;
Each day, you'd move the synonym to point to the new SalesResults table when it's ready.
I have been searching around but I cannot find the correct answer, probably I search wrong because I don't know what to look for :)
Anyway, I have a TSQL with a begin and commit transaction. In the transaction I add some columns and also rename some columns.
Just after the renames and added column statement i also run some update statements to load data into the newly created columns.
Now the problem is that for some reason the update gives an error that it cannot update the given column as it does not exist (YET???).
My idea is that the statement is still working out the rename and the adding of the columns but already goes ahead with the update statements. The table is very big and has a few million records so I can imagine it takes some time to add and rename the columns
If I run first the rename and add statements and than separate the update statements, it does work. So it has to do with some wait time.
Is it possible to make sql force to execute step by step and wait until the complete statement is done before going to the next?
If you modify columns (e.g. add them), you have to finish the batch before you can continue with updating them. Insert the GO keyword between table structure changes and updates.
To illustrate that, the following code won't work:
create table sometable(col1 int)
go
alter table sometable add col2 varchar(10)
insert into sometable(col2) values ('a')
But inserting go will make the insert recognise the new column
create table sometable(col1 int)
go
alter table sometable add col2 varchar(10)
go
insert into sometable(col2) values ('a')
If you do it in the code, you may want to create separate transaction for the structure changes and data migration. You can still wrap them it in one transaction for data integrity.
It doesn't have anything to do with wait time. The queries are run in order. It's because all the queries are submitted all at once and therefore when it tries to validate your update, the column doesn't exist at that point in time. To get around it, you need to send the update in a separate batch. The following keyword needs to be added between your alter and update statement
GO
You can try using select for update,
http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_10002.htm#i2130052
This will ensure that your query will wait for lock, bit it is recommended to Specify WAIT to instruct the database to wait integer seconds so that it will not wait for indefinate time.
I have an application that uses a SQL Server database with several instances of the database...test, prod, etc... I am making some application changes and one of the changes involves changing a column from a nvarchar(max) to a nvarchar(200) so that I can add a unique constraint on it. SQL Server tells me that this requires dropping the table and recreating it.
I want to put together a script that will do the table drop, recreate it with the new schema, and then reinsert the data that was there previously all in one go, if possible, just to keep things simple for use when I migrate this change to production.
There is probably a good SQL Server way to do this but I'm just not aware of it. If I was using Mysql I would mysqldump the table and its contents, and use that as my script for applying that change to production. I can't find any export functionality in SQL server that will give me a text file consisting of inserts for all data in a table.
Use SQL Server's Generate Scripts command
right click on the database; Tasks -> Generate Scripts
select your tables, click Next
click the Advanced button
find Types of data to script - choose Schema and Data.
you can then choose to save to file, or put in new query window.
results in INSERT statements for all table data selected in bullet 2.
No need to script
here are two ways
1 use alter table ....alter column.....
example..you have to do 1 column at a time
create table Test(SomeColumn nvarchar(max))
go
alter table Test alter column SomeColumn nvarchar(200)
go
2 dump into a new table while converting the column
select <columns except for the columns you want to change>,
convert(nvarchar(200),YourColumn) as YourColumn
into SomeNewTable
from OldTable
drop old table
rename this table to the same table as the old table
EXEC sp_rename 'SomeNewTable', 'OldTable';
Now add your index