Large update in Sybase - sybase

I need your help to solve a problem that I have with a large update. Hi we want to create a server for developers/testers and we want that have a copy from production, so we want to obfuscate the NSS with a random NSS generated at the moment that I execute the update, so I create a function that do this process. This its the update:
UPDATE CUSTOMERS SET
NAME = 'John',
LAST NAME ='Doe',
NSS = RandomNSS
plan '(i_scan PK_CUSTOMERS CUSTOMERS)'
The update works fine except that the table its 9 million long so I got the log suspended or the message that you are out of locks so the process never finish so I tried to implement the following:
SET ROWCOUNT 10000
WHILE (1 = 1)
BEGIN
BEGIN TRANSACTION
UPDATE CUSTOMERS SET
NAME = 'John',
LAST NAME ='Doe',
NSS = RandomNSS
plan '(i_scan PK_CUSTOMERS CUSTOMERS)'
IF ##ROWCOUNT = 0
BEGIN
COMMIT TRANSACTION
BREAK
END
COMMIT TRANSACTION
END
SET ROWCOUNT 0
But it doesn't solve the problem because I don't have a where clause so the update never finish even that all the customers name are John Doe, so can you help me to create a query that help me to finish this process?

First, on the DEV system, I would change the setting for the databse so that it will truncate the transaction log when it checkpoints. That will keep the log from filling up while you accomplish the update. Because the DEV database is restored from PROD database, it's not important to dump the transaction logs for recoverability.
sp_dboption DEV_DB, 'trunc log on chkpt', true
If you must do the update, try changing the while loop to continue until all the names are changed to "John", like the following.
SET ROWCOUNT 10000
while ( select count(*) from CUSTOMERS where name != "John" and LAST_NAME != 'Doe' ) > 0
BEGIN
BEGIN TRANSACTION
UPDATE CUSTOMERS SET
NAME = 'John',
LAST NAME ='Doe',
NSS = RandomNSS
WHERE name != "John" AND LAST_NAME != "DOE"
plan '(i_scan PK_CUSTOMERS CUSTOMERS)'
COMMIT TRANSACTION
END
SET ROWCOUNT 0
Alternatively, you can create a view on the database, grabbing the Production data you want, while obscuring the fields you don't want the devs to see. This view can then be used to select into the DevelopementDB, or exported/imported with bcp
create view DEV_VIEW (NAME, LAST_NAME, NSS, colA, colB) as
select "John", "Doe", RandomNSS, other_colA, other_colB
from CUSTOMERS

Related

SQL Trigger Works in Play but not Production

I created an SQL trigger in my Play database and it worked great. When I moved it over to Production, it suddenly won't work. We want the trigger to kick off whenever someone edits one of two custom fields in our database. The company who created the software already set up a trigger that kicks of any time a change is made to the database object (it just didn't track the changes made to custom fields). If I let my new trigger create a new record, I wound up with two audit records, so I changed my trigger to update the audit record the software company's trigger created. Could anyone tell me what I have done wrong? Here is my trigger:
USE [TmsEPrd]
GO
/****** Object: Trigger [dbo].[tr_Biograph_Udef_Audit_tracking] Script Date: 11/23/2020 10:22:57 AM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER TRIGGER [dbo].[tr_Biograph_Udef_Audit_tracking] ON [dbo].[BIOGRAPH_MASTER] FOR UPDATE AS
BEGIN
IF EXISTS (SELECT 1 FROM deleted d
JOIN inserted i ON d.ID_NUM = i.ID_NUM
JOIN (SELECT ID_NUM, binary_checksum(UDEF_10A_1, UDEF_2A_4) AS inserted_checksum
FROM inserted) a ON i.ID_NUM = a.ID_NUM
JOIN (SELECT ID_NUM, binary_checksum(UDEF_10A_1, UDEF_2A_4) AS deleted_checksum
FROM deleted) b ON d.ID_NUM = b.ID_NUM
WHERE a.inserted_checksum <> b.deleted_checksum)
BEGIN
Update BIOGRAPH_HISTORY
set archive_job_name = 'UDEF_Change',
udef_2a_4 = i.udef_2a_4,
udef_2a_4_CHG = i.udef_2a_4_chg,
udef_10a_1 = i.udef_10a_1,
udef_10a_1_chg = i.udef_10a_1_chg
from
(select i.ID_NUM, SYSDATETIME()as job_time_a,
i.UDEF_10A_1, case when i.UDEF_10A_1 = d.UDEF_10A_1 then 0 when i.UDEF_10A_1 is null and d.UDEF_10A_1 is null then 0 else 1 end as UDEF_10A_1_CHG,
i.UDEF_2A_4, case when i.UDEF_2A_4 = d.UDEF_2A_4 then 0 when i.UDEF_2A_4 is null and d.UDEF_2A_4 is null then 0 else 1 end as UDEF_2A_4_CHG,
d.USER_NAME,d.JOB_NAME,d.JOB_TIME
FROM deleted d JOIN inserted i ON d.ID_NUM = i.ID_NUM) i
join BIOGRAPH_HISTORY b on i.ID_NUM = b.ID_NUM
where DATEDIFF(Minute, i.job_time_a, b.ARCHIVE_JOB_TIM) = 0
and b.ARCHIVE_JOB_NAME not like 'UDEF_Change%'
END;
END;
Try specifying #order = 'LAST' for your trigger. It might be that your trigger is executing first and not finding a record to update. In your test system, the trigger execution order might be reversed.
The order that triggers are created might affect trigger execution order, but this is not something to rely upon. When you think about it, this can be a headache. A test system that looks just like production can behave differently.
This is similar to relying upon a "natural" record order of a clustered index and not using a ORDER BY clause. A different execution plan can use a different index or go parallel resulting in a different or no order.

How do I set the correct transaction level?

I am using Dapper on ADO.NET. So at present I am doing the following:
using (IDbConnection conn = new SqlConnection("MyConnectionString")))
{
conn.Open());
using (IDbTransaction transaction = conn.BeginTransaction())
{
// ...
However, there are various levels of transactions that can be set. I think this is the various settings.
My first question is how do I set the transaction level (where I am using Dapper)?
My second question is what is the correct level for each of the following cases? In each of these cases we have multiple instances of a web worker (Azure) service running that will be hitting the DB at the same time.
I need to run monthly charges on subscriptions. So in a transaction I need to read a record and if it's due for a charge create the invoice record and mark the record as processed. Any other read of that record for the same purpose needs to fail. But any other reads of that record that are just using it to verify that it is active need to succeed.
So what transaction do I use for the access that will be updating the processed column? And what transaction do I use for the other access that just needs to verify that the record is active?
In this case it's fine if a conflict causes the charge to not be run (we'll get it the next day). But it is critical that we not charge someone twice. And it is critical that the read to verify that the record is active succeed immediately while the other operation is in its transaction.
I need to update a record where I am setting just a couple of columns. One use case is I set a new password hash for a user record. It's fine if other access occurs during this except for deleting the record (I think that's the only problem use case). If another web service is also updating that's the user's problem for doing this in 2 places simultaneously.
But it's key that the record stay consistent. And this includes the use case of "set NumUses = NumUses + #ParamNum" so it needs to treat the read, calculation, write of the column value as an atomic action. And if I am setting 3 column values, they all get written together.
1) Assuming that Invoicing process is an SP with multiple statements your best bet is to create another "lock" table to store the fact that invoicing job is already running e.g.
CREATE TABLE InvoicingJob( JobStarted DATETIME, IsRunning BIT NOT NULL )
-- Table will only ever have one record
INSERT INTO InvoicingJob
SELECT NULL, 0
EXEC InvoicingProcess
ALTER PROCEDURE InvoicingProcess
AS
BEGIN
DECLARE #InvoicingJob TABLE( IsRunning BIT )
-- Try to aquire lock
UPDATE InvoicingJob WITH( TABLOCK )
SET JobStarted = GETDATE(), IsRunning = 1
OUTPUT INSERTED.IsRunning INTO #InvoicingJob( IsRunning )
WHERE IsRunning = 0
-- job has been running for more than a day i.e. likely crashed without releasing a lock
-- OR ( IsRunning = 1 AND JobStarted <= DATEADD( DAY, -1, GETDATE())
IF NOT EXISTS( SELECT * FROM #InvoicingJob )
BEGIN
PRINT 'Another Job is already running'
RETURN
END
ELSE
RAISERROR( 'Start Job', 0, 0 ) WITH NOWAIT
-- Do invoicing tasks
WAITFOR DELAY '00:01:00' -- to simulate execution time
-- Release lock
UPDATE InvoicingJob
SET IsRunning = 0
END
2) Read about how transactions work: https://learn.microsoft.com/en-us/sql/t-sql/language-elements/transactions-transact-sql?view=sql-server-2017
https://learn.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql?view=sql-server-2017
You second question is quite broad.

Oracle trigger implementation

I have to implement a trigger which would:
7) Show the DDL for how a trigger can be used to transfer all rental copies from a store being deleted from the Store information table to the central store
8) Show how this trigger can be extended to make
sure that the central store is never deleted from the database
So far I have done this:
CREATE OR REPLACE TRIGGER stores BEFORE DELETE ON stores FOR
EACH ROW BEGIN IF DELETING WHERE cvr = 123456789 THEN
Raise_Application_Error ( num => -20050, msg => 'You can
not delete Main Store.'); END IF; IF DELETING THEN
UPDATE store_id=123456789 ON movies WHERE isActive = 0 END
IF; END;
so main store is with cvr which is written, but it gives me a compilation error. Any help? Thanks in advance.
You have two errors in your code.
there is no "DELETING WHERE" expression, you have to use two boolean exceptions like this:
IF DELETING AND :old.cvr = 123456789 THEN...
:old.cvr refers to value of cvr column in deleted record
Syntax of UPDATE clause is
UPDATE table_name
SET column_name1 = value1,
column_name1 = value2
WHERE where_clause;
in your case probably somethink like this:
UPDATE movies
set store_id = 123456789
WHERE store_id = :old.cvr
I guess, I don't know required functionality

Why triggers try to insert NULL value when using a field from 'inserted' table?

I have to sync changes done in MSSQL with a remote MySQL database. The changes to be synced are adding invoices and users to the system. The remote server is not expected to be always reachable so I'm trying to set up a kind of log table for storing changes done in MSSQL.
Here is a fully working trigger for that:
CREATE TRIGGER [dbo].[dokument_insert]
ON [dbo].[dokument]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [bcg_ekodu].[dbo].[sync_stack] (event,sql, table_name, import_priority)
SELECT
'INSERT',
'INSERT INTO bills SET
date = "'+CONVERT(VARCHAR(19),dok_kuup,120)+'",
total = "'+CAST(kokkusum AS nvarchar)+'",
number = "'+RTRIM(dok_nr)+'",
created = "'+CONVERT(VARCHAR(19),savetime,120)+'",
rounded = "'+CAST(ymardus AS nvarchar)+'",
currency = "'+CAST(valuuta AS nvarchar)+'",
due_date = "'+CONVERT(VARCHAR(19),tasupaev,120)+'",
pk_joosep = "'+CAST(dok_kood AS nvarchar)+'",
joosep_hankija = "'+CAST(hankija AS nvarchar)+'";
UPDATE
bills, users, companies
SET
bills.user_id = users.id,
bills.imported = NOW()
WHERE
bills.imported IS NULL
AND companies.id = users.company_id
AND companies.pk_joosep = 10
AND bills.user_id = users.pk_joosep',
'bills',
'200'
FROM inserted
END
It inserts a row into 'sync_stack' table every time a row is inserted to 'dokument' table. The 'sql' column will contain an SQL to create the same kind of row in another (MySQL) database.
But this trigger is not working:
CREATE TRIGGER [dbo].[klient_insert]
ON [dbo].[klient]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [bcg_ekodu].[dbo].[sync_stack] (event,sql, table_name, import_priority)
SELECT
'INSERT',
'INSERT INTO users SET
username =10'+CAST(kl_kood as nvarchar)+',
password = NULL,
name ="'+LTRIM(RTRIM(kl_nimi))+'",
email ="'+CAST(LTRIM(RTRIM(kl_email)) as nvarchar)+'",
reference_no ="'+CAST(LTRIM(RTRIM(kl_viide)) as nvarchar)+'",
phone ="'+CAST(LTRIM(RTRIM(kl_tel1)) as nvarchar)+'",
logins ="'+CAST(0 as nvarchar)+'",
last_login = NULL,
created ="'+CONVERT(VARCHAR(19),savetime,120)+'",
updated = NULL,
deleted ="0",
address ="'+CAST(LTRIM(RTRIM(kl_aadr1)) as nvarchar)+'",
pk_joosep ="'+CAST(kl_kood as nvarchar)+'"',
'users',
'210'
FROM inserted
END
While the execution of the above SQL to create that trigger completes just fine, when I try to insert some rows to the 'triggered' table, I get the following error:
No row was updated.
The data in row 175 was not committed.
Error Source: .Net SqlClient Data Provider.
Error Message: Cannot insert the value NULL into column 'sql', table 'mydb.dbo.sync_stack'; column does not allow nulls. INSERT fails.
The statement has been terminated.
Correct the errors and retry or press ESC to cancel the change(s).
If I delete this trigger, this error does not occur.
If I insert just plain text for 'sql' column, it works as expected.
If I use any field from the inserted row, even just a text field, it fails again.
If I allow NULL values in 'sql' column, inserting rows succeeds but I get a NULL value in 'sql' column.
How to make the second trigger work as expected, too?
I suspect that at least one of the values from inserted that you are concatenating into your SQL statement is NULL. You can circumvent this by using COALESCE, e.g.
username =10'+COALESCE(CAST(kl_kood as nvarchar), '')+',
Of course you shouldn't be declaring nvarchar without specifying a length, right?
Bad habits to kick : declaring VARCHAR without (length)
Concatenating any value to NULL is NULL:
select 'test' + NULL
Results in null, you should use something like that for your columns:
select isnull(column, '')
This would result in an empty string.

Why does an UPDATE take much longer than a SELECT?

I have the following select statement that finishes almost instantly.
declare #weekending varchar(6)
set #weekending = 100103
select InvoicesCharges.orderaccnumber, Accountnumbersorders.accountnumber
from Accountnumbersorders, storeinformation, routeselecttable,InvoicesCharges, invoice
where InvoicesCharges.pubid = Accountnumbersorders.publication
and Accountnumbersorders.actype = 0
and Accountnumbersorders.valuezone = 'none'
and storeinformation.storeroutename = routeselecttable.istoreroutenumber
and storeinformation.storenumber = invoice.store_number
and InvoicesCharges.invoice_number = invoice.invoice_number
and convert(varchar(6),Invoice.bill_to,12) = #weekending
However, the equivalent update statement takes 1m40s
declare #weekending varchar(6)
set #weekending = 100103
update InvoicesCharges
set InvoicesCharges.orderaccnumber = Accountnumbersorders.accountnumber
from Accountnumbersorders, storeinformation, routeselecttable,InvoicesCharges, invoice
where InvoicesCharges.pubid = Accountnumbersorders.publication
and Accountnumbersorders.actype = 0
and dbo.Accountnumbersorders.valuezone = 'none'
and storeinformation.storeroutename = routeselecttable.istoreroutenumber
and storeinformation.storenumber = invoice.store_number
and InvoicesCharges.invoice_number = invoice.invoice_number
and convert(varchar(6),Invoice.bill_to,12) = #weekending
Even if I add:
and InvoicesCharges.orderaccnumber <> Accountnumbersorders.accountnumber
at the end of the update statement reducing the number of writes to zero, it takes the same amount of time.
Am I doing something wrong here? Why is there such a huge difference?
transaction log file writes
index updates
foreign key lookups
foreign key cascades
indexed views
computed columns
check constraints
locks
latches
lock escalation
snapshot isolation
DB mirroring
file growth
other processes reading/writing
page splits / unsuitable clustered index
forward pointer/row overflow events
poor indexes
statistics out of date
poor disk layout (eg one big RAID for everything)
Check constraints with UDFs that have table access
...
Although, the usual suspect is a trigger...
Also, your condition extra has no meaning: How does SQL Server know to ignore it? An update is still generated with most of the baggage... even the trigger will still fire. Locks must be held while rows are searched for the other conditions for example
Edited Sep 2011 and Feb 2012 with more options
The update has to lock and modify the data in the table, and also log the changes to the transaction log. The select does not have to do any of those things.
Because reading does not affect indices, triggers, and what have you?
In Slow servers or large database i usually use UPDATE DELAYED, that waits for a "break" to update the database itself.

Resources