I wanted to perform a simple delete statement like this:
DELETE
FROM table
WHERE table.value = 123
and I am expecting the delete 512 rows from the table since those 512 rows have value 123.
However, there are 5 lines of "xxx rows affected" displayed after running the delete statement.
The last two lines are identical, "512 lines affected", which is expected.
The first "512 rows affected" was the actual deletion.
The second "512 rows affected" was a trigger(the only delete trigger) inserting 512 rows into table_AUDIT.
What about the first 3 lines of "xxx rows affected"?
I don't know which tables are affected so I don't know how to use OUTPUT(googled) to figure out which rows/tables are affected.
My question is: how to display detailed information of the rows deleted? Insert of meaningless "123 rows effected", I like to see which rows from which tables are deleted.
The best you can do is get a query plan which shall include the triggers. Which rows is something left to your intelligence - query plans generally do not provide this information.
Related
I am executing MERGE query to perform CDC operation. I have a target table which is holding around 50 million records and the incoming file which is source for MERGE contains 230 records. There is simple join in ID of table and id column from file data. After execution , the History view shows records inserted 200 and records updated 30. However its showing rows_produced as 5K. I need to understand what does rows_produced in this case. Does it show the rows return as a part of join ? if its yes, then it should be matching the row count of file.
I believe that rows_produced is the total number of records that were created when the underlying micropartitions were written out.
For example, if you updated 1 record, you are actually recreating the entire micropartition of data that this 1 record exists in (micropartitions are immutable, so therefore never updated). If that 1 record exists in a micropartition that contains 100 records, then you'd get an output that has 1 record updated, but 100 rows_produced.
This information is "interesting" but not helpful when trying to make sure the right outcome of your MERGE statement. Using the insert, update, and delete output for the MERGE is the accurate way to look at that.
If you notice, I am just keeping the first three blocks of a row same and deleting the others as they are same but not deleting the entire row as the balance qty keeps on changing. I want the first entire row to be retained, however succeeding rows should have only balance qty output for the part no , product descry and weight
Can someone please help me in suggesting a query in Microsoft SQL where I get an output shown in table 2 from table 1 output which I am getting through a query
This is really a presentation-level question. I'd never do it in SQL, not that it's impossible. Look up the LAG function. you can use that to look at the previous row's description and set it to blank if equal.
I have Firebird table with 60 milions rows and i need delete ca. half of table.
Table rows has gps position of cars, timestamp of record and other data. Table has primary key IdVehicle+TimeStamp and one foreign key (into Vehicle table). There is no other key or index or trigger. One vehicle has 100 000 - 500 000 records.
I need delete older data, eg. from all vehicles delete data older than 1 March 2015. I tried different ways and actually use my fastest comes with 'execute block' (use primary key). First I read for one vehicle records older then 1.3.2015. Then I am going through the individual records and prepare sql execute a block and then perform it into firebird for every 50 entries.
EXECUTE BLOCK AS BEGIN
DELETE FROM RIDE_POS WHERE IdVehicle = 1547 and date = '4.5.2015 8:56:47'
DELETE FROM RIDE_POS WHERE IdVehicle = 1547 and date = '4.5.2015 8:56:59'
DELETE FROM RIDE_POS WHERE IdVehicle = 1547 and date = '4.5.2015 8:57:17'
...... a total of 50 line
END
Thus delete 1 million lines per 800 seconds (about 1 record for 1 ms).
Is there another quicker way to delete records?
Additionally, this way I can delete only a few million lines, and then I have to restart firebird, otherwise starts to slow down and jam (on the test server there is no other database / application). From early records cleared quickly and gradually takes longer and longer time.
For orientation, how quickly you erasing records routinely in large tables (not completely erase the table, but only a part of the record).
If you want to delete all records older than given date, no matter the vehicle, then there is no point including the Idvehicle in the query, just the date is enough. Ie following should do, just straight query, no need for execute block either:
DELETE FROM RIDE_POS WHERE date < '2015-03-01'
If you have to delete many thousands (or millions) records do not do it in one single transaction. You better do it in several steps - delete for example 1000 records and commit, then delete other 1000 and commit - it should be faster than delete one million of records in one transaction. 1000 is not a rule, it depends on your particular situation (how large are your records, how many linked data they have via foreign keys with "on delete cascade"). Also check whether you have "on delete" triggers and maybe it is possible to temporary deactivate them.
Maybe a combined approach would help.
Add (temporarily) index on date:
CREATE INDEX IDX_RIDE_POS_date_ASC ON RIDE_POS (date)
Write an execute block:
EXECUTE BLOCK
AS
DECLARE VARIABLE V_ID_VEHICLE INTEGER;
BEGIN
FOR SELECT
DISTINCT ID_VEHICLE
FROM
RIDE_POS
INTO
:V_ID_VEHICLE
DO BEGIN
DELETE FROM RIDE_POS WHERE IdVehicle = :V_ID_VEHICLE AND date < '1.3.2015'
END
END
Drop index if you don't want to have it anymore.
DROP INDEX IDX_RIDE_POS_date_ASC'
I think that even taking into account a time that is needed for creating index, you would still save some time on deleting records.
Finally, I found where the problem was. The main problem was that I am using the classic Winforms application (or IBExpert) and that causing jams and slowing query. I used to execute block and erases the data portions, which has solved the problem of jams, but it was slow.
The solution was to create a simple console application and run query from it. I left primary key and erases through it (no adding or deleting indexes) and the speed of deleting the records was some 65 per milisecond (1 million rows per 16 second).
When I tried to delete primary and add index on datetime column, than erasing speed up just little about 5-10%.
I'm using SQL Server 2008 R2 and am trying to insert some records from one table to another using the form:
INSERT INTO table2
SELECT ... From table1
And am receiving the dreaded error.
"string or binary data would be truncated"
table1 is a view to another database from which we are taking records and doing some processing of the data before inserting them into table2 which is a table in the database.
I assumed that this message meant that one of the fields I'm populating is too small to hold the data I'm inserting, but it doesn't appear to be the case.
The select does some parsing of the fields from table 1 and inserts those values into table2. However the inserted data is smaller than the original fields (being only part of the original field) and I have checked to confirm that they all fit into the fields they are being inserted into.
I'm struggling to find the real cause of this problem as the fields in table2 are all of sufficient size and type to hold the data that will be added to them.
Additionally, when I limited the number of records to be added (doing it in batches to try and find the offending record) it would run without error for 2-3 batches of 10k records before getting this error. So i thought it was in the next 10k records where there was some anomalous data.
However, when I reduced the batch size to 6k records, instead of getting the error at the second batch (records 6k-12k) which is where the problem data should have been, I was able to run the insert without error another 10 times (60k records in total) before the error happened again.
Then I reduced the batch size to 3k records and was able to run it another 4 times (12k records) before seeing the error.
I kept reducing the size of the batch and each time I could insert more records using multiple smaller batches than I could with a single bigger batch.
This would suggest that it's not actually the size of the data being inserted that is causing the problem, but some other aspect.
It has now got to the point where I can no longer insert a single record. Inspecting the record to be inserted confirms that it complies with the sizes of all the fields and should not be causing this error.
Given that initially I couldn't insert the next 10k records because of this error, but have managed to insert over 100k records (in ever decreasing batch sizes) I was wondering if anyone could suggest where the real problem lies?
The question is:
Is there some system table, log or some other limit that may be blocking these inserts?
Any suggestions would be greatly appreciated.
Scenario of Records Insertion from one DB to another:
Suppose I have 100,000 records in db1 and I am selecting a batch of records from database db1 and
inserting it into another database db2. By selecting a batch of records I mean I am selecting 1000 records
at a time. Next time, I run the same query to select another batch of 1000 records, I want to start from
record number 1001 as all the previous records 1000 records starting from row #1 have already been
inserted into db2.
So, basically, second time, I want to avoid copying the same number of record again in order to avoid
the duplicate records in the database db2.
In order to avoid the insertion of duplicate records, one approach, I am following right now is,
I have a integer value column named flag in my database db1 with a negative value, say for example -1 and as soon
as I grab 1000 records, I am updating the value to positive value , that is +1. So that, the second time
I will run the query to grab the next set of 1000 records, it will start from the record #1001.
I am wondering whether this is an efficient approach as everytime I run my query, my query will be checking
all the records in db1 right from the start and then ultimately start grabbing the data from where it
will notice the start of negative value in the flag column. Please suggest if there is much more efficient
approach.
P.S: I am using JDBC to grab the records and insert it into another database.
Thanks
I would not update the table in db1 just for this purpouse.
I'd rather create another table either in db1 or, maybe better, in db2, for control purpouses.
This way you will not need to cope with write access rigths in the original table in db1 and/or with race or collision or multiuser conflict conditions.
The second table would record, for each extraction, the timestamp, tables involved, range extracted, user name, process name... and any other control information you might be interested.