I want to insert 10 million rows into my Oracle Database via Database Link.
What will be an optimized way to do that?
Doing INSERT by SELECT * FROM [dblink_table_name] would be an optimized way of doing it?
Related
I have a SQL Server instance which contains multiple databases. I have 2 tables which exist in all databases on the server:
Refresh Log
Detailed refresh log.
I want to union all the tables across all databases on the server so the final result will be 2 tables which are the union refresh log and detailed refresh log.
I need help to write the function which runs across all databases.
I'm also a little uncertain as to what you're hoping for, for example if you need the resulting output to be in two permanent tables or if you just need the result when queried. Of course once you build your SELECT you can return it to the caller or put it into a table, so I'll leave that up to you.
If your databases are unchanging, then of course you can just write your query and maybe put it into a VIEW for convenience:
SELECT columns from database1.dbo.RefreshLog
UNION ALL
SELECT columns from database2.dbo.RefreshLog
...
and so on
But if you're saying that your databases are themselves dynamic, in other words that databases may be created or dropped over the lifetime of your project, then you could consider using the "undocumented" procedure sp_msforeachdb to build up a list of databases, and then use THAT list to build your UNION query. Here's a quick script that captures the names of all databases that include a specific table ("Products" in the example):
IF object_id('tempdb..#DatabaseNames') IS NOT NULL
DROP TABLE #DatabaseNames
CREATE TABLE #DatabaseNames (DatabaseName SYSNAME)
execute sp_msforeachdb #command1=
N'IF EXISTS(SELECT * FROM [?].sys.tables WHERE Name = ''Products'')
INSERT #DatabaseNames VALUES(N''Database [?]'')'
SELECT * FROM #DatabaseNames
I am using bulk inserts to insert a collection of large files (up to 100 million records) into a database replacing existing data.
Initially this took about 5 minutes, which is a bit too slow since it locks production tables during update (for the purposes 2-3 minute wait should be max). In order to speed up things I created a staging table from which I copy data internally on the SQL Server (which is about 5 times faster when testing). The thought was that at least the production tables were locked for a shorter time. However now it seems that the entire database is locked during the update process, even tables not in use in the update process.
Is this normal behavior? and how can this be prevented? I have googled a lot and all issues are related to the locking of tables being updated and not unrelated tables.
For a single table this it the bulk insert:
truncate table SomeSchema.TradesStaging
BULK INSERT SomeSchema.TradesStaging
FROM '\\SomePath\SomeSchema.Trades.tab'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n',
TABLOCK
)
And the internal copying looks like this:
if EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'SomeSchema' AND TABLE_NAME = 'Trades') begin drop table SomeSchema.Trades end
select * into SomeSchema.Trades from SomeSchema.TradesStaging
--a lot of code recreating indexes etc. down here.
All code is wrapped into a stored procedure like this (example with only the above):
SomeSchema.StageTradesFromFile
SomeSchema.LoadTradesFromStaging
Are you sure you need to drop the SomeSchema.Trades table? This will result in a schema lock. Why not just truncate the table and reload it? In addition you could disable and reenable the indexes to help speed it up.
Why the not logged initially option during the data movement is faster than using the LOAD utility in DB2?
Not logged initially Method:
db2 alter table tablename activate not logged initially
db2 insert into table tablename select * from tbname
Load Utility:
db2 declare source cursor for select * from tablename
db2 load from source of cursor insert into tablename nonrecoverable
Based on your database size and performance question I am going to assume that you are using the Database Partitioning Feature (DPF) with DB2.
When you perform INSERT INTO ... SELECT, this occurs in parallel on all database partitions – each partition is working independently. With logging turned off, this will be quite fast (albeit dangerous – if there is a problem, the not-logged-initially table will have to be dropped and recreated).
When you use LOAD FROM CURSOR, all of the database partitions execute the SELECT statement and return the rows to the coordinator partition, which then feeds them into the LOAD utility. The LOAD utility then performs hash partitioning to send the data back to all of the database partitions again. As you can imagine, with a large volume of data, shipping all of this data back and forth can be quite inefficient.
I am new to Sql Server but I have always used Oracle a lot. I see that some functionalities are not suppported in Sql Server. Lets say, I want to watch a table for insertion, and what is inserted into that table, I want to copy that same inserted row into another table.
here is my code
Create TRIGGER Post_Trigger ON Posts For Insert
AS
INSERT INTO EmailQueueRaw (UserID,CreatedBy,EmailTypeId,EmailTablePrimaryKey) VALUES('','Arif','1','1');
GO
In Oracle, I used to use New and Old function which are great. But we dont have it in Sql Server and I am not sure what to do here. Please help me how to copy the same data to another table?
You would use INSERTED (and, if needed DELETED) but you need to be aware that they are pseudo-tables and could contain 0, 1, or multiple rows:
Create TRIGGER Post_Trigger ON Posts For Insert
AS
INSERT INTO EmailQueueRaw (UserID,CreatedBy,EmailTypeId,EmailTablePrimaryKey)
SELECT '',ColumnA,'1',ColumnB FROM inserted;
GO
I want to create trigger for one table for insertion...If i insert records in one table then that same records should be inserted to another one......how?
and also explain about triggers
All you need to know about triggers, without having to wade through Microsoft speak, including how to create them in SQL Server.
http://en.wikipedia.org/wiki/Database_trigger
It should be something like
CREATE TRIGGER TableInserted ON TableInsert
AFTER INSERT
AS
INSERT INTO OtherTable SELECT * FROM inserted
This article covers all the basics of creating triggers.
Though for you task i think a stored procedure doing both inserts will be better, because what if one insert succeeded and the other failed? it will create inconsistency in your data. See here for stored procedures.