We have 2 databases and we need data to be transferred from db 1 to db 2. How can I do that (in SYBASE there are proxy tables) in SQL Server?
As #Nathan says just BULK INSERT the data. Assuming both databases are on the same server then you reference the table usually as databasename.schema.tablename thus db1.dbo.table1 or db2.dbo.table1
As such you can also just create a view in the destination data to use as a 'proxy' and pull the data without actually copying it. The view would be in db2 and be something like:
CREATE VIEW table1 AS SELECT * FROM db1.dbo.table1
I think INSERT INTO would be a good way to go.
http://msdn.microsoft.com/en-us/library/aa933206(v=sql.80).aspx
First of all, you can create a linked server in your destination server to the other server. Then you can do the INSERT INTO.
If you don't want to do that (or are unable), then dump the data to a file and do the very fast BULK INSERT to get the data into your new table.
Related
Here's the basic idea of what I want to do in SSIS:
I have a large query against a production Oracle database, and I need the following where clause that brings in a long list of ids from SQL Server. From there, the results are sent elsewhere.
select ...
from Oracle_table(s) --multi-join
where id in ([select distinct id from SQL_SERVER_table])
Alternatively, I could write the query this way:
select ...
from Oracle_table(s) --multi-join
...
join SQL_SERVER_table sst on sst.ID = Oracle_table.ID
Here are my limitations:
The Oracle query is large and cannot be run without the where id in (... clause
This means I cannot run the Oracle query, then join it against the ids in another step. I tried this, and the DBA's killed the temp table after it became 3 TB in size.
I have 160k id's
This means it is not practical to iterate through the id's one by one. In the past, I have run against ~1000 IDs, using a comma-separated list. It runs relatively fast - a few minutes.
The main query is in Oracle, but the ids are in SQL Server
I do not have the ability to write to Oracle
I've found many questions like this.
None of the answers I have found have a solution to my limitations.
Similar question:
Query a database based on result of query from another database
To prevent loading all rows from the Oracle table. The only way is to apply the filter in the Oracle database engine. I don't think this can be achieved using SSIS since you have more than 160000 ids in the SQL Server table, which cannot be efficiently loaded and passed to the Oracle SQL command:
Using Lookups and Merge Join will require loading all data from the Oracle database
Retrieving data from SQL Server, building a comma-separated string, and passing it to the Oracle SQL command cannot be done with too many IDs (160K).
The same issue using a Script Task.
Creating a Linked Server in SQL Server and Joining both tables will load all data from the Oracle database.
To solve your problem, you should search for a way to create a link to the SQL Server database from the Oracle engine.
Oracle Heterogenous Services
I don't have much experience in Oracle databases. Still, after a small research, I found something in Oracle equivalent to "Linked Servers" in SQL Server called "heterogeneous connectivity".
The query syntax should look like this:
select *
from Oracle_table
where id in (select distinct id from SQL_SERVER_table#sqlserverdsn)
You can refer to the following step-by-step guides to read more on how to connect to SQL Server tables from Oracle:
What is Oracle equivalent for Linked Server and can you join with SQL Server?
Making a Connection from Oracle to SQL Server - 1
Making a Connection from Oracle to SQL Server - 2
Heterogeneous Database connections - Oracle to SQL Server
Importing Data from SQL Server to a staging table in Oracle
Another approach is to use a Data Flow Task that imports IDs from SQL Server to a staging table in Oracle. Then use the staging table in your Oracle query. It would be better to create an index on the staging table. (If you do not have permission to write to the Oracle database, try to get permission to a separate staging database.)
Example of exporting data from SQL Server to Oracle:
Export SQL Server Data to Oracle using SSIS
Minimizing the data load from the Oracle table
If none of the solutions above solves your issue. You can try minimizing the data loaded from the Oracle database as much as possible.
As an example, you can try to get the Minimum and Maximum IDs from the SQL Server table, store both values within two variables. Then, you can use both variables in the SQL Command that loads the data from the Oracle table, like the following:
SELECT * FROM Oracle_Table WHERE ID > #MinID and ID < #MaxID
This will remove a bunch of useless data in your operation. In case your ID column is a string, you can use other measures to filter data, such as the string length, the first character.
I have an Access 2003 database that data is inserted into throughout the day and I want to convert to a SQL Server database. There are many different instruments and processes that insert into the Access database, so it would take a lot of modification to change all of them to insert into SQL Server.
I know Microsoft has a tool to convert an Access database to a SQL Server one, but as far as I can tell, it can't be run automatically and it wouldn't be able to just import the new data since the last "merge" (please correct me if I am wrong)
What would be the best way to go about this problem? Maybe some sort of script?
Thanks.
There are only inserts on the Access side, and the SQL will be read-only
Then I would do this:
Link the SQL Server tables into the Access database
Run INSERT queries to copy the new data into the server tables. This assumes that all tables have primary keys.
The queries would e.g. look like this (air code):
INSERT INTO dbo_Customers
SELECT src.*
FROM Customers AS src LEFT JOIN dbo_Customers AS tgt
ON src.ID = tgt.ID
WHERE tgt.ID IS NULL
Customers is the local table, dbo_Customers the server table.
The query selects all records from Customers that don't exist in dbo_Customers.
Since several other processes access the original database, it's probably best to set up a new Access database, link both the original and the Server tables, and run the INSERT queries in the new db. You don't want any code running in the "data" db.
It can be automated e.g. with a Form_Timer procedure.
I have a database in SQL Server 2008 and the same database (backup) in SQL Server 2016. All the schema and tables are exactly the same.
I need to copy all rows from a table in SQL Server 2008 to the same table in SQL Server 2016. How to achieve this?
Linked Server
BCP
Create new DB, copy the data to that new DB, back it up, restore on target server as a new DB, update the data from Target new DB to existing target DB.
create your table on the target server using your scripts from the Script Table As / Create Script step
on the target server, you can then issue a T-SQL statement:
INSERT INTO dbo.YourTableNameHere
SELECT *
FROM [SourceServer].[SourceDatabase].dbo.YourTableNameHere
If it's a 1-time extract only, I'd not bother setting up a linked server, but instead I would use the Export Data functionality in SQL Server Management Studio.
To do this, right-click the Source Database in SSMS, then select "Tasks" -> "Export Data". You can use the wizard to specify exactly what you want to export (I.e. one or more tables, or even data using a query), specify the target, and select whether you want to run the package or save it for later use.
Another Option you can use OpenRowSet like:
SELECT * into insertTable FROM
OPENROWSET('SQLNCLI', 'Server=ServerIP\ServerExtentionNameIfhas;Database=dbname;Uid=uname;PWD=password;'
,'SET FMTONLY OFF;SET NOCOUNT ON;SELECT * FROM fromTable')
this will create automatically a table name which is inserTable and copy all rows into it.
I have two different databases in two different SQL Servers. The databases are identical in schema but contain different data in one of the tables.
I want to copy all the data from one table in one database to the same table in the other database so that I can get rid of the database from which I am copying the data.
The data is too large so I cannot create data scripts and run it onto other database.
How can I achieve this?
There are many ways like ssis transfer,select * into ,but i prefer below way if you are just transferring data
create a linked server on source server for destination server,then you could refer destination server with four part name
Assuming linked server of source is A and destination server is B,data moving is as simple as
insert into B.databasename.Schema.Table
select * from table---this is in source server and db
if data is huge and you may worry about time outs,you can write a simple script which can do in batches like
While (1=1)
begin
insert into B.databasename.Schema.Table
select top 10000* from table---this is in source server and db
if (##rowcount=0)
break
end
Creating linked server ,you can follow this
You have the following options available to you. Not all of these will work, depending on your exact requirements and the networking arrangements between the servers.
SQL Server Management Studio - Import & Export Wizard: this is accessed from the right-click menu for a database > Tasks > Import Data (or Export Data).
SQL query using a Linked Server: a Linked Server configured between the two servers allows you to reference databases on one from the other, in much the same way as if they were on the same server. Any valid SQL query approach for transferring data between two tables within one database will then work, provided you fully-qualify the table names as Server.Database.Schema.Table.
SSIS: create an SSIS package with both servers as connections, and a simple workflow to move the data from one to the other. There is plenty of information available online on how to use SSIS.
Export to flat-file format then import: this could be done using the Import/Export Wizard above or SSIS, but instead of piping the data directly between the two servers, you would output the data from the source table into a suitable flat-file format on the filesystem. CSV is the most commonly used format for this. This file can then be moved to the destination server using any file transfer approach (compressed e.g. to a Zip file if desired), and imported into the destination table.
Database backup and restore: Similar to (4), but instead of using a flat file, you could create a backup of the source database via Tasks > Back Up... You then move that backup as a file (just like the CSV approach), and restore it onto the destination server. Now you have two databases on the destination server, and can move data from one to the other locally.
I hope, this query helps you!!!
INSERT INTO [dbo].[tablename] (Column1, Column2,Column3)
(select Column1, Column2,Column3, from [Database1].[dbo].[tablename]
Thanks!!!
I have two databases, one MSSQL and the other in Access.
Right now, inside the access file, the mssql tables are set up as linked tables so queries can be written using tables from both databases. (e.g. "select * db1.table1 where db1.table1.somevalue not in db2.table1", or select into tables like that one)
These queries need to be moved into a VB.NET project, but still be linked to the access file.
I think what I am needing is a Database object that can have tables from 2 different connections, (ie the SqlClient and OleDb connections)
Is this possible? If so, how? Or do I need to rewrite the queries using loops or something?
What I would do is query your access database to get some result set and that pass that result set as a parameter to a stored procedure in your MS SQL database. You would just have to transform your results from access into XML to be passed as a xml variable as a parameter. And then in your stored procedure you can convert the XML to be a table variable and you can use it just like a regular table.
THere is no reason you can't create an MS Access .mdb with Links to your MS Access Database and your SQL Server database
Access Db #1 Contains Access Tables and Data.
SQL Db Contains your MS SQL Tables.
Access Db #2 contains links to the tables in Access DB #1 as well as links to the tables in your SQL Server Db. This .mdb files ALSO contains your query defs required by your vb.net project.
I'm pretty sure you can just connect to the Access database. All the internal objects--including the links to SQL Server tables--should be accessible to your vb.net project.