My goal is to migrate table with all its rows into sybase database.
I want to use bulk like using the sqlBulk object, my problem is that i don't think that odbcBulk object exist, i've searched the net and all i found there is some reference to aseBulk, can someone post a code of how to create this aseBulk object and use it in order to save data or datatable from sql server to sybase
Related
I have a database in SQL Server which contains collected data during one day, and a database in PostgreSQL with OSM data. I need to modify collected data in order to create reports for my users.
Now, I imagined that somehow call PostgreSQL procedure from SQL Server, pass collected data to PostgreSQL, do something with that data, and return another result set to SQL Server for creating reports.
What is the 'most efficient' way for achieving this? OR, better question atm, what is the way for achieving this functionality?
My idea is to connect SQL Server and PostgreSQL with PostgreSQL ODBC driver, then copy data from SQL Server to PostgreSQL table, run that stored procedure on PostgreSQL, and return data to SQL Server result table. But, it is not scheduled task. Data to be transferred to PostgreSQL contains latitude, longitude and bearing for about 2-3 million of rows and function which analyses them requires one per one record, not all at once.
My way is using C# and npgsql to connect to postgres.
I create an app:
check SQL Server every min.
check Postgres for the last id inserted.
create a dataset from MSQL with the news ids
insert the new records into Postgres.
run postgres store procedure to generata new data
create separated webservice to consume the report generated on postgres.
I open a Foxpro file using OleDb
Dim connection As New OleDbConnection(String.Format("Provider=VFPOLEDB.1;Data Source={0};", dbfPath))
connection.Open()
Dim tables As DataTable = connection.GetSchema(System.Data.OleDb.OleDbMetaDataCollectionNames.Tables)
Then I analyze the structure of the FoxPro data
For Each rowTables As System.Data.DataRow In tables.Rows
Dim columns As DataTable = connection.GetSchema(System.Data.OleDb.OleDbMetaDataCollectionNames.Columns, New [String]() {Nothing, Nothing, rowTables("table_name").ToString(), Nothing})
For Each rowColumns As System.Data.DataRow In columns.Rows
Console.Out.WriteLine(String.Format("{0};{1};{2};{3};{4};{5}", rowTables("table_name").ToString(), rowColumns("column_name").ToString(), rowColumns("data_type").ToString(), rowColumns("CHARACTER_MAXIMUM_LENGTH").ToString(), rowColumns("NUMERIC_PRECISION").ToString(), rowColumns("NUMERIC_SCALE").ToString()))
Next
Next
Now I must create a Sql server table with a compatible structure in order to load the FoxPro data (I'll use a DataReader and SqlBulkCopy).
I could use the information collected inside the For Each to build an SQL script to create the table but I would like to know if there is a smarter way to automatically map data types of FoxPro on Sql server, avoiding a Select Case that implements this table https://msdn.microsoft.com/en-us/library/ms130984.aspx
By project requirement I cannot use Entity framework (and i think it is not the right tool to load millions of records).
If you have VFP, and this will be a one time upgrade process, have you looked into VFP from the menu Tools -> Wizards -> Upsizing?
This will allow you to define a connection to SQL Server, then pick your database you want to upsize to. Pick the tables and go.
If your tables are "free" tables (not associated with an actual database container), I think you can still upsize them, but it has been a while since I dug into that process, but can offer additional direction if this is an avenue you have available to run with.
I have a table in sql server, and another table in ms access. The two tables are different tables. but one column in sql server need fresh data from the table in ms access. Is that possible that when ms access have update or new record, the table in sql server will be updated automatically.
Absolutely yes! Just link the SQL Server table in Access via ODBC (after setting up a DSN). On ribbon, see External Data tab (ODBC).
Then, run an append and update query in the After Insert or After Update VBA trigger event on form from the local table into the linked table.
Just wondering what is best practice to achieve:
We have stored procedure that runs in SQL Server and needs to do some calculations with a huge fact table to be stored on Netezza.
The flow:
Stored procedure will create temp tables on SQL Server
These will be sent to Netezza to be joined with the fact table
Calculations will be made in Netezza
Results will be passed back to SQL Server
What are the ways to pass the temp tables from SQL Server to Netezza?
Thanks
The Only way that I know of to get data in and out of Netezza is through flat files, or using the ODBC drivers to work with Netezza directly and hold a recordset in memerory and write to Netezza through an odbc connection..
Regardless of if you are on Windows or Unix, Netezza comes with client tools that you can use to connect to Netezza. Fyi, don't bother looking for them on the internet. You will have to get them from Netezza directly, or from the person that manages that relationship.
I would suggest looking into how I might be able to use the ODBC drives in SSIS to do the work for you. I'm not a pro at SSIS so I can't say I would know how to do that, but I would look into that first.
If I had to accomplish the task I would write something in C# to perform the following tasks.
Create flat files from sql server
Connect to Netezza Create external table that links to the flat file.
Call procedures in Netezza to do the work and generate the data for export in a temp table.
Export the new data to a flat file and import that back into sql server.
Now that I think about it you might also try the following, it is untested however. I wonder if you can create a linked table in sql server and an external table in Netezza that uses the same flat file. Baring a file lock, if they can, you can create a quasi-link to netezza from sql server.
To find out more about external tables in netezza. Look in the doc Netezza User Guide in chapter 5.
Netezza User Guide
Also, if you are interested in the coding side there is a very good link below to how to connect to netezza via c#.
Stack Post
I ended up using some of that post to build the method below to execute commands against Netezza.
OdbcDataReader GetReaderForCommand(string strCmd, string dbname)
{
var conn = new OdbcConnection();
conn.ConnectionString = "Driver={NetezzaSQL};servername=<servername>;port=5480;database="+dbname+"; username=<username>;password=<pwd>;";
OdbcDataReader rdr = null;
conn.Open();
System.Data.Odbc.OdbcCommand cmd = new System.Data.Odbc.OdbcCommand(strCmd, conn);
rdr = cmd.ExecuteReader();
return rdr;
}
Lastly, here are a couple of links that I would follow up with.
enzeecommunity.com - User base to search and ask questions of.
A free management studio to use with Netezza.
http://www.aginity.com/ProductivityTools/WorkbenchOverview.aspx -
I have two databases, one MSSQL and the other in Access.
Right now, inside the access file, the mssql tables are set up as linked tables so queries can be written using tables from both databases. (e.g. "select * db1.table1 where db1.table1.somevalue not in db2.table1", or select into tables like that one)
These queries need to be moved into a VB.NET project, but still be linked to the access file.
I think what I am needing is a Database object that can have tables from 2 different connections, (ie the SqlClient and OleDb connections)
Is this possible? If so, how? Or do I need to rewrite the queries using loops or something?
What I would do is query your access database to get some result set and that pass that result set as a parameter to a stored procedure in your MS SQL database. You would just have to transform your results from access into XML to be passed as a xml variable as a parameter. And then in your stored procedure you can convert the XML to be a table variable and you can use it just like a regular table.
THere is no reason you can't create an MS Access .mdb with Links to your MS Access Database and your SQL Server database
Access Db #1 Contains Access Tables and Data.
SQL Db Contains your MS SQL Tables.
Access Db #2 contains links to the tables in Access DB #1 as well as links to the tables in your SQL Server Db. This .mdb files ALSO contains your query defs required by your vb.net project.
I'm pretty sure you can just connect to the Access database. All the internal objects--including the links to SQL Server tables--should be accessible to your vb.net project.