I have a Visual FoxPro table that I need to access from Sql Server. In Sql Server x86, I would just create a linked server. Unfortunately, there is no x64 driver for VFP - so Sql Server x64 can't create a linked server to it.
So far, I've come up with following options - none of which I'm particularly fond of:
Set up an x86 Sql Server to be used as a relay, so that queries go from x64 -> x86 -> VFP.
I don't really care for this, as in addition to being dev, I'm also sysadmin. So, this means I need to patch, maintain, and monitor yet another Sql Server - and possibly yet another server (assuming I don't just use a separate instance).
Also, since the VFP provider doesn't work with 4 part syntax, I have to use OPENQUERY. Thinking of all the single quote escaping that'd need to happen to have an OPENQUERY statement embedded into another OPENQUERY statement makes my head spin....
Create a CLR Table Valued Function, though the assembly would (presumably?) also be x64 - so I'd have to go out of proc (IPC? Webservice?) to actually run queries
Turns out that TVFs require a schema, so this option isn't as clean as I initially thought. I did a spike to get a WCF client into MSSQL, which returns a single column of XML that can then be parsed with the Sql XML datatype functions. It works, and is actually a little bit nicer to query than OPENQUERY since it actually takes variables as parameters. That saves me most of the single quote and EXEC dance.
Of course, WCF inside Sql is wholly unsupported, and smells like a pretty big hack. I have pretty serious reservations on performance and reliability.
Stop making queries from Sql Server to VFP, and rewrite a good bit of client code
Obviously, this is the "right" answer. But, there is a good deal of client code that relies on joins between Sql Server tables and VFP tables. Rewriting this stuff to populate a temp table or do client side joins seems like a rather large burden.
Here's hoping someone can suggest a better alternative, or some similar experiences.
It's a nasty problem, I agree.
SSIS run in 32-bit mode to import the data on a regular basis (perhaps on demand, in a job triggered by the same SP) to a SQL Server native table is another option if you can stand the delay. It would depend on the frequency of data change and problems with chance of slightly out of date data.
I think I found an alternative. Microsoft has released an updated driver for Access, which comes in both 32bit and 64bit flavors. Like the original Jet OleDB driver, this will allow you to access dBase file formats from SQL Server x64.
The only restriction is that the DBF must be in one of the dBASE formats supported by ISAM. I have done a few tests using a dBASE IV format and it seems to work, using the following connection string.
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\folder;Extended Properties=dBASE IV;User ID=Admin;Password=;
Related
I have a scenario where I get queries on a webservice that need to be executed on a database.
The source for these queries is from a physical device so I cant really change the input to my queries.
I get the queries from the device in MSSQL. Earlier the backend was in SQL Server, so things were pretty straight forward. Queries would come in and get executed as is on the DB.
Now we have migrated to Postgres and we don't have to the option to modify the input data (SQL queries).
What I want to know is. Is there any library that will do this SQL Server/T-SQL translation for me so I can run the SQL Server queries through this and execute the resulting Postgres query on the database. I searched a lot but couldn't find much that would do this. (There are libraries that convert schema from one to another but what I need is to be able to translate SQL Server queries to Postgres on the fly)
I understand there are quite a bit of nuances that will be different between SQL and postgres so a translator will be needed in between. I am open to libraries in any language(that preferably runs on linux : ) ) or if you have any other suggestions on how to go about this would also be welcome.
Thanks!
If I were in your position I would have a look on upgrading your SQL Sever to 2019 ASAP (as of today, you can find on Twitter that the officially supported production ready version is available on request). Then have a look on the Polybase feature they (re)introduced in this version. In short words it allows you to connect your MSSQL instance to other data source (like Postgres) and query the data in as they would be "normal" SQL Server DB (via T-SQL) then in the background your queries will be transformed into the native pgsql and consumed from your real source.
There is not much resources on this product (as 2019 version) yet, but it seems to be one of the most powerful features coming with this release.
This is what BOL is saying about it (unfortunately, it mostly covers the old 2016 version).
There is an excellent, yet very short presentation by Bob Ward (
Principal Architect # Microsoft) he did during SQL Bits 2019 on this topic.
The only thing I can think of that might be worth trying is SQL::Translator. It's a set of Perl modules that have been around for ages but seem to be still maintained. Whether it does what you want will depend on how detailed those queries are.
The no-brainer solution is to keep a SQL Server Express in place and introduce Triggers that call out to the Postgres database.
If this is too heavy, you can look at creating a Tabular Data Stream (TDS is SQL Server network transport) gateway with limited functionality and map each possible incoming query with any parameters to a static Postgres query. This limits any testing to a finite, small, number of cases.
This way, there is no SQL Server, and you have more control than with the trigger option.
If your terminals have a limited dialect demand then this may be practical. Attempting a general translation is very likely to be worth more than the devices cost to replace (unless you have zillions already deployed).
There is an open implementation FreeTDS that you could use if you are happy with C or Java.
We're building an application for a client with the assumption that they'd be upgrading to a minimum of SQL Server 2005 from SQL Server 2000. We're finished our application, built on 2005, and are ready to integrate. Turns out that they're not going to upgrade their DB server.
So, now we're stuck with trying to sort out what will break.
We don't have access to SQL Server 2000, so we can only change the compatibility of the database to 80.
Aside from complete testing and reviewing every stored procedure (and I've read that changing the compatibility mode is not foolproof - so testing wouldn't be bombproof), is there any other way to determine what will break? Any tools out there? Scripts?
Edit
I'd prefer not to try restoring this onto their production DB server to see what errors are spit out, so that's not a good option.
Suggest you look in Books online for the page that spells out the differences between the two and look for those things. YOu can look over the list and then search for some new keywords in the table where the sp text is stored. That will give you a beginning list.
#rwmnau noted some good ones, I'll add two more
SQL Server 2000 does not have varchar(max) or nvarchar (max), use text instead.
SQl Server 2000 also does not have SSIS - if you are creating SSIS packages to import data or move data to a data warehouse or export data, all of those need to be redone in DTS.
Also it looks to me like you can still download the free edition of SQL Server 2000:
http://www.microsoft.com/downloads/details.aspx?familyid=413744d1-a0bc-479f-bafa-e4b278eb9147&displaylang=en
You might want to do that and test on that.
I wouldn't be worried about your ANSI-SQL (setting the database compatibility level should take care of most of that), but there are a few big features you may have used that aren't available in SQL 2000 (there are many more, but these are the ones I've seen that are most popular):
Common Table Expressions (CTE) - http://msdn.microsoft.com/en-us/library/ms190766.aspx
TRY...CATCH blocks
CLR-integrated stored procs
Also, though you shouldn't be, any selections directly from system tables (objects that begin with "sys" or are in the "sys." schema) may have changed dramatically between SQL 2000 and 2005+, so I'd see if you're selecting from any of those:
SELECT *
FROM syscomments --I know, using a sys table to figure it out :)
WHERE text like '%sys%'
Also, it's worth noting that while extended support is available for a hefty fee, Microsoft has officially ended mainstream support for SQL 2000, and will end extended support in the near future. This leaves your client (and you) without any updates from Microsoft in the case of security patches, bugs, or anything else you discover. I'd strongly encourage them ot upgrade to a newer version (at least 2005), though I suspect you've already been down that road.
The application part of my database is in Access 2003, and I use tables that are linked from SQL server. Now, I have some tables that I have to link from an Oracle database. I link them through and ODBC connection and it works fine. Is it possible to link that Oracle link in SQL and then link that table as it is already linked in Access 2003? So, I want to use just one ODBC connection to SQL server and in SQL Server link that Oracle link.
Yes, I believe the double indirection structure you suggest should work "OK". That's because MS_SQL linked server sources are handled very much like local databases and can be queried individually, i.e. within queries not involving local databases.
Do note, however, that it could be much less efficient, since you introduce an extra "hop". Also, do look for possible issues with regards to type mapping as some types in oracle may get mapped to a slightly different type in SQL than when accessed directly from MS-Access. Such type mapping issues would however be easy to work-around.
Edit: To "establish a connection" between MS-SQL and Oracle servers
This concept is known as "linked server" in MS-SQL lingo. See this MSDN article for an overview and details about sp_addlinkedserver Stored Procedure. This latter document provides the connection parameters required for various sources, including Oracle or ODBC (i.e. for Oracle you can either use ODBC, which is generally easier but less efficient, and for Oracle versions 8 and up, an OLE DB driver, which as implied may be harder to confure, but provide better performance).
Again, even with the gain associated with the Oracle OLE DB driver, the extra hop may hinder the overall performance of your setup...
I've looked around and can't seem to find anything that answers this specific question.
What is the simplest way to move data from an MS SQL Server 2005 DB to a Postgres install (8.x)?
I've looked into several utilities like "Full Convert Enterprise", etc, and they all fail for one reason or another, ranging from strange errors that make it blow up to inserting nulls rather than actual data (wth?).
I'm looking at a DB with all table except for a single view, no stored procs, functions, etc.
At this point I'm about to write a small utility to do it for me, I just can't believe that's necessary. Surely there's something somewhere that can do this? I'm not even too worried about cost, although free is preferable :)
I don't know why nobody has mentioned the simplest and easiest way using robust MS SQL Server Management Studio.
Simply you just need to use the built-in SSIS Import/export feature. You can follow these steps:
Firstly, you need to install the PostgreSQL ODBC Driver for Windows. It's very important to install the correct version in terms of CPU arch (x86/x64).
Inside Management Studio, Right click on your database: Tasks -> Export Data
Choose SQL Server Native Client as the data source.
Choose .Net Framework Data Provider for ODBC as the destination driver.
Set the Connection String to your database in the following form:
Driver={PostgreSQL ODBC Driver(UNICODE)};Server=;Port=;Database=;UID=;PWD=
In the next page, you just need to select which tables you want to export. SQL Server will generate a default mapping and you are free to edit it. Probably you`ll encounter some Type Mismatch problems which take some time to solve. For example, if you have a boolean column in SQL Server you should export it as int4.
Microsoft Docs hosts a detailed description of connecting to PostgreSQL through ODBC.
PS: if you want to see your installed ODBC Driver, you need to check it via ODBC Data Source Administrator.
Take a look at the Software Catalogue. Under Administration/development tools I see DBConvert for MS SQL & PostgreSQL. Probably there are other similar tools listed.
You can use the MS DTS functionality (renamed to SSIS in the latest version I think). One issue with the DTS is that I've been unable to make it do a commit after each row when loading the data into pg. Which is fine if you only have a couple of 100k rows or so, but it's really very slow.
I usually end up writing a small script that dumps the data out of SQLServer in CSV format, and then use COPY WITH CSV on the PostgreSQL side.
Both those only take care of the data though. Taking care of the schema is a bit harder, since datatypes don't necessarily map straight over. But it can easily be scripted together with a static load of the schema. If the schema is simple (just varchar/int datatypes for example), that part can also easily be scripted off the data in INFORMATION_SCHEMA.
Well there are .NET bindings for MS SQL Server 2005 (obviously) and also for PostgreSQL. So it would only take a few lines of code to code up a program that could transfer data safely from one to the other. The view would probably have to be done manually as Postgres doesn't use the same language for views as SQL Server.
This answer is to help summarize current connection string because someone may overlooked the comment.
Current version of ODBC connection string is:
For 32-bit system
Driver={PostgreSQL UNICODE};Server=192.168.1.xxx;Port=5432;Database=yourDBname;Uid=postgres;Pwd=admin;
For 64-bit system
Driver={PostgreSQL UNICODE(x64)};Server=192.168.1.xxx;Port=5432;Database=yourDBname;Uid=postgres;Pwd=admin;
You can check the driver name by typing ODBC in windows search.
And open ODBC Data Source Administrator
I have a SQL Server 2005 machine with a JDE DB2 set up as a linked server.
For some reason the performance of any queries from this box to the db2 box are horrible.
For example. The following takes 7 mins to run from Management Studio
SELECT *
FROM F42119
WHERE SDUPMJ >= 107256
Whereas it takes seconds to run in iSeries Navigator
Any thoughts? I'm assuming some config issue.
In certain searches SQL Server will decide to pull the entire table down to itself and sort and search the data within SQL Server instead of sending the query to the remote server. This is usually a problem with collation settings.
Make sure the provider has the following options set:
Data Access,
Collation Compatible,
Use Remote Collation
Then create a new Linked Server using the provider and select the following provider options
Dynamic Parameters,
Nested Queries,
Allow In Process
After setting the options change the query slightly to get a new query plan.
It might be a memory issue on your SQL Server machine. I recently learned that linked server queries use memory allocation by the OS. Whereas native SQL Server queries use memory pre-allocated by SQL Server. If your SQL Server machine is configured to use 90% or more of the server's memory, I would scale that back a bit. Maybe 60% is the right place to be.
Another thing to check is the SQL Server processor priority. Make sure "Boost SQL Server priority" is not enabled.
I assume you are going through ODBC for access. Remember that you are not writing native db2 queries here, but instead ODBC sql queries. If you only need read-only data, you may want to try configuring your ODBC datasource to read-only mode (if that is an option).
In a project with DB2 integration, I replaced every query via direct select or view by stored procedures calling the OPENQUERY function.
My interpretation is that SqlServer fetches the whole table before applying the WHERE conditions, whereas OPENQUERY passes the SQL statement directly to the db driver.
Anyway, performance was acceptable after the modifications.
My first thought would go to the drivers. Years ago I had to link DB2 to SQL Server 2000 and it was extremely difficult to find the correct combination of drivers and setup parameters that would work...
So maybe I'm biased because of that, but I would try upgrading or downgrading the driver or changing the setup so that the DB2 driver can run INPROC (if it's not already doing so).
I've had several issues with DB2 as a linked a server. I do not know if it will address your problems, but here is what fixed mine:
1) Enabled lazy close support and pre-fetch during EXECUTE in the ODBC settings
2) Add "FOR FETCH ONLY" on all selects
3) Query using the SELECT * FROM OPENROWSET(LinkedServerName, 'SQL Command') method