I have a scenario where I get queries on a webservice that need to be executed on a database.
The source for these queries is from a physical device so I cant really change the input to my queries.
I get the queries from the device in MSSQL. Earlier the backend was in SQL Server, so things were pretty straight forward. Queries would come in and get executed as is on the DB.
Now we have migrated to Postgres and we don't have to the option to modify the input data (SQL queries).
What I want to know is. Is there any library that will do this SQL Server/T-SQL translation for me so I can run the SQL Server queries through this and execute the resulting Postgres query on the database. I searched a lot but couldn't find much that would do this. (There are libraries that convert schema from one to another but what I need is to be able to translate SQL Server queries to Postgres on the fly)
I understand there are quite a bit of nuances that will be different between SQL and postgres so a translator will be needed in between. I am open to libraries in any language(that preferably runs on linux : ) ) or if you have any other suggestions on how to go about this would also be welcome.
Thanks!
If I were in your position I would have a look on upgrading your SQL Sever to 2019 ASAP (as of today, you can find on Twitter that the officially supported production ready version is available on request). Then have a look on the Polybase feature they (re)introduced in this version. In short words it allows you to connect your MSSQL instance to other data source (like Postgres) and query the data in as they would be "normal" SQL Server DB (via T-SQL) then in the background your queries will be transformed into the native pgsql and consumed from your real source.
There is not much resources on this product (as 2019 version) yet, but it seems to be one of the most powerful features coming with this release.
This is what BOL is saying about it (unfortunately, it mostly covers the old 2016 version).
There is an excellent, yet very short presentation by Bob Ward (
Principal Architect # Microsoft) he did during SQL Bits 2019 on this topic.
The only thing I can think of that might be worth trying is SQL::Translator. It's a set of Perl modules that have been around for ages but seem to be still maintained. Whether it does what you want will depend on how detailed those queries are.
The no-brainer solution is to keep a SQL Server Express in place and introduce Triggers that call out to the Postgres database.
If this is too heavy, you can look at creating a Tabular Data Stream (TDS is SQL Server network transport) gateway with limited functionality and map each possible incoming query with any parameters to a static Postgres query. This limits any testing to a finite, small, number of cases.
This way, there is no SQL Server, and you have more control than with the trigger option.
If your terminals have a limited dialect demand then this may be practical. Attempting a general translation is very likely to be worth more than the devices cost to replace (unless you have zillions already deployed).
There is an open implementation FreeTDS that you could use if you are happy with C or Java.
Related
I am a developer and performance tester but not a DBA. My team is working on a performance testing tool that is specific to our software. One of the features we want it to have is the ability to generate a database report immediately after the test. Our software is database agnostic. For Oracle, I can easily create a snapshot id before and after the test and programmatically create an AWR report for those snapshots, write to a file and save with other artifacts we gather. Works great.
For SQL Server, however, there is no AWR equivalent (that I know of). I know the MDW as part of the SSMS has a UI for getting things like top 10 slow SQL and things like that. But, I have not yet found a way to programmatically create and extract a SQL performance report (preferably similar to Oracle's AWR) for SQL Server.
I am even willing to create the report myself if I can find a way to extract the raw data.
Any ideas would be greatly appreciated because searching online is not getting me anywhere.
P.S. I'm trying to do this in Java, by the way, but will accept help in any language. Thanks again!
Good news! In SQL Server 2016, you can use Query Store. This is like your flight recorder blackbox.. finding long running queries and waits. Capture baseline built in to SQL Server. You can compare before and after hardware changes and/or upgrades on queries. Maybe this similar to Oracle AWR.
Only available SQL Server 2016 and up.
Is there seriously no way of using a shared access non-server driven database file format without having to use an SQL Server? The Entity Framework is great, and it's not until I've completely finished designing my database model, getting SQL Server Compact Edition 4.0 to work with Visual Studio that I find out that it basically cannot be run off a network drive and be used by multiple users. I appreciate I should have done some research!
The only other way as far as I can tell is to have to set up an SQL server, something which I doubt I would be able to do. I'm searching for possible ways to use it with Access databases (which can be shared on a network drive) but this seems either difficult or impossible.
Would I have to go back to typed DataSets or even manually coding the SQL code?
Another alternative is to try using SQL
Install SQL Server express. Access is not supported by EF at all and my experience with file based databases (Access, SQL Server CE) is mostly:
If you need some very small mostly readonly data to persist in database you can use them (good for code tables but in the same time such data can be simply stored in XML).
If you expect some concurrent traffic and often writing into DB + larger data sets their performance and usability drops quickly. They are mostly useful for local storage for single user.
I'm not sure how this relates for example to SQLite. To generate database from model for SQLite you need special T4 template (using correct SQL syntax).
Have you tried SQLite? It has a SQL provider, and as far as I know EF supports any provider. Since it's file-based, that might be a plausible solution. It's also free.
We're building an application for a client with the assumption that they'd be upgrading to a minimum of SQL Server 2005 from SQL Server 2000. We're finished our application, built on 2005, and are ready to integrate. Turns out that they're not going to upgrade their DB server.
So, now we're stuck with trying to sort out what will break.
We don't have access to SQL Server 2000, so we can only change the compatibility of the database to 80.
Aside from complete testing and reviewing every stored procedure (and I've read that changing the compatibility mode is not foolproof - so testing wouldn't be bombproof), is there any other way to determine what will break? Any tools out there? Scripts?
Edit
I'd prefer not to try restoring this onto their production DB server to see what errors are spit out, so that's not a good option.
Suggest you look in Books online for the page that spells out the differences between the two and look for those things. YOu can look over the list and then search for some new keywords in the table where the sp text is stored. That will give you a beginning list.
#rwmnau noted some good ones, I'll add two more
SQL Server 2000 does not have varchar(max) or nvarchar (max), use text instead.
SQl Server 2000 also does not have SSIS - if you are creating SSIS packages to import data or move data to a data warehouse or export data, all of those need to be redone in DTS.
Also it looks to me like you can still download the free edition of SQL Server 2000:
http://www.microsoft.com/downloads/details.aspx?familyid=413744d1-a0bc-479f-bafa-e4b278eb9147&displaylang=en
You might want to do that and test on that.
I wouldn't be worried about your ANSI-SQL (setting the database compatibility level should take care of most of that), but there are a few big features you may have used that aren't available in SQL 2000 (there are many more, but these are the ones I've seen that are most popular):
Common Table Expressions (CTE) - http://msdn.microsoft.com/en-us/library/ms190766.aspx
TRY...CATCH blocks
CLR-integrated stored procs
Also, though you shouldn't be, any selections directly from system tables (objects that begin with "sys" or are in the "sys." schema) may have changed dramatically between SQL 2000 and 2005+, so I'd see if you're selecting from any of those:
SELECT *
FROM syscomments --I know, using a sys table to figure it out :)
WHERE text like '%sys%'
Also, it's worth noting that while extended support is available for a hefty fee, Microsoft has officially ended mainstream support for SQL 2000, and will end extended support in the near future. This leaves your client (and you) without any updates from Microsoft in the case of security patches, bugs, or anything else you discover. I'd strongly encourage them ot upgrade to a newer version (at least 2005), though I suspect you've already been down that road.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Has anyone used all three of these databases? What are your experiences with them? PostgreSQL looks pretty tempting for a project but I'm curious to learn more about it (We're a .NET Shop). I've also heard of quite a lot of people being satisfied with DB2.
I work in a very large organization that uses DB2 primarily on Linux (Red Hat). We have a number of large databases and have investigated moving to other RDBMS solutions, such as Oracle and SQL Server. I did a lot of work on the SQL server end of that.
We found that SQL server performs much better and requires less tuning than DB2, especially when tables frow larger than 1M records. HADR is also difficult and bulky, to say the least.
We found many differences between DB2 and SQL Server, and too many to list here. I was responsible for doing a lot of the engineering behind code conversion from one platform to another, and can't say I found anything in DB2 to be superior to SQL server, but did find many things I liked better about SQL server. Here are some off the top of my head:
Better selection data types in SQL Server, such as MONEY and SMALL MONEY.
Mixed character encoding in SQL Server. Some columns can be ANSI and others UNICODE (char and nchar, respectively). Setting this up in DB2 is neither straightforward nor easy.
Better tools in SQL server, mainly SSIS for ETL (As opposed to the insanely priced IBM Data Stage).
SQL server has more forgiving syntax. For example, you don't need semi-colons everywhere. Maybe just a personal preference but I found it much easier to code in T-SQL.
Many advanced features seem to work better in SQL server. For example, SQL server lets you do page-level compression, where DB2 is limited to row-level compression.
It was easier to tune SQL Server queries with the SQL Server IDE
There are more, but honestly I suggest that anyone who is considering one over the other should set both up and spend some time working with both systems. Right now it seems like SQL server is a better overall solution, but DB2 may one day take the crown.
Lastly, when dealing with data warehousing, SQL, SSIS, and SSAS made a much better solution than InfoSphere, DataStage, and DB2. I could write a whitepaper on it, but my suggestion here is to set it up on your own and spend a week or to playing with each solution. Microsoft's solution here was faster and cheaper than IBM's. I don't know of any other basis upon which to make a decision.
Platform shouldn't be an issue since databases generally run on their own machines, but there are always those "no microsoft!" and "no linux!" shops around. It's a shame, really. I'd recommend SQL server.
As a DB2 person, I can offer a few details about what you could expect from running DB2 for Windows and developing .NET applications for it. The following details were current as of version 9.7, which was released in June 2009.
Drivers and API support for just about any Windows programming language and IDE, including .NET and Visual Studio extensions
A no-cost, production-ready database engine (Express-C) that has no database size limit and is the least restricted when compared to Oracle Express and SQL Server Express
A self-tuning database engine for Windows that automatically handles the sizing of several memory buffers that are critical to good performance
Rock-solid support for XML as a native datatype, handled by its own dedicated query engine that is optimized for XML's hierarchical nature. Queries can access any combination of XML and tabular data with any combination of SQL and XQuery expressions
Avoid microsoft like the plague. Always push to use PostgreSql even on windows, way better support for developing applications for e.g. Java/Python and still has good support in .NET. Also of course is completely free which given the current license fees for SQL Server is nothing to be sniffed at even if you're a multi-million dollar company.
For the cost of 1 SQL Server license you are going to save £30,000 (say $40,000) or more - buy better hardware to run Postgres on and still have a net benefit.
As far as performance, really if this is such a massive issue we should not be using either DB2, SQL Server or Postgres anyway. The difference between the three is negligible for their design purposes.
Edit: On the .NET integration, actually this is really poor in SQL Server anyway, it does have more features than Postgres/DB2 admittedly but it's not really hugely advantageous over SSIS or stored procedures. I could see the main use case in my work as accessing classes and functions from a CLR .dll but then you're implementing logic in the database which may or may not be a good idea for you.
If you're a .NET shop, and are either using a small database (i.e. Sql Server Express), or have the money for the full SQL Server, use it. SQL Server will perform better than PostgreSQL for most actions, and about the same as DB2.
PostgreSQL is fantastic if you need multi-platform support, are Linux-based, or need a free product that's not Microsoft.
I haven't used DB2 in over 10 years, other than running an in-house performance test vs. other databases (where it was about the same for a transactional database as Oracle/SQL Server, where were better than MySQL, PostgreSQL, etc).
If you are a .net shop stay with SQL Server.
Using any other database platform would require non-Windows to get the best out of it. On Windows, SQL Server is king simply because MS own both OS and SQL Server (Like Oracle/Red Hat).
Hello,
I'm new here, so sorry, if my question is too basic. However, maybe you have some advice, example, links, which could help me... I'm trying to find something helpfull for few days, but no results as for now.
I'm working in a distributed environment. I have a Oracle server hundreds of miles away and a MS SQL server close to me. I'm writing a application using Visual Web Developer 2008 Express. I need some data from Oracle. It's not worth to query the Oracle server every time i need some data from it. I'd prefer to run some Oracle queries once each night and store results in some local (SQL Server) tables. I assume, I should run queries through standard windows scheduler (Windows Server 2008). I have the basic connectivity - I can open Oracle Database from local Visual Studio.
The questions are:
How to write a query/procedure/function that would get data from Oracle and put them into a SQL Server table (possibly recreated before each query run)?
How can I run such a query from command line (or in other way run from scheduler)
What naming conventions are applicable? In VS I use something like //IP.IP.IP.IP/Name and a user with password.
Thanks for any help or advice.
Regards,
Matteo
I suggest you speak to the DBA's of the Oracle and SQL Server databases, as there may be other considerations you need to bear in mind. (Data Integrity, Security, ownership etc.)
One route you could follow would be to implement DTS (For older databases) or SSIS (for new versions of SQL Server) processes to copy the data across on the schedule you want. (This is pretty much what they were built for.)
How much data are we talking about?
If there is a small quantity that you need to transfer every day, you can write a stupid fetch and insert script in language of your choice.
You only need to search for better solutions if "sync" would take too much resources.
Thanks...
I'm the DBA for the SQL Server, which will serve only for my application. For Oracle I just want to read data and I have enough privileges and agreement with DBA's. Security, ownership and integrity are not an issue for now. I just need some technical advise how to get data from Oracle to MSSQL tables on a schedule.
I use MS SQL Server 2008 Express SP1. I'm very close to solve my problem - I have established connections and everything installed and working. I just don't know, how to run a query, which would get data from Oracle and put into MSSQL, on regular basis, without manual interaction.
I've some experience in programming, but not much in databases (except creating complex SQl queries). Therefore some example or links to detailed description would be helpful. I'm not sure about naming conventions, differences between procedures, functions and queries, command line options to run db automation procedures and so on. I'm also not sure, about which mechanisms or technologies are available in MS SQL Server 2008 Express edition.