At work we are having a bit of trouble with spatial support of SQL Server 2008.
In SQL Server 2008 we have a big system in production going on, managing a bunch of important stuff. In some of the tables, I have pairs of coordinates and need to display them to ArcGIS and other GIS software.
My question here really is: Is it possible to use DBI-Link (PostgreSQL tool) to connect to SQL Server 2008?
What kind of performance loss should I expect? I don't expect to conduct complicated queries. It's just a matter of reading from PostgreSQL a view inside SQL Server 2008 (a simple view, such as SELECT * FROM foo).
So, what are you thoughts about this? I know this is a bit haxor solution, but inside SQL Server I lose a lot of spatial handling functions, and all my databases in SQL Server stores are coordinate pairs.
Yes, that should work fine, as long as you have a DBI driver properly set up.
Performance - depends on what you're doing. DBI-link doesn't have the ability to push down restrictions, so if your view is on "SELECT * FROM foo", it will always do that. If your app does "SELECT * FROM myview WHERE pk=1", it will still request the whole table with SELECT * and then filter it on the pg side. You may be better off using functions that can adapt the query.
As long as your queries don't shuffle lots of data, performance is usually pretty decent.
Related
I am a developer and performance tester but not a DBA. My team is working on a performance testing tool that is specific to our software. One of the features we want it to have is the ability to generate a database report immediately after the test. Our software is database agnostic. For Oracle, I can easily create a snapshot id before and after the test and programmatically create an AWR report for those snapshots, write to a file and save with other artifacts we gather. Works great.
For SQL Server, however, there is no AWR equivalent (that I know of). I know the MDW as part of the SSMS has a UI for getting things like top 10 slow SQL and things like that. But, I have not yet found a way to programmatically create and extract a SQL performance report (preferably similar to Oracle's AWR) for SQL Server.
I am even willing to create the report myself if I can find a way to extract the raw data.
Any ideas would be greatly appreciated because searching online is not getting me anywhere.
P.S. I'm trying to do this in Java, by the way, but will accept help in any language. Thanks again!
Good news! In SQL Server 2016, you can use Query Store. This is like your flight recorder blackbox.. finding long running queries and waits. Capture baseline built in to SQL Server. You can compare before and after hardware changes and/or upgrades on queries. Maybe this similar to Oracle AWR.
Only available SQL Server 2016 and up.
I have a scenario where I get queries on a webservice that need to be executed on a database.
The source for these queries is from a physical device so I cant really change the input to my queries.
I get the queries from the device in MSSQL. Earlier the backend was in SQL Server, so things were pretty straight forward. Queries would come in and get executed as is on the DB.
Now we have migrated to Postgres and we don't have to the option to modify the input data (SQL queries).
What I want to know is. Is there any library that will do this SQL Server/T-SQL translation for me so I can run the SQL Server queries through this and execute the resulting Postgres query on the database. I searched a lot but couldn't find much that would do this. (There are libraries that convert schema from one to another but what I need is to be able to translate SQL Server queries to Postgres on the fly)
I understand there are quite a bit of nuances that will be different between SQL and postgres so a translator will be needed in between. I am open to libraries in any language(that preferably runs on linux : ) ) or if you have any other suggestions on how to go about this would also be welcome.
Thanks!
If I were in your position I would have a look on upgrading your SQL Sever to 2019 ASAP (as of today, you can find on Twitter that the officially supported production ready version is available on request). Then have a look on the Polybase feature they (re)introduced in this version. In short words it allows you to connect your MSSQL instance to other data source (like Postgres) and query the data in as they would be "normal" SQL Server DB (via T-SQL) then in the background your queries will be transformed into the native pgsql and consumed from your real source.
There is not much resources on this product (as 2019 version) yet, but it seems to be one of the most powerful features coming with this release.
This is what BOL is saying about it (unfortunately, it mostly covers the old 2016 version).
There is an excellent, yet very short presentation by Bob Ward (
Principal Architect # Microsoft) he did during SQL Bits 2019 on this topic.
The only thing I can think of that might be worth trying is SQL::Translator. It's a set of Perl modules that have been around for ages but seem to be still maintained. Whether it does what you want will depend on how detailed those queries are.
The no-brainer solution is to keep a SQL Server Express in place and introduce Triggers that call out to the Postgres database.
If this is too heavy, you can look at creating a Tabular Data Stream (TDS is SQL Server network transport) gateway with limited functionality and map each possible incoming query with any parameters to a static Postgres query. This limits any testing to a finite, small, number of cases.
This way, there is no SQL Server, and you have more control than with the trigger option.
If your terminals have a limited dialect demand then this may be practical. Attempting a general translation is very likely to be worth more than the devices cost to replace (unless you have zillions already deployed).
There is an open implementation FreeTDS that you could use if you are happy with C or Java.
I've just migrated from Access where I would "write" SQL constantly and quickly using their GUI layout. I'm very new to SQL Server and am puzzled why is there no query GUI in SQL Server. Is it just that much more powerful that a GUI wouldn't be able to address the majority of queries? Or is it like a purist thing where once one is no longer in watered down databases like Access that he or she should just know SQL well enough to write it on the fly. Or am wrong altogether, and there IS a graphical way that most programmers compose SQL?
Sorry for the umbrella question, but I think some theory here would help me understand the big picture better.
I think there is a Query Designer as part of Sql Server Management Studio: see for example SQL Server 2005 Management Studio Query Designer.
The SQL Server is a database server. That it self does not have any editor text or GUI. You can mange it from the console, or any dedicated software that might have or not a query designer.
The main difference between Access and SQL Server or any other database, is that the MS Access is application that store the data in a file, when You close "Access" anything can happen inside the database by it self, while you are using SQL Server it is alway on You only connect to it. That why access have build nice query, report form designers. The SQL Server has totally different job to do and does not need this. With SQL Server we have Sql Server Management Studio, whit serve to global management not only for table creation and query design, there is a lot of background staff going on right there for example user management, rights jobs.
So it is not that SQL Server does not have a query designer, You just don't use proper software to work with it. A power full application is SSMS that ChrisW presented.
Actually there is a GUI-Designer for Views and Queries. Just use the SQL Server Management Studio.
For Example on the "Views" right-click and choose "Add View...". The view GUI is looking very similar to the one in access.
AFAIK about MySQL. Earlier there was the MySQL suite of tools and now there is the MySQL Workbench.
Say what you may, but it's definitely a timesaver.
First off, I believe the query designer in SQL Server Management Studio is probably the equivalent of what you've used in Access. I'm not terribly well-versed in using Access except through code, but I think you can do what you want there (in query designer). I've never found it to be very flexible, so I always end up back in the code itself, but there you go.
My other answer is a little bit off topic (and posted at this point mostly for googlers), but if you're a developer (especially using SQL Server), it probably still applies: Linq to SQL in Visual Studio.
I've been developing against SQL Server databases for about 7 years now, and most of that time has been spent writing the SQL statements out by hand. Last year, I changed jobs and finally had an environment where the databases my programs used were local and didn't change structure. I finally had a good opportunity to begin using Linq!
Linq is an amazingly useful tool!!! I now use it for almost (not quite) all of my database interactions from code.
The basic gist is that you create a graphical database layout in a dbml file. This layout includes tables (or views) as objects and association (PK/FK relationships) as links between the objects. How you set up the associations determines whether that property is a single object or an entity set of that object. Imagine you've got the following:
TABLE (ORDERS) TABLE (CUSTOMERS)
-------------- ----------------
PrimKey
CustomerID --(FK)--> PrimKey
OrderDate CompanyName
Using Linq to SQL, when you run something like the following:
Using db as new DataBaseContext
Dim orders = (From iOrder In db.ORDERS Order By iOrder.Customer.CompanyName).ToList
End Using
You end up with a list of Order objects that each own its corresponding Customer object. You can set this up to have object hierarchies and it pretty much works like you'd expect.
Anyway, it's a fairly big topic and more to get into than is appropriate here, but suffice to say that I think it's a great technology to bridge graphical interaction with true, nitty-gritty code.
I m using Access 97 database.It has lots of forms. Data bulked. I have to upgrade it quickly. I bought SQL Server 2005 enterprise edition.
I want to use SQL Server for data holder. I m going to use Access forms regularly. I just want to export data to the sql server.
Is it possible to use "Linked" data storing?
While I agree with HLGEMs first paragraph I respectfully disagree with HLGEMs second paragraph. There are quirks you need to know about of which I'm somewhat ignorant. Such as changing boolean fields to LittleInt. But otherwise it's a lot of tedious work to recreate the database schema. And it'll be error prone such as missing indexes or relationships.
There is a tool from the SQL Server group which is a lot better than the Upsizing Wizard especially the Access 97 version.
SQL Server Migration Assistant for Access (SSMA Access)
http://www.microsoft.com/sql/solutions/migration/access/default.mspx
As you discover these quirks you can change the scripts to recreate the database with the appropriate changes.
I concur with Tony Toews (and you should trust him on this, he's an Access guru): use SSMA to help you move data to SQL Server, it does a more complete job than the upsizing Wizard integrated in Access (which doesn't work for upsizing to SQL Server 2008 anyway).
You have to be wary of a few caveat though; I've made a blog post about some of the things you should check out.
The point is that if the original Access database was designed without relying too much on the liberties that Access allows (strange characters in table and column names for instance), then the process will be much easier.
Pay special attention to all the warning and errors reported by SSMA, they are really useful in helping you focus on the issues you must solve.
With regards to performance, moving to SQL Server isn't necessarily going to make things faster.
In some areas it will actually be slower, sometimes much much slower:
Access is pretty good at optimizing certain forms of data access but once the database moves outside of its reach, it doesn't have as much control.
Most things will work fine though.
You will probably have to rewrite a few queries, maybe move them as views on SQL Server instead of keeping them in your Access application.
Little things such as using % instead of * as wildchars in queries using LIKE in their WHERE clause can also cause strange issues like queries not returning any records.
By the way, I'll post a very good resource Tony has on his own website regarding SQL upsizing: My random thoughts on SQL Server Upsizing from Microsoft Access.
There is also a good and detailed read about things to consider when using SQL Server from Access: Optimizing Microsoft Office Access Applications Linked to SQL Server
YOu can add the SQL Server tables to access as linked tables. Then you will want to start looking at your slowest queries and convert them to stored procs.
Do not use the upgrade wizard in Access to create the SQl Server tables becasue it will make poor choices for datatypes. Do the work yourself to create the scripts, choosing the best datatypes. It takes longer this way, but your database will perform better and you will gain a better understanding of how to do things in SQL Server. You should start right now, learning to do everything through a script and never from the GUI. Best to learn good habits in SQL Server from the start.
Microsoft Access is a slick way to access data in a MS SQL Server backend database, but I've always had problems accessing (so to speak) large tables of data, especially when trying to toggle between results and design mode in Access.
Access gives me a number of nifty things, not the least of which is Crosstabs, but this hung connection to the server drives me a little crazy!
Does any MS Access gurus know how to optimize the ODBC connection so it isn't doing what appears to be full table scans when I just want to tweak and build my queries?
The ODBC driver will pass as much work as possible to SQL Server but as soon as you use a vba function like Nz or non-SQL Server syntax like PIVOT then the ODBC driver must pull back more data and indexes to get the work done on the client side.
As per other answer either build your views in SQL Server and link to the views or else use an Access Data Project.
NB: PIVOT queries with unknown number of columns cannot be handled in SQL Server in the same way that Access will do this natively - so if you run a pivot in Access against SQL Server data you will likely pull the whole table back. Pivot queries must be built in SQL Server using dynamic SQL techniques or else pre-saved views that have all the columns hard coded. Check out this link for one way to do this:
http://www.sqlservercentral.com/articles/Advanced+Querying/pivottableformicrosoftsqlserver/2434/
As others have said, the only way to improve performance on large tables is to have the SQL Server database engine do the work for you. A method of doing this which hasn't been mentioned is to use a pass-through query, which will enable you to keep all your code in MS Access, without having to create objects on the SQL Server:
http://support.microsoft.com/kb/303968
You will have to write SQL Server T-SQL rather than the Access dialect; however, SQL 2005 (when running in compatibility mode 90) does support a PIVOT command.
My similar problem was that the ORACLE ODBC connection hung after selecting the Link table/ODBC connection. Task manager said not responding after 10's of minutes. The connection then pings ORACLE for all available tables. I had turned on logging on the ORACLE ODBC Administrator, so it had to write all these things to the log, slowing any results by perhaps hours. The log was 60 MB one hour later, when I turned it off, then everything was fine!
To turn it off go to the Oracle installation/Network Administration/MS ODBC Adminstrator/Tracing tab and turn it OFF!
A good resource on ODBC is here: http://eis.bris.ac.uk/~ccmjs/odbc_section.html
Unfortunately Access is not able to push a lot of that work to the server, and yes, it will do huge table scans when designing queries against multiple tables or views in SQL Server.
You can build and tweak queries (views) in SQL Server using SSMS and store the views in SQL Server for a massive performance boost and still use Access for your front end.