I have a rather large query that builds three tables and union all's them together. All of the tables used for this are remote, the output is used locally.
Would I see a performance gain by creating a view on the remote server with my union query, then selecting from that, rather than have a lot of remote calls from my local query?
Related
My goal is to pull data from 5 tables into one resultset, using a UNION Query.
Problem is that my tables are distributed across two separate servers (SQL Server v11.0 and SQL Server v13.0). They are not linked, they cannot be linked, and they have no relationship whatsoever.
Is there anyway to do that?
That is not going to happen in 11 and 13. But if it's really just a union you need, import to a staging area using bcp or your favorite ETL tool; if staging is in one or the other servers then you can union right there and save the transfer of any duplicates that would have been removed from the union (assuming we want a union and not union all).
You can try througt ACCESS. You can connect ACCESS with ODBC to both server, do a union in ACCESS and then upload the result where you want.
But this depend on the dimensions of the table of course.
I am investigating the possibility to split one DB into multiple. We decided to move some tables into another database, but we have queries with join on these tables. I found a few solutions about how to achieve that:
Azure SQL Database elastic query
EXTERNAL DATA SOURCE
But I don`t know what the difference between them and what to choose.
Thanks for any help!
Azure SQL Database Elastic Queries and External data sources are two names for the same concept.
My suggestion is to avoid cross database queries and avoid splitting one database into multiples because query performance involving external data sources won't be the same no matter what strategy you choose to query those external tables.
If you still want to stick with the plan of splitting the database into multiple databases, then know that cross database queries show good performance when the remote tables are not big. When remote tables are big, this article shows you how to perform joins remotely using table variables and improve performance. This other article shows you also how to push parameterized operations to remote databases and improve performance.
if you are thinking to split your DB into multiple SQL server DB with the different host then you can prefer Linked server which has flexible to join across SQL servers
What is the best solution when I would like to use an Access front-end application with some linked table (via ODBC) from MSSQL Server?
The difficulty of this for me is that I have to use complex queries with many multiple joins (and functions called from queries).
It is very-very slow because of the joins between the two DB (and there is a lot of data in some tables, the 2 GB Access mdb limit is the reason of the MSSQL DB upgrade).
Pass-through query doesn't help because of the joined Access tables.
With OPENDATASOURCE('Microsoft.ACE.OLEDB.12.0'... it is still slow in SQL Server too. I tried ODBC linked view with WHERE clause from MSSQL, but it
seems as slow as the full table.
I have to move all of joined Access tables to the MSSQL DB and convert all queries to Pass-Through? Is there any other solution?
I have to move all of joined Access tables to the MSSQL DB
Yes, definitely.
and convert all queries to Pass-Through?
Not necessarily, only those that are still slow.
"Normal" INNER JOIN queries, using only linked tables from one server database, are handled by Access and the ODBC driver in a way that everything is processed on the server. They should be (more or less) as fast as when run on the server (or as Pass-Through query).
Only "complex" queries, especially involving multiple INNER and OUTER JOINs, won't work like that. You'll notice that they are still very slow when running on linked tables. These need to be changed to Pass-Through queries.
Edit: I just noticed
functions called from queries
You can't call VBA functions from PT queries, and they will again kill performance when called from Access queries running on linked MSSQL tables (because they have to be processed locally).
You'll need to learn to create views in MSSQL, probably also user defined functions and/or stored procedures.
In the long run, you'll find that views are actually easier to manage than PT queries.
I use MS Access 2010 on my PC, to link MS SQL tables from our server on cloud.
When I run a query I wrote on Access on my PC involving the lined tables, does the query retreive data from the SQL server over the connection, or is there a cashed data locally on my PC that is used instead?
Simply the question is: In case of using linked tables in Access, does querying these tables run locally where Access database is in use or on the server?
The query will run locally with ordinary queries and linked tables. This means that Access will need to pull all of the data from the linked tables and filter the rows locally. This can be very bad for performance if your WHERE clause only returns a small percentage of rows.
An alternative would be to use pass through queries where the query is executed on the server. This means only the data you need will pass over the network and the performance could be far better.
Here's my scenario:
I have to query two PeopleSoft Databases on different servers (both are SQL Server 2000) and do a join of the data. My application is a .Net application (BizTalk).
I'm wondering what the best option is with regards to performance?
use standard select queries to get data
and do the join in memory (e.g. LINQ) for example
generated complex dynamic queries using LINKED Server, e.g.
select blah
from Server1.HRDB.dbo.MyTable1
left join Server2.FinanceDb.dbo.MyTable2
use standard select queries to get the data into an intermediate / staging sql server database and do my queries / joins on this database instead.
should I consider using SSIS? ( are there features here that might be better than doing an in-memory, e.g. LINQ? )
I wish I could use stored procedures on the source database, but the owners of the PeopleSoft database refuse it
The main constraints we have is that the source database is old (SQL Server 2000) and that performance of the source database is paramount. Whatever queries I run on this server must not block the other users. Hence, the DBAs are adamant about no Stored Procedures. They also believe that queries involving Linked Servers will trump (i.e. take higher priority) to other queries being run against the the database.
Any feedback would be greatly appreciated.
Thanks!
Update: additional background information on the project
We are primarily integrating PeopleSoft databases (the HR and Finance) into another product. Some are simple - like AccountCode and Department. Others are more complex, like the personal data, job, and leave accrual. Some are real-time, other's are scheduled, and other's are 'batch' (e.g. at payroll runs).
Regardless, we have to get source data out of PeopleSoft database -- and my hope had been to let the (source) database do the 'heavy' lifting by executing SQL Queries. I don't really want BizTalk, or SSIS, or C# LINQ to be the ones doing the transformations/filtering.
Definitely open to suggestions.