We are using a MSSQL 15 database for our application. Access to the database is established using OLEDB.
While running a series of 10,000 queries
SELECT * FROM table WHERE primary_key = "some_id"
we experienced a rather large drop in performance (factor of around 10) when we switched from running the application directly on the database server to running it on another machine that connects to the server via TCP/IP.
We then investigated the packets sent to and from the server using wireshark. We found that the server replied to every SELECT with the metadata (fieldname, datatype, etc.) of every field of the result followed by the single row of data we had selected. We do not need the metadata information as we already know the structure of table we run the query on.
Because the metadata block is much larger than the actual data, most of the data sent through the network are (to us) completely redundant.
Is there any way to tell the server not to include the metadata in the query result?
Related
I'm dealing with a three piece datapath: client application, host integration server, db server. Client application (MS Mashup Engine) is generating queries that pass through MS SQL Server to a legacy IBM iSeries DB backend.
I'm running into issues where the client is generating queries like
select * from x where numericValue = 1.46510+003
I'm checking the execution plan for these on the SQL Server and they result in a full data load with the comparison occurring on SQL Server (which is acting as the Host Integration Server).
By comparison, a human generated query
select * from x where numericValue = 1465.1
results in no scan and performance two orders of magnitude faster.
I have tried playing with the client application to force it to generate something like the human generated query, but I've had no luck.
I'm not sure if I can massage the way the query plan is generated in SQL server by playing with column data types. I.e. exposing a view over the backend DB with explicitly defined data types. Or otherwise forcing query plan generation?
Any thoughts?
I'm not sure if I can massage the way the query plan is generated in SQL server by playing with column data types. I.e. exposing a view over the backend DB with explicitly defined data types. Or otherwise forcing query plan generation?
No, the answer is no using all of the following:
casting types prior to delivering to power query/power bi changes nothing
adapter properties set for DB2OLEDB or IBMDASQL ( f.ex. Decimal As Numeric=True; Derive Parameters=True; etc...) have no effect
creating stored procedures over the data requires parametrization in power query which is fine, but does not integrate with the ui. I.e. - using a parameter field instead of the column filters is not clean.
TBH, this has become less and less of a problem as backend performance improves and frontend caching is available.
Hope it helps,
-Alex T.
I use MS Access 2010 on my PC, to link MS SQL tables from our server on cloud.
When I run a query I wrote on Access on my PC involving the lined tables, does the query retreive data from the SQL server over the connection, or is there a cashed data locally on my PC that is used instead?
Simply the question is: In case of using linked tables in Access, does querying these tables run locally where Access database is in use or on the server?
The query will run locally with ordinary queries and linked tables. This means that Access will need to pull all of the data from the linked tables and filter the rows locally. This can be very bad for performance if your WHERE clause only returns a small percentage of rows.
An alternative would be to use pass through queries where the query is executed on the server. This means only the data you need will pass over the network and the performance could be far better.
I am a SQL Server database developer. We have a current requirement that we need to link our product with an existing application of a client. Our product has a SQL Server 2012 database while the client's existing application uses Oracle 11g. Since the time limit for the project completion is limited we cannot migrate our application to Oracle.
The requirement is that we have to get the customer details from the Oracle database to do billing activities in our system.
So I went through a few links and found that SQL Server linked server can be used to do this. I have successfully created a view which uses the Customer table from the Oracle database using a linked server. It will solve our problem.
Now here are my questions:
Is there any better solutions to do this other than linked server?
Are there any drawbacks when using linked server for this?
Thanks in advance
One drawback to consider is that the filtering on your view may take place at "your" end, rather than in Oracle. e.g. if you have a linked server (using, say, an OPENQUERY statement) and a view based on that and you do this:
select id from myView where id = 4711
expecting that the result will be very quick (assuming id is indexed etc.etc.), then you may be in for a shock as what will actually happen is:
the entire contents of the Oracle table are passed to SQL Server
SQL Server then filters this data, i.e. the filtering cannot be "pushed
down" into the view objects (as they are remote).
N.B.: I know there are two ways to define linked server (openquery and the other one, I forget the details), so this may not always apply, but you should be aware of the potential performance hit.
I have an MS Access application with data in a separate access mdb. We need to move the data to MS SQL. Previously when I did this is worked very well - dramatic speed improvements. In this new upgrade however, we are seeing some problems with translating of the SQL from Access to SQL.
We have the solution separated into an APP mdb and a DATA MDB. We use Linked-tables to join the two. After migrating the data to SQL server, we created an ODBC link, then re-pointed the linked-tables to the SQL data source over ODBC. We would expect the SQL command generated by Access, to be simply transferred to SQL server, for SQL server to operate on same using its normal efficiency and power.
When I put a trace on SQL Profiler, instead of seeing for example a simple "Delete * from table", we see a series of delete statements, one for each record in the table ...
We see the same thing for updates - where we would expect for example "insert into table1 (select a,b,c from table2)...) to be sent as an SQL string, to be executed on the server, we instead see a series of insert statements, one per row being inserted, being sent to the server.
It seems that Access is trying to work out the logic for everything but select statements client side and not letting MS SQL server take care of things as one would expect.
Anyone experienced this behaviour before and can offer a suggestion to resolve?
thanks.
I have an access 2003 database that holds all of my business data. This access database gets updated every few hours during the day.
We're currently writing a website that will need to use the data from the access database. This website (for the time being) will have only read only capabilities. Meaning there will only need to be one way transfer of data (Access -> SQL).
I'm imaging there's a way to perform this data migration from access to SQL server programatically. Does anyone have any links to something I can read about?
If this practice sounds odd, and you'd like to suggest another way to do this (or a situation where data can go both ways (Access -> SQL, SQL -> Access), that's perfectly fine.
The company is going to continue using Access 2003 for their business functionality. There's no way around that. But I'd like to build the (readonly) website on top of SQL Server.
The strategy you outlined can be very challenging. You could use INSERT queries to copy new Access rows to SQL Server, as described in another answer.
However, if you have changes to existing Access rows, and you also want those changes propagated to SQL Server, it won't be so simple. And it will be more complicated still if you want deleted Access rows deleted from SQL Server, too.
It seems more reasonable to me to use a different approach. Migrate the data to SQL Server once. Then replace the tables in your Access database with ODBC links to the SQL Server tables. Thereafter, changes to the data from within your Access application will not require a separate synchronization step ... they will already be in SQL Server. And you won't need to write any code to synchronize them.
If your concern is that the connections between the web server and SQL Server be read-only, just set them up that way. You can still independently allow read-write permissions for your Access application.
To do the initial data migration and set the SQL Server automatically, I would use the SQL Server Migration Assistant. The only thing you should definitely change that I can think of would be to turn off the Identity property on any columns that have it - to be explained below (MS Access calls Identity autonumber). Once you have your tables loaded, you can set up a dsnless connection to the database (and tables) you just created.
I haven't used the method just linked, but I believe it allows you to use SQL Server authentication to connect to the db. The benefit of using this method is you can easily change which SQL Server instance and/or database your are connecting to for development and testing.
There might be a better, automated way, but you can create several insert queries doing left joins from the primary key of the Access table to the SQL Server table, and putting a WHERE clause that specifies the SQL Server PrimaryKey must be null. This is why you need to turn off the Identity property in the SQL Server tables, so that you can insert the new data.
Finally, put the name of each query in one function, then run the function periodically.
I have used Microsoft's free SQL Server Migration Assistant (SSMA) to migrate Access to SQL Server. The tool is very simple to use. The only problem I have encountered with the tool was overloaded data types when migrating. What I mean by this is a small string will get converted to a NVARCHAR(MAX) in some instances. Otherwise, the tool is very handy and can be reused after setting up a 'profile'.