What is the fastest SQL Server connection protocol?
Related: which protocols are available remote versus local, and does that affect the choice of fastest protocol?
VIA. This is the fastest SQL Protocol, it runs on dedicated hardware and is used in doing SQL Server benchmarked records.
Note that the VIA protocol is deprecated
by Microsoft, and will be removed in a
future version of Microsoft SQL Server.
It is however supported in SQL Server 2008,
SQL Server 2008 R2 and SQL Server 2012.
Shared Memory is next as performance, but it only works between a client and a server that can actually share memory, so local only.
For remote connectivity on ordinary hardware, TCP is the way to go. Under normal operations, it has the same performance as Named Pipes. On slow or busy networks, it outperforms NP in robustness and speed, a fact documented in MSDN:
For named pipes, network
communications are typically more
interactive. A peer does not send data
until another peer asks for it using a
read command. A network read typically
involves a series of peek named pipes
messages before it starts to read the
data. These can be very costly in a
slow network and cause excessive
network traffic, which in turn affects
other network clients.
Named Pipes also can lead to client connect time outs:
TCP/IP Sockets also support a backlog
queue. This can provide a limited
smoothing effect compared to named
pipes that could lead to pipe-busy
errors when you are trying to connect
to SQL Server.
Unfortunately the normal client configuration tries NP first and this can cause connectivity problems (for the reasons cited above), where enforcing TCP on client network config (or in connection string, via tcp:servername) skips the NP connect attempt and goes straight to TCP for a much better experience under load.
Now is true that the same link I quoted above goes on to praise NP for its easy of configuration, most likely referring to no need to open SQL TCP port in firewall, but is there where me and BOL have different views.
Shared memory is fastest for local (client and server on same machine). Named pipes is probably 2nd fasted for local. For remote everyone is using TCP-IP and the remaining protocols are kind of turning into networking history.
Using Shared Memory Protocol
The network libraries you choose when installing SQL Server can affect the speed of communications between the server and its clients. Of the three key network libraries, TCP/IP is the fastest and Multi-Protocol is the slowest. Because of the speed advantage, you will want to use TCP/IP on both your servers and clients. Also, don't install unused network libraries on the server, as they only contribute unnecessary overhead**
Named Pipes is the fastest SQL Server protocol.
Related
I have developed a VBNET application that would require database connection (SQL Server 2005 Express) to a Windows Server 2000 PC. These applications will run in Windows XP and it is expected to be installed across at least 20 clients, all linked through LAN.
I would like to know if there are connection limits using Windows XPs or is the limit dependent on the server machine.
Also on a related note, are there limits for TCP/IP connections in the same case?
These are all so that I can decide whether to upgrade the client PCs to Windows 7.
Thanks in advance.
Yes, there are connection limits but there are a few things you need to consider:
The database-access paradigm in .NET is to pool and use database connections on an as needed basis. This way database connections are not held unnecessarily
From #1 above follows that a single database server should be able to service lots of clients simultaneously if you write your code correctly
20 clients is definitely within the realm of possibility - even for SQL Server Express which is not limited in terms of number client connections but is is limited by how much memory it can use and how many cores which indirectly limit the number of connections that it can handle. To the same point, there is a limit to how many TCP connections a Windows machine can handle but on a server OS the limitation will most likely come from resources (processing power and available memory) before you hit the arbitrary software limit of active TCP connections (which I believe is in the range of millions)
You are using seriously obsolete operating systems - these are no longer supported and you should move off of them as soon as possible!
The connection limits you're referring to only apply to incoming connections. For outgoing connections, you're fine. Additionally, the limits only apply to desktop operating systems like XP. Server 2000 does not have the same limitation.
In other words, this will work just fine... at least for a while.
However, the systems you're using are beyond obsolete... they are end of life. This means that no new patches are created or released for these systems, even when new critical vulnerabilities are discovered... and believe me, new vulnerabilities are discovered all the time. That makes these systems fundamentally insecure. It's irresponsible to continue using them, and only a matter of time until your network is hacked.
To give some context, I'm currently running a SQL Server 2012 instance on Amazon RDS and I've had to move to a larger instance twice already. The first time SQLAzureMW was the way to go, but at the time no table was that significantly large. The second time, SQLAzureMW always timed out the source server on the bcp command with large tables (a few over 5 GB). Similarly, SSIS Import / Export Wizard also timed out. I found the source server was always the problem so I tried increasing the instance's class from an m1.medium to an m1.xlarge to no avail, the source server still always timed out before making any significant progress on the large tables.
In the end I ended up writing my own .NET program that simply ran a "SELECT * FROM [table] ORDER BY [id] OFFSET {0} ROWS" on the large source tables and pushed the results into SQLBulkCopy on the destination server. Again the source server timed out repeatedly but I wrapped the try and catch statements in a loop that would simply resume the query from the last point where SQLBulkCopy. That being said, I'm not exactly thrilled with this solution.
I'm considering building a solution around the Microsoft.SqlServer.Management.Smo.Transfer class but I'm afraid there might be the same problems with lack of recovery from a broken source connection.
I'd much rather an out of the box solution for this like SQLAzureMW was before tables got too large and that I'd expect SSIS Import Export Wizard to be. There has to be a better way.
We were running into a similar situation: running SQLAZureMW on an Window server 2012 EC2 instance connecting to SQL Server 2012 RDS Instance. AWS support suggested the following changes on our EC2 instance and it seems to have solved all of our issues:
Increase TCP/IP timeout value as described here (i'm not sure this is actually necessary) http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-firewall-guidance.html
Disabling all TCP offloading for the network adapter.
Instructions from AWS:
Here are the steps to disable TCP Offloading: Go to the properties of
the Citrix PV ethernet adapter Click Configure Go to Advanced Disable
all of the following Properties:
IPv4Checksum Offload Large Receive Offload (IPv4), Large Send Offload
Version 2(IPv4), TCP Checksum Offload (IPv4), UDP Checksum Offload
(IPv4)
Then as a final step run the following command from the command
prompt:
netsh int ip set global taskoffload=disabled
netsh int tcp set global chimney=disabled
netsh int tcp set global rss=disabled
netsh int tcp set global netdma=disabled
This issue has been known and reported to MSFT. The problem here is not with SQL Server (your source). The NIC drivers for the network card have a feature called TCP chimney which offloads the bulk data movement from the CPU to the network card. i.e For large data movement, the CPU does not get involved and rather relies on the network card to process the data. But while doing so, the NIC card some times runs out of memory (known bug).
You can simply turn off the Chimney feature off and give it another try. If your source is a production box, you may want to create a backup of the DB before doing anything with that machine (just to be on the safe side). People have reported resolving this problem by turning the feature off. Here is a link you can follow.
I thought I answered this but it turns out the problem was the instances I chose. I believe the m1 class of instances shared the same hardware network device for SAN storage and networking. The result being that enough network activity caused the system drive, and thus the virtual memory, to become inaccessible at least for an instant. Spending the money on newer hardware, m2 and above, solved the problem.
For a project I need to read the incoming and outgoing bytes per second of an SQL-Server (2012) instance of an instance or database (doesn't matter). For this I found the following performance counters:
SQL Server, Broker / DBM Transport Object
Receive I/O bytes/sec
Send I/O bytes/sec
When starting SQL Server Management Studio and executing some select statements the values of the performance counters are staying 0. While when I included the client statistics I see that Bytes sent from client and Bytes sent from server are not 0. I'm executing these select statements to a default installed installed on the same pc.
Does anyone how to solve this issue?
Thanks in advance
The documentation explains what SQL Server, Broker / DBM Transport Object measures:
The Broker / DBM Transport performance object contains performance
counters that report networking information for Service Broker and
database mirroring.
There is no performance counter for Transact-SQL traffic. If it helps, the DMV sys.dm_exec_connections will aggregate the traffic size for a connection. If the traffic occurs over a network interface then you could use the system network counters, that is the Network Interface Object. But a local test would not register anything because the connection will use shared memory protocol.
That being said, it is unusual to have to measure SQL Server Transact-SQL network traffic. If the question ever arise, then you're doing it wrong. Network traffic should always be negligible. The dimension everybody is interested is IO, for which there is support in SQL Server, Buffer Manager Object, SQL Server, Databases Object and DMVs like sys.dm_io_virtual_file_stats.
We use sql express 2008 r2 for simple reporting and data storage. Data is written typically at the rate of 600 to 3000 records per minute. (one client one connection same machine). We need the optimal performing protocol. (good performance without hogging memory..) I read articles online and they are quite confusing when it comes to deciding between tcp/ip, shared mem and named pipes. To summarize msdn documentation
1) Shared memory has no configurable properties. Shared memory is always tried first, and cannot be moved from the top position of the Enabled Protocols list in the Client Protocols Properties list. Does this mean shared mem is preferred and fastest ?
2)For TCP/IP Sockets, data transmissions are more streamlined and have less overhead. Data transmissions can also take advantage of TCP/IP Sockets performance enhancement mechanisms such as windowing, delayed acknowledgements, and so on. ok. But is it faster than shared mem ?
3)If the server application is running locally on the computer running an instance of Microsoft® SQL Server™ , the local Named Pipes protocol is an option. Local named pipes runs in kernel mode and is extremely fast. When I read this I could not take the confusion anymore and decided to post my question on stack.
So sql Gurus please help me decide. thank you
Shared Memory is the fastest protocol, however if "performance" is your goal, the framework you use to access the database and how you transfer the data in, the sql query you write (or use another method) will all have a much bigger impact on performance.
If you are going to try and use ADO/ADO.NET or another heavy protocol like this most performance will be lost here. If extreme performance is what you are after you'll need to investigate and learn how to communicate with the database engine on a lower level.
See http://www.devart.com/sdac/ as a start.
If the following quote from your answer is actually correct then you your question already contains the answer:
Shared memory is always tried first, and cannot be moved from the top position of the Enabled Protocols list in the Client Protocols Properties list.
Given that your client and database are running on the same machine this means that shared memory will always work (without drastic measures that is). It doesn't matter what protocol you chooe to use - shared memory is the one that will end up being used! :-)
For what its worth the performance completely depends on the implementation used within SQL Server (I'm basing this on my knowledge of the communication methods rather than intimate knowledge of SQL Server), but the order (in terms of performance) almost certainly goes like this (fastest first):
Shared memory (backed up by the fact that SQL Server always uses this when it can)
Named pipes
TCP/IP
I can't find any citation that Shared memory actually is faster than named pipes, but TBH I don't think it matters that much - communication between client and database on a local machine will be incredibly efficient anyway, I doubt that you would be able to notice any performance difference.
I am upsizing an MS Access 2003 app to a SQL Server backend. On my dev machine, SQL Server is local, so the performance is quite good. I want to test the performance with a remote SQL Server so I can account for the effects of network latency when I am redesigning the app. I am expecting that some of the queries that seem fast now will run quite slowly once deployed to production.
How can I slow down (or simulate the speed of a remote) SQL Server without using a virtual machine, or relocating SQL to another computer? Is there some kind of proxy or Windows utility that would do this for me?
I have not used it myself, but here's another SO question:
Network tools that simulate slow network connection
In one of the comments SQL Server has been mentioned explicitly.
You may be operating under a misconception. MS-Access supports so-called "heterogeneous joins" (i.e. tables from a variety of back-ends may be included in the same query, e.g. combining data from Oracle and SQLServer and Access and an Excel spreadsheet). To support this feature, Access applies the WHERE clause filter at the client except in situations where there's a "pass-through" query against an intelligent back-end. In SQL Server, the filtering occurs in the engine running on the server, so SQL Server typically sends much smaller datasets to the client.
The answer to your question also depends on what you mean by "remote". If you pit Access and SQL Server against each other on the same network, SQL Server running on the server will consume only a small fraction of the bandwidth that Access does, if the Access MDB file resides on a file server. (Of course if the MDB resides on the local PC, no network bandwidth is consumed.) If you're comparing Access on a LAN versus SQL Server over broadband via the cloud, then you're comparing a nominal 100 mbit/sec pipe against DSL or cable bandwidth, i.e. against perhaps 20 mbit/sec nominal for high-speed cable, a fifth of the bandwidth at best, probably much less.
So you have to be more specific about what you're trying to compare.
Are you comparing Access clients on the local PC consuming an Access MDB residing on the file server against some other kind of client consuming data from a SQL Server residing on another server on the same network? Are you going to continue to use Access as the client? Will your queries be pass-through?
There is a software application for Windows that does that (simulates a low bandwidth, latency and losses if necessary). It not free though. The trial version has a 30-sec emulation limit. Here is the home page of that product: http://softperfect.com/products/connectionemulator/
#RedFilter: You should indicate which version of Access you are using. This document from 2006 shows that the story of what Access brings down to the client across the wire is more complicated than whether the query contains "Access-specific keywords".
http://msdn.microsoft.com/en-us/library/bb188204(SQL.90).aspx
But Access may be getting more and more sophisticated about using server resources with each newer version.
I'll stand by my simple advice: if you want to minimize bandwidth consumption, while still using Access as the GUI, pass-through queries do best, because then it is you, not Access, who will control the amount of data that comes down the wire.
I still think your initial question/approach is misguided: if your Access MDB file was located on the LAN in the first place (was it?) you don't need to simulate the effects of network latency. You need to sniff the SQL statements Access generates, rather than introducing some arbitrary and constant "network latency" factor. To compare an Access GUI using an MDB located on a LAN server against an upsized Access GUI going against a SQL Server back-end, you need to assess what data Access brings down across the wire to the client from the back-end server. Even "upsized" Access can be a hog at the bandwidth trough unless you use pass-through queries. But a properly written client for a SQL-Server back-end will always be far more parsimonious with network bandwidth than Access going against an MDB located on a LAN server, ceteris paribus.