I'm trying to identify what is using a large amount of network bandwidth that is degrading the performance of my DB.
I've run netstat -ta and keep seeing a foreign ip with something like: static-142-154-43:34860.
Why is this ip address incomplete? Also what is the static prefix?
I'm just trying to find the culprit for excessive usage of the DB.
Servers are on AWS. Thanks!
You could try
netstat -n
which will display the ip addresses for all active TCP connections and not attempt to determine the names, as per http://en.wikipedia.org/wiki/Netstat
Related
If I am writing constantly to a database and the following LOG message is displayed will any of the data I am writing by damaged or omitted?
LOG: using stale statistics instead of current ones because stats collector is not responding
No, this will not affect the integrity of data written to the database.
It just means that the statistics collector does not react fast enough, perhaps because of I/O overload.
You can probably get rid of the problem if you set stats_temp_directory to point to a directory in a RAM file system.
As already said in the previous answer, no it will not lose data. But you probably still want to fix the problem.
One possible cause for this problem is that the statistics collector process is bound to an IP:port which is not responding.
In such a case, restarting postgres will fix it.
This problem happened to me when I disabled IPv6 on the server without restarting Postgres. I eventually found a detailed explanation here (search for "The statistics collector" in the page), but in short:
PostgreSQL [...] will loop through all the addresses returned [for
localhost], create a UDP socket and test it until it has a socket
that works.
If the socket it had selected was IPv6 and it is later disabled, it stops working and you get that message in the logs.
You can check to which IP and UDP port the "postmaster" (or "postgres") service is bound with
netstat -n -u -p
The output is something like this:
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp 0 0 127.0.0.1:47780 127.0.0.1:47780 ESTABLISHED 2824/postmaster
or on another host where it is bound to IPv6 ("udp6"):
# netstat -n -u -p
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp6 0 0 ::1:51761 ::1:51761 ESTABLISHED 1006/postgres
I have a postgresql database on a machine with ip 10.240.81.76. I need to allow remote access to this database from machines 10.240.122.64, 10.240.243.143,...10.240.x.y . I know that i need to make an entry in the pg_hba.conf like the following:
host all all 10.10.29.0/24 trust
What should be the IP range i should be using for the ips in my case ?
Please Help
Thank You
The easy answer is
host <database_name> all 10.240.0.0/16 md5
But much depends on your network configuration. Also note the use of md5 for the authentication method; usually only local addresses should use trust.
This assumes that you know the machines on the 10.240.0.0/16 network. That is, obviously, a private range but there can be up to 16K computers in that range. If you are uncertain, make multiple entries in pg_hba.conf for individual addresses or C-class ranges of which you are certain that they need to connect to your server.
I am working on udp server/client applicataion. Since for communicating with the server, all the clients must know the ip address and port number of the server. For this purpose, I have hard coded the ip and port number of my server to the clients so that everytime, the client connects to same ip and port number. (found the ip address of the server machine using ipconfig command.)
But now, the problem is that I am working on DHCP network, and there is a chance that everytime sever machine is restarted, a new ip address may be assigned to it (different from the ip address known by the clients at which they will connect.)
So, I always want the ip address hard coded at client side to be assigned to the server machine, everytime it logs in.
Is there any way to do it? I have no idea about it. Searched internet but couldn't find anything relevant.
Looking forward to help :(
Assuming that your clients are local to the server, why not abandon the hard-coded server IP address, and borrow a page from DHCP and use some kind of service discovery method:
Your clients broadcast "where is the server" message when they first come online. The server responds with "I am at IP address X.X.X.X"
When the server comes up, it broadcasts "Server is now at IP address Y.Y.Y.Y" so that if the server crashed, the clients start using the new server.
Presuming you are working on a LAN, that's how I'd do it.
Presuming your DHCP server is configurable enough:
Assign a static map MAC address/IP address in the dhcp server, so
that the same machine always get the same IP (just for the server,
not for every client).
Most entry level all in one devices with DHCP have this functionality, if not it should be quite cheap to buy a new one that has it.
If your DHCP server is a real computer, you can surely configure it to do so.
Additionally you might want to tell your clients to use a local DNS and in this local DNS server define a name for your server, so you won't have to hardcode an IP address in your clients. But the address should be located in some configuration file rather than hardcoded in any case.
I have used dnsmasq to serve as both DNS server with local names, and as DHCP server, giving the servers always the same address and pointing all the DNS requests towards itself.
This questions could be useful to find a windows alternative for dnsmasq: https://stackoverflow.com/questions/7709744/is-there-something-like-dnsmasq-for-windows
By adding a reservation field in the DHCP server we can attain this. If you are using Windows DHCP server, there is a section named 'Reservations', there we can give the MAC address of your pc and the desired IP address. Then the server will provide the mentioned IP for you.
With the narrow focus of a developer a DHCP reservation might be the logical step. But using a nameserver is far better. If the network itself changes or maybe the server is moved to another subnet or maybe even into anoher zone, using an IP address from a DHCP reservation fails, because the server's address changes.
You don't have any of these problems if you use a nameserver. That is what DNS is meant to be doing. Think of it as a "serviceprovider finding service" that detaches your service from the host it is running on.
And, like already suggested, you should never hardcode an IP address or DNS name or anything else that might change (even if you think it will not change) unless it is a design goal that things aren't working anymore if something changes (=not configurable).
I'm doing some socket programming in C where I have one server and many clients on the local network. I want the client to find (search for) the server on the network by itself (i.e.not having to specify the server's address) and connect to it. The protocol between the clients and server should be TCP.
I've been searching the web for some time to find a good solution to this, but haven't found one yet.
Any suggestion to how I would tackle this problem would be greatly appreciated.
An IP and port number is necessary for a client to connect to the game server. However, if the IP address of the server is not available, a 'compromise' could be reached by:
Using a hostname to connect to the server, you can use gethostbyname() to resolve a hostname and obtain the IP address. This should solve the issue if the server's IP is not known or is not static.
Having a process running on a known IP which can give you a list of active servers.
Having the clients scan a particular range of IP addresses instead of trying to connect to a single IP (not a very good idea, but should be doable on a LAN).
Havinh your server broadcast packets (say UDP datagrams) at fixed intervals to all hosts on the current subnet (again not really a good idea, will lead to unnecessary network traffic).
A hybrid approach with more than one of the above could also employed, for example, try connecting via a hostname and if that fails, fallback to connecting to a known IP with list of active servers etc.
If you have control over the network layout and such, I would use a solution involving DHCP and DNS.
Basically, you want to connect your DHCP server to your DNS server so that it would automatically create entries for new computers on the network. This is a feature that most DHCP servers and DNS servers support, including BIND and named and Microsoft's solution.
Then you'll set the server hostname to some known value, and have the clients find the server using DNS. That is, gethostbyname() would work properly so you could use the first bullet point offered in Bhargav's answer.
Suppose the following:
I have a database set up on database.mywebsite.com, which resolves to IP 111.111.1.1, running from a local DNS server on our network.
I have countless ASP, ASP.NET and WinForms applications that use a connection string utilising database.mywebsite.com as the server name, all running from the internal network.
Then the box running the database dies, and I switch over to a new box with an IP of 222.222.2.2.
So, I update the DNS for database.mywebsite.com to point to 222.222.2.2.
Will all the applications and computers running them have cached the old resolved IP address?
I'm assuming they will have.
Any suggestions along the lines of "don't have your IP change each time you switch box" are not too welcome as I cannot control this aspect of the situation, unfortunately. We are currently using the machine name of the box, which changes every time it dies and all apps etc. have to be updated with the new machine name. It hurts.
Even if the DNS is not cached local to the machine, it will likely be cached somewhere along the DNS chain between the machine and the name servers, at least for a short while. My understanding is this situation would usually be handled with IP takeover where you just make the new machine 111.111.1.1.
Probably a question for serverfault.
You're looking for DNS TTL (Time To Live) I guess.. In my opinion applications may cache the IP for at most the value of the TTL. I'm afraid however that some applications/technologies might actually cache it longer (agian in my opinion completely wrong)
Each machine will cache the ip address.
The length of time it is cached is the TTL (Time To Live). This is a setting on your DNS server, if you set it very low say 5 mins, then you show be up and running fairly quikly. A bit of a hack but it should work.
Yes, the other comments are correct in that what controls this is the DNS TTL set for the hostname database.mywebsite.com.
You'll have to decide what the maximum amount of time you're willing to wait for if you have a failure on your primary address (111.111.1.1) after you make the switch to the secondary address. Lower settings will give you a quicker recovery time, but will also increase the load and bandwidth to your DNS server because clients will have to re-query it to refresh their cache more often.
You can use nslookup using the -d option from your cmd prompt to see what your default TTL times and remaining TTL times are for the DNS server you are querying.
%> nslookup -d google.com
You should assume that they are cashed for two reasons not clearly mentioned before:
1- Many "modern" versions of OS families do DNS caching.
2- Many applications do DNS caching or have poor error/failure detection on live connections and/or opening new connections. This would possibly include your database client.
Also, this is probably not well documented. I did some googling, and found this for MySQL:
http://dev.mysql.com/doc/refman/5.0/en/connector-net-programming-connecting-connection-string.html#connector-net-programming-connecting-errors
It does not clearly explain its behavior in this regard.
I had a similar issue with a web site that disables the application pool recycling features and runs for weeks on end. Sometimes, a clustered SQL Server box would restart and for some reason, my SqlConnection's were not reconnecting. I was getting the error:
A network-related or instance-specific
error occurred while establishing a
connection to SQL Server. The server
was not found or was not accessible.
Verify that the instance name is
correct and that SQL Server is
configured to allow remote
connections. (provider: Named Pipes
Provider, error: 40 - Could not open a
connection to SQL Server)
The server was there - and running - in fact, if I just recycled the app pool, the app would work fine - but I don't like recycling app pools!
The connections that were being held in the connection pool were somehow using old connection information, and that could have been old IP addresses. This is what seems so similar to the poster's question, that it appears to be cached DNS information, because as soon as some sort of a cache is cleared, the app works fine.
This is how I solved it - by forcing all of the connections in the pool to be re-created:
Try
' Example: SqlDependency, but this could also be any SqlConnection.Open call
Dim result As Boolean = SqlClient.SqlDependency.Start(ConnStr)
Catch sqlex As SqlClient.SqlException
SqlClient.SqlConnection.ClearAllPools()
End Try
The code sample is just the boiled-down basics - it should be tweaked for your situation!
The DNS gets cached, but for any server that resolves to the wrong ip address, you can update the HOSTS file of the server and the ip should be updated immediately. This could be a solution if you have a limited amount of servers accessing your database server.