I am trying to streamline a server of a clients. After downloading the access_log files, I noticed that there were an awful lot of entries that looked like:
::1 - - [11/May/2009:23:21:16 +0100] "GET / HTTP/1.0" 403 5043 "-" "Apache/2.2.3 (CentOS) (internal dummy connection)"
I have also checked the httpd.conf file, and I have seen the following settings:
# ServerLimit: maximum value for MaxClients for the lifetime of the server
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule prefork.c>
StartServers 8
MinSpareServers 8
MaxSpareServers 13
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 50
</IfModule>
# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
I've been reading that I need to set MaxSpareServes to a value greater than MinSpareServers. Opinions are greatly appreciated.
Kindest Regards.
Tom
As far as I know it's nothing to worry about, you can just stop them getting into the log if you want by using the info from the link given already by Andri...
If you wish to exclude them from your
log, you can use normal
conditional-logging techniques. For
example, to omit all requests from the
loopback interface from your logs, you
can use
SetEnvIf Remote_Addr "127\.0\.0\.1" loopback
and then add env=!loopback to
the end of your CustomLog directive.
Related
When a new node is added, we see that it is starting to receive new tablets (in the http://:7000/tablet-servers page) and the system is rebalancing. But the default rate seems low. Are there any knobs to determine this rate?
The rebalance in YugaByte DB is rate limited.
One of the parameters that governs this behavior is the yb-tserver gflag remote_bootstrap_rate_limit_bytes_per_sec which defaults to 256MB/sec and is the maximum transmission rate (inbound + outbound) related to rebalance that any one server (yb-tserver) may do.
To inspect the current setting on a yb-tserver you can try this:
$ curl -s 10.150.0.20:9000/varz | grep remote_bootstrap_rate
--remote_bootstrap_rate_limit_bytes_per_sec=268435456
This particular param can also be changed on the fly without needing a yb-tserver restart. For example to set the rate to 512MB/sec.
bin/yb-ts-cli --server_address=$TSERVER_IP:9100 set_flag --force remote_boostrap_rate_limit_bytes_per_sec 536870912
A second aspect of this is the cluster wide global settings on how many tablet rebalances can happen simultaneously in the system. These are governed by a few yb-master gflags.
$ bin/yb-ts-cli --server_address=$MASTER_IP:7100 set_flag -force load_balancer_max_concurrent_adds 3
$ bin/yb-ts-cli --server_address=$MASTER_IP:7100 set_flag -force load_balancer_max_over_replicated_tablets 3
$ bin/yb-ts-cli --server_address=$MASTER_IP:7100 set_flag -force load_balancer_max_concurrent_tablet_remote_bootstraps 3
I'm trying to leverage my existing (fully configured and working) Samba AD DC as authentication for XWiki, and other apps.
As such, I'm first trying to do a successful ldapsearch from the XWiki server.
The following command works on the Samba server, but not on the XWiki client:
ubuntu#xwiki:~$ ldapsearch -x -LLL -E pr=200/noprompt -H ldaps://10.0.1.191/ -D "CN=Administrator,CN=Users,DC=ad,DC=nitssolutions,DC=com" -w 'SambaNovi2018' -b 'DC=ad,DC=nitssolutions,DC=com' -s sub '(sAMAccountName=*)' cn mail memberOf
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
However, if I do:
ping 10.0.1.191
I get:
ubuntu#xwiki:~$ ping 10.0.1.191
PING 10.0.1.191 (10.0.1.191) 56(84) bytes of data.
64 bytes from 10.0.1.191: icmp_seq=1 ttl=64 time=135 ms
64 bytes from 10.0.1.191: icmp_seq=2 ttl=64 time=138 ms
64 bytes from 10.0.1.191: icmp_seq=3 ttl=64 time=146 ms
^C
--- 10.0.1.191 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 135.741/140.317/146.970/4.832 ms
and if I do:
telnet 10.0.1.191 636
I'm able to connect.
EDIT: Additional information:
I added a -d 1 to the ldapsearch command line, and now I get:
ubuntu#xwiki:~$ ldapsearch -d 1 -x -LLL -E pr=200/noprompt -H ldaps://10.0.1.191/ -D "CN=Administrator,CN=Users,DC=ad,DC=nitssolutions,DC=com" -w 'SambaNovi2018' -b 'DC=ad,DC=nitssolutions,DC=com' -s sub '(sAMAccountName=*)' cn mail memberOf
ldap_url_parse_ext(ldaps://10.0.1.191/)
ldap_create
ldap_url_parse_ext(ldaps://10.0.1.191:636/??base)
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 10.0.1.191:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 10.0.1.191:636
ldap_pvt_connect: fd: 3 tm: -1 async: 0
attempting to connect:
connect success
TLS: peer cert untrusted or revoked (0x42)
TLS: can't connect: (unknown error code).
ldap_err2string
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
Note, in particular, this line:
TLS: peer cert untrusted or revoked (0x42)
I'm going to try researching this error further, but as of now, I'm still stuck...
EDIT2: Still more additional information:
When I run this command, with the -d 1 parameter on the Samba server, the command works, in spite of having the:
TLS: peer cert untrusted or revoked (0x42)
in the debug output....
Continuing to dig....
Help?
And here I go, answering my own question again...sigh. I should post here more often. Helps me clearly lay out the problem, which inevitably leads to finding a solution.
Anyhow, the solution was:
I had a file, /etc/ldap/ldap.conf on my sambadc machine as well as my xwiki client machine, but the content differed.
The sambadc machine had:
#
# LDAP Defaults
#
# See ldap.conf(5) for details
# This file should be world readable but not world writable.
#BASE dc=example,dc=com
#URI ldap://ldap.example.com ldap://ldap-master.example.com:666
#SIZELIMIT 12
#TIMELIMIT 15
#DEREF never
# TLS certificates (needed for GnuTLS)
#TLS_CACERT /etc/ssl/certs/ca-certificates.crt
TLS_REQCERT allow
Which worked.
But, my xwiki machine had:
#
# LDAP Defaults
#
# See ldap.conf(5) for details
# This file should be world readable but not world writable.
#BASE dc=example,dc=com
#URI ldap://ldap.example.com ldap://ldap-master.example.com:666
#SIZELIMIT 12
#TIMELIMIT 15
#DEREF never
# TLS certificates (needed for GnuTLS)
TLS_CACERT /etc/ssl/certs/ca-certificates.crt
which failed.
When I commented out the TLS_CACERT line, and added the TLS_REQCERT line, it all started working as expected.
We use ping localhost -n 2 >nul to delay its following executions.
We can change 2 to the number of seconds needed.
How can I control this in a much broader way? I tried using 1.5 instead of 2 and it didn't work.
Is there any code by which we can change the unit of time?
EDIT: Instead of ping localhost -n 2 >nul. I'm using TIMEOUT 1 >nul.
The command timeout is the best choice for waiting a specific time in a batch file which is designed for execution on Windows 7 and later versions of Windows. It supports breaking the timeout by the user with any key except /NOBREAK is specified as parameter. And it shows a nice message with a seconds countdown for the user informing also the user how to break the timeout. But it supports only timeout values in seconds, not in milliseconds.
The command sleep could be also used on Windows XP and later versions of Windows when having access to Windows 2003 resource kit and this small executable is copied to all computers running the batch file. But this executable is deprecated because of being replaced by TIMEOUT and by default not installed on any Windows computer.
But a good choice for all Windows is using the command ping for pinging the loopback adapter or a not reachable IP address with using appropriate values of the options -n and -w for the delay.
The IP address of the loopback adapter of local machine is 127.0.0.1, see Wikipedia articles about Reserved IP addresses. localhost is just an RFC defined alias for 127.0.0.1 defined on Windows XP and former Windows versions in file %SystemRoot%\system32\drivers\etc\hosts and is defined built-in on Windows Vista and later Windows versions.
The first ping of 127.0.0.1 is always immediately successful. Therefore using command PING with -n 1 as option gives just a delay of approximately a millisecond in total.
For that reason using PING as delay on pinging 127.0.0.1requires a value greater 1 for option -n ... number of echo requests to send. After a successful request PING waits about 1 second before making the next request.
So for a delay of 5 seconds the following command line is necessary with 6 echo requests:
%SystemRoot%\System32\ping.exe 127.0.0.1 -n 6 >nul
Note 1: Windows is not a real-time operating system and for that reason the time is not 100% accurate, but should be good enough for a batch file.
The option -w defines in milliseconds how long ping (Microsoft documentation) waits for an echo on the request. It does not define the time between two successful requests. Therefore this option can't be used to fine tune the delay on pinging the IP address 127.0.0.1 as this request is successful in less than 1 millisecond and value of option -w does not matter.
So for a delay in milliseconds instead of seconds it is necessary to ping an IP address which is definitely or at least most likely not reachable and which is not routed via networks because of being a private network address according to RFC 6761.
An example is:
%SystemRoot%\System32\ping.exe 168.192.255.253 -n 1 -w 1500
The IPv4 address range from 168.192.0.0 to 192.168.255.255 is for private networks. The highest address 192.168.255.255 in this network is the broadcast address and is not used for devices. It is common to configure a router with local area network broadcast address minus 1 which means 192.168.255.254 could be assigned to a router in case of current computer is part of this private network. And other devices in a LAN get assigned usually the IPv4 addresses from lowest address plus 1 upwards. Therefore for IPv4 network 168.192.0.0/16 the IP address 168.192.255.253 is most likely not assigned to any device which would respond on the echo request of PING.
Well, the milliseconds delay is not very accurate. But is it really important on execution of a batch file to wait exactly 1500 ms?
Note 2: This approach does not work if the computer on which the batch file is running is currently not connected to any network. Without any network connection each echo request is always immediately terminated and PING outputs for each echo request the error message:
PING: transmit failed. General failure.
The general failure is no network (connection) present at all and therefore only echo requests to local loopback adapter work.
Unfourtanelty, #thx1138v2's solution only delays 0.04 seconds on my machine. Therefore, I've modified his solution to make it more accurate.
ping 1.1.1.1 -n 1 -w 1500 >nul
1500 stands for 1500 milliseconds, which is 1.5 seconds.
ping is inaccurate when pinging a small amount of time, see this table:
Milliseconds In Code | Actual Waited Time
1500 | 1.24 seconds - 1240 milliseconds
1600 | 1.34 seconds - 1360 milliseconds
1700 | 1.52 seconds - 1520 milliseconds
As you can see, 1700 ms's wait time is much precise than 1500 ms, so you may need to consider some extra milliseconds.
Note: ping only supports delay more than 99 milliseconds
-n is the (n)umber of times to ping, not the amount of time to wait. You can't ping 1.5 times.
-w is the time to (w)ait on each ping in milliseconds. To pause 1.5 seconds would be
ping -n 3 -w 500
If there is a web site set up on the machine running the batch file the ping will find it as localhost and the timeout will not apply. The timeout only applies to failed requests. It is better to ping 0.0.0.1 for a delay.
ping -n 3 -w 500 0.0.0.1
I operate this WordPress website - menstrualcupreviews(dot)net, it is on a VPS server with 2GB RAM and 2 core-CPU. I noticed I got many connections from the server to itself and also from the active users.
My guess was something with the sidebar of the website - it is manually included from each page\post with full path (include "http://...."), and the file also includes few other files in the same way (full path with http).
In some point when we got only 20-30 online visitors, It had 1800+ self connections (on a 2GB RAM) from the server IP to itself and it caused the server to crash.
I have changed it to be relative path (without the "http://"), and the number of self connections dropped from 1800+ to about 50-60 self connections, and also upgraded the RAM to be 8GB.
Now, we currently get around 100-200 online visitors and I get
Mem: 8059424k total, 6759236k used, which is not far from crashing again.
The 8GB RAM memory and CPU are on such a high load and I am sure the website need to be optimized.
For example in a given moment with about 150 visitors:
ss -tuna | awk '{print $6}' | cut -d : -f1 | sort | uniq -c | sort -nr | head
108 198.1.72.1 (==>The SERVER IP - 108 self connections!!!)
51 186.237.5.188
31 *
20 177.216.202.135
15 168.90.64.203
14 201.37.162.119
14 177.182.1.136
13
12 191.251.41.105
12 186.212.162.110
I am not sure what causing those self connections OR many connections from random IP's I guess are some visitors.
Any ideas how to solve the high load and if those connections are the reason?
Thanks in advance! (-:
I have apache virtualhost:
<VirtualHost IP_WAN:80>
ServerName test.localnet
DocumentRoot /srv/http/localnet/test/trunk/docroot
<Directory /srv/http/localnet/test/trunk/docroot>
Options Indexes
</Directory>
</VirtualHost>
The partition /srv is NFS3 mounted filesystem:
172.16.0.2:/srv /srv nfs nodev 0 0
at the server-side it's exported by:
/srv 172.16.0.0/255.255.255.240(rw,sync,no_subtree_check,no_root_squash)
directory /srv/http/localnet/test/trunk is absolute symlink to /srv/http/localnet/test/exports/trunk-v2
In the /srv/http/localnet/test/exports I have exported SVN trees (trunk-v1, trunk-v2, trunk-v3)
When I ask apache for http://test.localnet, it serves 200 OK and index from /srv/http/localnet/test/exports/trunk-v2/docroot
BUT:
If I remove symlink /srv/http/localnet/test/trunk and create a new one to another version (ln -s /srv/http/localnet/test/exports/trunk-v3 /srv/http/localnet/test/trunk), apache gives me 404 Not Found. It takes about seconds/minutes, then everything goes back to normal itself.
If I do ls -la /srv/http/localnet/test/trunk/ in this condition, it goes back to normal immediately. I think there's some problem with NFS cache, but I'm not able to find where exactly the problem occurs and how to prevent it. The symlink occupies the same inode as the previous one, both of targets (the old one and the new one) exists.
On the server side I have /srv partition mounted this way:
/dev/xvda7 on /srv type xfs (rw,nosuid,nodev)
I also use this parameters:
echo 262144 >/proc/sys/net/core/rmem_max
echo 262144 >/proc/sys/net/core/rmem_default
echo 262144 >/proc/sys/net/core/wmem_max
echo 262144 >/proc/sys/net/core/wmem_default
echo noop >/sys/block/xvda7/queue/scheduler
echo 0 >/sys/block/xvda7/queue/read_ahead_kb
I'm also trying to tune MTU of the network interface up to 9000, but without success.
Does anybody know, what's the problem? Why apache can't find the symlink until 'manually' refresh of the directory structure (ls)? Thanks a lot
Ondra
The solution seems to be:
http://publib.boulder.ibm.com/httpserv/manual70/mod/core.html#enablemmap
"With an NFS-mounted DocumentRoot, the httpd may crash due to a segmentation fault if a file is deleted or truncated while the httpd has it memory-mapped"
This is exactly the situation, when removing the symlink in the path to the page.
Hope it helps others :-)