From what I have read, I need to make changes to the configuration file postgres.conf and pg_hba.conf and after that we need to create a "replication" user.
How do I make changes to these configuration files and make sure that the changes to these configuration files persist through the restarts?
Since my postgres is running inside a kubernetes pod, I reckon that the user must be created inside the container.
But when I do createuser -replication -P replica, it wont allow me to create one since the current user after I did docker exec -it <container_id> bash is root and not postgres.
This gist has a detailed explanation on how to setup a master-slave replication with Postgres using docker.
Master configuration
Create replication_user
CREATE USER replication_user NOSUPERUSER;
ALTER USER replication_user WITH REPLICATION;
ALTER USER replication_user WITH PASSWORD 'CHANGEME';
postgresql.conf
Edit or add these lines:
listen_addresses = '*' # It can be more specific with the listen IP
wal_level = hot_standby
max_wal_sender = 5 # Specifies the maximum number of concurrent connections from standby servers
ssl = off
archive_mode = on
archive_command = "cp %p /path/to/archive/%f"
pg_hba.conf
Let's suppose that 192.168.0.2/32 is the IP of the slave and replication_user is the replication user.
case no authentication is required
host replication replication_user 192.168.0.2/32 trust
case authentication using password
host replication replication_user 192.168.0.2/32 scram-sha-256 # or MD5
Slave configuration
Let's suppose that the Master IP is: 192.168.0.1 and the port is the default (5432).
listen_addresses = '*'
primary_conninfo = 'user=replication_user password=CHANGEME host=192.168.0.1 port=5432'
hot_standby = on
Making the configuration persistent
You can use docker volumes (or for kubernetes).
Start by creating the configuration files in the local machine that start the containers and then:
docker run --rm --name IMAGE_NAME -v /local/path/to/postgresql.conf:/etc/postgresql/postgresql.conf -v /local/path/to/pg_hba.conf:/var/lib/postgresql/data/pg_hba.conf CONTAINER_NAME
The same for the slaves.
sources:
https://wiki.postgresql.org/wiki/Streaming_Replication
https://gist.github.com/avshabanov/eb8e03a050c79f8e77420b06f9b4abe5
Related
Requirement in detail:
So I have two databases (both are in sync) and somehow one goes down and Spring Boot application starts giving exceptions. In this case I want the application to connect to the second database.
Please help me with this.
Thanks in advance.
As you have a DataGuard implementation in Oracle with a Primary database and another one in Standby mode, Oracle Transparent Application Failover is the way to go.
Transparent Application Failover (TAF) is a feature of the Java
Database Connectivity (JDBC) Oracle Call Interface (OCI) driver. It
enables the application to automatically reconnect to a database, if
the database instance to which the connection is made fails. In this
case, the active transactions roll back.
Database Setup
I am assuming your implementation of DG uses Oracle Restart.
Datatase: TESTDB
Service in TAF: TESTDB_HA
Primary site
srvctl add service -d testdb -s testdb_ha -l PRIMARY -y AUTOMATIC -e select -m BASIC -z 200 -w 1
srvctl start service -d testdb -s testdbha
Standby site
srvctl add service -d testdb -s testdb_ha-l PRIMARY -y AUTOMATIC -e select -m BASIC -z 200 -w 1
srvctl modify service -d testdb -s testdb_ha -failovermethod basic
Your JDBC connection
jdbc:oracle:thin:#(description=(address=(host=primaryserver)(protocol=tcp)(port=yourdbport))(address=(host=standbyserver)(protocol=tcp)(port=yourport))(failover=yes)(connect_data=(service_name=testdb_ha)(failover_mode=(type=select)(method=basic))))
In this setup, in case a failover from Primary to Standby, the connection will keep working once the failover is completed without manual intervention.
I am using this configuration currently in applications store in Kubernetes, using Spring Boot and/or Hibernate, and in normal Jboss Java applications. I have personally tested failover scenarios totally transparent for the applications. Obviously, if you have a transaction or query running in the moment the failover is being performed, you will get an error. But you don't need to manually change any jdbc settings in case of switch from primary site to standby site.
I'm new to Grafana and trying to connect Grafana to Microsoft SQL Server. I run both Grafana and SQL server on the same machine with Windows OS. In Grafana, I selected SQL Server data source and provided Host and DB name. I created a user in SQL server and granted reader permission to the user as per https://grafana.com/docs/grafana/latest/datasources/mssql/. Either for SQL server Authentication or Windows Authentication, I get the error db query error: failed to connect to server - please inspect Grafana server log for details.
I checked then Grafana log file: lvl=eror msg="query error" logger=tsdb.mssql err="Unable to open tcp connection with host 'servername:1433': dial tcp [2a02:908:1391:9e80:c180:xxxx:xxxx:xxxx]:1433: connectex: No connection could be made because the target machine actively refused it."
How can I force SQL server to give access to Grafana?
I should mention that, I haven't changed Grafana conf file. Do I need to change the default conf or create another conf file?
The default DB configuration in Grafana conf file is:
[database]
# You can configure the database connection by specifying type, host, name, user and password
# as separate properties or as on string using the url property.
# Either "mysql", "postgres" or "sqlite3", it's your choice
type = sqlite3
host = 127.0.0.1:3306
name = grafana
user = root
# If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""
password =
# Use either URL or the previous fields to configure the database
# Example: mysql://user:secret#host:port/database
url =
# Max idle conn setting default is 2
max_idle_conn = 2
# Max conn setting default is 0 (mean not set)
max_open_conn =
# Connection Max Lifetime default is 14400 (means 14400 seconds or 4 hours)
conn_max_lifetime = 14400
# Set to true to log the sql calls and execution times.
log_queries =
# For "postgres", use either "disable", "require" or "verify-full"
# For "mysql", use either "true", "false", or "skip-verify".
ssl_mode = disable
# Database drivers may support different transaction isolation levels.
# Currently, only "mysql" driver supports isolation levels.
# If the value is empty - driver's default isolation level is applied.
# For "mysql" use "READ-UNCOMMITTED", "READ-COMMITTED", "REPEATABLE-READ" or "SERIALIZABLE".
isolation_level =
ca_cert_path =
client_key_path =
client_cert_path =
server_cert_name =
# For "sqlite3" only, path relative to data_path setting
path = grafana.db
# For "sqlite3" only. cache mode setting used for connecting to the database
cache_mode = private
The settings in Grafana's configuration file refer to its internal database so you do not need to change any of these to connect to MS SQL Server.
Try using "localhost" or "127.0.0.1" as the host name
Make sure authentication is SQL Server Authentication
Make sure Encrypt is false
Check the SQL server logs for any errors
Docker host using IP address of your machine follow below steps:
Open the CMD
IPCONFIG /ALL
Look for the IPV4 address under WiFi or
vEtherner; in my case, it's 192.168.1.24 and 172.45.202.1, respectively
Then try accessing the app hosted in the Docker container with the mapped port (e.g., 1433/5436)
It simply worked using 192.168.1.24:1433 and 172.45.202.1:1433 in the same way to access all container apps hosted using Docker
I've failed to set up postgreSQL to work with my Ruby-on-Rails project for the past week. I've tried to uninstall and reinstall, postgreSQL, twice now.But when I try to launch postgreSQL I keep getting the error below:
could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "localhost" (::1) and
accepting TCP/IP connections on port 5432?
could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "localhost" (127.0.0.1) and
accepting TCP/IP connections on port 5432?"
I've looked at many online resources, including stackoverflow and none seem helpful.The key parts of my pg_hba.conf file looks like this:
# TYPE DATABASE USER ADDRESS METHOD
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#host replication postgres 127.0.0.1/32 md5
#host replication postgres ::1/128 md5
And the key part of my postgresql.conf file is as follows:
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
#superuser_reserved_connections = 3 # (change requires restart)
#unix_socket_directories = '' # comma-separated list of directories
# (change requires restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal notation
Most of the suggestions, I've seen so far, were based on those two files. (For my case, they were already configured correctly). I also tried disabling the firewall and restarting postgreSQL but it didn't help. Does anyone have any suggestions for me? Thanks!
Got the same issue while settings up PostgreSQL 9.6.16 to work with Python/Django, but this is purely a database issue.
The solution lies in the error: In fact, I found this error mentioned within the official PostgreSQL documentation thus it's a common error.
And here is how I resolved this issue:
Always first start the postgres database server, use postgres or the wrapper program pg_ctl.I used the command, below, on windows 10.Remember, whatever comes after -D should be the path to where you installed PostgreSQL, to the data folder, which holds the pg_hba.conf and postgresql.conf files.
> pg_ctl start -D "C:/Program Files/PostgreSQL/9.6/data"
If that runs well, you are ready to access the database server.Open another cmd shell, and type the command below.Remember the password you entered while installing PostgreSQL?Enter that password when asked Password for user postgres:
> psql -U postgres
Once done, you can now go ahead to CREATE ROLE and CREATE DATABASE accordingly.
Spec:
Ubuntu 14.04
webmin/virtualmin 1.791
I am using following code to test remote mysql database connection:
<?php
$db_host = "123.456.789";
$db_name = "database";
$db_user = "user";
$db_pass = "password";
$db_table_prefix = "prefix_";
GLOBAL $errors;
GLOBAL $successes;
$errors = array();
$successes = array();
$mysqli = new mysqli($db_host, $db_user, $db_pass, $db_name);
GLOBAL $mysqli;
if(mysqli_connect_errno()) {
echo "Conn Error = " . mysqli_connect_error();
exit();
}
?>
I keep getting this error:
No connection could be made because the target machine actively
refused it.
Research shows this means the server is "not listening". Before I ran the above script I've already tried to allow remote mysql access through webmin gui. What I did is editting "database manage->host permissions" and make it as follows:
This was supposed to allow remote mysql access but it doesn't work. Also I read from somewhere else that to allow remote mysql access I need to edit /etc/mysql/my.cnf; I have thought that after I edit the "host permissions" in webmin this file would be changed, but it was not. On the other hand, I couldn't find the lines I was supposed to edit in my.cnf, so I am stuck here.
Any help is appreciated.
You can do this via webmin too,
Create your user account for remote access
Webmin > Servers > MySQL Database Server > User permissions
Allow the MySQL server to listen to remote requests
Webmin > Servers > MySQL Database Server > MySQL Server Configuration
MySQL server listening address - set it to any
Restart MySQL using service mysql restart or directly from webmin.
Allowing MySQL to listen to any port is not a good idea , unless you are the only one who can access that network ,
Don't leave it like this afterwards , Its better to Allow certain hosts on certain domains , like your IP , Or simply just dont listen on all ports if its not required i.e when you are finished with your session.
You can also do this via /etc/mysql/my.cnf
Just add a binding adress of your choice instead of localhost
I have got it to work, however not through webmin at all.
First I need to comment out the following line in /etc/mysql/my.cnf:
#bind-address = 127.0.0.1
I guess instead of simply commenting it out, I can also change 127.0.0.1 to my local IP address. Many google results stop here, but this is not enough. The next step is to grant the local user privileges: On remote server, I need to run the following commands:
$ mysql -u root -p
Enter password:
mysql> use mysql
mysql> GRANT ALL ON . to user#'localIP' IDENTIFIED BY
'password';
mysql> FLUSH PRIVILEGES;
Actually I have seen this when I was doing google search before I asked the question here, but I just ignored it because I thought I've done it. It turns out I not only need to grant privileges on server side but also need to do it for "local user".
Feel free to comment here if there's still something I missed out or you know how to do it throught webmin(I am still wondering what editting "host permissions" in webmin does).
Please don't move this question to askubuntu as I think this question is not OS-specific.
When I invoke the createuser postgres command (for now it doesn't matter if I provide any parameters or not), I'm getting this error:
createuser: could not connect to database postgres: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Normally it means that the postgres server is down but not this time:
pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.4 main 5432 online postgres /var/lib/postgresql/9.4/main /var/log/postgresql/postgresql-9.4-main.log
sudo service postgresql status
9.4/main (port 5432): online
But it's true that there is no /tmp/.s.PGSQL.5432 file because my configuration file (/etc/postgresql/9.4/main/postgresql.conf) has this line:
unix_socket_directories = '/var/run/postgresql'
So I don't really understand why createuser whants to access /tmp/.s.PGSQL.5432? Can this path can be hardcoded into the createuser binary? I don't see any command line argument to specify the settings file location for createuser...
Have you started the service?
service postgresql start
The postgresql.conf file is read by the database server, but not by client applications (such as createuser, psql, ...). (In fact, the server configuration file cannot be read by client applications because the client would have to connect to the server, which could be halfway across the world, before it could possibly know where that configuration file lives).
Instead, you have to tell your client application where to find the socket directory.
If your client application (createuser) is connecting to the local host (which is must be because you are not specifying a different host), you use the host parameter to specify the name of the socket directory.
For example:
createuser -h /var/run/postgresql newusername
See http://www.postgresql.org/docs/devel/static/libpq-connect.html#LIBPQ-CONNECT-HOST
Hope that helps.