hello everyone I create a cluster in postgres-XL but I use the command pg_ctl promote but I need a standby server how I can create it I need the cluster to be high availability
You can't do this. Postgres-XL does not support HA or any fail over protection. If you lose a single node the entire database is lost.
The best option they provide is to have a mirror instance of each node, but even with this if a node fails you will still have to manually take the database down and removed the failed node and replace it with its mirror.
Related
I don't want to use any other tools to promote or demote any node. I know pg_ctl able to promote any standby node to master ,but if I want to demote primary node to standby node using default PostgreSQL, then how can I do it?
If there is no direct command then what will be the actual steps to do it?
Let me make a statement first: I'm new to Kubernetes, please take it easy if I'm asking wrong questions.
Ok, here is what I'm gonna do. I'm planning to build a Kubernetes for my project using some physical machines.
I have 1 server for master and 2 worker nodes. My service dockers (pods) will be allocated by Kubernetes master, they will need storage for the database (MySQL).
After searching around, I came up with a solution of Persistent Volume, but I don't want to use those online cloud services out there such as Google Cloud or Azure Cloud, etc. It leads me to another solution - Local Persistent Volume (LPV), this is where I stuck currently.
The problem with LPV is it's attached with a specific node, so I wouldn't be able to replicate (backup) the database on other nodes, if something happens to this node, or something wrong with the physical disk, I'm gonna lose all the databases, right?
The question is, are there any solutions to set up replication on the database using Local Persistent Volume? For example, I have a database on Node 1, and a backup version on Node 2, so when Node 1 is not available, the pods will mount to the backup database on Node 2.
Thanks in advance!
You can deploy the database as statefulset using local volumes on nodes.Just create the volumes and put them in a StorageClass
For backup , you need to setup the replication on the database level ( not volume level ) to some other cluster /other database instance running somewhere else/or on some other cluster.
Pod failures are handled by kubernetes anyway , it will restart the pod if goes unhelathy.
Node failures can't be handled in statefulset ( one node can't replace another , in other words , in statefulset a pod will not be restarted on other node , kubernetes will wait for node to come back )
If you are going for simple single pod deployement rather than statefulset , you can deploy the database as single pod and another instance as single pod and use node selector to run those on different nodes , then setup the replication from one instance to another instance on database level , and configure your client app to failover to fallback instance in case the primary is not available , this needs to be synchronous replication.
Links:
Run a Single-Instance Stateful Application ( MYSQL)
Run a Replicated Stateful Application ( MYSQL )
I'm learning Docker Swarm mode and I managed to create a Swarm locally with a web application and a PostgreSQL database. I can scale them and I see Swarm creating replicas.
I think I understand how Docker Swarm can load balance regular web servers, but how does it deal out of the box with database containers?
Outside of the Swarm context, usually databases have their own ways to deal with replication, in the form of plugins or extended products like MySQL cluster. Other databases like Cassandra have replication built directly into their product.
On a Swarm context, do we still need to rely on those database plugins and features?
What is the expected pattern to handle data consistency between replicas of a database container?
I know it's a very open-ended question, but Docker's documentation is very open-ended too and I can't seem to find anything specific to this.
How does it deal out of the box with database containers?
It doesn't.
There is a pretty good description of Swarm services here: How services work (emphasis mine)
When you deploy the service to the swarm, the swarm manager accepts your service definition as the desired state for the service. Then it schedules the service on nodes in the swarm as one or more replica tasks.
Swarm has no idea what's inside the task, all it knows is how many instances of it there are, whether those instances are passing their health checks, and if there are enough of them to satisfy the task definition you gave it. The word overlap between this and database replicas is a little unfortunate, but they are different concepts.
What is the expected pattern to handle data consistency between replicas of a database container?
Setting up data replication is on you. These are probably as good a place to start as any
How to Set Up PostgreSQL for High Availability and Replication with Hot Standby
PostgreSQL Replication Example
Docker swarm currently scales well for the stateless applications. For database replication, you have to rely on every database's own replication mechanism. Swarm could not manage the datatbase replication. The volume or file system level replication could provide the protection for a single instance database, but are not aware of database replication/cluster.
For databases such as PostgreSQL, the additional works are required. There are a few options:
Use host's local directory. You will need to create one service for every replica, and use constraint to schedule the container to one specific host. You will also need custom postgresql docker image to set up the postgresql replication among replicas. While, when one node goes down, one PostgreSQL replica will go down. You will need to work to bring up another replica. See crunchydata's example.
Use the volume plugin, such as flocker, REX-Ray. You will still need to create one service for every replica, and bind one volume to one service. You need to create all services in the same overlay network and configure the PostgreSQL replicas to talk with each other via the dns name (the docker service name of the replica). You will still need to set up the postgresql replication among replicas.
I have a question for the DBA's out there: If I scale from a single web/DB server setup to two web/two DB server setup with a load balancer in front of the web servers to route incoming queries evenly... how do solutions like MySQL Cluster work so that a change made to one DB server is immediately known to the other (otherwise, users routed to the other DB server won't see the data or will outdated data), or at least so that the other web server is made aware of the fact that it's reading "dirty data" and it should try again in X seconds so as to get up-to-date data?
Thank you.
TWO ways of doing this.
Active/Active or Active/Passive.
Active/Passive is most prevalent
The data is kept in sync on the passive node.
The cluster is useful configuration in as much as the active node goes down the passive is immediately switched hence no downtime.
The clustering continuously synchronises the 2 nodes in the cluster.
I work with SQL server but I think the basic premise of clustering is the same for mySQL - that is no (or no noticeable) downtime on hardware failure.
EDIT: Additionally the clustering software handles the synchronisation. You don't need to worry. You view the cluster nodes as a virtual directory, which behaves like one server in windows.
here is document explaining this
http://www.sql-server-performance.com/articles/clustering/clustering_intro_p1.aspx
In Windows server clustering (to be distinguished from High Performance Clustering), there is a shared external storage array. The active node takes ownership/control of the storage, and when that node fails, the storage 'fails over' to the previously passive node (which is now the active node). There are also different schemes that allow for independent storage at each node, vs. shared storage. However, these require the application to have enough intelligence to know that it is clustered, and keep the two storage sets in sync.
Clustering is also where a number of nodes handle the workload, this is sometimes called active/active clusters i.e. all the nodes share the workload and are active. This is normally handled by specialist software like Oracle RAC (RAC#Wikipedia) for the Oracle RDBMS database. RAC allows Oracle to scale to very large workloads.
we are planning to implement sql server 2005 cluster in next few months. i wanted to know what steps / precautions need to be taken as a database developer when trying to achieve this? do we need to change any ado.net code (in front end) / stored procs etc etc? are there any best practices to be followed?
reason i am asking this question is : for asp.net load balancing, you have to ensure your code for sessions / application / cache all comply with load balanced environment. (so incase you are using inproc sessions, you have to rewrite that code so that it works on load balanced environment). now this is at your web server level. i just wanted to the right things to do when trying to scale out at database server level
i am sorry if this question is stupid. please excuse my limited knowledge on this subject :-)
You have to make no front end changes to implement a SQL Server cluster, you simply connect to a SQL Server instance as normal.
SQL Server failover clustering is not load balancing however. It is used to add redundancy should any hardware fail on your primary node. Your other (secondary) node is doing nothing until your primary fails, in which case the failover happens automatically and your database is serving connections again after a 10-20 second delay.
Another issue is the cache on the secondary node is empty, so you may see some performance impact after failover. You can implement a "warm" cache on your mirror server using SQL Server database mirroring, but there is no way to do something similar with clustering.
Database clustering is different to load balancing. It's high availability, not "scaling out"
Basically:
2 servers (or nodes) with shared disks (that can only be owned by one node at any time)
one is "active" running a virtual windows server and SQL Server instance
one is monitoring the other ("passive")
you connect to the virtual windows server.
If node 1 goes off line, node 2 takes over. Or it can be failed over manually.
This means: services are shut down on node 1, node 2 takes control of the disks and services and starts up. Any connections will be broken, and no state or session is transferred.