I want to connect a serverless MongoDB instance to MongoDB charts.
I don't see any of my serverless instances when I try to add data sources on my Mongo Chart dashboard.
If this is not possible, would somebody know if there is a way of upgrading a serverless instance to a dedicated one? Apart from running a script, and creating a new instance.
Note: I am not very optimistic on the last question, since of course the underlying data probably needs to go from one server to the other.
According to the Serverless Instance Limitations doc, this is unfortunately not possible.
As for the second part of your question, the answer is also negative, but apparently there's a room for optimism - this feature is marked as 'coming soon'.
Related
Why I'd want to have multiple replicas of my DB?
Redundancy: I have > 1 replicas of my app code. Why? In case one node fails, another can fill its place when run behind a load balancer.
Load: A load balancer can distribute traffic to multiple instances of the app.
A/B testing. I can have one node serve one version of the app, and another serve a different one.
Maintenance. I can bring down one instance for maintenance, and keep the other one up with 0 down-time.
So, I assume I'd want to do the same with the backing db if possible too.
I realize that many nosql dbs are better configured for multiple instances, but I am interested in relational dbs.
I've played with operators like this and this but have found problems with the docs, have not been able to get them up and running and found the community a bit lacking. Relying on this kind of thing in production makes me nervous. The Mysql operator has a note even, saying it's not for production use.
I see that native k8s statefulsets have scaling but these docs aren't specific to dbs at all. I assume the complication is that dbs need to write persistently to disk via a volume and that data has to be synced and routed somehow if you have more than one instance.
So, is this something that's non-trivial to do myself? Or, am I better off having a dev environment that uses a one-replica db image in the cluster in order to save on billing, and a prod environment that uses a fully managed db, something like this that takes care of the scaling/HA for me? Then I'd use kustomize to manage the yaml variances.
Edit:
I actually found a postgres operator that worked great. Followed the docs one time through and it all worked, and it's from postgres docs.
I have created this community wiki answer to summarize the topic and to make pertinent information more visible.
As Turing85 well mentioned in the comment:
Do NOT share a pvc to multiple db instances. Even if you use the right backing volume (it must be an object-based storage in order to be read-write many), with enough scaling, performance will take a hit (after all, everything goes to one file system, this will stress the FS). The proper way would be to configure clustering. All major relational databases (mssql, mysql, postgres, oracle, ...) do support clustering. To be on the secure side, however, I would recommend to buy a scalable database "as a service" unless you know exactly what you are doing.
The good solution might be to use a single replica StatefulSet for development, to avoid billing and use a fully managed cloud based sql solution in prod. Unless you have the knowledge or a suffiiciently professional operator to deploy a clustered dbms.
Another solution may be to use a different operator as Aaron did:
I actually found a postgres operator that worked great. Followed the docs one time through and it all worked, and it's from postgres: https://www.kubegres.io/doc/getting-started.html
See also this similar question.
Could someone explain the benefits/issues with hosting a database in Kubernetes via a persistent volume claim combined with a storage volume over using an actual cloud database resource?
It's essentially a trade-off: convenience vs control. Take a concrete example: let's say you pay Amazon money to use Athena, which is really just a nicely packaged version of Facebook Presto which AWS kindly operates for you in exchange for $$$. You could run Presto on EKS yourself, but why would you.
Now, let's say you want to or need to use Apache Drill or Apache Impala. Amazon doesn't offer it. Nor does any of the other big public cloud providers at time of writing, as far as I know.
Another thought: what if you want to migrate off of AWS? Your data has gravity as well.
Could someone explain the benefits/issues with hosting a database in Kubernetes ... over using an actual cloud database resource?
As previous excellent answer noted:
It's essentially a trade-off: convenience vs control
In addition to previous example (Athena), take a look at RDS as well and see what you would need to handle yourself (why would you, as said already):
Automatic backups
Multizone deployments
Snapshots
Engine upgrades
Read replicas
and other bells and whistles that come with managed service opposed to self-hosted/managed one.
But there is more to it than just convenience/control that this post I trying to shed light onto:
Kubernetes is adding another layer of abstraction there (pods, services...), and depending on way of handling storage (persistent volumes) you can have two additional considerations:
Access speed (depending on your use case this can be negligent or show stopper).
Storage that you have at hand might not be optimized for relational database type of I/O (or restrict you to schedule pods efficiently). The very same reasons you are not advised to run db on NFS for example.
There are several recent conference talks on kubernetes pointing out that database is big no-no for kubernetes (although this is highly opinionated, we do run average load mysql and postgresql databases in k8s), and large load/fast I/O is somewhat challenge to get right on k8s as opposed to somebody already fine tuned everything for you in managed cloud solution.
In conclusion:
It is all about convenience, controls and capabilities.
Is it possible to run postgres (essentially, a non-HTTP service) in a custom Google App Engine Flexible container? Or will I be forced to use Google's Cloud SQL solution?
TL;DR: You could do that, but don’t. It’s better to externalize the persistent data storage.
Yes, it is possible to run a PostgreSQL database as a microservice (named simply a 'service' in Google Cloud Platform) in a custom Google App Engine Flexible container. However, that raises another important question, namely why would you like to run an SQL database inside a container. This is a risky solution, unless you are perfectly sure about what you are doing and how to manage that.
Typical container orchestration is based on stateless services which means that they are not intended to store persistent data. This kind of containers do have some form of storage sometimes, like NoSQL databases for cache or user session information. This data is not persistent, it can be lost during restarts or destruction of instances in an agile containerized application environment. PostgreSQL databases are rather used as stateful services and do not suit the aforementioned model. Putting such database into a container, one can run into problems like data corruption or direct concurrency when accessing some shared data directory. Also, in Google App Engine Flexible it’s not possible to add a shared persistent disk, the volumes are attached to instances and destroyed together with them. Much safer solution is keeping the SQL database in an external, durable storage, as Cloud SQL that you have mentioned. There are numerous blog posts and articles that elaborate this issue with the stateless/stateful services, like this one.
It should be mentioned that if you are to use the container in a local environment or for test/development (and you are not looking for a durable state of the database), putting a PostgreSQL inside a container should be perfectly ok. Also, if you design a special way of splitting your data across instances this could work fine, as the guys did with their MySQL servers in this article. So once again, the idea of putting a PostgreSQL database in a container should be carefully thought-out, especially that there are so many options of a safe externalization of such a service.
And just as a side note, you are not forced to use Cloud SQL. The database can be hosted on Compute Engine, another cloud provider, on premises, or can be managed by a third-party vendor. In case of hosting it in Compute Engine the application is able to communicate with the database inside the same project using the internal IP of the Compute Engine instance. Using Cloud Launcher you can quickly deploy PostgreSQL and other popular databases to Compute Engine. Check these Google docs for more information about using third-party databases.
I am new to servers and online databases, so please bear with me.
I have a question regarding database server communication on mobile devices as follows:
I am currently developing a game application on iOS. I have set up a non-SQL database on Cloudant and I would like to access that data on my iOS device. I have to update multiple database entries each time I complete a round, and I also need to read multiple entries on my database to refresh the leaderboard. I have tried to access multiple entries on Cloudant individually via device before, but most of them returned as timeout.
Thus, right now I have written several PHP scripts on my application server so that my device only needs to access the script once, and do multiple updates on my database or filter through the data I require from Cloudant. However, this means I need an additional server, meaning higher costs. I feel there should be a better or more elegant solution out there, and thus I would like to ask for help from everybody out here. Is it better to do all the updates directly from the device, or to enlist the help of a 3rd party?
Thanks for your time!
For security reasons alone it is necessary to use a server in front of the cloudant database. I assume you don't want every user of your app to be able to access the whole database. Also, the reasons you gave seem valid to me. It's generally a good idea to reduce the number and size of requests for a mobile application. Also, this might allow you to do some caching in the PHP server, ultimately reducing your costs.
I am developing a mobile app and I am now getting to the stage where I want to put all my back-end server code online for the production version (I actually want to release my app). The back-end consists of 2 parts:
MySQL Database
Actual PHP files for app access.
Looking for a service that is popular, highly scaleable and proven, I came across Amazon Web Services. Now while RDS seemed very straight forward to use for the database part, EC2 was way too complicated for me and I decided on using a managed solution, which puts it out of the question.
Seeing as the simple PHP pages could be moved easily to any service at any time (unlike the database which is a bit harder to move with no downtime), I have decided to use a much cheaper solution for the time being for the PHP pages.
My Question is: Does it make senesce to use Amazon RDS with app hosted on other managed service (rackspace, azure, godaddy etc...)?
While it is technically possible, it depends on your application if it is generally okay to do so. Keep in mind that there will be extra delay in the communication between the RDS instance and your server - and this will happen with every database request you do. You can for example check the network latency with simple tools like ping or traceroute. In general you are looking for the lowest latency possible for database queries.
However if your app hardly does any reads from the database, this could work. If not, I would strongly advise against this. In this case you would either have to install a MySQL database on your managed server or start using EC2.