TDengine is a time-series database with cluster open-sourced.
but any specific feature related to cloud-native?
anyone could give me some information?
I think first we release the TDengine database cloud service .
then TDengine 3.0 supports separation between storage and computation
in the last ,The virtual data node support split and merge.
Related
We are planning to implement a project in Azure cloud where data storage will be Azure Data lake for now and in future HDP will be implemented and ADLS will be the extended datanode. From ADLS we want to expose data for Dashboard creation using Tableau. Initial plan was to use Hive and Tableau will connect to Data through Hive. But here comes the performance issue as:
There will be multiple users who will have access to Data through Tableau(100+)
We will also have to expose Data to different portal with API calls.
Which means multiple connectivity will be established at the same time which will hit hive . My question is:
Can hive serve the purpose with minimal time?
How can i measure the performance?
I dont want to let my users to sit back after running a query in tableau and wait for a long time to see the dashboard.
Would you please share your experiences in this design issue? Should we use Hive or should We use some other tools which have better performance to work with tableau and HDFS storage. Someone suggested me to use Azure SQL Server and connect Tableau to SQL server. But its again the old fashion and also matter of cost as price is related with the execution of each query.
If you have any better solution experience please share , would be greatly appreciated.
Thanks in advance.
Hive LLAP could work, if you can get it installed.
Otherwise, at my work, we've had good experience with PrestoDB and Tableau on S3 data.
Some teams use Spark SQL, and you can setup a Spark Thrift Server, that should be compatible with the Hive JDBC/ODBC drivers
I installed grafana as explained on the website. http://docs.grafana.org/installation/debian/ And since I'm using postgresql as database I wanted tp add may data source. Unfortunately I didn't found the option postgres (only Graphit, MySQL, InfluxDB and two other types were there). There is even any postgres plugin on the website (The one that I found didn't work https://github.com/sraoss/grafana-sqldb-datasource).
Do have a solution so that Grafana supports Postgres ?
Currently Grafana only supports Postgres as the data store for Grafana (for storing dashboards, users etc.) and there is no published plugin for using Postgres as a data source.
The MySQL data source plugin has just been added to Grafana and the Postgres plugin is on the Grafana roadmap and should be released later in the year.
If you are wondering why there are so many data sources supported but not Postgres, it is because relational databases are not traditionally what Grafana is used for. The main use case is visualizing time series data (often more data than a relational db can handle). Relational dbs are not built for this use case which is why most people use something like Graphite or InfluxDB. But it is quite common to have some data in a db like Postgres that you would like to show on a dashboard or combine with some data from a time series db. This is why the Grafana team is planning to release a plugin soon.
EDIT: The Postgres data source is now merged to Grafana master and will be released in Grafana 4.6: https://github.com/grafana/grafana/pull/9209
I'm looking for a tutorial or something that allow me to learn Presto step by step.
The idea is to start integrating file's and MSSQL, which is my knowledge area.
Unfortunately, since it is a relatively new area, I didn't find anything more than Facebook page or the Presto.io page, however it is not good enough for someone that want to start knowing the big data world from scratch.
I will appreciate your help and/or orientation in this area.
Presto has 2 primary use cases:
querying data stored in a cluster (on Hadoop's HDFS) or in a cloud (e.g. Amazon S3)
data federation, i.e. querying (and joining) data from multiple data sources (e.g. HDFS, S3, traditional RDBMS like PostgreSQL or SQL Server)
As far as SQL Server support is concerned -- Presto supports connecting to SQL Server since https://github.com/prestosql/presto/commit/072440cbb2c8df2a689c4c903dd325013eae41a0.
When it comes to querying files -- Presto uses Hive's Metastore to keep track of metadata (everything besides actually reading the data). Thus the files must reside on HDFS or S3 to be accessible (other cloud data stores like Azure's Blob are, AFAIK, not supported yet).
I know Aster Data leverages SQL Map Reduce, ncluster and analytic capability.
From Database architecture perspective which family does Aster belongs to?
Aster database doesn't formally belong to certain database family, but you can identify it with several database types:
it's distributed, parallel, relational database;
it's MPP (massively parallel processing) database;
it's based on PostgreSQL open source code (forked);
it's NOT based on Teradata database.
I do not know exactly about right name, but it is sharded db, what means one Queen server and several workers with running postgres instances.
Agree with topchef. Hope the below gives you some high level information.
The database is built on top of Postgres similar to other databases like Netezza and Greenplum.
Asterdata is built on postgres but in a distributed manner.
It has something called vproc's which are similar to a standalone postgres db instance.
A Node (worker) will have multiple vprocs and all the nodes are co-ordinated using a Queen node (Master).
Though its built on Postgres, not all features of postgres are ported to Asterdata because of the distributed nature of the system.
On this page it says
You can access a familiar, highly available SQL database
but then on the same page down below it says
Unsupported Features: MySql replication
Can anyone clarify this?
Each Cloud SQL instance replicates your data at the storage level, rather than within MySQL. So a single MySQL instance writes every byte multiple times in multiple geographic locations. Even if an entire datacenter becomes available a new MySQL instance can be spun up in a new location and carry on serving your data.
See https://developers.google.com/cloud-sql/faq#replication