I'm trying to create a social media type app using Codename One. In order to do this, I was wondering which database service I should use within my application: just the Storage class, SQLite, or MySQL? I expect to be storing significant amounts of data further down the road.
Storage and SQLite are device databases where in Storage you would normally want to store small amounts of data mainly for caching and offline purposes and SQLite for larger sets of data where you might need more sophisticated queries on the data.
MySQL is a server side database, if you're building a social app you can use mysql on your server to store your data and connect with http/https to the server and query the data to display on the device.
Related
I need to design a scalable database architecture in order to store all the data coming from flat files - CSV, html etc. These files come from elastic search. most of the scripts are created in python. This data architecture should be able to automate most of the daily manual processing performed using excel, csv, html and all the data will be retrieved from this database instead of relying on populating within csv, html.
Database requirements:
Database must have a better performance to retrieve data on day to day basis and it will be queried by multiple teams.
ER model, schema will be developed for the data with logical relationship.
The database can be within cloud storage.
The database must be highly available and should be able to retrieve data faster.
This database will be utilized to create multiple dashboards.
The ETL jobs will be responsible for storing data in the database.
There will be many reads from the database and multiple writes each day with lots of data coming from Elastic Search and some of the cloud tools.
I am considering RDS, Azure SQL, DynamoDB, Postgres or Google Cloud. I would want to know which database engine would be a better solution considering these requirements. I also want to know how ETL process should be designed- lambda or kappa architecture.
To store the relational data like CSV and excel files, you can use relational database. For flat files like HTML, which doesn't required to be queried, you can simply use Storage account in any cloud service provider, for example Azure.
Azure SQL Database is a fully managed platform as a service (PaaS) database engine that handles most of the database management functions such as upgrading, patching, backups, and monitoring without user involvement. Azure SQL Database is always running on the latest stable version of the SQL Server database engine and patched OS with 99.99% availability. You can restore the database at any point of time. This should be the best choice to store relational data and perform SQL query.
Azure Blob Storage is Microsoft's object storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Your HTML files can be stored here.
The ETL jobs can be performed using Azure Data Factory (ADF). It allows you to connect almost any data source (including outside Azure) to transform the stored dataset and store it into desired destination. Data flow transformation in ADF is capable to perform all the ETL related tasks.
I would like to build an application that serve a lot of users, so I decide to use cloud datastore because it is more scalable, but i also want to have an interface that will help me observe my data with some complex sql query.
so i decide to build my data with tow data base (cloud data store and cloud sql) and the users for my application will get the data from the datastore, and me with my interface i will use cloud sql.
The users will just read data they will not write to the datastore, but me with my interface I would read the data from my cloud sql so i can use complex query, and if i want to write or change the data, I will change both data in cloud sql and data sore.
what do you think? is there another suggestion ? thank you
I want to get an idea on how to achieve this;
I have an application that runs at 5 different geographical locations. Eg: Texas, NY,California, Boston, Washington
This application saves data to a local database, which is located at that location.
I want to do data warehousing, So is it a must to have just have one database (Where all the 5 applications will now save its data in a single database - without having local DBs)
Or is it possible to have 5 local databases, and do data warehousing by retrieving data from those local DBs to a central DB and then performing data warehousing.
Please give me your thoughts and references.
You have three options for this:
you use a single, centrally hosted database server. Typical relational database servers can be directly accessed via network these days: mySQL, Postgresql, Oracle, ... This means you can implement an application which opens a network connection to the database server and uses that remote server to store and retrieve the data as required. Multiple connections are possible at the same time.
you use a single, central database server but put a wrapper around it. So some small network layer application layer acting as a broker. This way you can address that central instance over network, but via standard protocols like for example http.
you use a decentralized approach and install a database instance at each location. Then you need some additional tool to perform a synchronization. For most modern database servers (see above) such tools exist, but the setup is not trivial.
If in doubt and if the load is not that high go with the first alternative.
I'm developing an application that needs to store large images
and about 100 image a day
about 3 people work on the same application DB (desktop application connected to same db)
any one of theme can view , edit, delete other work
which means all of theme can access certain image at once
now the question is what is the best way to store images this big
(currently I'm storing the images in a table but i know this eventually will suck)
I'm using sql server 2008
You could use the new in v2008 FILESTREAM data type to store your images. This is a hybrid approach that means the data can be queried from SQL, but the data itself is stored in the filesystem rather than in the database.
Hi i am building a window apps retailer pos but was wondering what is the best method to design the database. Should i just use 1 database to store all my clients data?
Meaning to say if i have 100 clients from different businesses using my App, all of their data will be stored in 1 database.
e.g. i will store 1 company column in the user table to indicate which company does the customer or transaction belongs to.
My current practice is i create new database for each business and put it installed into their local machine. (Got to manually install sqlserver + sqlexpress).
Do u think it is more easier for me to design in this way? and i can just put the database online to sql server. Will i be getting any latency ? how bad will it be? I heard Window Azure able to handle this well. In my case i think the speed and data size per business is not really a concern.
Could you advice?
You should definitely look at other alternatives within Azure for storing data, specifically Azure Storage Tables and Blobs.
Utilizing all of the Azure Storage Options with SQL Azure will allow you to choose different data tiers depending on your application's needs and your desired cost structure. Running everything inside of SQL Azure will cost you more in the long run, but it makes a good place to tie together federated data for relational reporting, whereas you can store each tenant's data inside of Azure Tables, using PartitionKeys which keep each client's data separated from the others.