Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'm building an app, that will need to store approx 100-200GB of JSON data per month with ~20.000 write operations per minute.
Is there any service that won't require millions of dollars to store this data?
One option is to use Azure's HDInsight. You'd pay for the HDInsight servers in addition to the storage of the data. Of course your costs will keep climbing as you add more and more data, so some form of archive would make sense. How long do you have to keep data easily available?
HDInsight Pricing
Storage Pricing
I think you may be overestimating your data growth. I would start with either AWS or Azure and build my own datacenter if volume goes near the level you are talking about. Yes this involves some migration later on but its always good to grow by observation.
Thanks everybody. After all, I decided to go with Azure Table Storage.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 10 months ago.
Improve this question
I have been thinking of making a program to use in my company. I would like to store information in a (local) database and use this to keep track of the payments of my clients. I am most experienced in programming in Java. Do you have any suggestions for these databases?
I believe you are probably looking for SQLite. It is very light, basic, works with SQL,but doesn’t have any built in relational methods to link multiple tables together(JOINS, etc). As you mentioned you’ll be using Java, here’s the SQLITEJDBCPackage. Also, here’s a blog that can help you get started.
On the other hand, there is a wide variety of databases present in the market like:
RDBMS: MySQL, PostgresSQL
NoSQL: MongoDB(can run on cloud and locally), Neo4J
Time Series Database(If you storing IOT or time dependant data): InfluxDB
Cloud Databases(Might not be relevant to you since you want a local setup, but just to help you understand better): Firebase, Neo4J, MongoDB, AWS RDS, etc.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
This question is a little more generic, a brainstorm one. I'm about to develop a small website, and I still don't know if it’s better for me to store my “Text” (to fill Labels, Messages, etc) data on the Database or just on the frontend.
I know that for a fact, consulting the BackEnd Database is slower than just searching a specific file, but it’s also better to update the list later-on (when the website is developed) by just running a script.
I want to know some opinions, experiences, advantages and disadvantages about both.
Edit: For the technologies, i was thinking in using ExtJS with a Java Backend, I'm not quite sure about the BD yet.
Consider what data you are storing and the purpose of your website.
Advantages of front end storage: quicker
Advantages of database storage: more secure/structured
If your strings are sensitive then I would secure them in your database. Any client information, including "Text" data should be stored on the back end. If the strings are only relevant to you as the site owner then I don't see a problem with storing them on the front end.
Also perhaps specify which technologies you are using to build this site to get more specific responses.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
For a small start-up mobile app/website what options are there for storing its data? I.e. Physical server or cloud hosted data base such as azure.
Any other options or insight would be helpful thank you!
Edit:
For some background I'm looking at something that users could regularly upload data to and consumers could query to find results through an app or website.
I guess it depends on your work load and also on the your choice of data store. Generally, SQL based storage are costlier on cloud based solution due to the fact that those can be only vertically upgraded whereas no-sql ones are cheaper.
So according to me you should first decide on your choice of data-store, which depends on following factors:
The type of data; is your data structured or it falls under non-structured category?
Operations that you will perform on the data. Do you have any transactional use-cases?
Write/Read pattern; is it a read heavy use case or a write heavy one ?
These factors should help you decide on an appropriate data-store. Each database has its own set of advantages and disadvantages. The trick is to choose one based on your use cases and above mentioned factors.
Hope it helps.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
Every time a certain form in a site dashboard loads, I generate a unique ID and store it into the database, when the form is submitted I mark the ID as "used" and don't allow to submit forms which are already used.
Question: Is it worth to store date/time of form load and remove unique IDs say two weeks old?
Or to keep this data in MySQL forever?
I think saving timestamp and clearing out old dara is an excellent idea.
Without that, your web site is not scalable. In fact it could be attacked by repeatedly loading the page and filling your database's disk to capacity, effectively killing your site.
Wether it is worth to store the date-time of the form in the database is up to your decision and your business needs, but, you should not keep any data stored, in the on-line database, which is not needed any longer.
If you need to keep such data for historic report purposes then create an off-line/report database and transfer the data, periodically, from the on-line to "report" DB.
The less amount of data you have in your db the faster your RDBMS will be to retrieve/manipulate it.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I was at a .NET development group meeting a couple weeks ago and the speaker was extolling the virtues of NoSQL and how even relational data doesn't have to be stored relationally if you just index lots of data. So, my questions are: was he blowing smoke? How does one craft an index to be more efficient than the last? Does indexing just logically store the information in a table in a logical format i.e. alphabetically?
Well relational data is needed more for data integrity than indexing. Speed is not the only consideration when choosing a database. SQL Server and other enterpise databases can perform very well if they are designed by people who know what they are doing. Unforuntately most relational databases are designed by data amateurs and their performance reflects that.
NoSQL databases and relational database are used for different things. I would never consider putting a financial application in noSQL for instance because of the need for data integrity and internal controls to prevent fraud and ensure records are consistent and correct. However a website where data quality doesnt matter so much (think Google - who would notice if they failed to serve up every single website that mentions Bill Gates in a query) then yes it is a good choice.