From high level usage, please help me with the following.
I am looking to utilize the database from Expedia EAN. It recommends you update the database weekly via cron job.
On the home page of my application, if I have a enter your location/dates/number of guests, and search for hotel availability, should I be fetching data via this local database or from via the live API?
It doesn't seem to make sense to fetch this from the local database, as how will it be up to date in terms of use availability. If that's the case, it doesn't really seem useful to use the local database unless its just for browsing hotels?
Thanks
Expedia's EAN provides the Database with pseudo-static (slowly changing) data such as hotel's location, address, images, description etc. These are the ones to be downloaded as files once per day or so.
Then they also provide dynamic data such as availability and prices via their API. Those data surely can be fetched at the time of request.
Related
This is the scenario:
I am developing a clinic management sofwate in vue, nodejs and mongodb, which each account (each client) may have their own data with their own users and their own patients in their page after they login. Shall I use a separate DB for each client or store everything in one DB and query all data from all the clients in every request ? I searched a lot but couldn't find a clue about which way to go .
In my experience, yes, separated DB(s) per customer is the best solution in the long run.
In your main administrative DB you can map customers to DB-servers giving you a lot of flexibility, for example to give more resources to big customers without impacting small ones.
Also consider using storage.directoryPerDB:true config option to reclaim disk space after deleting a DB (trial expired customer)
I have a situation where I have to save UK citizen data to UK server and US citizen data to US server.
Customer (citizen) will fill a form in a web application, telling which country citizen he/she is. The storage is then chosen by a web application.
Is it possible to achieve something like this, or not?
And there also must be a possibility to query all citizens from both storages at once.
What you need is called zone sharding in MongoDB. Essentially, given a shard key, you can tell MongoDB to store certain range of keys in certain part of the sharded cluster.
There is an example in Segmenting Data by Location which describes your use case.
Having said that, if you don't need to query all citizens very frequently, it's likely to be easier to deploy separate deployments under each jurisdiction, and perform the same query on each of them. This way, if you need the US servers to be maintained, it won't affect the UK operation.
I need to save requests data (main data : request id, date, url. additional data : headers and bodies).
Currently it's store in my databse with only 1 table.
The biggest part of memory take additional data. So when it will be to many requests stored, here can be problem with performance.
Is there any way to store additional data separately from main data? or any other advices?
Thanks.
Work with ASP.Net MVC4 and Ms SQL Server 2012.
The software like Apache web server stores access info in log files (access.log). You could probably implement something similar. Then you can rotate your logs daily.
This way you have your data separated and filesystem storage is really fast, so performance won't be a problem. The problem comes when you have to do some advanced querying on your log.
You could probably use ELMAH: https://code.google.com/p/elmah/ and set it to store info in XML.
I hope anyone can help me out in this topic, even if it's not a specific programming question.
I'm writing a bachelor thesis, where I compare MySQL to MongoDB and I want to write something about Youtube, as the platform has to handle many requests with heavy dataload.
The only good resource which I found was this video: Seattle Conference on Scalability: YouTube Scalability
As the conference was in 2007, I can imagine there were some updates regarding to the database.
The last information that I have from this talk is that the thumbnails are stored in a BigTable database and the metadata in MySQL. Are there any changes since then?
Where are the videos stored? Is there an entry in the MySQL table, which refers to the stored video?
Thanks in advance for the answer!
According to this, youtube still uses mysql: http://code.google.com/p/vitess/wiki/ProjectGoals
I am not sure of how things are at youtube but I am in process of developing a similar application for our client. So what we are doing is we are making the use of best of both worlds i.e SQL and NoSQL..
We store the videos on disk and store the path to these videos in MySQL db table. Then we have a separate table which holds the genre and video mapping i.e which video belongs to which particular genre.
Today with vast of pool of user data we are in position to leverage upon these data like we had never been before, so you see things are now way different then 2007 and with the popularity and dependency of people on internet when it comes to sites like you tube we have vast set of unstructured data which if used properly can give you great results. So in our project we store the site admin and reporting stuff like user db, video locations and genre mapping etc in MySQL and store the unstructured data about user interaction in NoSQL database. We then use the NoSQL data to do all the analytics and give appropriate results to the user.
They are using mysql with Bigdata.
The user information such has who uploaded the file,file information all will be stored in mysql and data will be stored in Bigdata.
I think they are using database that can use FileTable
How is this problem generally solved?
To give some context, I have a single database and multiple processes connected to that database (a mobile api, a custom content management tool and a custom front end website), they all run on different servers. Sometimes it is useful to get delta changes on the database; for instance say I update the database with new data using the content management tool (which then sets some metadata on the data that was changed, specifically its updated_at field), and all my mobile apps need a local copy of the database to work. It wouldn't make much sense to redownload the whole db, so it's useful for the app to send its local last_updated timestamp and retrieve the subset of fields which were updated since that time. But the mobile api's server's last_updated and the database server's updated_at are not calculated from the same time source, each server has its own clock.
Is using a time based "freshness indicator" just a dirty mess or is there a robust and efficient way to do it? Or is there a better approach? I'm thinking something like incrementing a version number. What's the best practice for this kind of thing?