I have started working on a School Management System for learning purposes.
Now I am using only mongoDB database, but I am thinking about a clean architecture on this project so I want to hear some suggestions from you.
I want to divide all the data to handle a big traffic in the fastest way.
I have the following modules in my app:
auth system
user types
subjects
classes
marks
absences
schedule
forum
chat (with sockets)
I thought to keep the first 8 points mentioned above in MySql and I am thinking about the chat to use RabbiMQ and mongo with caching in Redis.
1. What would be the difference between RabbiMQ and mongo with caching in Redis and using cassandra with RabbitMQ for chat?
2. What would be the best solution to handle a big traffic for chat?
3. If you would architect this app in another way, how would you do it and why it would be better than the way mentioned above?
Thank you!
Related
I'm designing an app which needs to have some collaboration functionality. So 1 to many users can edit certain attributes of an event they plan together. e.g. the main-admin can change the title, picture etc. while all admin-users could change the date for example.
I would like to get some ideas how one would approach this in the modern world. Are there fancy frameworks etc.
Q: Is the best way storing it centraly on some server or would some peer to peer data storage work?
Q: My gut-feeling is that a web-application would probably be the easiest way, where people work on the object stored on the server instead of trying to sync a local copy with some central repository.
Is this correct?
Q: Are there mobile frameworks which could do the syncing, locking etc for me?
Thank you for some hints and suggestions. I know the questions are a bit broad, but I'm looking for directions not finished solutions. Thank you.
Kind regards
Fred
Some thoughts:
1a) There is no "best" way without a metric for better/best. But yes, having a server is almost certainly simpler, which is probably part of 'best' for most of us.
1b) Actually, there is always a server. Even p2p systems have clients and servers, it's just that every node is both a server and a client.
2) Yes, a web app would certain give you a lot of plumbing for free, & would probably be fastest/cheapest route to a working app. An alternative would be an olde worlde client/server database. A shinier approach might be mobile apps which use a webservice to communicate with a central server.
3) databases do that. But actually, if you use a web app it's probably not hard.
Analogies:
Apart from web apps, version control systems do exactly what you've
just described. they even do offline editting and subsequent merging.
Straightforward CRUD applications against a database also do what
you've just described.
But perhaps I'm under-estimating what you mean by collaboration?
firstly I apologise if this is a ridiculously simple question to answer but it has been bothering me for a while.
I am trying to understand what salesforce actually is, I mean in technical terms. I have read the websites documentation and the wikipedia page but I am trying to understand what's behind all this fluffy terminology.
My understanding is that salesforce is a cloud based database which stores a very high volume of information and all salesforce apps consists of scripts that query this database and model them in different ways depending on the intended application, is this correct?
Thanks !
Software as a Service (SaaS)
To get program you need to download it, install, configure and so on. If your system have a lot of users it's very hard to configure ans support single user installation.
Imagine that you improved application, new release for example. You need update every instance.
With SaaS model you have a shared web application, that do the same thing as old downloadable one. But it's much easier to support it, because ideally there is just one instance of it.
Salesforce is a company that provides its own system by SaaS model, but not only. It is also a platform for developing new applications.
I was just wondering how many db queries might facebook be issuing to render a user's home page. Does anybody have some idea on how the facebook DB is designed. I've heard it runs MySql and there are thousands of replica plus more memcache server than DB Servers.
Is the facebook data shard-ed?
If it is does it go to every shard and search for the latest update of my friend. In worst case if I've 100 friends and suppose facebook has 101 shards, there is a possibility that each of my friend is in a different shard. How might facebook be handling this?
I'll be highly grateful if somebody can provide me seom hints or pointers towards something like "How to Design a DB for Social Networking Website". I'm just curious!
Facebook is using LAMP structure. Facebook’s backend services are written in a variety of different programming languages including C++, Java, Python, and Erlang and they are used according to requirement. With LAMP Facebook uses some technologies ,to support large number of requests, like
Memcache - It is a memory caching system that is used to speed up dynamic database-driven websites (like Facebook) by caching data and objects in RAM to reduce reading time. Memcache is Facebook’s primary form of caching and helps alleviate the database load. Having a caching system allows Facebook to be as fast as it is at recalling your data.
Thrift (protocol) - It is a lightweight remote procedure call framework for scalable cross-language services development. Thrift supports C++, PHP, Python, Perl, Java, Ruby, Erlang, and others.
Cassandra (database) - It is a database management system designed to handle large amounts of data spread out across many servers.
HipHop for PHP - It is a source code transformer for PHP script code and was created to save server resources. HipHop transforms PHP source code into optimized C++. After doing this, it uses g++ to compile it to machine code.
If we go into more detail, then answer to this question go longer. We can understand more from following posts:
How Does Facebook Work?
Data Management, Facebook-style
Facebook database design?
Facebook wall's database structure
Facebook "like" data structure
At this website you find lots of details about those big internet companies and their technical structures:
http://highscalability.com/
Adding to #Somnath Muluk's answer - Facebook uses few other technologies like Hadoop etc.
Refer to the following links for more details:
http://www.quora.com/Facebook-Engineering/What-is-Facebooks-architecture
Facebook Architecture
Hope it helps.
Zero. On average, that is. A highly interconnected network with a large number of users such as Facebook can only run effectively if it runs fully out of ram for the pages that are shown often. Nearly all data should already be in the memcache.
I'm in the process of designing an application that would tie into the database for multiple e-commerce stores.
I have the general business logic down but the problem is, I have no idea on the most painfree and effective way to do the data integration. I know how to access my own databases, but this is something I always just left to my hosting provider to handle the backend and I just developed the interface
Can someone point me in the right direction on this? There is a large emphasis on ease of installation. So my customers (once I get them) should be able to very easily integrate their system into my app.
RESTful web services are what many folks do for this.
Start here: http://en.wikipedia.org/wiki/Representational_State_Transfer
Then revise this to be a more specific question.
You might need to look at direct VPN/SSL connection from their server to yours.
We have a new django powered project which have a potential heavy-traffic characteristic(means a heavy db interaction). So we need to consider the database scalability in advance. With some researches, the following questions are still not clear to us:
coarse-grained: how to specify one db table(a django model) to a specific db(maybe in another server)?
fine-grained: how to specify a group of table rows to a specific db(so-called sharding, also can in another db server)?
how to specify write and read to different db?(which will be helpful for future mysql master/slave replication)
We are finding the solution with:
be transparent to application program(means we don't need to have additional codes in views.py)
should be in ORM level(means only needs to specify in models.py)
compatible with the current(or future) django release(to keep a minimal change for future's upgrading of django)
I'm still doing the research. And will share in this thread later if I've got some fruits.
Hope anyone with the experience can answer. Thanks.
Don't forget about caching either. Using memcached to relieve your DB of load is key to building a high performance site.
As alex said, django-core doesn't support your specific requests for those features, though they are definitely on the todo list.
If you don't do this in the application layer, you're basically asking for performance trouble. There aren't any really good open source automation layers for this sort of task, since it tends to break SQL axioms. If you're really concerned about it, you should be coding the entire application for it, not simply hoping that your ORM will take care of it.
There is the GSoC project by Alex Gaynor that in future will allow to use multiple databases in one Django project. But now there is no cross-RDBMS working solution.
There is no solution right now too.
And again - there is no cross-RDBMS solution. But if you are using MySQL you can try excellent third-party Django application called - mysql_replicated. It allows to setup master-slave replication scenario easily.
here for some reason we r using django with sqlalchemy. maybe combination of django and sqlalchemy also works for your needs.