Please fill me up with the possible architecture of Uber, Lyft like applications and their servers. What kind of technologies they use? The protocols for mobile-server communication? (esp tracking the user/car location and updating them live)
https://www.youtube.com/watch?v=vujVmugFsKc
This video explains a lot on server architecture and technology uber uses, but its bit old now. The speaker mentions 'mobiles update GPS data every 4 second', what would the protocol be in this case? Poker apps in common connects upto 9 players per table and updates the cards in realtime for each mobile, what protocol they use for mobile-server communication? technologies they use and architecture of their server?
Few useful links http://yalantis.com/blog/uber-underlying-technologies-works/
Stackshare lists this information:
http://stackshare.io/uber
http://stackshare.io/lyft
You can get a good understanding of a companies basic stack by looking at their jobs page:
https://www.uber.com/jobs/list?team=Engineering
https://www.lyft.com/jobs
Related
I have started working on a School Management System for learning purposes.
Now I am using only mongoDB database, but I am thinking about a clean architecture on this project so I want to hear some suggestions from you.
I want to divide all the data to handle a big traffic in the fastest way.
I have the following modules in my app:
auth system
user types
subjects
classes
marks
absences
schedule
forum
chat (with sockets)
I thought to keep the first 8 points mentioned above in MySql and I am thinking about the chat to use RabbiMQ and mongo with caching in Redis.
1. What would be the difference between RabbiMQ and mongo with caching in Redis and using cassandra with RabbitMQ for chat?
2. What would be the best solution to handle a big traffic for chat?
3. If you would architect this app in another way, how would you do it and why it would be better than the way mentioned above?
Thank you!
For a hobby project of me i'm looking to implement a group voice chat feature. Pretty straight forward: I'm running a server to which multiple clients (mobile) can connect to. Some Clients are in the same "group" and I want them to have an audio chat feature.
I already set up a UDP server client with C# to which clients can connect to. I have successfully implemented audio distribution between clients over the server and the rudimentary features are working pretty well.
I am not sure if i'm going in the right direction with this approach though.
I'm stuck with the implementation of mixing the different voices for example (two people talking simultaneously and another is listening to both). I don't really know how I can mix both voices together and generate different outputs for different clients - above mentioned example: The two people talking should only receive the input of the other person while the one not talking should receive a mix of the other two talking.
What is the best server sided structure for this? Should i maybe go in a completely different direction and work with SIP? I'm having a hard time finding suitable resources for this problem online and i'm really stuck.
Thanks for your help!
Let me suggest using standards in your application. If (as I strongly recommend) your application is a web application, WebRTC make your job dramatically easier. Please see in WebRTC samples some ideas that I'm sure will inspire you, including multiple peer connections
If you are only interested in group calls, you could install a PBX server as asterisk that has a powerfull set of conference capabilities. You could use a SIP library on top of WebRTC in clients (e.g. sip.js, sipml5) to connect via SIP to asterisk and obtain conference services. This could sound daunting but the code for calling a conference room can be reduced to very few lines and asterisk can be easily installed in a linux box in a real machine, or in a virtual one or in a docker container.
If you prefer fat clients, I suggest using a SIP library as PJSIP (that incidentally is the base of the new SIP stack for asterisk). Proprietary solutions ride against the future while standard ones are pushed by it.
In a os x app, which is the best way to store hundreds of user record with a lot of information for each record?
I know that i can use CoreData, SQLite, Realm, plist or other solution.
Which is the most compatible (i think for example for a web use) and clean/light/fast method/type of database to store data ?
thank
Core Data is written by Apple to be used in Cocoa based applications. It is already included on the operating system, has 10 years of development behind it and is used in hundreds of thousands of applications on the app store already.
Using a third party library has tremendous risks associated with it.
I always recommend using Core Data on iOS and OS X because it is written by Apple, has a fully supported team of developers behind it and is a mature code base.
Using raw SQLite is possible but you will not get the same performance benefits as you will with Core Data.
Realm is focusing on the wrong things. Their data migration system is scary, they focus on raw speed when they need to be focusing on the overall picture. They might be targeting mobile but I do not agree with their goals.
Hundreds of records is not hard for any database system. It is when you get to 10s of thousands of records on a device that many of them start to fall down due to memory pressure (even on OS X).
Core Data is designed to help with the memory pressure. I am not aware of any third part framework that will do so.
I think the most important thing is to compare Realm with the other database. Realm use its own core and the others are based on SQLite (CoreData use SQLite).
Realm is the first database designed for mobile use. So with your case (OSX) you must consider if your app have the same functionning as mobile App: device restrictions.
Realm:
Very fast methods: see example here
Rapid execution: see benchmark here, yes thousands of user records!
Light: the embeded database less than 1MB
But in beta state. This mean that they are constantlyy in development
SQLite:
the "lite" in the name mean it's a light database. You have many libraries available in github to help you, full documentation and full examples.
Personnaly, I use Realm for My OSX App and despite it being in a beta state, I have all the features I need. Realm is faster than CoreData and save me thousand of lines of code and months of work. :)
I hope that this will help you.
I was just wondering how many db queries might facebook be issuing to render a user's home page. Does anybody have some idea on how the facebook DB is designed. I've heard it runs MySql and there are thousands of replica plus more memcache server than DB Servers.
Is the facebook data shard-ed?
If it is does it go to every shard and search for the latest update of my friend. In worst case if I've 100 friends and suppose facebook has 101 shards, there is a possibility that each of my friend is in a different shard. How might facebook be handling this?
I'll be highly grateful if somebody can provide me seom hints or pointers towards something like "How to Design a DB for Social Networking Website". I'm just curious!
Facebook is using LAMP structure. Facebook’s backend services are written in a variety of different programming languages including C++, Java, Python, and Erlang and they are used according to requirement. With LAMP Facebook uses some technologies ,to support large number of requests, like
Memcache - It is a memory caching system that is used to speed up dynamic database-driven websites (like Facebook) by caching data and objects in RAM to reduce reading time. Memcache is Facebook’s primary form of caching and helps alleviate the database load. Having a caching system allows Facebook to be as fast as it is at recalling your data.
Thrift (protocol) - It is a lightweight remote procedure call framework for scalable cross-language services development. Thrift supports C++, PHP, Python, Perl, Java, Ruby, Erlang, and others.
Cassandra (database) - It is a database management system designed to handle large amounts of data spread out across many servers.
HipHop for PHP - It is a source code transformer for PHP script code and was created to save server resources. HipHop transforms PHP source code into optimized C++. After doing this, it uses g++ to compile it to machine code.
If we go into more detail, then answer to this question go longer. We can understand more from following posts:
How Does Facebook Work?
Data Management, Facebook-style
Facebook database design?
Facebook wall's database structure
Facebook "like" data structure
At this website you find lots of details about those big internet companies and their technical structures:
http://highscalability.com/
Adding to #Somnath Muluk's answer - Facebook uses few other technologies like Hadoop etc.
Refer to the following links for more details:
http://www.quora.com/Facebook-Engineering/What-is-Facebooks-architecture
Facebook Architecture
Hope it helps.
Zero. On average, that is. A highly interconnected network with a large number of users such as Facebook can only run effectively if it runs fully out of ram for the pages that are shown often. Nearly all data should already be in the memcache.
I was wondering if anyone knows of any good Open Source distributed computing projects? I have a computationally intensive program that could benefit from distributed computing (a la SETI#Home, etc.) and want to know if anyone has seen such a thing or will I be developing it from scratch?
I see that this is over a year old but this is a new and relevant answer
http://openstack.org/
Here's one for java and one for c# and here's an open source grid toolkit.
SETI#Home uses BOINC
MPAPI - Parallel and Distributed Applications Framework.
Sector 0 Article:
http://sector0.dk/?page_id=15.
Gives a good overview of the
framework, architecture and the
theory behind it.
Works on a single machine to 'n'
machines.
Design distributed logic into the
system.
Focuses on message passing to isolate
the state that each thread has access
to i.e. no shared state only
messages.
Is Open Source =] and is MONO
Compatible YAY!
Architecture in a Nutshell
Cluster
Single Main Node
Controls the cluster
Numerous Sub-Nodes (one per machine) which are the work horses of the cluster
Single Registration Server - Binds the cluster together by allowing nodes to register / unregister with cluster notifying
existing nodes
Communication
Node to Node directly. Each worker
communicates with others through the
node.
The messages are not
propagated down through the remoting
layer unless two workers are on
different nodes.
Hadoop if you want to run the machines yourself. Amazon Elastic MapReduce if you want to let others run your workers. Amazon Elastic MapReduce is based on Hadoop.
I have personally used BOINC which is a robust solution, widely used and offer you a great range of possibilities in term of customization.
This is the most complete solution I know. The only problems I had were that it was difficult to use for remote job submission (if you don't have access to the server) and it can be a bit long to setup. But overall it is a very good solution.
If you rather want to implement distributed computing just over a local grid, you can use GridCompute that should be quick to set up and will let you use your application through python scripts.
PS: I am the developer of GridCompute.