My organization is a large organization and they are need to create a online channel with 24 hours live streaming support. They are not interested to use 3rd party video hosting sites such as Youtube and Vimeo.
I am the responsible to create a website and mobile application for the channel. But I don't have much idea about the requirements. So please guide me to right directions on the followings,
Server configuration (mobile support)
Software requirements
Best Hosting services
Website requirements
Mobile application requirements
Adobe Media Server is a good fit for your requirements. There are a few alternatives like wowza, but Adobe's Server offers lots of bells and whistles. The idea with Adobe is simple, you set a publishing point, and push your video/audio contents to it, and use the link of the stream in your HTML5 video tag.
An article about this (Older version of Media Server) is here for your reference.
http://www.adobe.com/devnet/adobe-media-server/articles/beginner_live_fms3.html
Website requirements and mobile app requirements are not that complicated as long as you stick with HTML5, and HTML5 is almost 'mobile ready'. All you got to do is to use the streaming link from the Media Server, and you are good.
Well, the server hardware and network requirements depends on the amount of data you stream and simultaneous visitors viewing your stream. Its fine for you to run your own dedicated server at your premises if you have access to a high speed dedicated ISP.
CPU:Core 2 Quad(Quad Core Q6600) or i7(i7 920) are good rigs.
RAM: 4-8 GB
HDD: 1TB
Bandwidth: must be unmetered, dedicated, 1:1 (Start with 20Mbps then upgrade to 50Mbps, 100Mbps, etc when needed)
As for how much bandwidth you need, you just have to do simple math... one video played means (video bitrate + audio bitrate + 5% tolerance ) x number of people watching the video at the same time.
Example: 100 people watching a video at same time with the specs 160 kbps video and 48 kbps audio means you need at least (160 + 48) * 1.05 * 100 = 21840 kbps or 22 mbps.
If you cannot meet the above requirements, you can opt for hosted solutions, where they charge you per GB of bandwidth you stream.
Related
We are currently using Solr4.3 cloud in master slave mode and have been pretty happy with our initial solr POC. We are looking to store Social Data (tweets, blogs, Facebook feed) into Solr and make it searchable, also at the same time utilize the Faceting capabilities provided by Solr.
Going by the amount of social data that comes in, we were wondering what kind of infrastructure would be required to say store 2 TB or data and query them with minimum time.
Also, give the rate in which tweets come in what would be the best indexing strategy.
I will suggest you to choose some cloud platform .AWS is one good option in this matter since you can always try and then change your machine if it does'nt suit your requirement.
So what will suit your requirement?
I think for solr since there is a high I/O I will suggest you to go with high I/O machine and good processing power
I suggest using
c3.2xlarge of AWS with 8 ecu, 15 gb ram, 2 x 80 SSD.
and attach to it an EBS volume of atleast 4 TB.
This would solve your problem.
We need to develop a multi-player game with real-time performance.
This needs to be working worldwide (servers in America, Europe, Asia), and supporting a huge traffic. Using Google Cloud services for the hosting.
We're thinking of references like Jam with Chrome, Chrome Maze or Cube Slam.
The game :
2 players challenge a race
We need to simultaneously display the progression of the 2 players
Each match could last around 30 to 45 seconds
The hosting :
We will obviously host the website on AppEngine, automagically scaling,
but are thinking about 2 solutions for the real-time servers :
Using websocket servers with Compute Engine
Like they did for Jam with Chrome, Maze, etc.
Developing our own websocket servers (technology TBD), deploying on datacenters in Europe, US, Asia, handling scaling, syncing between them, computing latency issues on servers and clients, etc.
But it's pretty technically challenging as we are very short on time, and missing an admin sys and network guy for now.
Or using Channel API
We understand that it's not a websocket platform, and real-time performances are lower.
But it would be way more simple and secure for us and the time we have.
So, we would also like to know more about that.
In any case, we think we could use some graphical tricks on front ends, to make it look like real-time, but it really depends if we have a 100~500ms or a 500ms~10s latency.
Some questions :
What would the latency range values look like for the different solutions ?
(Jam w/ Chrome got 100ms with GCE, could Channel API reach several seconds ?)
How would Channel API servers handle high traffic, how does scaling work, could the latency go very high ? (no info about that on Channel docs ?)
What if someone in France play with someone in US, connecting to different servers, waiting them to sync, how to deal with it ?
Any advice or experience to share ?
Any interesting reading or viewing ? (seen some but not very precise)
Any other solution ?
Thank you for any helping comment !
EDIT :
Only 2 players connected together, potentially from different world zone, no broadcasting needed.
We could find some front side tricks to avoid server side processing. This is a race between 2 players, so we actually just need to compare their progression, and the real winner resolution is not that important as there is no real stuff to win, this is more for fun.
If you need a server for processing the data:
I would definitely go with websockets at Compute Engine!
The Channels API is much slower, and also quite unpredictable (latency differs from message to message)! Data has to go to the Channels server, which sends it to the App Engine instance, which has to do a request back to the Channels server, which will push the message to the client. There is too much going on there if you want to keep latency down!
Here is a Channels API stress test:
http://channelapistresstest.appspot.com/
Try clicking "send 5"-button a lot, and you will see latency numbers going up to several seconds.
The Channels API is also quite expensive under heavy load (it probably does not scale well, even if Google of course can solve that with more instances).
When keeping latency down, geolocation is quite important. With a websocket server at Compute Engine, you can send your european visitors to google's european datacenter and your american visitors to the US datacenter (using the geo location headers that AppEngine will provide). You have no such control with the Channels API (or app engine, which all your messages are relayed through). Maybe Google has edge servers for the Channels API (I don't know), but if your AppEngine instance is on the other side of the planet, that does not matter.
If you do NOT need a server for processing the data:
You should establish a peer-to-peer connection with WebRTC, sending stuff directly between the users' browsers. That is was Cube Slam does. (WebRTC requires some initial handshaking ("signaling") so the two peers can find each other, and Channels API would work fine for that handshaking, that's just a couple of messages to establish the peer-to-peer connection.)
WebRTC DataChannels API will give you a nice websocket-like interface like channel.onmessage = function(e) { yadayada()... }; and channel.send("yadayada"); to send your data between the peers.
Occasionally, WebRTC is not able to make a peer-to-peer connection. Then it will fall back to a TURN server, which relays traffic between the peers. Cube Slam is using TURN servers running on ComputeEngine (in both Europe and America to keep latency down), but that is just the fallback when true peer-to-peer is not possible.
It also depends on other things like scalability.
Ingress is built on app engine and a part from the occasional cache glitch it is pretty impressive.
Remember that the channel api is using talk.Google which is the service that hangouts is built on. Scalable and real time.
Personally if your traffic levels are going to be erratic and unpredictable, go app engine. If you think it can be controlled and predictable use compute engine or something else.
Alfred's answer is the best in the frame of the question I asked.
Thank you very much !
However, I forgot to mention a few important points and the scope changed a bit :
We have very little development time (about 1 week only)
This is for a campaign that will last 3 weeks only (we'll need to keep it online a few months afterward, but this is not like we need a long-lasting architecture)
We need to make it work on the broader browser audience as possible (WebRTC only runs on Chrome & Firefox for now)
According to these points, we eventually came up to a 3rd solution :
Using a real-time PAAS.
It's way easier and faster to develop, way cheaper as we don't need a solid backend developer and system/network admin, and we can concentrate more on the project than on the infrastructure and platform.
There are a couple of services that seems good out there, already hosting MMO RPG and the kind, worldwide, with low latency, and good scaling systems.
Here is a list of providers :
https://github.com/leggetter/realtime-web-technologies-guide/blob/master/guide.md
I have a 2 GB database and a front end that will likely handle 10-15 hits during the day. Is the AWS (with MySQL RDS) free-tier a good place to start?
Will CakePHP apps encounter time-outs or other resource issues, due to sizing of the Micro Instance?
Micro Instance (from Amazon): Micro instances (t1.micro) provide a small amount of
consistent CPU resources and allow you to increase CPU capacity in
short bursts when additional cycles are available. They are well
suited for lower throughput applications and web sites that require
additional compute cycles periodically. You can learn more about how
you can use Micro instances and appropriate applications in the Amazon
EC2 documentation.
Micro Instance 613 MiB of memory, up to 2 ECUs (for short periodic
bursts), EBS storage only, 32-bit or 64-bit platform
If you're only getting a very small amount of hits, you can probably run your application and mysql database on a micro instance.
The micro will be free, but you will have to pay for the RDS.
You should not notice any issues - we do most of our testing on micros, and our database is larger than yours.
It will work perfectly for your scenario.
I have deployed myself for other clients applications with at least twice requirements as yours and they worked fine.
If your application does operations saving and retrieving files from the disk I would like to suggest you giving a try to Amazon S3.
My problem is this...
I have two sites, one acting as an "Admin" site, the other as general "User" site. I need to broadcast live audio from the "Admin" site to all clients of the "User" site. I need to do this with <1 sec of latency.
Some restrictions include:
No install on "User" machines (the idea being the whole thing sits on the web)
If there needs to be a 3rd party plugin then Silverlight is preferred*
Any help much appreciated here
*I have tried IceCast with a flash client, IIS Smooth Streaming, Internet radio, all of which give us a latency of >5 secs.
Have you tried Flash with a server like Red5? You're generally going to get subsecond latency (though not much less than that), as it's designed for realtime communications. There's a learning curve with Flex and ActionScript, but if you're at all familiar with XAML, you can pick it up from the sample apps that come with Red5 pretty quickly.
Failing that, if there aren't too many clients, you can use one of the two real-time peer-to-peer solutions out there, namely Flash over RTMFP or WebRTC over JSEP/ICE/RTP. If you can ensure that all the clients are using Chrome, then WebRTC is probably your best bet. If you can ensure that they're not using Chrome, then Flash is a good choice. The current Flash Pepper client on Chrome is buggy up the wazoo when it comes to audio processing, and no sign of a fix in sight. (It doesn't support echo cancellation, and the volume of the audio goes up and down horribly.) So if you're using Flash, steer clear of recording and broadcasting your audio on Chrome. And I wouldn't recommend either approach if you have more than half a dozen clients - the number of audio streams is gonna overwhelm your "Admin" browser pretty quickly, I think. Better to push that out to something like a Red5 server.
Silverlight is a bad choice for more reasons than I can count. I'm saying this as a guy who spent several years trying to implement a realtime communication solution on Silverlight. Don't do it.
Google App Engine offers free quotas of 1 GB outbound traffic per day and 6.5 CPU-hours (based on a 1.2 GHz Intel x86 processor) per day.
How do those free quotas compare to web hosting plans? For example, the traffic supported by the free quotas -- is that generally higher or less than the traffic supported by a typical $5/mo shared hosting account?
Above the free quotas, Google charges $0.12 per GB outgoing traffic, $0.10 per GB incoming traffic, $0.10 per CPU-hour, $0.15 per GB storage per month.
How do those numbers translate to normal web hosting plans? For example, the traffic that can be supported by a $40/mo VPS plan and $200/mo dedicated server plan, what would they cost on Google App Engine?
I know it depends on a lot of factors, but if anyone has any ballpark estimates or experiences they're willing to share I'd really appreciate it.
I'm trying to decide between App Engine and standard web hosting for a DB-backed Python site. The site will start small, but if the traffic grows I want to see which would be a better option long term.
As you say, it depends on a lot. Not just the site you want to host, but who, specifically, you're comparing it to. To give you a rough idea, I host a site that gets from 20k to 30k pageviews a day on App Engine, and it costs me 17c a week. That's for extra storage over the default quota - all other quotas are well within the free zone.
As far as specific comparisons go, if we look at Linode, for example, their base plan is $20 for 12GB of storage and 120GB transfer per month. At App Engine prices, that would cost you at most $16.20 per month - assuming it was all outgoing traffic. If you do less than that in a month, it costs you less. Obviously, you don't have all the versatility on App Engine that you have with a VPS, but you also have better scalability and reliability.
The comparison is fairly easy to other hosting services. I think App Engine will generally come out ahead, so mostly the decision comes down to if your app is suitable for App Engine.
Man if your app is already in python go with app engine! You could not ask for a more reliable, scalable, efficient platform to host with. As far as the $ goes you cannot beat it. I am working on porting all the applications I have written to python now to take advantage of google app engine. Imagine, no more vps worries about server issues. Take advantage of it!
Try this free diagnostic. It compares Azure / AWS / Appengine and tells you exactly which platform you should use and for how much: http://www.whitestratus.com/cloud-platform-diagnostic-tool