For a hobby project of me i'm looking to implement a group voice chat feature. Pretty straight forward: I'm running a server to which multiple clients (mobile) can connect to. Some Clients are in the same "group" and I want them to have an audio chat feature.
I already set up a UDP server client with C# to which clients can connect to. I have successfully implemented audio distribution between clients over the server and the rudimentary features are working pretty well.
I am not sure if i'm going in the right direction with this approach though.
I'm stuck with the implementation of mixing the different voices for example (two people talking simultaneously and another is listening to both). I don't really know how I can mix both voices together and generate different outputs for different clients - above mentioned example: The two people talking should only receive the input of the other person while the one not talking should receive a mix of the other two talking.
What is the best server sided structure for this? Should i maybe go in a completely different direction and work with SIP? I'm having a hard time finding suitable resources for this problem online and i'm really stuck.
Thanks for your help!
Let me suggest using standards in your application. If (as I strongly recommend) your application is a web application, WebRTC make your job dramatically easier. Please see in WebRTC samples some ideas that I'm sure will inspire you, including multiple peer connections
If you are only interested in group calls, you could install a PBX server as asterisk that has a powerfull set of conference capabilities. You could use a SIP library on top of WebRTC in clients (e.g. sip.js, sipml5) to connect via SIP to asterisk and obtain conference services. This could sound daunting but the code for calling a conference room can be reduced to very few lines and asterisk can be easily installed in a linux box in a real machine, or in a virtual one or in a docker container.
If you prefer fat clients, I suggest using a SIP library as PJSIP (that incidentally is the base of the new SIP stack for asterisk). Proprietary solutions ride against the future while standard ones are pushed by it.
Related
mux.com (and also agora.io and so on) is a great service, but very expensive since it's a server solution. I can't use that.
Discord is a great client solution, that just uses gateways as a pass-through to hide IP addresses and so on. They described their entire architecture here: https://discord.com/blog/how-discord-handles-two-and-half-million-concurrent-voice-users-using-webrtc Discord ain't the only one with this approach, Instagram has AFAIK the same approach too, since it's cheap and does what it does
I want to use for my social media app (like instagram) this solution too, but without these many custom built things to increase performance. I am a one-man team and I can't handle that complexity; still i don't want to use mux because it's way too expensive for me
I am okay with the stock/standard performance. Does anyone know or can point me to a tutorial, where to start building such webRTC elixier gateway solution for m:m audio/video live streams calls?
maybe there already is code published that I can just copy paste
thanks a lot!!
edit
ive got an answer on their official forum https://elixirforum.com/t/how-to-build-webrtc-m-m-audio-video-live-streams-calls-like-discord-does-client-to-client-via-gateway-for-ip-protection/44956
Discord backend use SFU to forward streams for peers in a videoroom, the description from the discord post:
Discord Voice server contains two components: a signaling component and a media relay component called the selective forwarding unit or SFU. The signaling component fully controls the SFU and is responsible for generating stream identifiers and encryption keys, forwarding speaking indication, etc.
Note that the projects in answer are written in Elixir(based on Erlang) programming language, which is not very common used in neither live streaming nor WebRTC. For example, FFmpeg, x264, libopus, WebRTC, SRS, all these audio/video components are written in C++, you'd better think about it.
For a video chat product like discord:
The client app, no doubt, could be built on WebRTC, both H5 and mobile.
For SFU server, recommend C++ server, for example, SRS or mediasoup. Because the whole audio/video economy is C++ based, there're lots of stuff to handle for SFU.
About the signaling server, also called videoroom, could be written by nodejs or Go, because it depends on your business, so highly recommend your best skilled language, there're lots of work to do in this server.
And not all peers in a video room need to publish video stream, instead they only play or consume streams, so it's actually low latency live streaming. For more information about live streaming and video chat, please read this post.
I have an idea for a mobile service based project. I have read some stuff online, including the following tutorial: SMS Tutorial and find it to be pretty helpful but I have some basic questions so please bear with me.
I run a small (as in me and a friend) company and want to setup a situation where people can text a number and receive information back, or setup on my website that they receive text messages letting them know its time to do something, or "tech support" can text them if they wish, etc.
So from what I've gathered, I can use Kannel as my "SMS gateway" interacting with a GSM Modem that I can purchase. For this modem I can buy a texting plan SIM. I can then setup Kannel to use my GSM modem as a virtual smsc. So, users can text that SIMs phone number, which will go to the modem and be interpreted by Kannel. My application will only have to interact with Kannel. And in the future, if I decide I need more texting throuput and upgrade to a real SMSC my application does not need to change.
Is there anything I'm missing/misunderstanding?
Thanks!
Using Kannel as a SMS gateway is a good option for a small company. It does come with a lot of headaches as you have to build, configure, maintain, etc.. all of the services you need, This is what everyone is referring to as "A lot of work".
What you are looking to do is use the GSM modem as a Long Code (verses short Code) for text messaging.
I think this is an expectable solution for something small and where service, latency and availability might not be as important if it's for a local region. But if this is something that needs to be reliable I would think about getting a short code (or sharing a short code) or just a SMS Messaging service with No Long/Short Code (See Twilio Below).
Also if you're trying to rollout your own service there are some things to consider with the SMSC's. If your Kannel/GSM Modem doesn't support the Carrier, you would have to reach out to that Carrier and connect to thier SMSC. This is a hefty price to connect to a Carrier. This is way Aggregators are appealing as they Have all the Carrier connections and pay those fees.
As you transitioning from Kannel to a Gateway Service Provider, that's another headache as you would need to start from scratch and use whatever the service provider API is and replace the Kannel/GSM altogether. Your workflow might be the same but how you send and receive messages with differ greatly. Most (if not all) Aggregators will offer there own version of a SDK/API/Service that you would need to comply with to use their service.
If it's in the US there are some other options you might consider:
Twilio, this is the simplest solution I've seem for smaller companies looking for SMS functionality. Now they are currently offering SMS Short Codes by trial but if you need a short code I would go with a true messaging aggregator.
Zeep Mobile Offers a free SMS service with a Short Code, but they do send Ads with all your SMS Messages. This is a great way to subsidize costs if the Ads don't bother you. Not sure if you can pick the types of ads you want but it's another option for a service.
Clickatell offers a service where you can Share a Short Code and use Keywords to filter your SMS traffic to your account. This is another way to cut some costs if you're funds and traffic (how much SMS you send and receive) are limited
OpenMarket offers a full service SMS/MMS global platform, this is who you want if you doing Mass amounts of trafic and/or need to reach globally.
Note: These are just a few services as there are many, many more
There are also some caveats with having a Short Code as you will need to register a new Short Code if the country you are services needs it's own Short Code. Example: you can use your US Short Code to service Canada, You would also need a Canadian Short Code as well. This can get costly if your only doing small amounts of traffic.
I think you have got basic considerations. John is right, and using a sms gateway is better idea, you will get better reliability and thoughput. And you could get slow prices.
I am wondering if it's possible to create a graphical application in Silverlight which supports synchronisation between the different clients.
To be a bit more precise, I am drawing concepts of developing a Silverlight Game. Visitors would log-in, and see live, synchronised what the other vistors are doing.
If it is possible to have this implemented, I would like to know what is needed to create a fully synched Silverlight environment between multiple peers. Anything from links, code snippets, ideas and / or alternatives are more than appreciated !
Please do not suggest Flash, as I do not own a valid Flash building license, I prefer to have this created within Visual Studio 2010.
Edit:
I want it to be as lightweight for the clients as possible, I don't care much for the server, and also low bandwidth consuming. I don't know whether a broadcasting principal is the only option to have all the events taken place at the same time?
You may want to take a look at the Polling Duplex protocol of WCF. This is the subscription and publish concept. Support in SL has been about since version 2 so there's plenty of articles out there. An article I referenced for a message broadcast system we put in place at work can be found here...
http://tomasz.janczuk.org/2009/07/pubsub-sample-using-http-polling-duplex.html
which also mentions an interesting project on codeplex (I've not used)...
http://laharsub.codeplex.com/
A simple and working (but rather inefficient) solution would be for all clients to ask a WCF/Ria service on the server for status updates in regular intervals, perhaps once every X seconds or so, letting the server keep track of changes relevant to the calling clients.
I have a year's experience writing client code but none with server stuff. I want to restrain the question a bit so I'll simplify what I'm trying to achieve.
I want to write server code such that two clients (Browser or iPhone / Android) can connect, when two player have connected they see a timer count down to zero. The clock would be synchronize on the server and the clients would be uniquely identifiable.
The trouble here is with the word connect, what do people use for multiplayer games? Open a TCP socket for two way communications? You can tell I'm not really sure what I'm talking about. I was hoping to use AppEngine but I'm not sure if it's suitable as it's request based.
I have some experience with Java and although Erlang sounds like the best choice this is something I just want to play with and push out fast so Java would be easier. I just need to know the best way to connect players etc.
Thanks,
Gav
I suggest we regard desktop and mobile systems as equal clients. What options are then?
You write a socket server which will accept connections from clients. But then you need to write some socket client as well, even 2x - for a desktop and for a mobile OS. And the user will have to install this client.
You launch a web server (whatever technology you like). It will expose some web services which will be equally accessible to both desktop and clients OSes. But still you need to write a client application (again 2x).
You run a web server and make all functionality accessible via standard HTTP protocol. This way you won't even need a client - almost every desktop or a mobile OS has at least some web browser installed. JavaScript will provide for dynamic updates of your ticker.
There is a good series of articles about Networking for game programmers by someone who does that stuff for a living.
I'm by no means an expert on network communication, but if you don't mind loosing a few packets (or error checking in software) and you want fast, lean communication you could use UDP. I think most time-sensitive data programs and streaming media use this protocol to avoid delays and keep packet size down.
I realized a Client/ Server app a few years ago with java and ServerSocket (http://java.sun.com/j2se/1.4.2/docs/api/java/net/ServerSocket.html). You also have a SSL version.
So you create a ServerSocket and wait for connexion. When a client is connected, you create a thread that will discuss with this client with a protocol that you made.
http://www.developer.com/java/ent/article.php/1356891/A-PatternFramework-for-ClientServer-Programming-in-Java.htm
If found this little framework :
http://www.bablokb.de/jcs/jcs.html
One of the hardest thing is to create your protocol , a good way to learn how to create one would be to understand how work the FTP (or HTTP ...).
You are correct that the J2EE model breaks down with near-realtime or multi-player demands. You migth want to consider the RedDwarf game server project. It does for games what Servlets do for busienss logic and is open source.
http://www.reddwarfserver.org
I suggest we regard desktop and mobile
systems as equal clients. What options
are then?
RedDwarf has a pluggable trabsport layer and can support any sort of client you wish.
Web servers are great for web type apps. if your game acts like a web page-- is not multi-user, is turn based, and evolves very slowly-- then a web server is a fine chocie.
If it doesn't however, you need something a bit beefier in technology.
Oh, and for what its worth, whetevr you do, if you want to write a server from scratch then DON'T use"ServerSocket." That requries a thread per connection and will never scale. Use NIO or use an NIO framewoprk like Netty.
I have a need of implementing two apps that will exchange data with each other. Both apps will be running on separate PCs which are part of a LAN.
How we can do this in Delphi?
Is there any free component which will make it easy to exchange data between apps across PCs?
If I'm writing it myself, I (almost) always use sockets to exchange data between apps.
It's light weight, it works well on the same machine, across the local network or the Internet with no changes and it lets you communicate between apps with different permissions, like services (Windows messages cause problems here).
It might not be a requirements for you, but I'm also a fan of platform independent transports, like TCP/IP.
There are lots of free choices for Delphi. Here are a few that I know of. If you like blocking libraries, look at Indy or Synapse. If you prefer non-blocking, check out ICS.
Before you choose a technique, you should characterize the communication according to its throughput, granularity, latency, and criticality.
Throughput -- how much data per unit time will you need to move? The range of possible values is so wide that the lowest-rate and highest-rate applications have almost nothing in common.
Granularity -- how big are the messages? How much data does the receiving application need before it can use the message?
Latency -- when one aplication sends a message, how soon must the other application see it? How quickly do you want the receiving application to react to the sending application?
Criticality -- how long can a received message be left unattended before it is overrun by a later message? (This is usually not important unless the throughput is high and the message storage is limited.)
Once you have these questions answered, you can begin to ask about the best technology for your particular situation.
-Al.
I used to use Mailslots if I needed to communicate with more than one PC at a time ("broadcast") over a network, although there is the caveat that mailslots are not guaranteed.
For 1-to-1, Named Pipes are a Windows way of doing this sort thing, you basically open a communication channel between 2 PCs and then write messages into the pipe. Not straight forward to start with but very reliable and the recommended way for things like Windows Services.
MS offer Named Pipes as an alternative way of communicating with an SQL Server (other than TCP/IP).
But as Bruce said, TCP/IP is standard and platform independent, and very reliable.
DCOM used to be a good method of interprocess communication. This was also one of Delphis strong points. Today I would strongly advice against using it.
Depending on the nature of your project I'd choose either
using a SQL server
socket communication
Look at solutions that use "Remote Procedure Call" type interfaces. I use RemObjects SDK for this sort of thing, but there are open source versions of RealThinClient which would do just as well.
Both of these allow you to create a connection that for most of your code is "transparent", and you just call an interface which sends the data over the wire and gets results back. You can then program how you usually do, and forget the details of sockets etc.
This is one of those cases where there really isn't a "best" answer as just about any of the technologies already discussed can be used to accurately communicate between two applications. The choice of which method to use will really come down to the critical nature of your communication, as well as how much data must be transfered from one workstation to another.
If your communication is not time sensitive or critical, then a simple poll of a database or file at regular intervals might be sufficient. If your communication is critical and time sensitive then placing a TCPIP server in each client might be worth pursuing. If just time sensitive then mailslots makes a good choice, if critical but not time sensitive then named pipes.
I've used the Indy library's Multicast components (IdIPMCastClient/Server) for this type of thing many times. The apps just send XML to each other. Quick and easy with minimal connection requirements.
Probably the easiest way is to read and write a file (or possibly one file per direction). It also has the advantage that it is easy to simulate and trace. It's not the fastest option, though (and it definitely sounds lame ;-) ).
A possibility could be to "share" objects across the network.
It is possible with a Client-Server ORM like our little mORMot.
This Open Source libraires are working from Delphi 6 up to XE2, and use JSON for transmission. There is some security features included (involving a RESTful authentication mechanism), and can use any database - or no database at all.
See in particular the first four samples provided, and the associated documentation.
For Delphi application integration, a message oriented middleware might be an option. Message brokers offer guaranteed delivery, load balancing, different communication models and they work cross-platform and cross-language. Open source message message brokers include:
Apache ActiveMQ and ActiveMQ Apollo
Open Message Queue (OpenMQ)
HornetQ
RabbitMQ
(Disclaimer - I am the author of Delphi / Free Pascal client libraries for these servers)