Is there any standard architectural pattern (with code sample) that I can use for monitoring real-time sensor data in a web app?
I found this for monitoring real-time sensor data in mobile apps. https://aws.amazon.com/blogs/mobile/iot-with-aws-appsync/
Any more help would be appreciated!!
Regards, Ashish
Related
I am currently working on a streaming platform that accepts an unbounded stream from a source into Kafka. I am using Flink as my stream processing engine. I am able to ingest data successfully, window it on event time and do whatever I want to do in Flink. The output of this stream currently goes into a Kafka sink, which is ok for now since this data will not be streamed anywhere. This entire setup is deployed on AWS.
A external client is now interested in the data. The client wants the data in a streaming format instead of pulling the data from Kafka. We also do not want to expose our Kafka brokers to the external world. How can we achieve this? I tried pushpin proxy to "push" the data out. However, it's a pain to setup and manage.
Any idea how to approach this? I am really open to any ideas.
Thanks
Flink Web UI has a brilliant backpressure section. But I can not see any metrics, given by Prometheus reporter, which could be used to detect backpressure in the same way for a Grafana dashboard.
Is there some way to get the same metrics outside of the Flink Web UI? Using the metrics described here https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html. Or even having a prometheus scraper for scraping the web api?
The back pressure monitoring that appears in the Flink dashboard isn't using the metrics system, so those values aren't available via a MetricsReporter. But you can access this info via the REST api at
/jobs/:jobid/vertices/:vertexid/backpressure
While this back pressure detection mechanism is useful, it does have its limitations. It works by calling Thread.getStackTrace(), which is expensive, and some operators (such as AsyncFunction) do critical activities in threads that aren't being sampled.
Another way to investigate back pressure is to set this configuration option in flink-conf.yaml
taskmanager.network.detailed-metrics: true
and then you can look at the metrics measuring inbound/outbound network queue lengths.
Currently I observe that Apache Beam is released for flexible Stream processing. I am searching for api to process video stream, but not found any. Can anyone please suggest about any video processing API with Apache Beam/ Google Dataflow.
Thanks.
Apache Beam is a general purpose programming model for distributed data processing. While Beam doesn't have "video processing APIs", it would be great for many video processing tasks, particularly those that parallelize effectively.
Typically, you'd want to use Beam along with a standard video processing library. Beam programming model can help you with the parallelization aspect and the video processing library can simplify the underlying task on a small section of the input.
I would like to gather some thoughts on real-time audio streaming.
I want to stream audio from a live source (like a mic) to mobile devices. I thought about doing it via browser on the client side to keep it simple. A major concern of mine is the delay I might face. I would like to stream to multiple devices with little to none delay, so the phones would be almost synchronous when playing. I was thinking of using WebRTC, but I'm not too sure about that. Any experience or thoughts someone would like to share on this topic?
Because of your latency requirement, WebRTC is essentially the only good option for you.
Yes, WebRTC does support multiple users at the same time.
Look into OpenTok for iOS.
Is there any App/ Method / Process that i can use to get the battery consumption of a single app in blackberry? I am using 9300. My application uses GPRS and sends data over internet.
Till now i have been using a thread which tells me the difference between battery levels after an hour of a phone using my app and a phone not using my app.
Please suggest a better way?
Unfortunately BlackBery is not really open, clear and documented platform.
The best numbers that I got from conference in Amsterdam Info
There's no reliable way to tell how much battery your app has used. As you are already doing, you can retrieve the levels before and after calling DeviceInfo.getBatteryLevel, but the measured difference includes the battery used by other apps too.