we have a angularjs app and we want to track the android users way of using it in the fields. This app can be used for 2-3 days off-line and sync opportunistically when wi-fi connections or 3g-4g are available.
Our goal is to track user behaviour with UA and cache usage data to send at first wifi opportunity.
Does UA have a built-in support for this?
if not
Can we do this programatically using v3 apis? ie. cache data locally and use a service to sync the data if online?
In either case, how much data can UA cache?
How many days can the data stay cached locally before syncing?
Cheers,
T
While UA has support for this the support does not extend to multiple days - according to the parameter reference for the measurement protocol (which is the basis for UA) "queue time" is four hours at max.
Using the API (which is built on top of the measurement protocol) will not change that.
Related
Google is deprecating Cloud Iot, so not an option.
https://cloud.google.com/iot/docs/release-notes
Cloud IoT Core will be retired on August 16, 2023. After August 15, 2023, the >documentation for IoT Core will no longer be available.
I would like to use Firebase - Firestore for my backend. It takes all the hassles out of keeping a server up and running, scalability etc.
I managed to send data after login and authentication from an ESP32S3 using ESP-IDF in C, (note not Arduino, and not C++), and would like to know if I can rather use a websocket for the communication, once the Authentication done, and if so, can you give me a code example or pointers.
With a websocket, I can send data to my own server hosted in Europe, in less than 400ms.
With Firestore, there is a large HTTP header, that includes the API key, and also the Auth Token, a large amount of data, quite a lot of handshaking going on over HTTPS, and eventually the data is sent. This takes more than 1400ms.
We are weighing items in a farming scenario, and need to weigh very frequently, and the 1400ms with fast internet is not acceptable.
So if I could still go with Firebase Authentication, and Firestore for data, I probably would be able to speed it up to even faster than 400ms if I could use a WebSocket client connection with the Firestore document store. I can use the Refresh Token if needed to refresh the Auth Token, and thus keep the socket connection up, every 3600s as required by Firebase, (that also takes quite long) but less of a hassle, as only once every say 55 minutes.
Any pointers, advice will be appreciated.
Firestore supports multiple SDKs and wire protocols, but none of them work over web sockets. The closest you can get with Firestore would be its REST API, which is documented here. It's not the easiest protocol to work with though, so I recommend using the API explorer that is built into the documentation to create examples for yourself.
My question is about how to store data which once was received online and still can be processed after the mobile device got offline and/or was restarted.
I'm using AngularJS with Ionic (PhoneGap) for building apps. But my question is not explicitly adressing these technologies.
Best practices, patterns or algorythms would be very helpful to me or even some useful articles or key words.
1) The most simple challenge is to make my app more user-friendly by making its functionality usable not only if the device is online but also in offline mode. In my case this implies that I have to make the last fetched online data available for later use (while device is offline and also after restarting the device!).
2) A bit more difficult is to reduce the communication costs by only synchronizing the server side changed data when the device reconnects to the internet.
3) Entities can also be produced on client side while the device is offline and they must get synchronized to the server too. There are no potential risks of conflicts because the users don't share Entities with write access.
4) I use Googles and Apples push services to inform the devices about newer entity versions, which should get updated on client side. So polling isn't needed.
Client side technologies: Javascript, AngularJS Framework, Ionic Framework, SQLite (WebSQL) or IndexedDB, PhoneGap (Cordova)
Server side technologies: Java EE, JPA, MySQL
Data Format and communication: JSON over REST / http, Googles and Apples push services for server-to-client messaging
1) Store the needed data inside a local SQLite database, and pull it out when the app starts/resumes.
2)In the MySQL database you need a table that creates new entries when you update/change/create content. You would need to store an id and a timestamp (maybe a boolean value if the content was deleted).
On the device you would make a request to the server to send the data from that table and compare it with the locally stored data. If there is a new id or the timestamp has change make a new request to pull the updated data.
3)Store the created data locally with a flag that it isn't synced with the server. When the device then goes online again check for not synced flags and send the data to the server with an identifier to know which device it comes from and where to save it.
4)See 2)
You could make a Java script which checks every x minutes for updated entries and send an automatic push Notification with it. What you would need is 2 tables, one with the newest updates and one with the updates that got pulled by the device(just id's and timestamps, not all the data).
I hope this was helpful, if something new comes to my mind I will update this answer.
I am making an application based on Amazon Product API (or could be MWS), I will need to fetch data huge information again and again. So, in order to avoid Throttling limits, i would like to use Proxy ips.
Is this a valid thing to do.
1. Is the throttling limit apply per MWS account or per IP address
2. There are several proxy hosts avaialble for free/commerical. Is it ok to use them.
thanks
Valid? not really.
Amazon has some sharp people working there. I'm pretty sure this scheme wouldn't last very long. Your api access would be revoked pretty quickly.
User makes HTTP-request to the server. This request is processed with an object of some class, let's call it "Processor". Then the same user in two minutes makes another HTTP request. And I want it to be processed with the same instance of Processor as the first one. So basically I want to keep the state of some object among several requests.
I know that I can save it each time to the datastore and then load back, but this approach seems to be very slow. Is there a way to store objects in some RAM place?
How about using memcache?
You can't ensure that consecutive requests to your app will go to the same instance, but memcache can help reduce or eliminate the overhead of accessing the datastore for each request.
It sounds like you are describing is a session.
I am not sure which language runtime and web framework you are using, but it is sure to include support for a sessions. (If you are using Java you will need to enable it.)
The standard session mechanism puts a small ID in a cookie that is stored in the user's browser. On every request, each of which could be go to a different application server, this ID is used as a key to read and write persistent information from the data store.
If the datastore accesses are too slow for you I would suggest not using memcache for this session storage, because memcache is by design unreliable, so the user's session information could disappear at any time, which would be a bad experience for them.
If the amount of data you want to store is less than about a few kilobytes, then I recommend doing what Play Framework does, which is to encrypt your session data and store it directly in a cookie stored in the user's browser. This is fast and truly stateless.
If you have more data than can be stored in a cookie, and you don't want to use the data store, you could could use JavaScript local storage on the browser, and use AJAX calls to communicate with the server. (If you want to support older browsers you may need to use the jStorage wrapper library.)
If memcache isn't enough, you could use backends to maintain state. Use a resident backend (or a set of them) and route incoming requests from the frontend to the backend machine that has the state.
Docs: Python Java
I'm looking for options/alternative to achieve the following.
I want to connect to several data sources (e.g., Google Places, Flickr, Twitter ...) using their APIs. Once I get some data back I want to apply my "user-defined dynamic filters" (defined at runtime) on the fetched data.
Example Filters
Show me only restaurants that have a ratting more than 4 AND have more than 100 ratings.
Show all tweets that are X miles from location A and Y miles from location B
Is it possible to use a rule engine (esp. Drools) to do such filtering ? Does it make sense ?
My proposed architecture is mobile devices connecting to my own server and this server then dispatching requests to the external world and doing all the heavy work (mainly filtering) of data based on user preferences.
Any suggestions/pointers/alternatives would be appreciated.
Thanks.
Yes, Drools Fusion allows you to easily deal with this kind of scenario. Here is a very simple example application that plays around with twitter messages using the twitter4j API:
https://github.com/droolsjbpm/droolsjbpm-contributed-experiments/tree/master/twittercbr
Please note that there is an online and an offline version in that example. To run the online version you need to get access tokens on the twitter home page and configure them in the configuration file:
https://github.com/droolsjbpm/droolsjbpm-contributed-experiments/blob/master/twittercbr/src/main/resources/twitter4j.properties
check the twitter4j documentation for details.