We are following the theory that as users we must issue a command to the Context Broker to change the state of a device: Image 1
In our case, this command already works if we do it from the IoT Agent, but however, if we execute it from the Context Broker through a PATCH, it does not reach the IoT Agent.
Do you know why this could be happening?
Our Context Broker request is the following: Image 2
And finally, the request we make from the IoT Agent, which is the one that works, is this: Image 3
Another doubt that arises is, if the IoT Agent updates all the information in the Context Broker, why not execute the request from there instead of from the Contex Broker?
Your request to Context Broker seems to be ok. Sometimes, the lack of ?type in the request causes problems (see for instance this post) but it doesn't seem to be your case.
I'd suggest to check registrations at Orion. Registration is the mechanism in which the request forwarding from Orion to IOTAgent is based (more info in Orion documentation. IOTAgent should create and manage them, but something could be failing. You can get existing registrations in Orion with the GET /v2/registrations operation.
With regards:
Another doubt that arises is, if the IoT Agent updates all the information in the Context Broker, why not execute the request from there instead of from the Contex Broker?
The FIWARE data management model is context-centric. Thus, the Context Broker is the central piece of the architecture, to intermediate between context producer and context consumer. Commands are a kind of "context production", so it makes sense that Context Broker deals with commands. Note that the client issuing the command maybe doesn't even are able to access to the IOTAgent directly (they use to be "close" to the physical devices they manage and not typically open to direct client requests).
Related
I was implementing WebSockets just for practice and I encountered an architectural problem.
It's nice to have WebSockets, but I cannot figure out a simple scalable scenario.
Possible Scenario:
Browser users start some computationally difficult task over the frontend. It goes over the API server, API puts the task to a queue, some other GPU server with celery pulls the task and starts working on it. Somewhere on the way, possibly, there is a database saving a state. So I would say API and celery server writes in the DB under particular task information about what's going on.
Now the important part. There is a WebSocket server connected to the browser client. It would be great that WebSockets are simplex and only sends messages to browser clients about the progress of the task (status, progress bar % and etc). The WebSocket is clever and doesn't need periodical polling, but manages to send data to the browser client based on events that are triggered (by API and celery). Obviously, the WebSocket server needs to listen to this task state (Redis or something, certainly not something at the same place as is WebSocket server). This means that in the WebSocket loop there must be a listener for this state. But this ends up back to WebSocket server polling this redis or something for seeing the state of the task -> this is certainly connection killer in case of a lot of users as there will be a lot of WebSocket connections polling same database.
The question is then: How to solve this in terms of architecture(no polling, WebSockets sends messages only on the state change of some value in some DB)?
I'd propose that celery server also sends a task information to some queue. The WebSocket server would have to have a code responsible for reading from that queue and distributing that task information to its clients (WebSocket connections) that listen for that particular task information.
We are making a web application in Go with a MySQL database. Our users are allowed to only have one active client at a time. Much like Spotify allows you to only listen to music on one device at a time. To do this I made a map with as key the user ids and a reference to their active websocket connection as a value. Based on the websocket id that the client has to send in the header of the request we can identify weather the request comes from their active session.
My question is if it's a good practice to store data (in this case the map with the user ids and websockets) in a global space or is it better to store it in the database.
We don't expect to reach over 10000 simultaneously active clients. Average is probably gonna be around 1000.
If you only run one instance of the websocket server storing it in memory should be sufficient. Because if it for some reason goes down/restarts then all the connections will be lost and all the clients will have to create them again (and hence the list of connection will once again be populated by all the clients who want to use the service).
However, if you plan on scaling it horizontally so you have multiple websocket services behind a load balancer, then the connections may need to be stored in a database of some sort. And not because it necessarily needs to be more persistant but because you need to be able to check the request against all the services connections.
It is also possible to have a separate service which handles the incoming request and asks all the websocket services if any of them have the connection specified in the request. This could be done if you add a pub/sub queue and every websocket service subscribes to channels for all its websocket ids and the service that receives the request then publishes the websocket id, and the websocket services can then send back replies on a separate channel if they have that connection. You must decide how to handle if no one is responding (no websocket service has the websocket id). Either the channel does not exist, or you expect the answer within a specific time. Or you could publish the question on a general topic and expect all the websocket services to reply (yes or no).
And regarding whether you need to scale it I guess depends mostly on the underlying server you're running the service on. If I understand it correctly the websocket service will basically not do anything except from keeping track of its connections (you should add some ping pong to discover if connections are lost). Then your limitation should mainly be on how many file descriptors your system can handle at once. If that limit is much larger than your expected maximum number of users, then running only one server and storing everything in memory might be an OK solution!
Finally, if you're in the business of having a websocket open for all users, why not do all the "other" communication over that websocket connection instead of having them send HTTP requests with their websocket id? Perhaps HTTP fits better for your use case but could be something to think about :)
I'm pretty new to Kerberos. I'm testing the Single Sign On feature using Kerberos. The environment: Windows clients (with Active Directory authentication) connecting to an Apache server running on Linux machine. The called cgi script (in Perl) connects to a DB server using the forwarded user TGT. Everything works fine (I have the principals, the keytab files, config files and the result from the DB server :) ). So, if as win_usr_a on Windows side I launch my CGI request, the CGI script connects to the remote DB and queries select user from dual and it gets back win_usr_a#EXAMPLE.COM.
I have only one issue I'd like to solve. Currently the credential cache stored as FILE:.... On the intermediate Apache server, the user running the Apache server gets the forwarded TGTs of all authenticated users (as it can see all the credential caches) and while the TGTs lifetime are not expired it can requests any service principals for those users.
I know that the hosts are considered as trusted in Kerberos by definition, but I would be happy if I could limit the usability of the forwarded TGTs. For example can I set the Active Directory to limit the forwarded TGT to be valid only to request a given service principal? And/Or is there a way to define the forwarded TGT to make it able to be used only once, namely after requesting any service principal, become invalid. Or is there a way the cgi script could detect if the forwarded TGT was used by someone else (maybe check a usage counter?).
Now I have only one solution. I can define the lifetime of the forwarded TGT to 2 sec and initiate a kdestroy in the CGI script after the DB connection is established (I set that the CGI script can be executed by the apache-user, but it cannot modify the code). Can I do a bit more?
The credential caches should be hidden somehow. I think defining the credential cache as API: would be nice, but this is only defined for Windows. On Linux maybe the KEYRING:process:name or MEMORY: could be a better solution as this is local to the current process and destroyed when the process is exited. As I know apache create a new process for a new connection, so this may work. Maybe KEYRING:thread:name is the solution? But - according to the thread-keyring(7) man page - it is not inherited by clone and cleared by execve sys call. So, if e.g. Perl is called by execve it will not get the credential cache. Maybe using mod_perl + KEYRING:thread:name?
Any idea would be appreciated! Thanks in advance!
The short answer is that Kerberos itself does not provide any mechanism to limit the scope of who can use it if the client happens to have all the necessary bits at a given point in time. Once you have a usable TGT, you have a usable TGT, and can do with it what you like. This is a fundamentally flawed design as far as security concerns go.
Windows refers to this as unconstrained delegation, and specifically has a solution for this through a Kerberos extension called [MS-SFU] which is more broadly referred to as Constrained Delegation.
The gist of the protocol is that you send a regular service ticket (without attached TGT) to the server (Apache) and the server is enlightened enough to know that it can exchange that service ticket to itself for a service ticket to a delegated server (DB) from Active Directory. The server then uses the new service ticket to authenticate to the DB, and the DB see's it's a service ticket for win_usr_a despite being sent by Apache.
The trick of course is that enlightenment bit. Without knowing more about the specifics of how the authentication is happening in your CGI, it's impossible to say whether whatever you're doing supports [MS-SFU].
Quoting a previous answer of mine (to a different question, focused on "race conditions" when updating the cache)
If multiple processes create tickets independently, then they have no
reason to use the same credentials cache. In the worst case they would
even use different principals, and the side effects would be...
interesting.
Solution: change the environment of each process so that KRB5CCNAME
points to a specific file -- and preferably, in an
application-specific directory.
If your focus in on securing the credentials, then go one step further and don't use a cache. Modify your client app so that it creates the TGT and service tickets on-the-fly and keeps it private.
Note that Java never publishes anything to the Kerberos cache; it may either read from the cache or bypass it altogether, depending on the JAAS config. Too bad the Java implementation of Kerberos is limited and rather brittle, cf. https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/jdk_versions.html and https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/jaas.html
I want to send a particular HTTP request (or otherwise communicate a message) to every (dynamic/autoscaled) instance which is currently running for a particular App Engine application.
My goal is to trigger each instance to discard some locally cached data (because I have just modified the underlying data and want them to reload it).
One possible solution is to store a value in Memcache, and have instances check this each time they handle a request to see if they should flush their cache. But this adds latency to every request.
Another possible solution would be to somehow stop all running instances. No fixed overhead, but some impact while instances are restarted.
An even less desirable solution would be to redeploy the application code in order to cause all instances to be stopped. This now adds additional delay on my end as a deployment takes some time.
You could use the management API to list instances for a given version, but I'd suggest that you'd probably want to use something like the PubSub API to create a subscription on each of your App Engine instances. Since each instance has its own subscription, any messages sent to the monitored queue will be received by all instances.
You can create the subscription at startup (the /_ah/start endpoint may be useful), and then delete it at shutdown (using the /_ah/stop endpoint).
Can anyone think of a good way to allow the server to notify the client based upon server processing? For example, consider the following events:
A user requests a deletion of data, however, due to it's long-running time, we kick it off to a queue.
The client receives a "Yes we completed your transaction successfully".
The server deletes the item and now wants to update any local structures any clients may be using (I'd also like to notify the user).
I know this can be done by client-side polling. Is there a event bus type way to do this? Any suggestions are welcome, but please keep in mind I am using GWT with App Engine.
The standard AJAX interaction is that the client sends requests to the server and expects some sort of response back fairly quickly.
In order for the server to initiate a request to the client, you will need to use WebSockets, and experimental HTML5 feature currently only supported by Chrome.
Or, to simulate this kind of interaction, you can use Comet (long-polling), made available in GWT by the rocket-gwt project.
You want server events for GWT? Have a look at GwtEventService (they couldn't have chosen a better name): http://code.google.com/p/gwteventservice/wiki/StartPage
Of course, it uses a Comet implementation, but you can't do any different when using HTTP, the client always initiates the communication. Request, response.