I am working on an app which touches sensitive information, like money.
We have some calculators, and we want to prefill the values with whatever the user has entered last. Apart from increasing UX, we don't need those. But we cannot store it in web storage or cookie because of security.
We have
a JS frontend,
an API Gateway backend that is supposed to be "stupid", so it only handles authentication and sending messages to to corresponding services
some services that actually care about the business logic
These possibilities come to mind and I cannot decide which I should do (and foremost: why)
Add a table in backend, that is a catch all for implementing cookie-like functionality in backend
Add a specific table in the service it fits the most
Use a key value store in backend (don't know about this, a coworker put it out there)
As i read your requirements it seams that this is kind of a defaulting including some business logic (stupid or smart). Personally i see defaulting as part of business logic and based on this it's part of the service which cares about this functionality.
Add a table in backend, that is a catch all for implementing cookie-like functionality in backend
This sounds like a generic solution for a pretty generic requirement. what do you wanna achieve with this?
Add a specific table in the service it fits the most
Sounds reasonable especially because you put it there where it belongs. Does it have to be a table? why not calculate or copy the values on runtime?
Use a key value store in backend (don't know about this, a coworker put it out there)
This is maybe a technological decision but first you need a design decision.
Related
I have a SaaS platform where the user fills out a form and data entered into the form is saved to a database. The form UI has a large amount of config (originates from the DB but ends up in JavaScript) and business logic (in JavaScript). After a form is filled out and saved, the user can go back at any time and edit it.
The wrinkle is that an old form entry needs to behave like it did when it was first filled out - it needs the same config and business logic - Even if the SaaS has gone through a data schema change and changes to business logic since then.
To confirm, new forms filled out by the user would use the new/current data schema and business logic of course. But previous forms needs to behave as they did when they were created.
So I need a sensible way to version config, business logic and any dependencies.
The best I've come up with is, when the user saves their entry, to save the form's config as JSON along with the entry. When the user goes back to edit an old entry, I do not load the config from current database schema but simply dump the JSON config that was saved with the entry.
For the business logic, I save a system version number along with the entry, for example "01". When the user loads an old form, I check the version of the entry and I then load the form JavaScript from a path like "js/main_01.js". When I make a non-backwards-compatible change to the business logic, I increase the system's version number to, for example, "02". New forms would then use "js/main_02.js". I also use this cheap versioning approach for HTML view templates which is getting hairy.
This approach works but it seems a bit flimsy or homegrown. I'm trying to avoid conditionals in my business logic like if version==2: do this. This approach avoids that but also has it's downsides.
I don't think the stack really matters for this convo but just in case, I'm using django/mysql.
You're likely to get a tremendous amount of "opinion" on this, and no real clear answer.
You could develop an API to your config and logic in may ways, with versioning saved with the submitted data, thereby requiring an API-Manager solution.
However, you could instead store the entire DOM object in the record that the data was stored, thereby creating a static page that is recalled and resubmitted at will, with separation between view and model.
We are trying to use AngularJS with Breeze, with a .NET backend. We have gotten things hooked up working together. However, we are having trouble figuring out how to lock things down based on the user role and the user's own data.
Can anyone point us in the general direction? We couldn't find anything explicitly in Breeze's documention.
There is no reason why Breeze should be insecure. Security is orthogonal. My question remains: what are your concerns?
Update 2 March 2015
Thanks for the clarifying comment ... which reflects concerns that are widely share. I really am going to have to write about this at length in our documentation.
Fortunately, I believe I can ease your mind about the specific issues you raised.
BreezeJS, the client library, can only reach the data that your server allows the current user to access. It's the server's job to grant or refuse such requests.
This is fundamentally the same story for a client written with any technology talking to a server written with any technology. If the server has a "Customers" endpoint, than a client can request all of your customers and will receive them unless you guard that endpoint with logic on the server. This is true with or without Breeze.
You may be thinking that the metadata describes your entire database schema and therefore exposes the entire database to Breeze client requests. That statement is not true on a couple of counts.
First, even if the client knows about your entire database schema, it can't do anything with that knowledge unless you go to the trouble of exposing every table in your web api with unguarded endpoints. This is entirely within your control and its not something you can do by accident.
Second, there is no good reason to send metadata that describe your entire database. If you let the server generate the metadata based on the Entity Framework model, you can easily limit the size and shape of that model to the subset of the database that you want to expose in your client-facing api.
After you've narrowed the model and the web api to the size and shape appropriate for your application, you must take the next step ... the step you'd take for any web api imaginable ... which is to guard the endpoints.
At a minimum that means ensuring that the user is authenticated and authorized to make requests of each endpoint. But it also means preventing unwanted responses even to authorized user requests. For example, you might want to limit on the server the number of Customers that can be returned for any given customer query. You might want to throttle the number of requests that you'll process in a single interval of time. You might want to filter the customers down to just those few that the user is allowed to see.
The techniques for doing these things are all part of the ASP.NET Web API itself, having nothing to do with Breeze whatsoever. You'll want to investigate the options that Web API has to offer.
The update side of things is actually much easier to manage with Breeze components for ASP.NET Web API. The conventional Breeze update mechanism is a batch post to a single SaveChanges endpoint. In other words, the surface area of the attack can be limited to a single endpoint. The Breeze SaveChanges method for .NET offers two interception points for security and integrity checks:
BeforeSaveEntity where you can inspect and confirm every entity individually before it gets saved to the database.
BeforeSaveEntities where you can inspect the entire batch as a whole ... to ensure that the save request is cohesive and legitimate. This is something you can't do in a simple REST-ish api where PUT/POST/DELETE requests arrive as separate, autonomous events
The Breeze query language is highly expressive so it is possible that the client may query the server for something that you were not expecting. The expand clause is probably the most "dangerous" in this regard. Someone can query the Customer endpoint and get their related Orders, their related OrderDetails, the related Products, etc ... all at the same time.
That is a great power and with it comes responsibility. You may choose to withhold that power by refusing to allow expand queries. You can refuse select queries that can "expand" by selecting related entities. The ASP.NET Web API makes it easy to impose these restriction.
Alternatively, you can allow expand in some cases and not others. Or you can inspect the query request within the GET endpoint's implementation and refuse it if it fails your security checks.
You could decide that you don't want certain entity types to be "queryable" at all. You can create just the specialized GET endpoints you need to support safe access to those highly sensitive types. If the supporting methods don't return IQueryable, neither Breeze nor Web API will attempt to turn the OData query parameters into LINQ queries. These endpoints look and behave exactly like the traditional REST-ish apis that are familiar to you now. And the Breeze client will be happy to play along. When you compose a Breeze query, the client doesn't know whether the server will honor that request. With Breeze you can compose any request you want and send it to any HTTP endpoint you desire. You're not limited to OData-style queries.
You don't need ONE approach for querying. You decide what entity types are exposed, how, and under what conditions. You can and should write the guard logic to ensure that the proper data are returned by a query or updated by a save ... just as you would for any web api. Both Breeze and Web API give you a rich set of tools for writing such guard logic. Your options are unbounded.
Finally, I observe that Breeze-oriented apis tend to be much smaller than the typical RESTy api ... that is, they offer fewer endpoints and (in this sense) a smaller surface area. As a practical matter, that means you can concentrate your server-side security budget on fewer methods and potentially improve both the quality of that code and your capacity to scrutinize your api's security risks.
How does Zapier/IFTTT implement the triggers and actions for different API providers? Is there any generic approach to do that, or they are implemented by individual?
I think the implementation is based on REST/Oauth, that is generic from high level to see. But for Zapier/IFTTT, it defines a lot of trigger conditions, filters. These conditions, filters should be specific to different provider. Is the corresponding implementation in individual or in generic? If in individual, there must be a vast labor force. If in generic, how to do that?
Zapier developer here - the short answer is, we implement each one!
While standards like OAuth make it easier to reuse some of the code from one API to another, there is no getting around the fact that each API has unique endpoints and unique requirements. What works for one API will not necessarily work for another. Internally, we have abstracted away as much of the process as we can into reusable bits, but there is always some work involved to add a new API.
PipeThru developer here...
There are common elements to each API which can be re-used, such as OAuth authentication, common data formats (JSON, XML, etc). Most APIs strive for a RESTful implementation. However, theory meets reality and most APIs are all over the place.
Each services offers its own endpoints and there are no commonly agreed upon set of endpoints that are correct for given services. For example, within CRM software, its not clear how a person, notes on said person, corresponding phone numbers, addresses, as well as activities should be represented. Do you provide one endpoint or several? How do you update each? Do you provide tangential records (like the company for the person) with the record or not? Each requires specific knowledge of that service as well as some data normalization.
Most of the triggers involve checking for a new record (unique id), or an updated field, most usually the last update timestamp. Most services present their timestamps in ISO 8601 format which makes parsing timestamp easy, but not everyone. Dropbox actually provides a delta API endpoint to which you can present a hash value and Dropbox will send you everything new/changed from that point. I love to see delta and/or activity endpoints in more APIs.
Bottom line, integrating each individual service does require a good amount of effort and testing.
I will point out that Zapier did implement an API for other companies to plug into their tool. Instead of Zapier implementing your API and Zapier polling you for data, you can send new/updated data to Zapier to trigger one of their Zaps. I like to think of this like webhooks on crack. This allows Zapier to support many more services without having to program each one.
I've implemented a few APIs on Zapier, so I think I can provide at least a partial answer here. If not using webhooks, Zapier will examine the API response from a service for the field with the shortest name that also includes the string "id". Changes to this field cause Zapier to trigger a task. This is based off the assumption that an id is usually incremental or random.
I've had to work around this by shifting the id value to another field and writing different values to id when it was failing to trigger, or triggering too frequently (dividing by 10 and then writing id can reduce the trigger sensitivity, for example). Ambiguity is also a problem, for example in an API response that contains fields like post_id and mesg_id.
Short answer is that the system makes an educated guess, but to get it working reliably for a specific service, you should be quite specific in your code regarding what constitutes a trigger event.
I am building a mobile app using AngularJS and PhoneGap. The app allows the user to access a large amount of data-items. These data-items come with the app in form of a number of .json files.
One use-case is that a user can favorite any of those data-items.
Currently, I store the (ids of) the items that have been favorited in localStorage. It works and it's great and very simple.
But now I would like create an online-backend for the app. By this I mean that the (ids of) the items that have been favorited should also be stored on a server somewhere in some form of database.
Now my question is:
How do I best do this?
How do I keep the localStorage data and online-backend data in synch?
In particular, the user might not have an internet connection at the time were he favorites a data-item. Additionally, if the user favorites x data-items in a row, I would need to make x update calls to the server db, which clearly isn't great.
So, how do people do it?
Does Angular have anything build-in for this?
Is there any plugin?
Any other framework?
This very much seems like a common problem that must have a well-known solution?
I think you've almost got the entire solution. All you need to do is periodically (on app start load the data from the service if available, otherwise use current local storage, then maybe with a timer and on app close update the data if connected) send the JSON out to a service (I generally prefer PHP, but Python, Java, Ruby, Perl, whatever floats your boat) that puts it in a database. If you're concerned with merging synchronization changes you'll need to use timestamps in the data in local storage and in the database to make the right call on what should be inserted vs what should be updated.
I don't think there's a one size fits all solution to the problem, though I imagine someone may have crafted a library that handles the different potential scenarios the configuration may be as complicated as just writing the logic yourself.
I'm a beginner AngularJS user. I've been trying to pull hard coded JSON (backend and server data not ready) currently. It seems that in order to pull data, for instance when using the very common ng-repeat, I need to know the database structure (as the rendered JSON will mirror that structure, right?).
So while I can code independently of the back end, am I correct in my assumption that I must know the database structure? For instance... I might want to pull user comment data. This could be in its own database and I might do this: ng-repeat='comment in comments' and filter for the specific user within each comment entry in database. Whereas if comments are only within a user table it would be ng-repeat='comment in user[0].comments'. I would imagine the former is the correct approach but I honestly have never learned about proper database structure. It seems that it is something you must know in order to properly implement AngularJS though.
Any help is appreciated. I really want to make sure I approach things properly. Thanks!
I don't think you need to (or should) know the database structure. AngularJS is an MVC framework. A basic principle in this architecture is the separation of concerns. Simple put: do not mix stuff, but more specifically, you're talking about the communication between two systems: a local one (the browser running angularJS) and the remote one (a server that might, or might not, be the same that served the angular files to the client)
For example, your view should not be accessing your database (if you were working with, say, PHP, you should not have things like mysql_query(...) in a view).
You should also design components to be loosely coupled: make them as independent as possible. Unit tests help you think that way and AngularJS is particularly unit-tests-friendly with karma. Following this principle, what if you use the twitter API to show tweets in your angularJS application? you don't need to know about the internals of twitter. There is an API that serves this JSON in a format that you can use.
Your backend should provide this (for example, with a façade controller), and you should agree with the backend team what data will be available.
Instead of making your design depend on the database structure, make the backend API depend on your requirements. this way you'll have two systems loosely coupled and the backend team can do whatever they want without affecting you. For example, changing the DBMS or the structure of the tables.
If you want to pull comments, you might have a remote call ($http or ng-resource) that gets all the comments for a specific user (or for a few users, because you might want to minimize the number of remote calls) in a service or in a controller. The server responds with a json file that represents this (and probably some more things that will be needed soon, like profile picture urls, user id's, etc). Then you put the data you want to expose to a view (a subset of what you fetched from the server) in $scope.