My User object that I want to create and store in the datastore has an email, and a username. How do I make sure when creating my User object that another User object doesn't also have either the same email or the same username?
If I just do a query to see if any other users have already used the username or the email, then there could be a race condition.
UPDATE:
The solution I'm currently considering is to use the MemCache to implement a locking mechanism. I would acquire 2 locks before trying to store the User object in the datastore. First a lock that locks based on email, then another that locks based on username.
Since creating new User objects only happens at user registration time, and it's even rarer that two people try to use either the same username or the same email, I think it's okay to take the performance hit of locking.
I'm thinking of using the MemCache locking code that is here: http://appengine-cookbook.appspot.com/recipe/mutex-using-memcache-api/
What do you guys think?
Try storing your User with their email as the key_name. This can be done in one simple step:
MyUser.get_or_insert(email)
Getting your MyUser by an email is also easy:
MyUser.get_by_key_name(email)
See this similar question: add properties to users google app engine
So that solves the problem of two users with the same email. To do the same for usernames, perform the "get users with this username" query and the "insert a user with this username" in a transaction (this is what get_or_insert() does behind the scenes).
You can catch a TransactionFailedError to find cases when another user "takes" that username during your transaction.
You definitely don't want to use a memcache mutex, since that while loop waiting for the lock to free up can spend a lot of your memcache API call quota.
If you use the JPA impl of datastore, you just have to set the annotation
#Column(unique=true)
Field field
This way the db will reject your insert/update.. i guess it's implemented by gdatastore, it's not a big matter for me to give a "technical error" to the user on such a specific case... but anyway you could catch a constraintviolation exception. I guess JDO also have this.
But actually i don't even know if you'r using Python or Java...
In Java as far as i know you can do a select request in a transaction...
About transactions you should also check what is transaction isolation and how it works on GAE...
Related
Context
Developing a platform with contents and some files. Users can create a resource, and set it:
private: only accessible for him
protected: accessible via a password that can expire
public: no security rules to access
Problem
Considering we have that 2 tables:
We want to protect our element table if visibility = 'protected'. So creator should give a password for resource access, and set an expiration date.
Users can set many passwords with a different expiration date.
How can we store that password, with security, and thinking about the user can share a password, close password access, and get his password access to share resources when he needs them.
We talk here about a conception of the solution, don't care about languages or orm.
Potential solution
Cron job
Create a table joining passwords with entities, and when the user sets a password, launch a cron job that will set an attribute like active to false when the expiration date is reached.
Problem: making our app stateful, if cloud-based and the pod crash, cron job fell in void...
Store it in base64
To allow users to get back already set passwords, we have to get symmetric encryption algorithm to encrypt and decrypt the password, but that exposes resources if the database is compromised..
Also creating a table joining passwords with entities here.
No more ideas for the moment... How would you do ?
The best solution should be stateless and don't store passwords in clear.
I can think of other two solutions
A PubSub mechanism, which you can choose to trigger when to execute. For example, when you create a password today and want it to expire tomorrow. you can create a pubsub task that will trigger in a day. The task will remove that password.
A JWT token, which is a token that encoded the information in there, which includes the expiration date. When you verify that token, you will verify the signature to make sure it is not tampered and also if it is still valid. If you need to store secret inside it, you can use RS256.
This situation might not happen much, but I would rather be safe than sorry. I'm currently trying to make a sign in/sign up system with flutter MongoDB using API call to my database. On the signup screen, I am making checks to see if the data placed into the password and username text fields already exists within the database. In other words, if an account with those credentials already exists then I ask the user to try again, but if there isn't one then I allow for the user to go to the EULA and submit page.
The problem here is that I feel like that two or more users can do the same check at the same time, and will get the same result because, while their information might match, there isn't an instance of an account with those credentials uploaded to the database for the system to tell them to stop before they go to the submit page. And if they submit the data they wrote on their phones to the database at the same time, a situation is made where we have multiple instances of the same kind of account existing in the database.
I know that keys exist and can help in making each account unique from each other, but, in this situation, I am not sure on how to handle or prevent the concurrent uploading of similar or identical data from local devices to the same server. Are calls to my server and MongoDB asynchornous? Is this something that I need to worry about at all?
TL;DR: If you use a unique index on the username field, then it will raise an error on attempting to create a duplicate, and you can forward this to the user's sign up page and let them change their name.
The long answer:
This is more of a UX problem than a DB problem.
If I understand correctly, the user enters a username & password, then reads the EULA before the account is created.
I would disagree with this - you should make the account once they submit their name, and update it with a Boolean for eulaAccepted so that if this process fails (eg. bad connection) they can come back and accept it later.
You should do two sets of unique username validation:
When they type in the name, you can check for them and let them know the name is available. This still means someone else could take it before they hit save.
Once they hit save. Put a unique index on the username field. Then if a user tries saving a duplicate (even if submitted simultaneously, they will be processed sequentially) then it will raise an error. You can forward this onto the user, prompting them to try a different name.
Answering your other questions:
Are calls to my server and MongoDB asynchronous?
Yes, as they go over the network they are naturally asynchronous. But the saves inside the Database itself are sequential and atomic. MongoDB is ACID compliant. have a read through what it means, and what changed in version 4
Is this something that I need to worry about at all?
Yes, it's good to keep an eye on this sort problems. If you don't consider it now, you will have a nasty bug on your hand later.
In my gae application, a user can do an action (buy something). I need that information stored persistently and available imediately on all requests from all sessions of this user on multiple devices/browsers. I'm using webapp2_extras sessions.
The way I'm thinking of doing this is either:
1) adding the action_happened field to the User model and make it available in the session by adding it to the list in webapp2_extras.auth['user_attributes'] config. But this doesn't work unless the user is logged out on all sessions.
or 2) create a memcache entry (backed by the datastore) for each user like user_id_action_happened and check if it is true or false on each request. This is my preffered method.
Is there any other way to do this?
I think storing in database and doing a query on each request is the most natural option.
Don't know about your full requirements and specifications, but for keeping the sessions synchronized I think a solution like firebase makes a lot of sense, though it might be overkill in your case.
What's the best practice or the common way of keeping (or not keeping) Evernote users in your application's database?
Should I create my own membership system and create a connection to Evernote accounts?
Should I store Evernote user data (or only part of it) in my own app and let the user log in only with Evernote?
Summary: you must protect their data but how you protect it is up to you. Use the integer edam_userId to identify data.
I think the API License agreement covers protection in the terms:
you agree that when using the API you will not, directly or indirectly, take or enable another to take any of the following actions:...
1.8.4 circumvent or modify any Keys or other security mechanism employed by Evernote or the API;
If you cache people's data and your server-based app lacks security to prevent people looking at other's data, then I think you're pretty clearly violating that clause. I think it's quite elegantly written!
Couple that with the responsibility clause 1.2
You are fully responsible for all activities that occur using your Keys, regardless of whether such activities are undertaken by you or a third party.
So if you don't protect someone's cached data and another user is able to get at it, you're explicitly liable.
Having cleared up the question of your obligations to (as you'd expect) protect people's data, the question is how do you store it?
Clause 4.3 covers identifiers pretty directly although it's a bit out of date now that we are all forced to use oAuth - there are no passwords ever entered into anything other a web view. However, mobile or desktop client apps must provide a mechanism for the user to log out, which must completely remove the username and password from your application and its persistent storage.
For a web app, you can't even save the username: If your Application runs as an Internet service on a multi-user server, you must not ask for, view, store or cache the sign-in name or password of Evernote user accounts.
The good news is that you can rely on the edam_userId value which comes back to you in the oAuth token credentials response, as discussed here.
When you look at the Data Model, you can see the unique id under the User and going into the User struct, see the reassuring definition The unique numeric identifier for the account, which will not change for the lifetime of the account.
Thinking about the consequences, as you can't get the user id until you have logged into the service, if you want to provide a local login for people you will have to link your local credentials to the user id. That may irk some people if they have to enter a username twice but can't be helped.
You can allow users to log-in via OAuth. Here's a guide on how that process works.
But you'll probably also want to store a minimal amount of user data, at least a unique identifier, in your database so you can do things like create relationships between the user and their notebooks and tags. Refer to the Evernote data model for those relationships. If you're using rails, this will also help you take advantage of rails conventions.
I have a business requirement that calls for an end user to be able to login to an e-commerce system with a one-time use password, purchase a course, and then never be able to login again. At the moment the company that is allowing their employees, my client's students, is going to generate that password for them and send me a list of the passwords they are generating.
I do not like the concept but I have yet to come up with a better one that will work for all parties. If something happens mid-purchase they have to go back to the company to get new login credentials.
The ability to purchase a course cannot be open to the public, it has to be through the e-commerce portal, and for the moment there is only one company participating but there will be more in the future. I am seeing this as a total maintenance nightmare.
My question is has anyone run across this before, if you wouldn't mind sharing tell me how you solved it, or point me in a direction that will give me some insight. I have googled the concept to death and have not come up with any bright ideas.
You can't resolve the "problem during purchase" issue yourself - They need to modify their service so that the password is destroyed when the transaction completes.
No matter what you do with regards to the password, once the user is logged in, the password is disabled on their servers. After that, you have no control.
They should either:
Cancel password on transaction completion
Provide you with an API/Web service to allow you to request a new key.
I have the answer for this one and document it here for my own personal OCD completion purposes. I created a login form that allows for a two part key; username is based on company name plus internal company ID(not a guid), paired with a password that is cryptic key, ex: #SCD6-, plus employee id generated by the company. The rub is I don't know what the employee ID is, and only use it because it should be unique although it doesn't matter if it is not, once the user logs in once completes the only marginally secure process then checks out the login is invalid and can never be used again, unless manually unlocked (in case of a duplicate employeeID at some time in the future, which is unlikely). The username and key is emailed to the employees of the target company, which is generated by the target company.If I do have the employeeID in the system which is 50/50 then I can pre-fill the forms out.
The only thing this lock protects is a process and not secure information so I am not too worried about security and it's only real purpose is to keep John Q, from stumbling onto the process and paying money that my client would need to refund later. If it were a secure data process I would not use this method.