I have just started to look at Redis and would like to be able to store an Array of hashes, where I can pop a random key/value out and then put it back in when I need to.
So in Ruby I would have something like this
users = [{ username: "user1", password: "password"}, { username: "user2", password: 'password'}]
So if I wanted to get a random key/value object from the Array I would do something like this
#user = users.shuffle!.pop
And then to put it back into the array
users.push(#user)
The idea for using Redis is that I have two processes (Ruby based app) that need to share a pool of users at the same time. Once a process has finished with a user I want it to put it back into the pool.
Could anyone point me in the right direction please
Thanks
You could Redis Hash to store a user info and Redis Set to store all these hashes together.
Steps:
Make a redis Hash with HSET command:HMSET userId_653 username "Tom" password "gd36e3hd38d3jdj3yd3hd38"
Add this hash in the set called users: SADD users userId_653. This set contains all the users.
Get a random user key from the set: SRANDMEMBER users. It will return userId_653
Get the corresponding values from hash using HGET userId_653 username
If you need to pop the key simply do SPOP users after step 3. and SADD again after processing in step 4.
A similar question for better understanding: Redis how to store associative array
References:
http://redis.io/commands/srandmember
http://redis.io/commands/sadd
http://redis.io/commands/spop
http://redis.io/commands/hget
http://redis.io/commands/hmset
PS: I have no experience in Ruby. Look for suitable Redis Ruby API which would support all these operations!
Related
I am aiming to build a scalable infrastructure (for fun) for a user posting system where timelines exist, such as a feed system. Feeds would include posts of a users followings.
Anyways, the current approach is this.
UserA follows UserB
Save the follows relationship in the DB
Create a redis list <userBiD-followers> -> [userAid, ...]
UserB creates a post
Save that post into the DB
Push the <postId> into their Redis list <userId-posts> -> [postId1, postId2, postId3, ...]
Now since UserA has a feed redis list <userAid-feed> -> [postId1, postId2, ...]
I essentially iterate over UserB followings, and push UserBs new post into all of their timelines. (fan-out)
Is this the right approach? Anything better I can do?
Another question of mine is, is it much faster to retrieve a redis list of postIds, and pass them into Prisma to get the actual posts than doing queries with just Prisma?
Example below.
const redisUserPosts = redis.lrange(`${userId}-posts`, 0, -1) = ['1', '2', ...]
posts.findMany({
where: {
id: {
in: redisUserPosts
}
}
})
The alternative would be to find by user, do potential other joins, etc, rather than knowing the exact postId which belongs to a user.
I am simply asking for advice on design, this is already implemented.
I am trying to retrieve data in my MongoDB database. If I have the following below in my mongoDB database, I want to select the Password given the Username. So in this case, I will be looking through the database for a Username that is 'e' and retrieving the password associated with that specific Username. I've tried looking everywhere but I can't seem to find a solution on how to do it. I am using express, node, and mongoDB for this personal project. What I have so far is just looking up with database with .find({ Username: Username} and it outputs the entire JSON object.
To clarify, I will be sending a request with a Username of value 'e' and looking it up the database trying to retrieve the value of Password.
{
_id: 62d7712e6d6732706b46094e,
Username: 'e',
Password: 'hi',
__v: 0
}
find takes multiple inputs you can give the select statements also in find itself
so the query will be like
db.collectionName.find({username:'e'},{_id:0,password:1})
mongo by default fetch _id all the time by default so you need to specifically mention to not fetch _id thus _id :0
for such scenarios, there are 2 options if username is unique i would suggest to go with findOne rather then find
db.collectionName.findOne({username:'e'}).password
the same will work if you have multiple records with same username but you want only the first record
but if you want data of all the records as array
db.collectionName.find({username:'e'},{_id:0,password:1})..map( function(u) { return u.password; } )
I'm writing a program and before it loads I want the user to enter the correct password without storing the password anywhere in my code. I've implemented MD5 hashes before but from what I've read they're outdated and can be broken. There are a few sites out there that attempt to reverse engineer and MD5 hash. What's the strongest encryption I can use to keep prying eyes out of my program (e.g., The NSA)?
"Encryption" is not the right thing to do for storing user passwords - as by design an encrypted password can be decrypted. As you said - hashing is the way to go.
MD5 is outdated, and I believe the current recommendation is sha1.
Note that there are ways to reverse any hashing algorithm to acceptable input. The commonly accepted standard to make this much more dificult is to add a unique "salt" to all passwords before putting them through the hashing function. A common mistake made when adding salts to passwords is to use the same salt value on every password in the database.
When salting passwords, use a unique value, for example the user ID, or the created date/time string for the user record. This will prevent attacks based on rainbow tables because there will be no existing ready to use rainbow table for your stored password hashes.
I personally like the approach of using the created date / time string of the user as it's a value that should never change and will be available and will likely be different for each user the the database.
Eexamples below assume you are familiar with PHP - however the concepts can be applied to any language.
Example:
Before saving a new user into the database:
$date = date('Y-m-d H:i:s');
// save this same value into the user record somewhere
$passwordHash = sha1($user['created_date'].$_POST['password']);
// and save the $passwordHash value into the password field for that user
To authenticate a login attempt, use something like the following:
function authenticateUserLogin($email, $password) {
$user = $db->fetchRow('SELECT * FROM users WHERE email=?', array($email));
if (!$user) return false;
$passwordHash = sha1($user['created_date'].$password);
return $user['password_hash'] !== $passwordHash;
}
To update an existing users password, use something like...
$passwordHash = sha1($user['date_created'].$newPassword);
$db->query('UPDATE users set password_hash=? WHERE id = ?', array($passwordHash, $user['id']));
I'd like to store some trivial values for each user in the database, like if the user can see the new comers' banner, the instruction on how to use each of the features etc. The number of values can increase as we come across new ideas.
So I've thought about two solutions for storing these data. Either having a field for each of these values (So the structure of the table will change a few times at least), or have one field for all these types of data, so they're stored as a dictionary in the field (In this case, I'm worried about if it's hurting db performance, I also need to write more logics for parsing the dictionary in string and the way around, and if storing dictionaries in db contradicts with what db does).
models.py
class Instruction(models.Model):
user=models.ForeignKey('auth.User')
can_see_feature_foo_instruction=models.BooleanField()
can_see_feature_bar_instruction=models.BooleanField()
...
or
class Instruction(models.Model):
user=models.ForeignKey('auth.User')
instruction_prefs=models.CharField() #Value will be "{'can_see_foo_inst':True, 'can_see_bar_inst':False, ...}"
Which will be the best solution?
It depends if you need to be able to search on these fields. If so, the text field option is not really suitable, as the individual flags won't be indexed. But if not, then this is a perfectly good way of going about it. You might want to consider storing it as JSON, which is useful as a method of serializing dicts objects to text and getting them back. There are plenty of implementations around of "JSONField" in Django that will take of serializing/deserializing the JSON for you.
Django has a built-in permission system. Try reading this link https://docs.djangoproject.com/en/dev/topics/auth/#permissions
Update
I think if you really want to use an Instruction model. You can use something like a JSONField and use it to store instructions. This way you can do something like instruction.key to access a value. You can try using this. https://github.com/derek-schaefer/django-json-field
You can create model for key value pair of instructions/permissions per user.
E.g.
class Instruction(models.Model):
user=models.ForeignKey('auth.User')
key = models.CharField(max_length=20)
value = models.BooleanField()
Then you can create multiple instances of this for each user depending upon permissions he has.
>>>> instr1 = Instruction()
>>>> instr1.user = user1
>>>> instr1.key = 'can_see_feature_foo'
>>>> instr1.value = True
>>>> instr1.save()
>>>> instr2 = Instruction()
>>>> instr2.user = user1
>>>> instr2.key = 'can_see_feature_bar'
>>>> instr2.value = True
>>>> instr2.save()
....
#To query
>>>> Instructions.filter(user=user1, key='can_see_feature_bar')
If you use a Model with a CharField to store the instruction and a ManyToManyField to the users you can create and assign any number of instructions to any number of users.
class Instruction(models.Model):
user = models.ManyToManyField('auth.User')
instruction = models.CharField() # Value will be a single instruction
I would like to store some information as follows (note, I'm not wedded to this data structure at all, but this shows you the underlying information I want to store):
{ user_id: 12345, page_id: 2, country: 'DE' }
In these records, user_id is a unique field, but the page_id is not.
I would like to translate this into a Redis data structure, and I would like to be able to run efficient searches as follows:
For user_id 12345, find the related country.
For page_id 2, find all related user_ids and their countries.
Is it actually possible to do this in Redis? If so, what data structures should I use, and how should I avoid the possibility of duplicating records when I insert them?
It sounds like you need two key types: a HASH key to store your user's data, and a LIST for each page that contains a list of related users. Below is an example of how this could work.
Load Data:
> RPUSH page:2:users 12345
> HMSET user:12345 country DE key2 value2
Pull Data:
# All users for page 2
> LRANGE page:2:users 0 -1
# All users for page 2 and their countries
> SORT page:2:users By nosort GET # GET user:*->country GET user:*->key2
Remove User From Page:
> LREM page:2:users 0 12345
Repeat GETs in the SORT to retrieve additional values for the user.
I hope this helps, let me know if there's anything you'd like clarified or if you need further assistance. I also recommend reading the commands list and documentation available at the redis web site, especially concerning the SORT operation.
Since user_id is unique and so does country, keep them in a simple key-value pair. Quering for a user is O(1) in such a case... Then, keep some Redis sets, with key the page_id and members all the user_ids..