How do you fetch all documents (including non-existent ancestor documents) in a Firebase collection? - database

I am trying to pull all the documents in the collection 'users', but it only pulls 'fred' and 'lisa', and ignores all the italicized documents:
For this data:
Trying to get all documents:
Will yield:
info: length 2
info: fred => { gender: 'male', contacts: [ '' ] }
lisa => { contacts: [ '' ] }
According to the Firebase documentation (Firebase: Add and Manage Data):
Warning: Even though non-existent ancestor documents appear in the console, they do not appear in queries and snapshots. You must create the document to include it in query results.
Note: The non-existent ancestor users seem to be auto-created when the user hits the sign-up button that triggers a firebase.auth() function (fred and lisa were created manually).
How would I print the contacts of each user if some of the users do not show up in my queries? Would I need to periodically run a script that manually re-adds all the users or is there a more elegant solution?

As you have mentioned, these "documents" are displayed with an italic font in the Firebase console: this is because these documents are only present (in the console) as "container" of one or more sub-collection but they are not "genuine" documents.
As matter of fact, if you create a document directly under a col1 collection with the full path doc1/subCol1/subDoc1, no intermediate documents will be created (i.e. no doc1 document).
The Firebase console shows this kind of "container" (or "placeholder") in italics in order to "materialize" the hierarchy and allow you to navigate to the subDoc1 document but doc1 document doesn't exist in the Firestore database.
Let's take an example: Imagine a doc1 document under the col1 collection
col1/doc1/
and another one subDoc1 under the subCol1 (sub-)collection
col1/doc1/subCol1/subDoc1
Actually, from a technical perspective, they are not at all relating to each other. They just share a part of their path but nothing else. One side effect of this is that if you delete a document, its sub-collection(s) still exist.
So, if you want to be able to query for these parent documents, you will have to create them yourself, as jackz314 mentioned in the comments.

If you're trying to list all your registered users from Firebase auth, you can use the Firebase SDK function:
function listAllUsers(nextPageToken) {
admin.auth().listUsers(1000, nextPageToken)
.then(function(listUsersResult){
listUsersResult.users.forEach(function(userRecord) {
console.log('user', userRecord.toJSON());
})
if (listUsersResult.pageToken) {
// list next batch of users
}
})
.catch(function(err) {
console.log('Error listing users: ', error)
});
}
listAllUsers();
via http://firebase.google.com/docs/auth/admin/manage-users#list_all_users

Related

Query based on multiple filters in Firebase

I am working out the structure for a JSON database for an app like onlyFans. Basically, someone can create a club, then inside of that club, there are sections where the creator's posts are shown and another where the club members posts are shown. There is however a filter option where both can be seen.
In order to make option 1 below work, I need to be able to filter based on if isFromCreator=true and at the same time based on timstamp. How can I do this?
Here are the 2 I have written down:
ClubContent
CreatorID
clubID
postID: {isFromCreator: Bool}
OR
creatorPosts
postID: {}
MemeberPosts
postID: {}
Something like the below would be what I want:
ref.child("Content").child("jhTFin5npXeOv2fdwHBrTxTdWIi2").child("1622325513718")
.queryOrdered(byChild: "timestamp")
.queryLimited(toLast: 10)
.queryEqual(toValue: true, childKey: "isFromCreator")
I triedqueryEqual yet it did not return any of the values I know exist with the configuration I specified.
You can use additional resource locations within rules by referencing the parent/child directories specifically and comparing the val() of the respective node structure.
for example:
".write": "data.parent().child('postID').child('isFromCreator').val()"
Just be aware that Security Rules do not filter or process the data in the request, only allow or deny the requested operation.
You can read more about this from the relevant documentation:
https://firebase.google.com/docs/database/security/rules-conditions#referencing_data_in_other_paths
https://firebase.google.com/docs/database/security/core-syntax#rules-not-filters

When should I use _id in MongoDB?

MongoDB has a field for every document called "_id". I see people using it everywhere as a primary key, and using it in queries to find documents by the _id.
This field defaults to using an ObjectId which is auto-generated, an example is:
db.tasks.findOne()
{
_id: ObjectID("ADF9"),
description: "Write lesson plan",
due_date: ISODate("2014-04-01"),
owner: ObjectID("AAF1") // Reference to another document
}
But in JavaScript, the underscore behind a field in an object is a convention for private, and as MongoDB uses JSON (specifically, BSON), should I be using these _ids for querying, finding and describing relationships between documents? it doesn't seem right.
I saw that MongoDB has a way to generate UUID https://docs.mongodb.com/manual/reference/method/UUID
Should I forget that _id property, and create my own indexed id property with an UUID?
Use UUIDs for user-generated content, e.g. to name image uploads. UUIDs can be exposed to the user in an URL or when the user inspects an image on the client-side. For everything that is on the server/not exposed to the user, there is no need to generate a UUID, and using the auto-generated _id is preferred.
An simple example of using UUID would be:
const uuid = require('uuid');
exports.nameFile= async (req, res, next) => {
req.body.photo = `${uuid.v4()}.${extension}`;
next();
};
How MongoDB names its things should not interfere in how you name your things. If data sent by third-party hurts the conventions you agreed to follow, you have to transform that data into the format you want as soon as it arrives in your application.
An example based in your case:
function findTaskById(id) {
var result = db.tasks.findOne({"_id": id});
var task = {
id: result._id,
description: result.description,
something: result.something
};
return task;
}
This way you isolate the use of Mongo's _id into the layer of your application that is responsible to interact with the database. In all other places you need task, you can use task.id.

Question: What is the best DB structure when adding follow/followers feature?

I have an app with the following DB structure currently, it uses Firebase Database and is on Swift IOS:
"Posts" : {
"Dm8iyaXXdTOJGsymEiLNVO6OdDK2" : {
"post:570915537" : {
"Media" : {
"image" : {
"mediaUrl" : "https://firebaseURL",
"postTimeStamp" : 5.70915539085856E8,
"timeStamp" : 5.7091551482329E8
}, ...
I was now going to add followers to it. I was thinking that I would either add a whole new group:
"Followers" : {
"Dm8iyaXXdTOJGsymEiLNVO6OdDK2" : {
"Following" : {
follower1: "Dm8iyaXXdTOJGsymEiLNVO6OdDK2";
//other followers
}, ...
Or add a new node to the original group and add them there. The last option is to do something similar to what is done above but in the "Users" section.
What is the best course of action?
A follow/unfollow schema using Firestore could be this one:
2 roots collections, one that holds users and another one that holds the following relationship between users through a composite key.
users/{userID}
.. userData
.. followerCount
.. followedCount
following/{followerID_followedID}
.. followerId
.. followedId
.. createdAt
When a userA starts to follow a userB:
set a new document with the id userAuid_userBuid in the
following collection.
triggers a Cloud Function that will run a transaction to update the
counters of both users.
Do the reverse thing when a user stops to follow another user.
In the client, you can know if userA follows userB simply by checking if the document userAuid_userBuid exists inside the following collection.
You can also get the list of a user's followers by querying the collection where the followedId == the current user Id.
Hope that helps you.

Should I always add new objects to list of objects via $push()?

I'm creating a data structure for Firebase and AngularFire consisting of Users, Posts, and Comments. I was under the impression that the key/id for users would be the username, and that the key/id for comments and posts would be the auto-generated firebase key.
I've been working my through the angularfire documentation and am confused about the auto-generated keys (name()) that is added to an object when the $push() method is used.
Looking at some examples on the firebase website I see that an example of a Users object does not have the auto-generated key -- the key for an individual user is the username -- but at the same time a key is added whenever you add an object to the array via $push
My question is:
1) Should I always be using the firebase auto-generated keys? And if not, then how do I add a new user since $push() automatically creates the key, and $set() would reset all of my users?
2) What is the relationship between $id and name()?
Example Data
From https://www.firebase.com/docs/web/guide/saving-data.html
The docs show the following Users object:
{
"users": {
"alanisawesome": {
"date_of_birth": "June 23, 1912",
"full_name": "Alan Turing"
},
"gracehop": {
"date_of_birth": "December 9, 1906",
"full_name": "Grace Hopper"
}
}
}
How would I add more users without resetting my current users with $set() or adding the angularfire id with push()?
And then a Posts object with the generated id:
{
"posts": {
"-JRHTHaIs-jNPLXOQivY": {
"author": "gracehop",
"title": "Announcing COBOL, a New Programming Language"
},
"-JRHTHaKuITFIhnj02kE": {
"author": "alanisawesome",
"title": "The Turing Machine"
}
}
}
Thanks very much.
The short answer: you probably don't want to use push to store your users.
If you're getting your key from another source, like a uid from Simple Login, you will almost certainly want to use the uid to organize your users and their data in your firebase.
This is because, your users' ongoing sessions always provide you with that same uid which you can use to look up their user data and their stuff.
And you can safely use set in this case without resetting all of your users if you set based on that known user id.
But what I think you're getting at is, So in general, when do you set vs push?
A typical blog might look something like this in Firebase:
{
'users' : {
// uid from Simple Login, that you used with set()
'google-1234' : {
'displayName' : 'Jane Smith',
...
}
, ...
},
'posts' : {
// a blog post ID you pick and use for set()
'blog-post-id-i-use-in-the-url' : {
'title' : 'Blog Post Title',
'contents' : 'Four score and seven...'
}, ...
}
'postComments' {
'blog-post-id-i-use-in-the-url' : {
// Firebase generated ID done with push()
'_fe31ca1' : {
// uid from simple login (assuming comments require auth)
'commenterUserId': 'google-5678',
'commentBody': 'cats back for everyone!'
} ... other comments ...
}
}
}
In this example we use set when inserting new users and posts because we get a good unique ID from another source. These IDs are good because they allow us to easily recall the content later based on that ID.
We use push for comments, though. We don't have a good ID from another source, and order does matter, so we let Firebase generate a key for us. This works out OK because most of the time we're working with comments relative to an entry, so we can just grab them all as needed.
Following what mimmming said, I found a solution to this.
Have your add user function take an id as a parameter. this will be the authData.uid for the user you want to save.
Then append that id to the firebase link to make a new user using set.
Any other user you add using set will not wipe this since it is an entire new branch of your database under users. No firebase unique id too.
$scope.addUSer = function(id){
//pass the id in, andd append it to the end of your url link
var usersRef = new Firebase("https//<your fire base>.firebaseio.com/Users/"+id);
usersRef.set($scope.newUserData);
};

AngularFire - How do I query denormalised data?

Ok Im starting out fresh with Firebase. I've read this: https://www.firebase.com/docs/data-structure.html and I've read this: https://www.firebase.com/blog/2013-04-12-denormalizing-is-normal.html
So I'm suitably confused as one seems to contradict the other. You can structure your data hierarchically, but if you want it to be scalable then don't. However that's not the actual problem.
I have the following structure (please correct me if this is wrong) for a blog engine:
"authors" : {
"-JHvwkE8jHuhevZYrj3O" : {
"userUid" : "simplelogin:7",
"email" : "myemail#domain.com"
}
},
"posts" : {
"-JHvwkJ3ZOZAnTenIQFy" : {
"state" : "draft",
"body" : "This is my first post",
"title" : "My first blog",
"authorId" : "-JHvwkE8jHuhevZYrj3O"
}
}
A list of authors and a list of posts. First of all I want to get the Author where the userUid equals my current user's uid. Then I want to get the posts where the authorId is the one provided to the query.
But I have no idea how to do this. Any help would be appreciated! I'm using AngularFire if that makes a difference.
Firebase is a NoSQL data store. It's a JSON hierarchy and does not have SQL queries in the traditional sense (these aren't really compatible with lightning-fast real-time ops; they tend to be slow and expensive). There are plans for some map reduce style functionality (merged views and tools to assist with this) but your primary weapon at present is proper data structure.
First of all, let's tackle the tree hierarchy vs denormalized data. Here's a few things you should denormalize:
lists you want to be able to iterate quickly (a list of user names without having to download every message that user ever wrote or all the other meta info about a user)
large data sets that you view portions of, such as a list of rooms/groups a user belongs to (you should be able to fetch the list of rooms for a given user without downloading all groups/rooms in the system, so put the index one place, the master room data somewhere else)
anything with more than 1,000 records (keep it lean for speed)
children under a path that contain 1..n (i.e. possibly infinite) records (example chat messages from the chat room meta data, that way you can fetch info about the chat room without grabbing all messages)
Here's a few things it may not make sense to denormalize:
data you always fetch en toto and never iterate (if you always use .child(...).on('value', ...) to fetch some record and you display everything in that record, never referring to the parent list, there's no reason to optimize for iterability)
lists shorter than a hundred or so records that you always as a whole (e.g. the list of groups a user belongs to might always be fetched with that user and would average 5-10 items; probably no reason to keep it split apart)
Fetching the author is as simple as just adding the id to the URL:
var userId = 123;
new Firebase('https://INSTANCE.firebaseio.com/users/'+userId);
To fetch a list of posts belonging to a certain user, either maintain an index of that users' posts:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
fb.child('/my_posts/'+userId).on('child_added', function(indexSnap) {
fb.child('posts/'+indexSnap.name()).once('value', function(dataSnap) {
console.log('fetched post', indexSnap.name(), dataSnap.val());
});
});
A tool like Firebase.util can assist with normalizing data that has been split for storage until Firebase's views and advanced querying utils are released:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
var ref = Firebase.util.intersection( fb.child('my_posts/'+userId), fb.child('posts') );
ref.on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val();
});
Or simply store the posts by user id (depending on your use case for how that data is fetched later):
/posts/$user_id/$post_id/...
new Firebase('https://INSTANCE.firebaseio.com/posts/'+userId).on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val());
});

Resources