I am looking to fetch the data of collection named users which has documents with an id of logged-in users' uid. Further, these documents contain some data and a subCollection called posts.
which looks like -
So now, I need to fetch all four(4) of these documents along with the posts collection data together so that I can display it.
My approach -
( here I fetched the document ids - middle section of image IDs)
// Fetching Firestore Users Document IDs
const [userDocs, setUserDocs] = React.useState([]);
React.useEffect(() => {
try {
const data = firestore.collection('users')
.onSnapshot(snap => {
let docIDs = [];
snap.forEach(doc => {
docIDs.push({id: doc.id});
});
setUserDocs(docIDs);
})
}
catch(err) {
console.log(err);
}
}, [])
Now, I have tried to fetch the entire data using the following way (which isn't working)
// Fetching Firestore Posts Data
const [postData, setPostData] = useState([]);
React.useEffect(() => {
try {
userDocs.map(data => {
const data = firestore.collection('users/'+currentUser.uid+'/posts')
.onSnapshot(snap => {
let documents = [];
snap.forEach(doc => {
documents.push({...doc.data(), id: doc.id});
});
setPostData(documents);
})
})
}
catch(err) {
console.log(err);
}
}, [])
Finally, I should end up with postData array which I can map on my card component to render all posted images and captions to the UI.
I am not sure if this is the right way to achieve what I am doing here, please help me correct this error and if there's a more subtle and easy way to do it please let me know. Thank You.
I have tried to fetch the entire data
Looking at the code you wrote for fetching "the entire data" (i.e. the second snippet) it seems that you don't need to link a post document to the parent user document when fetching the post documents. In other words, I understand that you want to fetch all the posts collection independently of the user documents.
Therefore you could use a Collection Group query.
If you need, for each post document returned by the Collection Group query, to get the parent user doc (for example to display the author name) you can do as explained in this SO answer, i.e. using the parent properties.
I am building an app using Firebase Firestore as a BaaS.
But I am facing a problem when I try to create a feed/implement full-text-search on my app.
I want to be able to search through all the users posts, the problem is, the users posts are structured like this in the Firestore Database:
Posts(collection) -> UserID(Document) -> user posts(subcollection that holds all userID posts) -> actual posts(separate documents within that collection)
I want to loop through every user's user posts subcollection and fetch all data for the feed, and also to implement it with a full text search app like Algolia or ES.
I can loop through a specific user ID(code below), but being a beginner, I couldn't find a way to loop through all of them and fetch all of them.
firebase.firestore()
.collection('allPosts')
.doc('SPECIFIC_USER_ID') //-> Here I have to loop through all docs in that collection
.collection('userPosts')
.orderBy("creation", "asc")
.get()
.then((snapshot) => {
let posts = snapshot.docs.map(doc => {
const data = doc.data();
const id = doc.id;
return { id, ...data }
})
setUserPosts(posts)
})
}
Would love some help!
Collection Group Query
You can query in all collections named X using a collection group query.
var posts= db.collectionGroup('userPosts').orderBy('creation').limit(10);
posts.get().then((querySnapshot) => {
let posts = querySnapshot.map(doc => {
const data = doc.data();
const id = doc.id;
return { id, ...data }
})
setUserPosts(posts)
});
Source: https://firebase.google.com/docs/firestore/query-data/queries#collection-group-query
Algolia implementation
You will need to use Cloud Functions to migrate fields to a dedicated collection specifically for Algolia. Many users have found nested SubCollections to be problematic with Algolia's setup.
You do this by duplicating the user Post data as a 'source' to this new public collection, and using the Firebase Algolia Extension, you can sync it directly
exports.bakePosts= functions.firestore
.document('allPosts/{userID}/userPosts/{postID}')
.onWrite((change, context) => {
// Get an object with the current document value.
// If the document does not exist, it has been deleted.
const document = change.after.exists ? change.after.data() : null;
// Get an object with the previous document value (for update or delete)
const oldDocument = change.before.data();
if(document != null)
db.collection("posts/"+ context.params.postID).set(document);
if(document == null)
db.collection("posts/"+ context.params.postID).delete();
});
Algolia Extension:
https://firebase.google.com/products/extensions/firestore-algolia-search
You can avoid most of the above if you simply submit posts to a master collection and have the userID as the 'owner' property within the document. The above also have benefits, but more related to blog posts where users may have a "work in progress" version vs Live.
The Algolia Extension has the full guide on how to set it up and if you need to customize the extensions, the source code is also available.
i want to fecth all the id not just 888888888 and get the orders for each id
What you're describing is known as a collection group query in Firestore.
It'd looks like this:
db.collectionGroup("orders").get()
The above reads all collections called orders no matter where they are located (top-level, under an tels doc, or under any other doc.
To determine what tels doc a specific order is under, you can navigate up ref.parent.parent of the order snapshot:
db.collectionGroup("orders").get().then((querySnapshot) => {
querySnapshot.forEach((orderSnapshot) => {
const orderRef = orderSnapshot.ref;
const ordersRef = orderRef.parent;
const telRef = orderRef.parent;
console.log(refRef.id); // 8888888888
})
})
I want to return a list of documents created by a certain user, how do I? I'm using Firestore.
It's currently like this:
firestore.get({ collection: 'jobs' });
You'll need to specify the user id you want to search by. If its the current user you could do something like...
const userId = firebase.auth().currentUser.uid
then you search for all the jobs collection documents where the userId field matches you userId above...
firebase.firestore()
.collection('jobs')
.where('userId', '==', userId)
.get()
.then(collection => {
const docs = collection.docs.map(doc => doc.data());
console.log(docs)
});
I console logged the docs, but you can return them or set a field in components state, or whatever else you want to do with them.
This question already has answers here:
Cloud Firestore collection count
(29 answers)
Closed 10 months ago.
In Firestore, how can I get the total number of documents in a collection?
For instance if I have
/people
/123456
/name - 'John'
/456789
/name - 'Jane'
I want to query how many people I have and get 2.
I could do a query on /people and then get the length of the returned results, but that seems a waste, especially because I will be doing this on larger datasets.
You currently have 3 options:
Option 1: Client side
This is basically the approach you mentioned. Select all from collection and count on the client side. This works well enough for small datasets but obviously doesn't work if the dataset is larger.
Option 2: Write-time best-effort
With this approach, you can use Cloud Functions to update a counter for each addition and deletion from the collection.
This works well for any dataset size, as long as additions/deletions only occur at the rate less than or equal to 1 per second. This gives you a single document to read to give you the almost current count immediately.
If need need to exceed 1 per second, you need to implement distributed counters per our documentation.
Option 3: Write-time exact
Rather than using Cloud Functions, in your client you can update the counter at the same time as you add or delete a document. This means the counter will also be current, but you'll need to make sure to include this logic anywhere you add or delete documents.
Like option 2, you'll need to implement distributed counters if you want to exceed per second
Aggregations are the way to go (firebase functions looks like the recommended way to update these aggregations as client side exposes info to the user you may not want exposed) https://firebase.google.com/docs/firestore/solutions/aggregation
Another way (NOT recommended) which is not good for large lists and involves downloading the whole list: res.size like this example:
db.collection("logs")
.get()
.then((res) => console.log(res.size));
If you use AngulareFire2, you can do (assuming private afs: AngularFirestore is injected in your constructor):
this.afs.collection(myCollection).valueChanges().subscribe( values => console.log(values.length));
Here, values is an array of all items in myCollection. You don't need metadata so you can use valueChanges() method directly.
Be careful counting number of documents for large collections with a cloud function. It is a little bit complex with firestore database if you want to have a precalculated counter for every collection.
Code like this doesn't work in this case:
export const customerCounterListener =
functions.firestore.document('customers/{customerId}')
.onWrite((change, context) => {
// on create
if (!change.before.exists && change.after.exists) {
return firestore
.collection('metadatas')
.doc('customers')
.get()
.then(docSnap =>
docSnap.ref.set({
count: docSnap.data().count + 1
}))
// on delete
} else if (change.before.exists && !change.after.exists) {
return firestore
.collection('metadatas')
.doc('customers')
.get()
.then(docSnap =>
docSnap.ref.set({
count: docSnap.data().count - 1
}))
}
return null;
});
The reason is because every cloud firestore trigger has to be idempotent, as firestore documentation say: https://firebase.google.com/docs/functions/firestore-events#limitations_and_guarantees
Solution
So, in order to prevent multiple executions of your code, you need to manage with events and transactions. This is my particular way to handle large collection counters:
const executeOnce = (change, context, task) => {
const eventRef = firestore.collection('events').doc(context.eventId);
return firestore.runTransaction(t =>
t
.get(eventRef)
.then(docSnap => (docSnap.exists ? null : task(t)))
.then(() => t.set(eventRef, { processed: true }))
);
};
const documentCounter = collectionName => (change, context) =>
executeOnce(change, context, t => {
// on create
if (!change.before.exists && change.after.exists) {
return t
.get(firestore.collection('metadatas')
.doc(collectionName))
.then(docSnap =>
t.set(docSnap.ref, {
count: ((docSnap.data() && docSnap.data().count) || 0) + 1
}));
// on delete
} else if (change.before.exists && !change.after.exists) {
return t
.get(firestore.collection('metadatas')
.doc(collectionName))
.then(docSnap =>
t.set(docSnap.ref, {
count: docSnap.data().count - 1
}));
}
return null;
});
Use cases here:
/**
* Count documents in articles collection.
*/
exports.articlesCounter = functions.firestore
.document('articles/{id}')
.onWrite(documentCounter('articles'));
/**
* Count documents in customers collection.
*/
exports.customersCounter = functions.firestore
.document('customers/{id}')
.onWrite(documentCounter('customers'));
As you can see, the key to prevent multiple execution is the property called eventId in the context object. If the function has been handled many times for the same event, the event id will be the same in all cases. Unfortunately, you must have "events" collection in your database.
Please check below answer I found on another thread. Your count should be atomic. Its required to use FieldValue.increment() function in such case.
https://stackoverflow.com/a/49407570/3337028
firebase-admin offers select(fields) which allows you to only fetch specific fields for documents within your collection. Using select is more performant than fetching all fields. However, it is only available for firebase-admin and firebase-admin is typically only used server side.
select can be used as follows:
select('age', 'name') // fetch the age and name fields
select() // select no fields, which is perfect if you just want a count
select is available for Node.js servers but I am not sure about other languages:
https://googleapis.dev/nodejs/firestore/latest/Query.html#select
https://googleapis.dev/nodejs/firestore/latest/CollectionReference.html#select
Here's a server side cloud function written in Node.js which uses select to count a filtered collection and to get the IDs of all resulting documents. Its written in TS but easily converted to JS.
import admin from 'firebase-admin'
// https://stackoverflow.com/questions/46554091/cloud-firestore-collection-count
// we need to use admin SDK here as select() is only available for admin
export const videoIds = async (req: any): Promise<any> => {
const id: string = req.query.id || null
const group: string = req.query.group || null
let processed: boolean = null
if (req.query.processed === 'true') processed = true
if (req.query.processed === 'false') processed = false
let q: admin.firestore.Query<admin.firestore.DocumentData> = admin.firestore().collection('videos')
if (group != null) q = q.where('group', '==', group)
if (processed != null) q = q.where('flowPlayerProcessed', '==', processed)
// select restricts returned fields such as ... select('id', 'name')
const query: admin.firestore.QuerySnapshot<admin.firestore.DocumentData> = await q.orderBy('timeCreated').select().get()
const ids: string[] = query.docs.map((doc: admin.firestore.QueryDocumentSnapshot<admin.firestore.DocumentData>) => doc.id) // ({ id: doc.id, ...doc.data() })
return {
id,
group,
processed,
idx: id == null ? null : ids.indexOf(id),
count: ids.length,
ids
}
}
The cloud function HTTP request completes within 1 second for a collection of 500 docs where each doc contains a lot of data. Not amazingly performant but much better than not using select. Performance could be improved by introducing client side caching (or even server side caching).
The cloud function entry point looks like this:
exports.videoIds = functions.https.onRequest(async (req, res) => {
const response: any = await videoIds(req)
res.json(response)
})
The HTTP request URL would be:
https://SERVER/videoIds?group=my-group&processed=true
Firebase functions detail where the server is located on deployment.
Following Dan Answer: You can have a separated counter in your database and use Cloud Functions to maintain it. (Write-time best-effort)
// Example of performing an increment when item is added
module.exports.incrementIncomesCounter = collectionRef.onCreate(event => {
const counterRef = event.data.ref.firestore.doc('counters/incomes')
counterRef.get()
.then(documentSnapshot => {
const currentCount = documentSnapshot.exists ? documentSnapshot.data().count : 0
counterRef.set({
count: Number(currentCount) + 1
})
.then(() => {
console.log('counter has increased!')
})
})
})
This code shows you the complete example of how to do it:
https://gist.github.com/saintplay/3f965e0aea933a1129cc2c9a823e74d7
Get a new write batch
WriteBatch batch = db.batch();
Add a New Value to Collection "NYC"
DocumentReference nycRef = db.collection("cities").document();
batch.set(nycRef, new City());
Maintain a Document with Id as Count and initial Value as total=0
During Add Operation perform like below
DocumentReference countRef= db.collection("cities").document("count");
batch.update(countRef, "total", FieldValue.increment(1));
During Delete Operation perform like below
DocumentReference countRef= db.collection("cities").document("count");
batch.update(countRef, "total", FieldValue.increment(-1));
Always get Document count from
DocumentReference nycRef = db.collection("cities").document("count");
I created an NPM package to handle all counters:
First install the module in your functions directory:
npm i adv-firestore-functions
then use it like so:
import { eventExists, colCounter } from 'adv-firestore-functions';
functions.firestore
.document('posts/{docId}')
.onWrite(async (change: any, context: any) => {
// don't run if repeated function
if (await eventExists(context)) {
return null;
}
await colCounter(change, context);
}
It handles events, and everything else.
If you want to make it a universal counter for all functions:
import { eventExists, colCounter } from 'adv-firestore-functions';
functions.firestore
.document('{colId}/{docId}')
.onWrite(async (change: any, context: any) => {
const colId = context.params.colId;
// don't run if repeated function
if (await eventExists(context) || colId.startsWith('_')) {
return null;
}
await colCounter(change, context);
}
And don't forget your rules:
match /_counters/{document} {
allow read;
allow write: if false;
}
And of course access it this way:
const collectionPath = 'path/to/collection';
const colSnap = await db.doc('_counters/' + collectionPath).get();
const count = colSnap.get('count');
Read more: https://code.build/p/9DicAmrnRoK4uk62Hw1bEV/firestore-counters
GitHub: https://github.com/jdgamble555/adv-firestore-functions
Use Transaction to update the count inside the success listener of your database write.
FirebaseFirestore.getInstance().runTransaction(new Transaction.Function<Long>() {
#Nullable
#Override
public Long apply(#NonNull Transaction transaction) throws FirebaseFirestoreException {
DocumentSnapshot snapshot = transaction
.get(pRefs.postRef(forumHelper.getPost_id()));
long newCount;
if (b) {
newCount = snapshot.getLong(kMap.like_count) + 1;
} else {
newCount = snapshot.getLong(kMap.like_count) - 1;
}
transaction.update(pRefs.postRef(forumHelper.getPost_id()),
kMap.like_count, newCount);
return newCount;
}
});