When using Cloud Functions with the Firebase Realtime Database you are able to target a specific field with a cloud function. For example, given this JSON structure, I could target when user1's email field changed with a ('/user/{userId}/email').onUpdate cloud function.
{
"user": {
"user1": {
"name": "John Doe",
"phone": "555-5555",
"email": "john#gmail.com"
}
}
}
Now with Firestore it seems that I can't target a specific field and can only target documents. If this is the case a Firestore cloud function to target the email field would have to look like this, ('/user/{userId}').onUpdate, and fire every time any document in the user collection changed. This would result in a lot of wasted cloud functions firing. Is this how Firestore works and is there an elegant work around to it?
You are correct, due to the different data models Cloud Firestore only allows you to trigger Cloud Functions on document level events rather than field level.
One method is to store email in a separate document (e.g. in a subcollection called Email), so updating email is tye only change that will fire. This does require you reading an extra document each time you need the email though.
Another similar method is to still have it in the same document, but also write it into the subcollection as a second write to trigger the function. Use email as the doc I'd and have a timestamp field in the document to make it easy to clean up the old document ( select oldest email doc to delete, maybe even in the function)
Related
For a RESTful API, consider a model schema as follows:
MyCoolObject {
field_a
field_b
field_c
}
Is it better to create one update endpoint to update one or many fields on the model (PUT)? Or create one endpoint per field that would only update that one field (PATCH)?
Heuristic: how do you GET the information from your API?
Typically if you get the information a single resource with all of the information included in its representation...
GET /my-cool-object
Then you should also edit that information using the same resource
PUT /my-cool-object
PATCH /my-cool-object
POST /my-cool-object
In cases where you get the information from multiple resources (presumably via links)
GET /my-cool-object
GET /my-cool-object/a
GET /my-cool-object/b
GET /my-cool-object/c
Then you would normally edit the information in its own resource
PUT /my-cool-object/a
PATCH /my-cool-object/a
POST /my-cool-object/a
I have a large collection where each document is of a substantial size and therefore cannot be embedded into a single document. For my feature, I need to sort the collection using orderBy and filter the collection using "array-contains" and "==" where the sort and filter parameters are provided by the user. I was wondering if there was an efficient way to do this by cacheing documents that had already been fetched in previous queries. That being said, does firebase do any cacheing itself and does it already optimize what I'm trying to do in this case, or is there any custom cacheing/optimization I can do?
This is what my implementation looks like right now. This works fine for now however it's not very scalable as it creates a new realtime listener each time any of the filter/sort state changes. Is there any way can I improve this to minimize the total number of documents read?
useEffect(() => {
if (!sort || !status || !search) return;
const unsubscribe = firebase.collection("users")
.where("searchTerms", "array-contains", search)
.where("status", "==", status)
.orderBy(sort)
.onSnapshot((snapshot) => {
// update state hook with snapshot data...
});
return () => unsubscribe();
}, [sort, status, search]);
Thank you for any and all help!
Typically, you will want to use a search indexer to enable functionality like this.
The Firebase documentation recommends using Algolia for full-text search. You want to create a cloud function that indexes the data you want to search on, along with the Firestore document ID. On the frontend, you can use the Algolia API to get search results and then fetch the whole document From Firestore when you need to display it.
An alternative to pagination for "big data" and still supporting complex queries can be done by baking the data for the client into a simplified search collection.
This displays only key information that represents a source document by combining the essential data into a dedicated collection of all the results.
Each document can hold up to 1MB of data each and that can equate to roughly 10k-200k entries based on your data size. It does take some time to set up but it has been effective for handling long-lived data within firebase without additional 3rd party solutions.
The key takeaways are as follows:
This is ideal for data that doesn't update too frequently, multiple changes at once can hit the 1 second limit per document.
All search documents contain two properties, a counter to maintain the current entries and an array of strings that represent your essential data.
Each source document needs to maintain a document ID of its entry document for future updates
On update, you find the search index ID, and use arrayUnion and arrayRemove methods, preferably with a transaction and update the source document.
Optionally, you can use the new Bundle Method to bundle this collection with your app
Resources:
https://firebase.google.com/docs/functions/firestore-events#event_triggers
https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
https://firebase.googleblog.com/2021/04/firestore-supports-data-bundles.html
This question already has answers here:
How can I write a firebase rule to allow reading only part of a collection/document? [duplicate]
(1 answer)
How to allow only particular fields of a firestore document to be accessed publicly
(2 answers)
Firestore security rules for public and private fields
(2 answers)
Closed 2 years ago.
I want to get a Document from Firestore, but just with some fields of the doc.
I found 2 use cases for this. One of these is:
Users in Firestore
UsersCollection
UserDocument
email
uid
photoUrl
name
If I want to make a chat with all of the users in the users collection, I need to get all the docs in the user collection. But these docs contain some important info that's not useful to our chat, like the e-mail. Can't I just request from firebase the user document without the e-mail field? Can't a hacker find the user document in memory and get their e-mail? The e-mail is not useful for the chat, I only need the name, uid and photoUrl, the e-mail is no use for this feature, but I still get it with the whole document.
How do I "except" or get the document without the e-mail? This is more about security and user privacy.
Another use case for this would be a like system. Let's say we have a post structured like this in firebase:
PostsCollection
PostDocument
number of likes
arrayOfUsersThatLiked
photoUrl
number of likes is incremented by a cloud function that when arrayOfUsersThatLiked is modified, the function adds the length of the array in the number of likes field.
(arrayOfUsersThatLiked is used to not let the user like more than 1 time, so if someone liked the post, arrayOfUsersThatLiked will keep track of who and will not let them like again)
When we get the post document from the db, we will actually get that usersThatLiked array with a lot of uids along with the things that we need, we only need the photoUrl and the number of likes on the post, not the usersThatLiked array. (it would be a pain also to store such a huge string array).
So the main question remains, how do I only get the document with only the fields that I want to? It's a matter of privacy/security and performance.
With the Client SDKs (including the FlutterFire plugin) it is not possible to get only a subset of the fields for a Document. When you fetch a Document you get it with all its fields.
Here are three possible approaches:
Dernomalize your data: You create another collection which contains documents that only have the fields you want to display in the front end. The complexity is that you need to keep the two collections in sync: when you create/modify/delete a doc in the master collection, you need to update the other collection. This can be done from your front-end (e.g. you write to the two collections in a batched write) or with a Cloud Function, triggered in the back-end with an onWrite trigger. In terms of security, you open read access only to the second collection, not to the master one.
Use the Firestore REST API: With the REST API you can use a DocumentMask when fetching documents, which will "restrict a get operation on a document to a subset of its fields. ". Note, however, that a "hacker" could get the collection ID and fetch the documents without the Mask.
Use a Cloud Function to access the Firestore documents. With a Cloud Function, you have full control on what you want to send back to the front-end, since you query the collection from within the Cloud Function and build the response in the Cloud Function. However, using a Cloud Function instead of a Client SDK has several drawbacks: see this article for more details.
I am sending requests to the the Firebase Realtime API. When I try to delete an from item within from a specific collection (e.g. journals) and specify which journal title using axios.delete("https://topxxx-xxx.firebaseio.com/journals.json", {title: "Nature"}}) all of the journals in the collection are deleted, not just "Nature". Similarly, if I do a get request and specify a title, all journals are returned.
I have also tried using "https://topxxx-xxx.firebaseio.com/journals.json", {params: {title: "Nature"}}. but that also returns all journals. Here are an example delete request:
axios.delete("https://topxxx-xxx.firebaseio.com/journals.json", {title: this.state.title})
.catch(error=> console.log("Error" + error))
The Firebase Realtime Database API doesn't support conditional deletes. To delete a node, you must specify the entire part to that node, and (in the REST API) either use the DELETE verb or use the X-HTTP-Method-Override: DELETE header.
So the two step process:
Perform a query to find the nodes with the title you're looking for:
https://topxxx-xxx.firebaseio.com/journals.json?orderBy="title"&equalTo="the title"
Delete the matching nodes by sending a DELETE to URLs like:
https://topxxx-xxx.firebaseio.com/journals/journalId1.json
You cannot delete a specify item from a json file like that in firebase, and also using GET will return the entire collection. It is working amazingly correct from firebase side. If you want something like how you are expecting, go for a small flask server or checkout cloud functions in firebase
You probably want to send that request to a back-end services that's connected to firebase (i'm assuming since you're using axios). Then call the remove method as demonstrated here
Intro
I have a FireStore database similar to a social media db, with 3 collections Users, Events, and EventUpdates. My goal is to create a feed with eventUpdates created by me and my friends. So i have to expand my database with friendship connections. But i struggle with 3 problems, and hopefully somebody here can push me in the right direction to solve these.
Problem/Question 1:
I added username and user image to the EventUpdate model so it's easier to query. I've heard denormalise is the way to go in a NoSQL database. But if a user updates his user image, i've to update all eventUpdates created by that user. Sounds like something you don't wanne do. But is there a better way to do this?
Problem/Question 2:
How can i create a data structure that is optimised for performing the following query: get eventUpdates from me and my friends ordered by date.
Problem/Question 3:
How to store likes? I can keep a counter in a eventUpdate. But this becomes a problem when i denormalise eventUpdates (see current solution underneath EDIT)..
Data structure example .
{
"users": {
"1": { "name": "Jack", "imageUrl": "http://lorempixel.nl" }
},
"events": {
"A": {
"name": "BeerFestival",
"date": "2018/09/05",
"creatorId": "1"
}
},
"eventUpdates": {
"1": {
"timestamp": "13243543",
"creatorId: "1",
"creatorName": "Jack",
"creatorImageUrl": "http://lorempixel.nl",
"eventId": "A",
"message": "Lorem ipsum"
}
}
}
EDIT
OK, after some trial and error i ended up with the following structure. This structure seems work, but my problem with this solution is that i need to make a lot of write calls to update a single eventUpdate because of all the copies in each feed (1000 followers means 1000 copies). And it looks like i need to do that a lot.
I would like for example to add a like button to each event update. This trigger an update on all EventUpdate copies. For me it looks like firebase is not suited for my project and i'm thinking of replacing it with a SQL DB, or can anyone here change my mind with a better solution?
{
"users": {
"user1": { "name": "Jack",
"imageUrl": "http://lorempixel.nl",
"followers": ["user1"]
}
},
"feeds": {
"user1": {
"eventUpdates": {
"1": {
"timestamp": "13243543",
"creatorId: "1",
"eventId": "A",
"message": "Lorem ipsum"
}
},
"following": {
"user1": {
"name": "Jack",
"imageUrl": "http://lorempixel.nl",
"followers": ["user1"]
}
}
},
"events": {
"A": {
"name": "BeerFestival",
"date": "2018/09/05",
"creatorId": "1"
}
}
}
I added username and user image to the EventUpdate model so it's easier to query. I've heard denormalise is the way to go in a NoSQL database.
That's right, denormalization and is a common practice when it comes to Firebase. If you are new to NoQSL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase realtime database but same rules apply to Cloud Firestore.
But if a user updates his user image, i've to update all eventUpdates created by that user. Sounds like something you don't wanne do. But is there a better way to do this?
Yes, that's also correct. You need to update all the places where that image exists. Because you have chosen google-cloud-firestore as a tag, I recommend you see my answer from this post because in case of many write operations, Firestore might be a little costly. Please also see Firestore pricing plans.
Regarding Firestore, instead of holding an entire object you can only hold a reference to a picture. In this case, there is nothing that you need to update. It's always a trade between these two techniques and unfortunately there is no way between. You either hold objects or only references to objects. For that, please see my answer from this post.
How can i create a data structure that is optimised for performing the following query: get eventUpdates from me and my friends ordered by date.
As I see, your schema is more a Firebase realtime database schema more than a Cloud Firestore. And to answer to your question, yes you can create. So talking about Firestore, you can create a collection named eventUpdates that can hold eventUpdate objects and to query it according to a timestamp, a query like this is needed:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference eventUpdatesRef = rootRef.collection("eventUpdates");
Query query = eventUpdatesRef.orderBy("timestamp", Query.Direction.ASCENDING);
But please note that the timestamp field should be of type Date and not long. Please also take a look at my answer from this post, to see how you can add a date property in a Cloud Firestore database.
How to store likes? I can keep a counter in a eventUpdate. But this becomes a problem when i denormalise eventUpdates (see current solution underneath EDIT)
You can simply add likes but I recommend you see the last part of my answer from this post. So you might consider adding that count in a Firebase realtime database rather than in Cloud Firestore. Both databases work very well together.
This structure seems work, but my problem with this solution is that i need to make a lot of write calls to update a single eventUpdate because of all the copies in each feed (1000 followers means 1000 copies). And it looks like i need to do that a lot.
You might also take a look at my answer from this post.
For me it looks like firebase is not suited for my project and i'm thinking of replacing it with a SQL DB, or can anyone here change my mind with a better solution?
I don't think this way. There are many apps out there that have the exact mechanism as yours and are working very well.
If you want your feed items to be in sync with the real users data (new profile image when the user changes it for example) you can simply store the user ID in the eventUpdate document. This way you don't have to keep them in sync manually, and every time you have to display the item in the feed you could simply fetch user data, and easily query many eventUpdates on userId and created_at fields ( assuming you have them ).
To implement likes in your feed the solution depends on a bunch of things like traffic volume.
The simplest way is to update a likes field with a transaction, but Firestore has a maximum updates frequency on a single document of 1 second. Plus, a transaction can easily fail if more than 5 transactions are trying to update the same document.
To implement a more solid likes system take a look at this page from the official Firebase docs.
Firestore has a different approach to the NoSQL world. Once you know the data you will use (as You already do) there are some very important points about what architecture the data will have. And It depends a lot about how the data grows, what kind of queries you will need and how often you will use them. Some cases You can create a root collection that aggregates data and queries might be easier.
There is a great video from Firebase Channel that might help. Check it out!
How to Structure Your Data | Get to Know Cloud Firestore #5
https://www.youtube.com/watch?v=haMOUb3KVSo
[UPDATED] December 26th
Others videos that might help to model and query your data is these videos:
How to Connect Firebase Users to their Data - 3 Methods
https://www.youtube.com/watch?v=jm66TSlVtcc
How to NOT get a 30K Firebase Bill
https://www.youtube.com/watch?v=Lb-Pnytoi-8
Model Relational Data in Firestore NoSQL
https://www.youtube.com/watch?v=jm66TSlVtcc