I am making a mobile app using flutter with firebase as my backend.
I have a collection of user document that stores user information. one of the fields is an array of references (reference documents in another collection) which I want to use in an operation like batch that in that would then allow be to read all the documents.
I know batch only allows writes to the database, My second option would be Transaction, which requires writes after reads which I am trying to avoid.
Is there a way to read multiple documents in one operation without having to use Transaction?
Firestore doesn't offer a formal batch read API. As Frank mentions in his comment, there is a way to use IN to fetch multiple documents from a single collection using their IDs. However, all of the documents must be in the same collection, and you can't exceed 10 documents per query. You might as well just get() for each document individually, as the IN query has limitations, and isn't guaranteed to execute any faster than the individual gets. Neither solution is guaranteed to be "consistent", so any one of the documents fetched could be "more fresh" than the others at any given moment in time.
If you know the document IDs and the collection paths of the documents needed to be fetched, you could always use the getAll() method which is exposed in the firebase Admin SDK (at least for Node.js environments).
Then, for example, you could write an HTTPS Callable Function that would accept a list of absolute document paths and perform a "batch get" operation on them using the getAll() method.
e.g.
// Import firebase functionality
const functions = require('firebase-functions');
const admin = require('firebase-admin');
// Configure firebase app
admin.initializeApp(functions.config().firebase);
// HTTPS callable function
exports.getDocs = functions.https.onCall((data, context) => {
const docPathList = data.list; // e.g. ["users/Jkd94kdmdks", "users/8nkdjsld", etc...]
const firestore = admin.firestore();
var docList = [];
for (var i = 0; i <= docPathList.length - 1; i++) {
const docPath = docPathList[i];
const doc = firestore.doc(docPath);
docList.push(doc);
}
// Get all
return firestore.getAll(...docList)
.then(results => {
return { data : results.map(doc => doc.data()) };
})
.catch(err => {
return { error : err };
})
});
Not sure what the limit (if any) is for the number of documents you can fetch using getAll(), but I do know my application is able to fetch at least 50 documents per call successfully using this method.
Firestore has a REST API that allows you to do batch GETs with document paths that may be what you need.
See https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents/batchGet
Related
I am using express to create my firebase functions, and I understand how to create regular callable functions. I am lost however on the exact way to implement trigger functions for the background (i.e. onCreate, onDelete, onUpdate, onWrite), as well as how Reactjs in the frontend is supposed to receive the data.
The scenario I have is a generic chat system that uses react, firebase functions with express and realtime database. I am generally confused on the process of using triggers for when someone sends a message, to update another user's frontend data.
I have had a hard time finding a tutorial or documentation on the combination of these questions. Any links or a basic programmatic examples of the life cycle would be wonderful.
The parts I do understand is the way to write a trigger function:
exports.makeUppercase = functions.database.ref('/messages/{pushId}/original')
.onWrite((change, context) => {
// Only edit data when it is first created.
if (change.before.exists()) {
return null;
}
// Exit when the data is deleted.
if (!change.after.exists()) {
return null;
}
// Grab the current value of what was written to the Realtime Database.
const original = change.after.val();
console.log('Uppercasing', context.params.pushId, original);
const uppercase = original.toUpperCase();
// You must return a Promise when performing asynchronous tasks inside a Functions such as
// writing to the Firebase Realtime Database.
// Setting an "uppercase" sibling in the Realtime Database returns a Promise.
return change.after.ref.parent.child('uppercase').set(uppercase);
});
But I don't understand how this is being called or how the data from this reaches frontend code.
Background functions cannot return anything to client. They run after a certain event i.e. onWrite() in this case. If you want to update data at /messages/{pushId}/original to other users then you'll have to use Firebase Client SDK to listen to that path:
import { getDatabase, ref, onValue} from "firebase/database";
const db = getDatabase();
const msgRef = ref(db, `/messages/${pushId}/original`);
onValue(msgRef, (snapshot) => {
const data = snapshot.val();
console.log(data)
});
You can also listen to /messages/${pushId} with onChildAdded() to get notified about any new node under that path.
I have a collection called list which I'm trying to order by 3 different fields: important, unimportant and date, so the unimportant come first (they'll be at the bottom of the list, important the last, and all that sorted by timestamp.
If i query to order my docs by any of the above on its own, then it works, trouble starts when I try to put them together as per firestore documentation. So I have the following code:
const q = query(listRef, orderBy("important", "desc"), orderBy("unimportant"), orderBy("date"));
Which gets me Uncaught Error in snapshot listener. This is how i get my data from firestore:
const getData = () => { // get data from firestore to app
onSnapshot(q, (snapshot) => {
firestoreList = [];
firestoreIds = [];
snapshot.docs.forEach((doc) => {
firestoreList.push({ ...doc.data(), id: doc.id });
!firestoreIds.includes(doc.id) && firestoreIds.push(doc.id);
});
if (firestoreList.length === 0) {
setItems(items.concat(newItem));
} else {
setItemIds(firestoreIds);
setItems(firestoreList);
}
});
}
useEffect(() => {
getData();
}, []);
I'm using onSnapshot, because i need the user to be able to add, remove and do other stuff with data and see the outcome reflected immediately for them and for other users who'll be using the app simultaneously.
Ordering/filtering on multiple fields requires that your database contains a so-called composite index on those fields. And unlike single-field indexes, which are created automatically, composite indexes are only created when you explicitly tell the database to do so.
If you log the warning/error message you get, it contains a long direct link to the Firestore console to create the exact index the query needs. The link has all details already filled in, so all you have to do is click the link and then click the button to start creating the index.
Also see:
Firestore order by two fields
How to query one field then order by another one in Firebase cloud Firestore?
the Firestore documentation on ordering a Firestore query on multiple fields.
I'm trying to update 2 firebase collections that will contain an array element that is the same.
For example, I'm building a job app, so when a user creates a job, it pushes that job object into a firebase collection called alljobs under a document called alljobs. In addition, the same job is pushed to a firebase collection called created jobs. Where each user on the app has their individual created jobs, each doc is named the users id.
Is there an easy way to update this specific job in both alljobs collection and the createdjobs collection?
For example, my approach of doing it would be like this.
Individual Job component (obtained by previously mapping through all the jobs)
const [userjobs, setUserjobs] = useState([])
const {job, createdjobs} = props
function updateJob(){
createdjobs?.map(job1=>{
if(job1.jobid===job.jobid){
const jobindex = createdjobs.indexOf(job1)
createdjobs[jobindex].jobtitle = 'New title'
db.collection('createdjobs').doc(user.uid).update({
jobs: createdjobs
})
}
})
}
I'll basically have to repeat this same process to update once again the job that has just been updated in the createdjobs collection. This gets repetitive and messy. So looking for a solution to this. By mapping through alljobs this time.
useEffect(()=>{
db.collection('alljobs').doc('alljobs').onSnapshot(snap=>{
setAlljobs(snap.data().jobs)
})
},[])
There is no shortcut for your problem I think. But I suggest you to write a sync function to Firebase Function.
It will watch changes of one source and sync to others. So that your logic code only needs to manage one source of trust.
As #Thanh Le suggested you can write a Google Cloud Function to fulfill this purpose. In cloud functions there is a function type named triggers. You can use this triggers.
Cloud Function triggers
We can write functions which will automatically trigger when the specfied document or set of documents,
onCreate - Trigger when document creating
onUpdate - Triggered when a document already exists and has any value changed.
onDelete - Trigger when document deleting
onWrite - Triggered when onCreate, onUpdate or onDelete is triggered.
From these triggers you can use onWrite Trigger to to implement the function.
exports.updateAllJobsTrigger = functions.firestore.document('createdJob/{userId}')
onWrite(async (change, context) => {
// Check if document is deleted
if (!change.after.exists) {
logger.log('Document not existing. Function exited.');
return;
}
const allJobsRef = admin.firestore().collection('alljobs').doc('alljobs');
// Check if document created
if (!change.before.exists) {
try {
// This is a newly created document. Therefore newjob should be in first element
const newJob = change.after.data().jobs[0];
const data = (await allJobsRef.get()).data();
if (data) {
const jobs = data.jobs;
await allJobsRef.update({
jobs: [...jobs, newJob]
});
logger.info('Job added to All jobs queue.');
}
} catch (exception) {
logger.error(exception)
}
return;
}
try {
// This is a updating document.newly added job is in the last element of the array
const newJob = change.after.data().jobs[change.after.data().jobs.length - 1];
const data = (await allJobsRef.get()).data();
if (data) {
const jobs = data.jobs;
await allJobsRef.update({
jobs: [...jobs, newJob]
});
logger.info('Job added to All jobs queue.');
}
} catch (exception) {
logger.error(exception)
}
});
As #Nimna Perera said, you can use Cloud Functions to solve this issue. Your CF should be triggered when a document is updated, created or deleted (so the onWrite option). Another way to do this is through transactions, when you need to read and write the documents or batched writes when you only need to write in one or various documents. In both cases you are not limited to a single collection, so it should work for your issue.
Actually am new in react and am trying to create an event app in which a user can join an event
here is code for joining an event
export const JoinEvent = (id) => {
return async dispatch => {
let data = await firebase.firestore().collection('Events').doc(id).get()
let tmpArray = data.data()
let currentUser = firebase.auth().currentUser
let newArray = tmpArray.PeopleAttending
await firebase.firestore().collection('Events').doc(id).update({
PeopleAttending : {...newArray, [currentUser.uid]: {displayName : currentUser.displayName}}
})
}
}
actually i have created an action bascailly in JoinEvent an id is passed of the particular event which is clicked.
here is my firestore structure look like this..
so basically i have to download the whole data and store in local array and then add new user and then finally update
So here am basically download the whole data is there any way to just simply add new Object without downloading whole data??
thankyou
You are doing it wrong. Firestore document size limit is Maximum size for a document 1 MiB (1,048,576 bytes), so sooner or later you're going to reach that limit if you keep adding data like this. It may seems that you're not going to reach that limit, but it's very unsafe to store data that way. You can check Firestore query using an object element as parameter how to query objects in firestore documents, but I suggest you don't do it that way.
The proper way to do it, is to create a subcollection PeopleAttending on each document inside the Events collection and then use that collection to store the data.
Also you can try document set with merge or mergeFields like documented here https://googleapis.dev/nodejs/firestore/latest/DocumentReference.html#set and here https://stackoverflow.com/a/46600599/1889685.
I'm confused as to the appropriate way to access a bunch of images stored in Firebase storage with a react redux firebase web app. In short, I'd love to get a walkthrough of, once a photo has been uploaded to firebase storage, how you'd go about linking it to a firebase db (like what exactly from the snapshot returned you'd store), then access it (if it's not just <img src={data.downloadURL} />), and also how you'd handle (if necessary) updating that link when the photo gets overwritten. If you can answer that, feel free to skip the rest of this...
Two options I came across are either
store the full URL in my firebase DB, or
store something less, like the path within the bucket, then call downloadURL() for every photo... which seems like a lot of unnecessary traffic, no?
My db structure at the moment is like so:
{
<someProjectId>: {
imgs: {
<someAutoGenId>: {
"name":"photo1.jpg",
"url":"https://<bucket, path, etc>token=<token>"
},
...
},
<otherProjectDetails>: "",
...
},
...
}
Going forward with that structure and the first idea listed, I ran into trouble when a photo was overwritten, so I would need to go through the list of images and remove the db record that matches the name (or find it and update its URL). I could do this (at most, there would be two refs with the old token that I would need to replace), but then I saw people doing it via option 2, though not necessarily with my exact situation.
The last thing I did see a few times, were similar questions with generic responses pointing to Cloud Functions, which I will look into right after posting, but I wasn't sure if that was overcomplicating things in my case, so I figured it couldn't hurt too much to ask. I initially saw/read about Cloud Functions and the fact that Firebase's db is "live," but wasn't sure if that played well in a React/Redux environment. Regardless, I'd appreciate any insight, and thank you.
In researching Cloud Functions, I realized that the use of Cloud Functions wasn't an entirely separate option, but rather a way to accomplish the first option I listed above (and probably the second as well). I really tried to make this clear, but I'm pretty confident I failed... so my apologies. Here's my (2-Part) working solution to syncing references in Firebase DB to Firebase Storage urls (in a React Redux Web App, though I think Part One should be applicable regardless):
PART ONE
Follow along here https://firebase.google.com/docs/functions/get-started to get cloud functions enabled.
The part of my database with the info I was storing relating to the images was at /projects/detail/{projectKey}/imgs and had this structure:
{
<autoGenKey1>: {
name: 'image1.jpg',
url: <longURLWithToken>
},
<moreAutoGenKeys>: {
...
}, ...}
My cloud function looked like this:
exports.updateURLToken = functions.database.ref(`/projects/detail/{projectKey}/imgs`)
.onWrite(event => {
const projectKey = event.params.projectKey
const newObjectSet = event.data.val()
const newKeys = Object.keys(newObjectSet)
const oldObjectSet = event.data.previous.val()
const oldKeys = Object.keys(oldObjectSet)
let newObjectKey = null
// If something was removed, none of this is necessary - return
if (oldKeys.length > newKeys.length) {
return null
}
for (let i = 0; i < newKeys.length; ++i) {// Looking for the new object -> will be missing in oldObjectSet
const key = newKeys[i]
if (oldKeys.indexOf(key) === -1) {// Found new object
newObjectKey = key
break
}
}
if (newObjectKey !== null) {// Checking if new object overwrote an existing object (same name)
const newObject = newObjectSet[newObjectKey]
let duplicateKey = null
for (let i = 0; i < oldKeys.length; ++i) {
const oldObject = oldObjectSet[oldKeys[i]]
if (newObject.name === oldObject.name) {// Duplicate found
duplicateKey = oldKeys[i]
break
}
}
if (duplicateKey !== null) {// Remove duplicate
return event.data.ref.child(duplicateKey).remove((error) => error ? 'Error removing duplicate project detail image' : true)
}
}
return null
})
After loading this function, it would run every time anything changed at that location (projects/detail/{projectKey}/imgs). So I uploaded the images, added a new object to my db with the name and url, then this would find the new object that was created, and if it had a duplicate name, that old object with the same name was removed from the db.
PART TWO
So now my database had the correct info, but unless I refreshed the page after every time images were uploaded, adding the new object to my database resulted (locally) in me having all the duplicate refs still, and this is where the realtime database came in to play.
Inside my container, I have:
function mapDispatchToProps (dispatch) {
syncProjectDetailImages(dispatch) // the relavant line -> imported from api.js
return bindActionCreators({
...projectsContentActionCreators,
...themeActionCreators,
...userActionCreators,
}, dispatch)
}
Then my api.js holds that syncProjectDetailImages function:
const SAVING_PROJECT_SUCCESS = 'SAVING_PROJECT_SUCCESS'
export function syncProjectDetailImages (dispatch) {
ref.child(`projects/detail`).on('child_changed', (snapshot) => {
dispatch(projectDetailImagesUpdated(snapshot.key, snapshot.val()))
})
}
function projectDetailImagesUpdated (key, updatedProject) {
return {
type: SAVING_PROJECT_SUCCESS,
group: 'detail',
key,
updatedProject
}
}
And finally, dispatch is figured out in my modules folder (I used the same function I would when saving any part of an updated project with redux - no new code was necessary)