How to get all subcollection documents with subcollection name as a date? [duplicate] - reactjs

Say I have this minimal database stored in Cloud Firestore. How could I retrieve the names of subCollection1 and subCollection2?
rootCollection {
aDocument: {
someField: { value: 1 },
anotherField: { value: 2 }
subCollection1: ...,
subCollection2: ...,
}
}
I would expect to be able to just read the ids off of aDocument, but only the fields show up when I get() the document.
rootRef.doc('aDocument').get()
.then(doc =>
// only logs [ "someField", "anotherField" ], no collections
console.log( Object.keys(doc.data()) )
)

It is not currently supported to get a list of (sub)collections from Firestore in the client SDKs (Web, iOS, Android).
In server-side SDKs this functionality does exist. For example, in Node.js you'll be after the ListCollectionIds method:
var firestore = require('firestore.v1beta1');
var client = firestore.v1beta1({
// optional auth parameters.
});
// Iterate over all elements.
var formattedParent = client.anyPathPath("[PROJECT]", "[DATABASE]", "[DOCUMENT]", "[ANY_PATH]");
client.listCollectionIds({parent: formattedParent}).then(function(responses) {
var resources = responses[0];
for (var i = 0; i < resources.length; ++i) {
// doThingsWith(resources[i])
}
})
.catch(function(err) {
console.error(err);
});

It seems like they have added a method called getCollections() to Node.js:
firestore.doc(`/myCollection/myDocument`).getCollections().then(collections => {
for (let collection of collections) {
console.log(`Found collection with id: ${collection.id}`);
}
});
This example prints out all subcollections of the document at /myCollection/myDocument

Isn't this detailed in the documentation?
/**
* Delete a collection, in batches of batchSize. Note that this does
* not recursively delete subcollections of documents in the collection
*/
function deleteCollection(db, collectionRef, batchSize) {
var query = collectionRef.orderBy('__name__').limit(batchSize);
return new Promise(function(resolve, reject) {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
}
function deleteQueryBatch(db, query, batchSize, resolve, reject) {
query.get()
.then((snapshot) => {
// When there are no documents left, we are done
if (snapshot.size == 0) {
return 0;
}
// Delete documents in a batch
var batch = db.batch();
snapshot.docs.forEach(function(doc) {
batch.delete(doc.ref);
});
return batch.commit().then(function() {
return snapshot.size;
});
}).then(function(numDeleted) {
if (numDeleted <= batchSize) {
resolve();
return;
}
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(function() {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
})
.catch(reject);
}

This answer is in the docs
Sadly the docs aren't clear what you import.
Based on the docs, my code ended up looking like this:
import admin, { firestore } from 'firebase-admin'
let collections: string[] = null
const adminRef: firestore.DocumentReference<any> = admin.firestore().doc(path)
const collectionRefs: firestore.CollectionReference[] = await adminRef.listCollections()
collections = collectionRefs.map((collectionRef: firestore.CollectionReference) => collectionRef.id)
This is of course Node.js server side code. As per the docs, this cannot be done on the client.

Related

Retrieve collection + subcollection give me an empty render

The problem
At web app startup, the main task is to retrieve all user informations that are stored in various Firestore's collections and sub-collections.
The problem is that, even if I uses promises and a Loading State to prevent an empty app rendering, the app is rendered with all collections data, except for collections that have sub-collections.
The process
- If user is logged
-- Set Loading Status Active
-- Load Collection A
-- Load Collection B and forEach, load all sub-collection
-- Load Collection C
-- Set Loading Status Inactive
At this point, the app is rendered, but only with Collection A and Collection C. The collection B is loaded (I can see it by Redux Logs) but can't be seen in the app.
Those datas appears only if I change the component status (open/close a menu, for example).
Some code
Here's how I retrieve a collection with sub-collections:
export function setCompanyJobs(user) {
return {
type: "SET_COMPANY_JOBS",
payload: loadCompanyJobs(user),
};
}
Function that retrieve the main collection
export function loadCompanyJobs(user) {
return new Promise((resolve, reject) => {
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
loadJobLinkedServices(user, job).then((jobLinkedServices) => {
companyJobs.push({
id: job.id,
...
});
});
});
resolve(companyJobs);
}).catch(function (error) {
...
});
});
}
Function that retrieve all collection's sub-collections
export function loadJobLinkedServices(user, job){
return new Promise((resolve, reject) => {
let jobLinkedServices = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").doc(job.id).collection("linkedServices").get().then((linkedServices) => {
linkedServices.forEach((linkedService) => {
jobLinkedServices.push({
id: linkedService.id,
...
});
});
resolve(jobLinkedServices)
}).catch(function (error) {
...
});
})
When you do
return new Promise((resolve, reject) => {
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
loadJobLinkedServices(user, job).then((jobLinkedServices) => {
companyJobs.push({
id: job.id,
...
});
});
});
resolve(companyJobs);
}).catch(function (error) {
...
});
});
nothing ensure that your Promise resolves only after ALL the queries triggered in the jobs.forEach() loop are done (i.e. the promises returned by the calls to the loadJobLinkedServices function have resolved).
I don't know reactjs but I think you can use the JavaScript Promise.all() method along the following lines:
return new Promise((resolve, reject) => {
let promises = [];
let companyJobs = [];
db.collection("company").doc(user.selectedCompany).collection("jobs").get().then((jobs) => {
jobs.forEach((job) => {
promises.push(loadJobLinkedServices(user, job));
});
Promise.all(promises).
then(results => {
//Loop over the results array to populate the companyJobs array
resolve(companyJobs);
})
}).catch(function (error) {
...
});
});
Also, don't forget to correctly chain your calls to the different asynchronous functions, i.e. something like:
query Collection A
THEN query Collection B
THEN query all sub-collections (with Promise.all())
THEN query Collection C
THEN set Loading Status Inactive
Finally, a last remark: Note that the get() method returns a promise, so I am not sure that you need to wrap the calls to the get() method into some new Promises (Again, I am not versed in reactjs, so this remark may be wrong).
In other words, I think you could do something like the following (for example for the loadJobLinkedServices function):
export function loadJobLinkedServices(user, job){
let jobLinkedServices = [];
return db.collection("company").doc(user.selectedCompany).collection("jobs").doc(job.id).collection("linkedServices").get()
.then((linkedServices) => {
linkedServices.forEach((linkedService) => {
jobLinkedServices.push({
id: linkedService.id,
...
});
});
return jobLinkedServices;
}).catch(function (error) {
...
});
})

Pagination in Watson Assistant api

I am trying to paginate the result from the watson conversation api. But I am unable to do that.
I have tried creating a function where I pass the url of the next_page but it doesn't work.
function callWatson(url) {
let watson_url = 'https://gateway.watsonplatform.net/assistant/api';
if (!url) {
watson_url = `${watson_url}/v1/workspaces/[workspace_id]/logs?version=2018-09-20`;
} else {
watson_url = `${watson_url}/${url}`;
}
assistant = new watson.AssistantV1({
username: ************,
password: ************,
url: watson_url,
rejectUnauthorized: false,
version: '2018-09-20'
});
return assistant;
}
let input = [];
let pagination = [];
assistant.listAllLogs(param, function(err, response) {
if (err) {
console.error(err);
} else {
for (var i = 0; i < response.logs.length; i++) {
pagination.push(response.pagination);
input.push(response.logs[i]);
}
// logger.debug(input);
logger.debug(pagination);
}
});
Based on the docs you need to use "next_url" - The URL that will return the next page of results, if any.
https://cloud.ibm.com/apidocs/assistant#pagination
You need to pass the next_cursor field from the pagination object in the current response as the value for the cursor parameter in the subsequent request.

How to for loop all documents in a collection - Azure CosmosDB - Nodejs

I have looked around at a few answers/questions regarding this issue but yet to find a solution.
I have a collection with documents (simplified) as such:
{
"id": 123
"stuff": "abc"
"array":[
{
"id2":456
"properties": [
{
"id3": 789
"important": true
}
]
}
]
}
I want to check for each document in my collection, for each array object within array, for each properties, if it has important: true for example. Then return:
"id": 123
"id2": 456
"id3": 789
I have tried using:
client.queryDocuments(self.collection._self, querySpec).toArray(function(err, results) {
if (err) {
callback(err);
} else {
callback(null, results[0]);
}
});
But the issue is an array has a maximum character limit. If my collection has millions of documents, this would presumably be exceeded. (Javascript Increase max array size)
Or, am I misunderstanding the above question? Is it talking about the number of objects in an array (of which, each can have unlimited object character length?)
Thus I am looking a for loop-esque solution, where each document is returned, I do my analysis, then move to then next/do them in parallel.
Any insight would be greatly appreciated.
But the issue is an array has a maximum character limit. If my
collection has millions of documents, this would presumably be
exceeded. (Javascript Increase max array size)
Based on my research,the longest possible array in js could have 232-1 = 4,294,967,295 = 4.29 billion elements. However, it is perfectly enough to meet your millions data volume requirements. In addition,you can't query such huge volume data directly surely,that's impossible you do that.
Whether about throughput constraints(RUs settings) or query efficiency factors, you should consider batching large volumes of data anyway.
Thus I am looking a for loop-esque solution, where each document is
returned, I do my analysis, then move to then next/do them in
parallel.
Maybe you could use v2 js sdk for cosmos db sql api.Please refer to the sample code:
const cosmos = require('#azure/cosmos');
const CosmosClient = cosmos.CosmosClient;
const endpoint = "https://***.documents.azure.com:443/"; // Add your endpoint
const masterKey = "***"; // Add the masterkey of the endpoint
const client = new CosmosClient({ endpoint, auth: { masterKey } });
const databaseId = "db";
const containerId = "coll";
async function run() {
const { container, database } = await init();
const querySpec = {
query: "SELECT r.id,r._ts FROM root r"
};
const queryOptions = {
maxItemCount : -1
}
const queryIterator = await container.items.query(querySpec,queryOptions);
while (queryIterator.hasMoreResults()) {
const { result: results, headers } = await queryIterator.executeNext();
console.log(results)
console.log(headers)
//do what you want to do
if (results === undefined) {
// no more results
break;
}
}
}
async function init() {
const { database } = await client.databases.createIfNotExists({ id: databaseId });
const { container } = await database.containers.createIfNotExists({ id: containerId });
return { database, container };
}
run().catch(err => {
console.error(err);
});
More details about continuation token ,please refer to my previous case.Any concern,please let me know.
I am using Cosmos DB SQL API Node.js library. I am unable to find the Continuation Token from this library so that I can return it to client. The idea is to get it back from the client for the next pagination request.
I have a working code which iterates multiple times to get all the documents. What changes will be required here to get the continuation token?
function queryCollectionPaging() {
return new Promise((resolve, reject) => {
function executeNextWithRetry(iterator, callback) {
iterator.executeNext(function (err, results, responseHeaders) {
if (err) {
return callback(err, null);
}
else {
documents = documents.concat(results);
if (iterator.hasMoreResults()) {
executeNextWithRetry(iterator, callback);
}
else {
callback();
}
}
});
}
let options = {
maxItemCount: 1,
enableCrossPartitionQuery: true
};
let documents = []
let iterator = client.queryDocuments( collectionUrl, 'SELECT r.partitionkey, r.documentid, r._ts FROM root r WHERE r.partitionkey in ("user1", "user2") ORDER BY r._ts', options);
executeNextWithRetry(iterator, function (err, result) {
if (err) {
reject(err)
}
else {
console.log(documents);
resolve(documents)
}
});
});
};

mongoose update array and add new element

I pretty sure this question was asked many items, however I cannot find an answer to my particular problem, which seems very but I just can't seem to get it working. I have a user model with cart schema array embedded in it. I am trying to add an object to an array and if it exists only update quantity and price, if it is doesn't add to an array. what happens with my code is that it adds a new item when array is empty, it updates the item's quantity and price but it doesn't want to add a new item. I read a bit a bout and as far as I understood I cannot use two different db methods in one request. I would appreciate any help on this, this is the first time I am actually using mongooose.
const CartItem = require('../models/cartModel');
const User = require('../models/userModel');
exports.addToCart = (req, res) => {
const cartItem = new CartItem.model(req.body);
const user = new User.model();
User.model
.findById(req.params.id)
.exec((err, docs) => {
if (err) res.sendStatus(404);
let cart = docs.cart;
if (cart.length == 0) {
docs.cart.push(cartItem);
}
let cart = docs.cart;
let isInCart = cart.filter((item) => {
console.log(item._id, req.body._id);
if (item._id == req.body._id) {
item.quantity += req.body.quantity;
item.price += req.body.price;
return true;
}
});
if (isInCart) {
console.log(cart.length)
} else {
cart.push(cartItem);
console.log(false);
}
docs.save(function (err, docs) {
if (err) return (err);
res.json(docs);
});
});
};
I actually managed to get it working like this
exports.addToCart = (req, res) => {
const cartItem = new Cart.model(req.body);
const user = new User.model();
User.model
.findById(req.params.id)
.exec((err, docs) => {
if (err) res.sendStatus(404);
let cart = docs.cart;
let isInCart = cart.some((item) => {
console.log(item._id, req.body._id);
if (item._id == req.body._id) {
item.quantity += req.body.quantity;
item.price += req.body.price;
return true;
}
});
if (!isInCart) {
console.log(cart.length)
cart.push(cartItem);
}
if (cart.length == 0) {
cart.push(cartItem);
}
docs.save(function (err, docs) {
if (err) return (err);
res.json(docs);
});
});
};
don't know if this is the right way to do it, but I can both add a new product into my array and update values of existing ones
Maybe your problem hava a simpler solution: check this page at the mongoose documentation: there is a compatibility issue with some versions of mongoDB and mongoose, maybe you need to edit your model code to look like this:
const mongoose = require("mongoose");
const CartSchema = new mongoose.Schema({
//your code here
}, { usePushEach: true });
module.exports = mongoose.model("Cart", CartSchema);
You can find more information here: https://github.com/Automattic/mongoose/issues/5924
Hope it helps.

CloudantDB & NodeJS: Query data with specific id

I just created a NodeJS cloudantDB web starter on bluemix. Then, I have a API get data from cloudantDB and get successfull but it returns all data. Please see js file:
js file:
app.get('/api/provider', function(request, response) {
console.log("Get method invoked.. ")
db = cloudant.use(dbCredentials.dbProvider);
var docList = [];
var i = 0;
db.list(function(err, body) {
if (!err) {
var len = body.rows.length;
console.log('total # of docs -> '+len);
if(len == 0) {
// error
} else {
body.rows.forEach(function(document) {
db.get(document.id, { revs_info: true }, function(err, doc) {
if (!err) {
if(doc['_attachments']) {
// todo
} else {
var responseData = createResponseDataProvider(
doc._id,
doc.provider_type,
doc.name,
doc.phone,
doc.mobile,
doc.email,
doc.logo,
doc.address
);
}
docList.push(responseData);
i++;
if(i >= len) {
response.write(JSON.stringify(docList));
console.log('ending response...');
response.end();
}
} else {
console.log(err);
}
});
});
}
} else {
console.log(err);
}
});
If I want to add parameter to API to get specific data from DB , Do we need create search index or query on cloudant, afer that call API the same : app.get('/api/provider/:id'). Please help me review and sharing. Thanks
you could get the document by id/name:
db.get(docID, function(err, data) {
// do something
});
references:
https://github.com/apache/couchdb-nano#document-functions
https://github.com/cloudant/nodejs-cloudant#api-reference
You can use a search function of Cloudant.
You need to create search index. In search index you can manage what data you want to get.
Example: https://cloudant.com/for-developers/search/
Following this code after create search index.
...
var query = {q: "id:doc.id"};
db.search('design document name', 'index name', query, function(er, result) {
if (er) {
throw er;
}
console.log(result);
});

Resources