I am trying to paginate the result from the watson conversation api. But I am unable to do that.
I have tried creating a function where I pass the url of the next_page but it doesn't work.
function callWatson(url) {
let watson_url = 'https://gateway.watsonplatform.net/assistant/api';
if (!url) {
watson_url = `${watson_url}/v1/workspaces/[workspace_id]/logs?version=2018-09-20`;
} else {
watson_url = `${watson_url}/${url}`;
}
assistant = new watson.AssistantV1({
username: ************,
password: ************,
url: watson_url,
rejectUnauthorized: false,
version: '2018-09-20'
});
return assistant;
}
let input = [];
let pagination = [];
assistant.listAllLogs(param, function(err, response) {
if (err) {
console.error(err);
} else {
for (var i = 0; i < response.logs.length; i++) {
pagination.push(response.pagination);
input.push(response.logs[i]);
}
// logger.debug(input);
logger.debug(pagination);
}
});
Based on the docs you need to use "next_url" - The URL that will return the next page of results, if any.
https://cloud.ibm.com/apidocs/assistant#pagination
You need to pass the next_cursor field from the pagination object in the current response as the value for the cursor parameter in the subsequent request.
Related
Say I have this minimal database stored in Cloud Firestore. How could I retrieve the names of subCollection1 and subCollection2?
rootCollection {
aDocument: {
someField: { value: 1 },
anotherField: { value: 2 }
subCollection1: ...,
subCollection2: ...,
}
}
I would expect to be able to just read the ids off of aDocument, but only the fields show up when I get() the document.
rootRef.doc('aDocument').get()
.then(doc =>
// only logs [ "someField", "anotherField" ], no collections
console.log( Object.keys(doc.data()) )
)
It is not currently supported to get a list of (sub)collections from Firestore in the client SDKs (Web, iOS, Android).
In server-side SDKs this functionality does exist. For example, in Node.js you'll be after the ListCollectionIds method:
var firestore = require('firestore.v1beta1');
var client = firestore.v1beta1({
// optional auth parameters.
});
// Iterate over all elements.
var formattedParent = client.anyPathPath("[PROJECT]", "[DATABASE]", "[DOCUMENT]", "[ANY_PATH]");
client.listCollectionIds({parent: formattedParent}).then(function(responses) {
var resources = responses[0];
for (var i = 0; i < resources.length; ++i) {
// doThingsWith(resources[i])
}
})
.catch(function(err) {
console.error(err);
});
It seems like they have added a method called getCollections() to Node.js:
firestore.doc(`/myCollection/myDocument`).getCollections().then(collections => {
for (let collection of collections) {
console.log(`Found collection with id: ${collection.id}`);
}
});
This example prints out all subcollections of the document at /myCollection/myDocument
Isn't this detailed in the documentation?
/**
* Delete a collection, in batches of batchSize. Note that this does
* not recursively delete subcollections of documents in the collection
*/
function deleteCollection(db, collectionRef, batchSize) {
var query = collectionRef.orderBy('__name__').limit(batchSize);
return new Promise(function(resolve, reject) {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
}
function deleteQueryBatch(db, query, batchSize, resolve, reject) {
query.get()
.then((snapshot) => {
// When there are no documents left, we are done
if (snapshot.size == 0) {
return 0;
}
// Delete documents in a batch
var batch = db.batch();
snapshot.docs.forEach(function(doc) {
batch.delete(doc.ref);
});
return batch.commit().then(function() {
return snapshot.size;
});
}).then(function(numDeleted) {
if (numDeleted <= batchSize) {
resolve();
return;
}
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(function() {
deleteQueryBatch(db, query, batchSize, resolve, reject);
});
})
.catch(reject);
}
This answer is in the docs
Sadly the docs aren't clear what you import.
Based on the docs, my code ended up looking like this:
import admin, { firestore } from 'firebase-admin'
let collections: string[] = null
const adminRef: firestore.DocumentReference<any> = admin.firestore().doc(path)
const collectionRefs: firestore.CollectionReference[] = await adminRef.listCollections()
collections = collectionRefs.map((collectionRef: firestore.CollectionReference) => collectionRef.id)
This is of course Node.js server side code. As per the docs, this cannot be done on the client.
I'm having trouble figuring out why can't I see a prompt to enter API key. I can connect directly without any authentication. Why is the API key ignored?
auth.js file:
function getAuthType() {
return {
type: 'KEY'
};
}
function validateKey(key) {
var url = 'http://myapi.com/endpoint?api_key=' + key;
var options = {
"method": "post",
"contentType":"application/json"
};
var response = JSON.parse(
UrlFetchApp.fetch(url, options)
);
Logger.log(response.data.length > 0)
return response.data.length > 0;
}
function isAuthValid() {
var userProperties = PropertiesService.getUserProperties();
var key = userProperties.getProperty('dscc.key');
return validateKey(key);
}
function resetAuth() {
var userProperties = PropertiesService.getUserProperties();
userProperties.deleteProperty('dscc.key');
}
function setCredentials(request) {
var key = request.key;
var userProperties = PropertiesService.getUserProperties();
userProperties.setProperty('dscc.key', key);
return {
errorCode: 'NONE'
};
}
function isAdminUser() {
return false;
}
The Logger.log output:
I was using a http url. We've moved our API to https and the problem is solved. Data Studio doesn't show any error messages and skips the auth step. This is very strange.
Edit: A month later, while reviewing the document I noticed that Data Studio is already asking for us an https url.
Each prefix must use https://, not http://. source
I am trying to make a node express app where I fetch data from different url's making a call to node-fetch to pull the body of some pages and other information about certain url endpoints. I want to then render a html table to display this data through an array of information. I am having trouble with the call to render the information as all the functions are asynchronous making it difficult to make sure all the promise calls have been resolved before making my call to render the page. I have been looking into using bluebird and other promise calls of .finally() and .all() but they don't seem to work on my data as it is not an array of promise calls, but an array of objects. Each object was 4 promise calls to fetch data relating to a column of my table all in one row. Is there a function or specific way to render the page after all promises are resolved?
var express = require('express');
var fetch = require('node-fetch');
fetch.Promise = require('bluebird');
var router = express.Router();
const client = require('../platform-support-tools');
function makeArray() {
var registry = client.getDirectory();
var data_arr = [];
for (var i = 0; i < registry.length; i++) {
var firstUp = 0;
for (var j = 0; i < registry[i]; j++) {
if (registry[i][j]['status'] == 'UP') {
firstUp = j;
break;
}
}
var object = registry[i][firstUp];
data_arr.push({
'name': object['app'],
'status': object['status'],
'swagUrl': object['homePageUrl'] + 'swagger-ui.html',
'swag': getSwag(object),
'version': getVersion(object['statusPageUrl']),
'timestamp': getTimestamp(object['statusPageUrl']),
'description': getDescription(object['healthCheckUrl'])
});
}
return data_arr;
}
function getSwag(object_in) {
var homeUrl = object_in['homePageUrl'];
if (homeUrl[homeUrl.length - 1] != '/'){
homeUrl += '/';
}
var datum = fetch(homeUrl + 'swagger-ui.html')
.then(function (res) {
return res.ok;
}).catch(function (err) {
return 'none';
});
return datum;
}
function getVersion(url_in) {
var version = fetch(url_in)
.then(function(res) {
return res.json();
}).then(function(body) {
return body['version'];
}).catch(function (error) {
return 'none';
});
return version;
}
function getTimestamp(url_in) {
var timestamp = fetch(url_in)
.then(function(res) {
return res.json();
}).then(function(body) {
return body['timestamp'];
}).then(function (res) {
return body['version'];
}).catch(function (error) {
return 'none';
});
return timestamp;
}
function getDescription(url_in) {
var des = fetch(url_in)
.then(function(res) {
return res.json();
}).then(function(body) {
return body['description'];
}).catch(function (error) {
return 'none';
});
return des;
}
/* GET home page. */
router.get('/', function (req, res, next) {
var data_arr = makeArray();
Promise.all(data_arr)
.then(function (response) {
//sorting by app name alphabetically
response.sort(function (a, b) {
return (a.name > b.name) ? 1 : ((b.name > a.name) ? -1 : 0);
});
res.render('registry', {title: 'Service Registry', arr: response})
}).catch(function (err) {
console.log('There was an error loading the page: '+err);
});
});
To wait on all those promises, you will have to put them into an array so you can use Promise.all() on them. You can do that like this:
let promises = [];
for (item of data_arr) {
promises.push(item.swag);
promises.push(item.version);
promises.push(item.timestamp);
promises.push(item.description);
}
Promise.all(promises).then(function(results) {
// all promises done here
})
If you want the values from all those promises, back into the object that's a bit more work.
let promises = [];
for (item of data_arr) {
promises.push(item.swag);
promises.push(item.version);
promises.push(item.timestamp);
promises.push(item.description);
}
Promise.all(promises).then(function(results) {
// replace promises with their resolved values
let index = 0;
for (let i = 0; i < results.length; i += 4) {
data_arr[index].swag = results[i];
data_arr[index].version = results[i + 1];
data_arr[index].timestamp = results[i + 2];
data_arr[index].description = results[i + 3];
++index;
});
return data_arr;
}).then(function(data_arr) {
// process results here in the array of objects
});
If you had to do this more often that just this once, you could remove the hard coding of property names and could iterate all the properties, collect the property names that contain promises and automatically process just those.
And, here's a more general version that takes an array of objects where some properties on the objects are promises. This implementation modifies the promise properties on the objects in place (it does not copy the array of the objects).
function promiseAllProps(arrayOfObjects) {
let datum = [];
let promises = [];
arrayOfObjects.forEach(function(obj, index) {
Object.keys(obj).forEach(function(prop) {
let val = obj[prop];
// if it smells like a promise, lets track it
if (val && val.then) {
promises.push(val);
// and keep track of where it came from
datum.push({obj: obj, prop: prop});
}
});
});
return Promise.all(promises).then(function(results) {
// now put all the results back in original arrayOfObjects in place of the promises
// so now instead of promises, the actaul values are there
results.forEach(function(val, index) {
// get the info for this index
let info = datum[index];
// use that info to know which object and which property this value belongs to
info.obj[info.prop] = val;
});
// make resolved value be our original (now modified) array of objects
return arrayOfObjects;
});
}
You would use this like this:
// data_arr is array of objects where some properties are promises
promiseAllProps(data_arr).then(function(r) {
// r is a modified data_arr where all promises in the
// array of objects were replaced with their resolved values
}).catch(function(err) {
// handle error
});
Using the Bluebird promise library, you can make use of both Promise.map() and Promise.props() and the above function would simply be this:
function promiseAllProps(arrayOfObjects) {
return Promise.map(arrayOfObjects, function(obj) {
return Promise.props(obj);
});
}
Promise.props() iterates an object to find all properties that have promises as values and uses Promise.all() to await all those promises and it returns a new object with all the original properties, but the promises replaced by the resolved values. Since we have an array of objects, we use Promise.map() to iterate and await the whole array of those.
I am new in firebase and angularjs and i am having difficulties in getting download url from firebase storage and store them in firebase realtime database.
I was able to upload multiple files to firebase storage. the problem is when i store the download url into firebase realtime database, all database url value are same.It should different based each files downloadURL.
Here my script:
$scope.submitPhotos = function(file){
console.log(file);
var updateAlbum = [];
for (var i = 0; i < file.length; i++) {
var storageRef=firebase.storage().ref(albumtitle).child(file[i].name);
var task=storageRef.put(file[i]);
task.on('state_changed', function progress(snapshot){
var percentage=( snapshot.bytesTransferred / snapshot.totalBytes )*100;
if (percentage==100){
storageRef.getDownloadURL().then(function(url) {
var galleryRef = firebase.database().ref('gallery/'+albumkey);
var postkey = firebase.database().ref('gallery/'+albumkey).push().key;
updateAlbum={img:url};
firebase.database().ref('gallery/'+ albumkey+'/'+postkey).update(updateAlbum);
});
};
})
};
};
As you can see i was able store the url into database but all of the urls are same. What i need is every key store each different links from storage.
Any helps appreciated. Thanks
function uploadImg(file,i) {
return new Promise((resolve,reject)=>{
var storageRef=firebase.storage().ref("store-images/"+file[i].file.name);
task = storageRef.put(file[i].file);
task.on('state_changed', function progress(snapshot){
var percentage=( snapshot.bytesTransferred / snapshot.totalBytes )*100;
console.log(percentage);
// use the percentage as you wish, to show progress of an upload for example
}, // use the function below for error handling
function (error) {
console.log(error);
},
function complete () //This function executes after a successful upload
{
task.snapshot.ref.getDownloadURL().then(function(downloadURL) {
resolve(downloadURL)
});
});
})
}
async function putImage(file) {
for (var i = 0; i < file.length; i++) {
var dd = await uploadImg(file,i);
firebase.database().ref().child('gallery').push(dd);
}
}
Try using the code below:
$scope.submitPhotos = function(file){
console.log(file);
var updateAlbum = [];
for (var i = 0; i < file.length; i++) {
var storageRef=firebase.storage().ref(albumtitle).child(file[i].name);
var task=storageRef.put(file[i]);
task.on('state_changed', function progress(snapshot)
{
var percentage=( snapshot.bytesTransferred / snapshot.totalBytes )*100;
// use the percentage as you wish, to show progress of an upload for example
}, // use the function below for error handling
function (error) {
switch (error.code) {
case 'storage/unauthorized':
// User doesn't have permission to access the object
break;
case 'storage/canceled':
// User canceled the upload
break;
case 'storage/unknown':
// Unknown error occurred, inspect error.serverResponse
break;
}
}, function complete () //This function executes after a successful upload
{
let dwnURL = task.snapshot.downloadURL;
let galleryRef = firebase.database().ref('gallery/'+albumkey);
let postkey = firebase.database().ref('gallery/'+albumkey).push().key;
updateAlbum={img:dwnURL};
firebase.database().ref('gallery/'+ albumkey+'/'+postkey).update(updateAlbum);
});
};
};
All the best!
I just created a NodeJS cloudantDB web starter on bluemix. Then, I have a API get data from cloudantDB and get successfull but it returns all data. Please see js file:
js file:
app.get('/api/provider', function(request, response) {
console.log("Get method invoked.. ")
db = cloudant.use(dbCredentials.dbProvider);
var docList = [];
var i = 0;
db.list(function(err, body) {
if (!err) {
var len = body.rows.length;
console.log('total # of docs -> '+len);
if(len == 0) {
// error
} else {
body.rows.forEach(function(document) {
db.get(document.id, { revs_info: true }, function(err, doc) {
if (!err) {
if(doc['_attachments']) {
// todo
} else {
var responseData = createResponseDataProvider(
doc._id,
doc.provider_type,
doc.name,
doc.phone,
doc.mobile,
doc.email,
doc.logo,
doc.address
);
}
docList.push(responseData);
i++;
if(i >= len) {
response.write(JSON.stringify(docList));
console.log('ending response...');
response.end();
}
} else {
console.log(err);
}
});
});
}
} else {
console.log(err);
}
});
If I want to add parameter to API to get specific data from DB , Do we need create search index or query on cloudant, afer that call API the same : app.get('/api/provider/:id'). Please help me review and sharing. Thanks
you could get the document by id/name:
db.get(docID, function(err, data) {
// do something
});
references:
https://github.com/apache/couchdb-nano#document-functions
https://github.com/cloudant/nodejs-cloudant#api-reference
You can use a search function of Cloudant.
You need to create search index. In search index you can manage what data you want to get.
Example: https://cloudant.com/for-developers/search/
Following this code after create search index.
...
var query = {q: "id:doc.id"};
db.search('design document name', 'index name', query, function(er, result) {
if (er) {
throw er;
}
console.log(result);
});