In Microsoft examples I saw two ways to check if DocumentDb object like Database, DocumentCollection, Document etc. exists :
First is by creating a query:
Database db = client.CreateDatabaseQuery().Where(x => x.Id == DatabaseId).AsEnumerable().FirstOrDefault();
if (db == null)
{
await client.CreateDatabaseAsync(new Database { Id = DatabaseId });
}
The second one is by using "try catch" block:
try
{
await this.client.ReadDatabaseAsync(UriFactory.CreateDatabaseUri(databaseName));
}
catch (DocumentClientException de)
{
if (de.StatusCode == HttpStatusCode.NotFound)
{
await this.client.CreateDatabaseAsync(new Database { Id = databaseName });
}
else
{
throw;
}
}
What is the correct way to do this procedure in terms of performance?
You should use the new CreateDatabaseIfNotExistsAsync in the DocumentDB SDK instead of both these approaches, if that's what you're trying to do.
In terms of server resources (request units), a ReadDocumentAsync is slightly more lightweight than CreateDatabaseQuery, so you should use that when possible.
I've just seen the try/catch example in one of the Microsoft provided sample project and it got me baffled, as it is plain wrong: you don't use try/catch for control flow.
Never.
This is just bad code. The new SDK provides CreateDatabaseIfNotExistsAsync which I can only hope doesn't just hide this shit. In older lib just use the query approach, unless you want to get shouted at by whoever is going to review the code.
Related
I have been using squel as query builder for our project and have to migrate due to it being outdated and vulnerable. The squel doc suggested to use knex, I have bit of trouble trying to use it simply just as querybuilder, as database we are using snowflake and use the native sdk.
If someone could help me with a basic insert query in knex without connection creation.
Also the feature/ability to something similar like below in knex.
const insertQuery: squel.Insert = squel
.insert()
.into('TENANT')
.set('URI', 'dummy')
.set('CREATEDBY', 'ADMIN');
if (uuid != null) {
insertQuery.set('UUID', uuid);
}
so basically create the query dynamically and later pass it snowflake query, values.
Any leads would help a lot. Thanks.
Hopefully this is what you are asking for!
const insertQuery = squel.Insert = squel
.insert()
.into('TENANT')
.set('URI', 'dummy')
.set('CREATEDBY', 'ADMIN');
let sqlNative = insertQuery.toSQL().toNative()
//this is the sql in the format sent to the SQL driver.
console.log(sqlNative)
if (uuid != null) {
insertQuery.set('UUID', uuid);
}
https://github.com/knex/knex/issues/2378
I have a nodejs application where i connect to my couchdb using nano with the following script:
const { connectionString } = require('../config');
const nano = require('nano')(connectionString);
// creates database or fails silent if exists
nano.db.create('foo');
module.exports = {
foo: nano.db.use('foo')
}
This script is running on every server start, so it tries to create the database 'foo' every time the server (re)starts and just fails silently if the database already exists.
I like this idea a lot because this way I'm actually maintaining the database at the application level and don't have to create databases manually when I decide to add a new database.
Taking this approach one step further I also tried to maintain my design docs from application level.
...
nano.db.create('foo');
const foo = nano.db.use('foo');
const design = {
_id: "_design/foo",
views: {
by_name: {
map: function(doc) {
emit(doc.name, null);
}
}
}
}
foo.insert(design, (err) => {
if(err)
console.log('design insert failed');
})
module.exports = {
foo
}
Obviously this will only insert the design doc if it doesn't exist. But what if I updated my design doc and want to update it?
I tried:
foo.get("_design/foo", (err, doc) => {
if(err)
return foo.insert(design);
design._rev = doc._rev
foo.insert(design);
})
The problem now is that the design document is updated every time the server restarts (e.g it gets a new _rev on every restart).
Now... my question(s) :)
1: Is this a bad approach for bootstrapping my CouchDB with databases and designs? Should I consider some migration steps as part of my deployment process?
2: Is it a problem that my design doc gets many _revs, basically for every deployment and server restart? Even if the document itself hasn't changed? And if so, is there a way to only update the document if it changed? (I thought of manually setting the _rev to some value in my application but very unsure that would be a good idea).
Your approach seems quite reasonable. If the checks happen only at restarts, this won't even be a performance issue.
Too many _revs can become a problem. The history of _revs is kept as _revs_info and stored with the document itself (see the CouchDB docs for details). Depending on your setup, it might be a bad decision to create unnecessary revisions.
We had a similar challenge with some server-side scripts that required certain views. Our solution was to calculate a hash over the old and new design document and compare them. You can use any hashing function for this job, such as sha1 or md5.
Just remember to remove the _rev from the old document before hashing it, or otherwise you will get different hash values every time.
I tried the md5 comparison like #Bernhard Gschwantner suggested. But I ran into some difficulties because im my case I'd like to write the map/reduce functions in the design documents in pure javascript in my code.
const design = {
_id: "_design/foo",
views: {
by_name: {
map: function(doc) {
emit(doc.name, null);
}
}
}
}
while getting the design doc from CouchDb returns the map/reduce functions converted as strings:
...
"by_name": {
"map": "function (doc) {\n emit(doc.name, null);\n }"
},
...
Obviously md5 comparing does not really work here.
I ended up with the very simple solution by just putting a version number on the design doc:
const design = {
_id: "_design/foo",
version: 1,
views: {
by_name: {
map: function(doc) {
emit(doc.name, null);
}
}
}
}
When I update the design doc, I simply increment the version number and compare it with the version number in database:
const fooDesign = {...}
foo.get('_design/foo', (err, design) => {
if(err)
return foo.insert(fooDesign);
console.log('comparing foo design version', design.version, fooDesign.version);
if(design.version !== fooDisign.version) {
fooDesign._rev = design._rev;
foo.insert(fooDesign, (err) => {
if(err)
return console.log('error updating foo design', err);
console.log('foo design updated to version', fooDesign.version)
});
}
});
Revisiting your question again: In a recent project I used the great couchdb-push module by Johannes Schmidt. You get conditional updates for free, alongside with many other benefits inherited from its dependency couchdb-compile.
That library turned out to be a hidden gem for me. HIGHLY recommended!
I have looked around in many MANY threads, and through various documentation however for what seems like such an incredibly SIMPLE task, this is driving me insane.
I have a node.js webapp which generates a userId upon login, and is stored within a session object.
req.user.id <== my local variable for the user id.
A snippet of the code I have so far is along these lines:
var query = client.query("SELECT * FROM programs WHERE authorid = req.user.id", function (err, result) {
if (err){
//Do erranous things
} else {
// Do good things
}
});
What am I doing wrong? How can I do this simple task of comparing a database entry to a value stored in a local variable?
Any / all help appreciated - I've been trying to do this for 6 hours.
From the github page for the Node.js PostgreSQL client, it looks like you can pass and use arguments like:
client.query("SELECT * FROM programs WHERE authorid = $1",
[req.user.id], function(err, result) { ...
Here is the javascript object I'm trying to convert to a query string
{$and: [{topic: categoryIds} , {$or :[ {'groups 1': {$ne: ''}}, {groups: $scope.myGroups}]}]};
Basically I'm looking to match a topic that equals a categoryIds and grab documents that have an empty groups array or that the groups array has values and matches one in the array $scope.mygroups
My question is what would be best practice to convert this in an easily parseable format so I can append it to a GET request, and how would you go about parsing it on the express server.
I am using this code in production for querying against a server with a MongoDB backend (using angular and lodash):
.factory('mongoQuery', function() {
return {
fromJson: function(json) {
return JSON.parse(json, fromJsonReviver);
},
toJson: function(object) {
return JSON.stringify(object, toJsonReplacer);
}
};
function fromJsonReviver(key, value) {
var val = value;
if (_.isPlainObject(value)) {
if (_.isNumber(value.$date)) {
val = new Date(0);
val.setUTCMilliseconds(value.$date * 1000);
} else if (_.isString(value.$regexp)) {
var match = /^\/(.*)\/([gimy]*)$/.exec(value.$regexp);
val = new RegExp(match[1], match[2]);
}
}
return val;
}
function toJsonReplacer(key, value) {
var val = value;
if (_.isPlainObject(value)) {
val = _.extend({}, value);
for (var k in value) {
val[k] = toJsonReplacer(k, val[k]);
}
} else if (_.isDate(value)) {
val = {$date: (new Date(value)).valueOf() / 1000};
} else if (_.isRegExp(value)) {
val = {$regexp: value.toString()};
}
return val;
}
})
It includes many of the suggestions mentioned by others in the comments and supports dates and regular expressions.
Other than that, if you need to send the query with a GET request, just use encodeURIComponent like others have mentioned.
Here is a working example: http://plnkr.co/edit/b9wJiUkrHMrDKWFC1Sdd?p=preview
What you're thinking here is basically to redevelop an API server, coupled with some mongodb database, and querying it with some format.
REST is the "best practise" you're looking for. It's a standard that encapsulates some common actions to http ressources.
You should know that you don't need to redevelop an ecosystem based on such a standard. Full-featured REST API servers exist, some are even based on express.js. Loopback and sails.js. These one provide some extra features like
Model abstraction through ORM's, and database-engine agnostism
Automatic REST actions, from database schema or model finition
Advanced querying through "extended REST" ("where", "limit", "order", ...)
Realtime with REST-like websockets
Client side libraries that help you query the server
Some standalone exernal libraries, such as js-data or Restangular can deal with REST server pretty well, and act as a frontend connector to backend
Now, if I had to purely answer your question, and if you realy wanted to go with your solution, I'd just add the mongo query to a where query param on the http call with encodeURIComponent as stated before.
I see jslinq, and see tds js libraries for using node and SQL together... So has anyone ever used those technologies together?
I want to be able to write linq to sql queries in a nodejs app...
I've started with JS variant of LINQ to Entities this week. Check UniMapperJS
Example
var comments = await Comment.getAll()
.where(e => e.author.endsWith("something") || e.author.startsWith("somethingElse"))
.orderByDescending("created")
.limit(10)
.skip(0)
.exec();
But cuz of limitation of JS, it was hard to figure out, how to accept variables in where because that arrow functions are parsed as string. I've solved it by "virtual" variable ($) and list of args as last param.
var comments = await Comment.getAll()
.where(e => e.author.endsWith($) || e.author.startsWith($), filter.endsWith, filter.name)
.orderByDescending("created")
.limit(10)
.skip(0)
.select(e => e.author)
.exec();
Today, JayData http://jaydata.org/ do it.
You can use the new ES6 Arrow Function syntax that looks very similar to C# (with a browser that supports it, like lastest Firefox, or a transpiler, like TypeScript or Traceur):
todoDB.Todos
.filter(todo => todo.Completed == true)
.map(todo => todo.Task )
.forEach(taskName => $('#list')
.append('Task: ' + taskName + ' completed'));
The query will be converted to a SQL Query (select Task from Todos where Completed = true) or a $filter URL parameter (http://.../?$filter=Completed%20eq%201&$select=Task), depending of the data source...
You should checkout the edge.js framework, it connects node.js with .Net. One way would be to use the built in T-SQL support in edge, and then use something like linq.js to manipulate the results.
var getTop10Products = edge.func('sql', function () {/*
select top 10 * from Products
*/});
getTop10Product(null, function (error, products) {
if (error) throw error;
console.log(products);
});
Otherwise, you could setup an EF datacontext in a .Net library and call out to that using Linq
var getTop10Product = edge.func(function () {/*
async (input) => {
using (var db = new ProductContext())
{
//Linq to EF query
var query = from b in db.Products
orderby b.Name
select b;
return query.Take(10).ToList();
}
}
*/});
I came across this question today as I was wondering the same thing, and after a little more searching I came across Squel.
Quoting directly from their website:
//this code
squel.select()
.from("students")
.field("name")
.field("MIN(test_score)")
.field("MAX(test_score)")
.field("GROUP_CONCAT(DISTINCT test_score ORDER BY test_score DESC SEPARATOR ' ')")
.group("name")
);
/* will return this SQL query as a string:
SELECT
name,
MIN(test_score),
MAX(test_score),
GROUP_CONCAT(DISTINCT test_score ORDER BY test_score DESC SEPARATOR ' ')
FROM
students
GROUP BY
name
*/
Hopefully this will help anyone who comes across this question while searching.
Good luck!
PS: <Insert standard disclaimer about not generating SQL queries in the browser...>
If you are creating client-side javascript that will send a SQL query to your database you open a whole can of worms - if the browser can dictate the SQL query, and a user can manipulate what comes from the browser, then you are vulnerable to SQL injection attacks.
Bottom line: don't use this in the browser!
This is not possible, at least if you want to create ad-hoc queries in Javascript. Linq (of any flavor) is a compiler technology. A query using Linq syntax or Linq expressions is handled by the C# or VB compiler, not directly interpreted by the database.
A conventional way to do this would be through a web service in C#, using Linq to fetch and store data, and presenting a clean API to clients. Then the client could consume the service through AJAX calls.