I have been developing a node.js app that connects to a SQL Server database using the mssql module but I have run into a wall.
Basically, mssql seems to have some kind of bug where it simply crashes the app if the results of a query of any kind returns a certain number of records. Nothing too heavy. I'm talking about 50 to 100 records!
This is not query specific either. It is happening on ALL my queries, no matter what the results are.
The queries run fine if I limit them to return 10, 20, 40 records (using "SELECT TOP x ..."), but as soon as I increase the limit to a larger number of records, the app simply crashes without a single error message. No exceptions. Nothing.
The actual number of records where this starts to happen varies from query to query. It looks as if mssql has either a bug or a by-design limitation that affects the amount of data that it can return.
Am I missing something? Is there I setting I should be changing to avoid this? Alternatively, is there any other npm that I could use to connect to SQL Server?
Needless to say, this is a show-stopper for me. Should I abandon node.js altogether?
The point is, that if I cannot find a proper way to connect to SQL Server, then I will not be able to use node.js for this app and will have to switch to something else.
Thank you!
UPDATE 1
Here is part of the code that is causing this issue:
// Basic modules
var express = require("express");
var bodyParser = require("body-parser");
// Custom modules
var settings = require("./lib/settings.js").GetSettings();
var app = express();
app.use(bodyParser.json());
app.use('/', express.static(__dirname + "/public"));
/***************************************************************************************************************/
// Routes
app.get("/GetBrands", function(req, res) {
var sql = require('mssql');
var config = {
user: settings.DatabaseConfiguration.user,
password: settings.DatabaseConfiguration.password,
server: settings.DatabaseConfiguration.server,
database: settings.DatabaseConfiguration.database
};
var cmd = "SELECT TOP 5 * FROM Brands WHERE Status = 'a'";
var connection = new sql.Connection(config, function(err) {
// ... error checks
if (err) {
console.log(err);
}
// Query
var request = new sql.Request(connection); // or: var request = connection.request();
request.verbose = true;
request.query(cmd, function(err, recordset) {
// ... error checks
if (err) {
console.log(err);
}
console.log(recordset);
connection.close();
});
});
});
/***************************************************************************************************************/
// Enable the port listening.
app.listen(process.env.PORT || 8050);
If I change the the SQL statement that says "SELECT TOP 5 * ..." to a bigger number, like 60, 80 or 100, the app crashes. Also, the response is simply the name of each brand and an ID. Nothing too complicated or heavy.
UPDATE 2:
These are the steps I am following which always crash the app:
Run the app by typing in command-line: node app.js
In a web browser, go to http://localhost:8050/GetBrands. The very first time, I get the results just fine. No crashes.
Run it a second time. The app crashes. Every time.
I also discovered something else. I am using WebStorm for editing the code. If I start the debugger from there, I get no crashes, no issues whatsoever. The app works just as it should. It only crashes when I run it directly from command-line, or from WebStorm without the debugger listening... how crazy is this??
I tried applying the same command-line parameters that the WebStorm debugger uses but it made no difference.
I Hope somebody can shed some light soon because I am very close to dropping node.js altogether for this project thanks to this.
I am OK with switching to use a different SQL Server npm package, but which one then? I already tried mssql, node-sqlserver-unofficial and tedious, they all have the same issue so I am guessing that it is a problem with TDS.
By using the streaming interface, you can reduce your overhead significantly, and allow for better handling of very large query results. I use the streaming interface, for example with piping to export files (csv and xml).
With a relatively simple test, I'm able to crash node itself loading a very large array with 36-character strings (generated with uuid.v4()), at it happens for me around 1GB of use. My guess is there's a hard limit for a number of references allowed in in a running instance.
Related
I'm just trying to simply connect to a neo4j database (on the desktop app) from a C# console app
My username is 'neo4j' and my password is 'root'
Most sites say do:
var client = new GraphClient(new Uri("http://localhost:7474"), "neo4j", "root");
client.Connect();
I then check with:
bool amIConnected = client.IsConnected // false
I was expecting to connect and then try some cypher queries in the code.
So it doesn't work (amIConnected is false, I get something about parsing or a 404 error). I've tried replacing http with neo4j or bolt but no joy.
I've also tried using var client = new Boltgraphclient...and bolt in the uri, but also no joy.
I've also tried using the Driver which is or isn't necessary, I don't know, but anyway, didn't work.
I've also tried tweaking the Uri to localhost:7474/data/db and I've also tried different ports.
Any help, much appreciated.
I have to use SQL Server with Flutter and I don't have another database option because my client has it. I was looking for packages but I only found a package that doesn't run on mobile. Is there any option to do that without web services or api?
The first thing that you need to consider is that there is no immediate and extremely effective solution and you have to decide what frameworks and tools to be used. And as mentioned in the comment that the market for this scenario is very small. But there are some ways that you can handle this.
Remote storage sample solution:
Here is a basic example of how you should implement this. It was also cited in this SO post:
Client application
The client application can be any application the user typically uses.
Some examples:
Mobile app (written in native, Dart, Xamarin, ...)
Desktop app (Electron, WPF, ...)
Website app (Angular, React, Vue, ...)
API
The API is there to retrieve data, and change data. But It will also
handle authentication, authorization, logging, doing business logic
Database
Your API will then execute queries, inserts, updates, deletes, execute
stored procedures on the Database of your choice. In your example SQL
Server.
There are many possibilities on how to set this up, depending on your
skills, knowledge of frameworks, how you want to deploy things.
How you want to deploy this will limit your choices as well. For your
API:
Serverless API (Via Azure Functions, AWS Lambda)
Cloud Website (Azure Web Apps)
Website hosted on premise
Docker container
In real life scenarios this often gets more complex with Firewalls,
Application Gateways, Virtual networks, clusters.
You can install a SQLServerSocket on your server:
https://github.com/nippur72/SqlServerSocket
Install and execute SqlServerSocket.exe in the background on the server machine where SQL Server is installed.
Also, you need a client:
https://github.com/nippur72/SqlServerSocket/tree/master/DartClient
And you can try some connections and queries directly to your DDBB:
// creates a connection
var conn = new
SqlConnection("SERVER=localhost;Database=mydb;Trusted_connection=yes");
// open connection
await conn.open();
// runs a query returning a single value
var howmany = await conn.queryValue("SELECT COUNT(*) FROM Customers");
// runs a query returning a single row
var myFirstCustomer = await conn.querySingle("SELECT name,age FROM Custormers");
print(myFirstCustomer["name"]);
// runs a query returning all rows
var customers = await conn.query("SELECT TOP 10 name,age FROM Custormers");
for(var customer in customers)
{
print(customer["name"]);
}
// execute a command, returning the number of rows affected
var n = await conn.execute("UPDATE Customers SET age=0");
print("zeroed $n customers");
// disconnect
await conn.close();
Hey guys hopefully the title wasn't too bad, hard to describe succinctly.
So I have an ASP.net core 2.0 server running (via Kestrel) on a VPS hosted in Australia (I'm in New Zealand, ~50ms latency to server). An HTTP Get action causes the following code on the ASP.net server to run, where I query an SQL server (also running on the vps) and return the result:
public async Task<string> GetTopContactPhoto(int contactUID)
{
// Open connection if not already open
if (conn.State != System.Data.ConnectionState.Open) { conn.Open(); }
// Get number of rows
string sqlRequest = string.Format("select top 1 imageData from IMAGES where contactUID=#contactID;");
// Return value
using (SqlCommand cmd = new SqlCommand(sqlRequest, conn))
{
cmd.Parameters.AddWithValue("#contactID", contactUID);
return Convert.ToBase64String((byte[])await cmd.ExecuteScalarAsync());
}
}
Now this whole process takes pretty consistently ~5-6 seconds from the time the HTTP Get request is made, to the time the result is given, image sizes ~2MB.
But here's the thing, when I compile and run the ASP.Net core server on my development PC, on a separate continent altogether to the VPS (still running the SQL server), the whole process takes only ~2 seconds, less if I resize the image before responding to Get request.
Any ideas what could be going wrong? This problem has vexed me for months, with no amount of googling doing the trick.
I am using serilog as logging framework in .net core 2.0 project and i am trying to store the logs in sql server but serilog is not storing any data in database and it is not even returning error.
can any one help how to resolve this issue and is it possible to add file approach to store logs when database fails to store
Serilog.Debugging.SelfLog
You can use the SelfLog property to tell serilog where to log it's own errors (all of us have had to debug the logger at some point).
Sample Code
Because I hate providing an answer without sample code that others might find useful ... here is the code we use to "initialize" our logger (including serilog and seq -- a great combo for generating centralized logs that the devops team can monitor).
Serilog.Debugging.SelfLog.Enable(Console.Error);
ILoggerFactory factory = new LoggerFactory();
factory.AddConsole();
factory.AddDebug();
var env = "PROD"; //MyEnvironment: PROD, STAGE, DEV, ETC
var seqLogger = new LoggerConfiguration()
.MinimumLevel.Information()
.Enrich.FromLogContext()
.Enrich.WithProperty("Environment", env)
.WriteTo.Seq(
"logserveraddress",
Serilog.Events.LogEventLevel.Verbose,
1000,
null,
"LogServerApiKey")
);
if (env.ToLower() == "prod") { seqLogger.MinimumLevel.Warning(); }
factory.AddSerilog(seqLogger.CreateLogger());
}
return factory.CreateLogger("NameThisLogInstaceSomethingUseful");
I might be missing something but there doesn't seem to be anything in the official documentation, so I'm stuck asking here;
I'm trying to set up some stuff asynchronously (connections to databases, etc) when the application starts. I want this to complete before the app becomes ready to accept connections (for obvious reasons).
Is there a way to do this in Express 4.x?
Here is an example of basically what I want, however it's unsupported and 4 years out of date at this point.
Any help is appreciated.
Edit: I feel I should point out that I used express-generator to setup my application, so all the server listening is done inside bin/www. Should I just modify this file? Or can I control it from app.js?
For example you can use promises, something like this:
var express = require('express');
var app = express();
var Q = require('q');
var d = Q
mongoose.connect('mongodb://localhost/db', function(){
return d.resolve();
});
d.promise.then(function(){
app.listen(8080);
});