I am trying to insert a new note into a database with the following flutter code. final noteId = await db.insert(noteTable, { userIDColumn: owner.id, textColumn: text, isSyncedWithCloudColumn: 1, });
I get the following exception Exception has occurred. SqfliteDatabaseException (DatabaseException(NOT NULL constraint failed: note.id (code 1299 SQLITE_CONSTRAINT_NOTNULL)) sql 'INSERT INTO note (user_id, text, is_synced_with_cloud) VALUES (?, ?, ?)' args [1, , 1])
I tried the code described above and got the exception instead of a successful result.
Related
I use CockroachDb and Npgsql driver.
I have a simple Users table. When I insert new record, every other query fails with syntax error, which seems bizarre to me.
CREATE TABLE Users (
RequestIdentifier BYTEA NOT NULL UNIQUE,
Identifier UUID PRIMARY KEY DEFAULT gen_random_uuid(),
Id INT8 NOT NULL DEFAULT unique_rowid(),
Email BYTEA NOT NULL UNIQUE,
Username BYTEA NOT NULL,
PasswordHash BYTEA NOT NULL
);
var q = #"
INSERT INTO Users (RequestIdentifier, Email, Username, PasswordHash)
VALUES (#RequestIdentifier, #Email, #Username, #PasswordHash)
ON CONFLICT (RequestIdentifier)
DO NOTHING
RETURNING Identifier
";
byte[] userIdentifier = null;
using (var cmd = new NpgsqlCommand(q, dbConn)) {
cmd.Parameters.Add("RequestIdentifier", NpgsqlDbType.Bytea);
cmd.Parameters.Add("Email", NpgsqlDbType.Bytea);
cmd.Parameters.Add("Username", NpgsqlDbType.Bytea);
cmd.Parameters.Add("PasswordHash", NpgsqlDbType.Bytea);
await cmd.PrepareAsync();
cmd.Parameters[0].Value = msg.RequestIdentifier;
cmd.Parameters[1].Value = msg.Email;
cmd.Parameters[2].Value = msg.Username;
cmd.Parameters[3].Value = passwordHash;
try {
userIdentifier = ((Guid) await cmd.ExecuteScalarAsync()).ToByteArray();
} catch (PostgresException e) when (e.SqlState == SqlErrorCodes.UniqueViolation) {
logger.Information("Email {Email} already in use", UTF8.GetString(msg.Email));
} catch (PostgresException e) {
logger.Error("{Exception}", e);
throw;
}
}
Npgsql.PostgresException (0x80004005): 42601: at or near "close": syntax error
at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
at Npgsql.NpgsqlCommand.ExecuteReaderAsync(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
at Npgsql.NpgsqlCommand.ExecuteScalar(Boolean async, CancellationToken cancellationToken)
at OwlAuthentication.Services.CockroachDbStorageService.CreateOrRetrieveExistingUser(SignUpMessage msg) in C:\Users\Che Khen Kho\Desktop\dotnet\OwlAuthentication\Services\CockroachDbStorageService.cs:line 94
Exception data:
Severity: ERROR
SqlState: 42601
MessageText: at or near "close": syntax error
Detail: source SQL:
CLOSE ALL
^
File: lexer.go
Line: 175
Routine: Error
If I try, say, 10 queries, 5 of them would fail with this exception, but quite often more than 5 rows would actually get inserted (sometimes 6, sometimes 8, etc.).
I tested it with PostgreSQL as well (using uuid_generate_v4 instead of gen_random_uuid and BIGSERIAL for Id column), and everything works fine.
I think this is covered by this issue (https://github.com/cockroachdb/cockroach/issues/45490) which is fixed in CockroachDB v20.1
I'm having issues quickly inserting large volumes of data from Python3 into SQL Server.
The target table has 9 columns with 3 indexes and 1 primary key.
The below code works but it's a lot slower than I would like. See timing below:
-- 1,000 records
In [35]: %time connection_factory.executemany(sql, args)
CPU times: user 30.2 ms, sys: 40.9 ms, total: 71.1 ms
Wall time: 3.54 s
-- 5,000 records
In [46]: %time connection_factory.executemany(sql, args)
CPU times: user 110 ms, sys: 55.8 ms, total: 166 ms
Wall time: 17 s
I've tried using sql_alchemy and am currently using Turbodbc - but open to anything else that works faster.
Below is a sample of my code
from turbodbc import connect, make_options
class ConnectionFactory:
def __init__(self):
self.connection = self.initialize()
#staticmethod
def initialize():
options = make_options(autocommit=True)
return connect(driver="FREETDS",
server="",
port="",
database="",
uid="",
pwd="",
turbodbc_options=options)
def execute(self, query, params=None):
try:
cursor = self.connection.cursor()
cursor.execute(query, params)
except Exception as e:
print(e)
finally:
cursor.close()
return
def executemany(self, query, params=None):
try:
cursor = self.connection.cursor()
cursor.executemany(query, params)
except Exception as e:
print(e)
finally:
cursor.close()
return
sql = """
INSERT INTO table1 (value1,
value2,
value3,
value4,
value5,
value6,
value7)
VALUES (?, ?, ?, ?, ?, ?, ?); """
args = df.to_records().tolist()
connection_factory = ConnectionFactory()
connection_factory.executemany(sql, args)
Is anyone familiar with this exact combination of SQL Server and python that could point me in the right direction?
Sorry, my mistake, I posted information about mySQL. You're looking for msSQL.
Here is an equivalent bulk insert statement for msSQL:
BULK INSERT MyTable
FROM 'path\myfile.csv'
WITH
(FIELDTERMINATOR = ';',
ROWTERMINATOR = '\n')
There are a few options:
You may write your data to a .csv file and then leverage mySql's very fast LOAD DATA INFILE command.
OR
You may also use another form of the insert command, which is:
INSERT INTO tbl_name
(a,b,c)
VALUES
(1,2,3),
(4,5,6),
(7,8,9);
See these optimization links:
Load Data Infile
mySQL Insert Optimization
I can see that you already have function for execute(). it should be faster same to bulk insert.
args= ', '.join(map(str, df.to_records().tolist()))
sql = "
INSERT INTO table1 (value1,
value2,
value3,
value4,
value5,
value6,
value7)
VALUES {}".format(args)
connection_factory = ConnectionFactory()
connection_factory.execute(sql)
Create new method to execute query from string without params.
def execute2(self, query):
try:
cursor = self.connection.cursor()
cursor.execute(query)
except Exception as e:
print(e)
finally:
cursor.close()
return
So I want to create a database for Users and insert values into the fields using variable. Initially I tried using it calling the do function, but it wasn't reading the variables properly so I decided to just use prepare and execute separately. This is my code:
$dbh->do("DROP TABLE IF EXISTS Users");
$dbh->do("CREATE TABLE Users(
zid TEXT,
Name TEXT,
Email TEXT,
password TEXT,
Mates TEXT,
Program TEXT,
Courses TEXT,
Suburb TEXT,
Birthday TEXT)");
$zid = "z33432523";
$name = "John Doe";
$email = "email#gmail.com";
$password = "alien";
$mates = "z3459148 z3458291";
$program = "";
$courses = "";
$suburb = "";
$birthday = "13/5/1992";
$sth = $dbh->prepare('INSERT INTO Users VALUES (?, ?, ?, ?, ?. ?, ?, ?, ?)');
$sth->execute($zid, $name, $email, $password, $mates, $program, $courses, $suburb, $birthday);
$dbh->disconnect();
However, if I try running this code I get the following error:
DBD::SQLite::db prepare failed: near ".": syntax error at ./dbm.pl line 35.
I'm not sure exactly what the problem is?
near ".": syntax error
INSERT INTO Users VALUES (?, ?, ?, ?, ?. ?, ?, ?, ?)
^
I am currently trying to connect my iOS app with SQLite database I have on my computer locally, and while I have found a series of codes that work, I would like to understand how they work.
The below is from the post
Use sqlite3_exec to perform SQL (e.g. create table).
if sqlite3_exec(db, "create table if not exists test (id integer primary key autoincrement, name text)", nil, nil, nil) != SQLITE_OK {
let errmsg = String.fromCString(sqlite3_errmsg(db))
print("error creating table: \(errmsg)")
}
I understand the if statement but why is there 3 nils in the parameter and where is sqlite3_exec coming from? is it bundled with C when I imported sqlite3.dylib?
The next line as I understand defines errmsg as a string that displays the error message returned by sqlite3 along with the database name.
var statement: COpaquePointer = nil
if sqlite3_prepare_v2(db, "insert into test (name) values (?)", -1, &statement, nil) != SQLITE_OK {
let errmsg = String.fromCString(sqlite3_errmsg(db))
print("error preparing insert: \(errmsg)")
}
if sqlite3_bind_text(statement, 1, "foo", -1, SQLITE_TRANSIENT) != SQLITE_OK {
let errmsg = String.fromCString(sqlite3_errmsg(db))
print("failure binding foo: \(errmsg)")
}
if sqlite3_step(statement) != SQLITE_DONE {
let errmsg = String.fromCString(sqlite3_errmsg(db))
print("failure inserting foo: \(errmsg)")
}
The above also raises some questions for me.
What is COpaquePointer? is it just a pointer?
Again the same questions about the parameter of the first statement (db, "insert into test (name) values (?)", -1, &statement, nil) Why is there -1 or nil?
And is SQLITE_OK returned by SQLite?
I want to do a simple WebOS Mojo app that adds dates to a database, but before that it needs to check few conditions by interrogating the database. Given the asynchronous mode of accessing the database in WebOS I'm wondering what is the most efficient and short way of writing such code.
For example I need to make sure the new date is not already in the database. Then I need to get the closest date so that I can compute the difference in days and throw an error if the difference is too small. Then I need to insert the new date and then compute an average.
This would be easy to do in a synchronous way of accessing the db, but I don't really like the idea of writing parts of the code in multiple success handlers of the different sql statements executed. Is there a more elegant solution?
You can use inline callbacks to the HTML 5 relational database functions:
function createProject(project, onSuccess) {
if (project.projectId)
throw new StorageError("project already exists");
if (project.path)
throw new StorageError("project already has a path");
project.projectId = ++Tracker.maxLocalId * 4096;
project.path = calcNextProjectPath();
project.normalize();
Tracker.db.transaction(
function (transaction) {
transaction.executeSql(
"INSERT OR ROLLBACK INTO item (dbId, path, hasChildren, kind, summaryText, place, work, responsible, responsibleClass) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
[project.projectId, project.path, project.hasChildren, project.kind, project.title, project.place, project.work, project.responsible, project.responsibleClass],
function (transaction, resultSet) {
Mojo.Log.info("DB: inserted new project item", resultSet.insertId, project.title);
transaction.executeSql(
"INSERT OR ROLLBACK INTO project (projectId, accountId, title, backendKind, backendStatus, backendLastChanged, lastSuccessfulDown) \
VALUES (?, ?, ?, ?, ?, ?, ?)",
[project.projectId, project.accountId, project.title, project.backendKind, project.backendStatus, project.backendLastChanged, project.lastSuccessfulDown],
function (transaction, resultSet) {
Warn.logInfo("created new project", "projectId=" + resultSet.insertId, project.title);
if (onSuccess)
onSuccess();
},
function (transaction, sqlError) {
Warn.bannerError("create", $L("Quit and reset your phone."), "project row insertion failed: ", sqlError.message, sqlError.code, project);
return true; // abort whole transaction
}
);
},
function (transaction, sqlError) { // failure of insert project item
if (sqlError.code === 1 && sqlError.message === "constraint failed" && Mojo.appInfo.id !== "com.outlinetracker.outlinetracker") {
upgradeAlert(true);
} else {
Warn.bannerError("create", $L("Quit and reset your phone."), "project item insertion failed: ", sqlError.message, sqlError.code);
}
return true; // abort whole transaction
}
);
}, // end transaction function
function (sqlError) { // seems to only be called for exceptions in callbacks
Warn.logError($L("Quit and reset your phone."), "transaction 'insert project' failed", sqlError.message, sqlError.code);
}
); // end transaction call
}