I have installed the latest version of CI 2.1.3
Now after running a query, I am getting a very slow response time for something very simple such as:
function Bash(){
$sql = “SELECT * FROM Contacts”;
$stmt = sqlsrv_query( $conn, $sql );
if( $stmt === false) {
die( print_r( sqlsrv_errors(), true) );
}
after querying a remote database. (Sql server 2008)
When I run this same query in a simple PHP script against the same remote database. I get results instantly.
a) Has anyone else experienced this problem with the sqlsrv drivers in codeigniter?
If so, how did you solve it?
Here is my connection string:
$db['default']['hostname'] = "xxxxx,1433";
$db['default']['username'] = "xx";
$db['default']['password'] = "xxxxxx-xx";
$db['default']['database'] = "xxxxxxxxx";
$db['default']['dbdriver'] = "sqlsrv";
$db['default']['dbprefix'] = '';
$db['default']['pconnect'] = TRUE;
$db['default']['db_debug'] = TRUE;
$db['default']['cache_on'] = TRUE;
$db['default']['cachedir'] = '';
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
$db['default']['swap_pre'] = '';
$db['default']['autoinit'] = TRUE;
$db['default']['stricton'] = FALSE;
UPDATE:
I have found the following from running the profiler.
DATABASE: database QUERIES: 1 (Hide)
0.0659 select * from Contacts
Loading Time: Base Classes 0.0428
Controller Execution Time ( Welcome / AzureBash ) 58.2173
Total Execution Time 58.2602
It seems as though the query is executing in 0.06 secs but the controller is taking a minute to load.
No idea why this is happening.
Solution
The active records interface for the latest SQLSRV drivers are buggy.
So, download and overwrite the existing interface with these (overwrite your sqlsrv folder in the database folder in CI):
http://www.kaweb.co.uk/blog/mssql-server-2005-and-codeigniter/
Note: These have been tested with SQL Azure and works.
$query->num_rows(); does not work with these drivers, so I suggest you use count instead. Or create your own wrapper.
In addition date is now a date object type in your result set.
I hope this helps.
Solution 2
If for whatever reason you find a bug that makes this completely unusable. Revert back to the sqlsrv interface originally provided. You will find what is causing the problem is the way the original interface are executing the query, thus, create a database helper class; use $sql = $this->db->last_query(); to get the query you was about to execute and then within the database_helper class execute it yourself:
function MakeDbCall ($sql)
{
$serverName = "xxxxx-xxxx-xxx,1433"; //serverName\instanceName
$connectionInfo = array( "Database"=>"xxx", "UID"=>"xx", "PWD"=>"xxxxx","ConnectionPooling" => "1");
$conn = sqlsrv_connect($serverName,$connectionInfo);
$stmt = sqlsrv_query($conn, $sql);
while( $row = sqlsrv_fetch_array( $stmt, SQLSRV_FETCH_ASSOC) ) {
$result_array[] = $row;
}
return $result_array;
}
Create one for row_array.
You should be able to call this function directly, from anywhere in your app. Whilst taking advantage of the way active_records constructs your query.
Not an ideal solution, but until codeigniter sort their SQLSRV class, there is not a lot we can do.
Adding an answer to this after the answer has already been accepted because I found a different solution. I was having the same problem ... looping through the result set was very very slow. i opened system/database/drivers/sqlsrv/sqlsrv_driver.php and found the connection function. i noticed that is was using the SQLSRV_CURSOR_STATIC option. i changed this to SQLSRV_CURSOR_CLIENT_BUFFERED and my slowness problems went away. See documentation for this here:
http://msdn.microsoft.com/en-us/library/hh487160(v=sql.105).aspx
I honestly have no idea what the sql server driver for php is doing, however, given the speed up, etc i can guess that the driver might be using a cursor by default. this seems like an awful idea. i also am assuming that by choosing client_buffered the data for the query would b e read without a cursor and accessed in memory on the client as if it were a cursor. If this is the case, bad things might happen if you try to execute a query that has many many rows to read. Perhaps another option (SQLSRV_CURSOR_FORWARD?) can be used to read data without a cursor - but i'm sure the methods used to access the query will be more limited (e.g. not using result_array())
-Don
Solution
The active records interface for the latest SQLSRV drivers are buggy.
So, download and overwrite the existing interface with these (overwrite your sqlsrv folder in the database folder in CI):
http://www.kaweb.co.uk/blog/mssql-server-2005-and-codeigniter/
Note: These have been tested with SQL Azure and works.
$query->num_rows(); does not work with these drivers, so I suggest you use count instead. Or create your own wrapper.
In addition date is now a date object type in your result set.
Solution 2
If for whatever reason you find a bug that makes this completely unusable. Revert back to the sqlsrv interface originally provided. You will find what is causing the problem is the way the original interface are executing the query, thus, create a database helper class; use $sql = $this->db->last_query(); to get the query you was about to execute and then within the database_helper class execute it yourself:
function MakeDbCall ($sql)
{
$serverName = "xxxxx-xxxx-xxx,1433"; //serverName\instanceName
$connectionInfo = array( "Database"=>"xxx", "UID"=>"xx", "PWD"=>"xxxxx","ConnectionPooling" => "1");
$conn = sqlsrv_connect($serverName,$connectionInfo);
$stmt = sqlsrv_query($conn, $sql);
while( $row = sqlsrv_fetch_array( $stmt, SQLSRV_FETCH_ASSOC) ) {
$result_array[] = $row;
}
return $result_array;
}
Create one for row_array.
You should be able to call this function directly, from anywhere in your app. Whilst taking advantage of the way active_records constructs your query.
Not an ideal solution, but until codeigniter sort their SQLSRV class, there is not a lot we can do.
What is your connection string? You can specify the "network protocol" explicitly, which somtimes can affect speed.
http://www.connectionstrings.com/articles/show/define-sql-server-network-protocol
"Provider=sqloledb;Data Source=190.190.200.100,1433;Network Library=DBMSSOCN;Initial Catalog=pubs;User ID=myUsername;Password=myPassword;"
By specifying the IP address, the port number (1433) and the Network Library, you are providing a very granular connection string.
Your details may vary of course.
Alot of times, you don't need this. But I've been on a few client trips where this was the magic dust.
You might want to turn db_debug to FALSE which should save time debugging the database.
Also, would suggest to turn cache_on to FALSE and specify cachedir and use $this->db->cache_on(); for queries that are less dynamic, i.e. does not change frequently.
For speed up fetch up to 3 times please use "MultipleActiveResultSets"=>'0' in your sqlsrv_connect connection options.
Ex:
$db = sqlsrv_connect('127.0.0.1', array('Database'=>'dbname','UID'=> 'sa','PWD'=> 'pass',"CharacterSet" =>"UTF-8","ConnectionPooling" => "1"
,"MultipleActiveResultSets"=>'0'
));
Related
I am having issues attempting to connect to two different databases in one Qt Application. I have my information database that stores all the information collected by the application and the new Log database which allows me to track all the changes that occur to the Application, button presses, screen loads etc, for easy debugging after its release. Separately, the databases work perfectly, but when I try to use both of them, only one will work. I read that this could be because I wasn't naming the connections and obviously only the most recently connected database could use the default connection. However when I give the databases names they wont work at all, isOpen() will return true on both, but as soon as they attempt to execute a query I get the errors
"QSqlQuery::prepare: database not open"
"QSqlError(-1, "Driver not loaded", "Driver not loaded")"
My two database declarations are:
database_location = filepath.append("/logger.sqlite");
logDB = QSqlDatabase::addDatabase("QSQLITE", "LoggerDatabaseConnection");
logDB.setHostName("localhost");
logDB.setDatabaseName(database_location);
for the Logger Database connection and :
database_location = filepath.append("/db.sqlite");
db = QSqlDatabase::addDatabase("QSQLITE", "NormalDB");
db.setHostName("localhost");
db.setDatabaseName(database_location);
Also when I am running the first query on the databases to see if their tables exist I am using
QSqlQuery query("LoggerDatabaseConnection");
and likewise for the normal database, but I am still getting connection issues even after declaring the database connection to run the query on.
The database used for the application is declared as a static QSqlDatabase in a namespace to create a global effect, so everyone can access it, that was a previous programmer, and I created the Log database as Singleton with a private database connection. Like I said both versions of the code work separately but when they are together they are fighting each other. I know there is a huge debate over the proper design of Singleton vs Dependecy Injection, but again the code works separately so I am happy with how it is designed for now. If there is any missing information or if you have any ideas, please let me know. Thank you.
QSqlQuery query("LoggerDatabaseConnection");
The first parameter of the constructor is the query, not the connection name. It will use the default connection since you specified no database object.
Try something like this:
QSqlQuery query1("YourFirstQuery", db);
QSqlQuery query2("YourSecondQuery", logDB);
Important: Also do not forget to open and close the database before / after using it by calls to QSqlDatabase::open() and QSqlDatabase::close().
The correct way to have multiple databases is to not use the pointer returned from the static addConnection method. You should use the connectionName argument:
https://doc.qt.io/qt-5/qsqldatabase.html#addDatabase-1 during initilization and query usage:
example:
void MyClass::initDb(QString dbPath, QString connName)
{
// initial db usage, etc
QSqlDatabase db = QSqlDatabase::addDatabase(YOUR_DRIVER, connName);
db.setDatabaseName(dbPath);
// open it, etc
}
void MyClass::updateThing(QString val, QString name, QString connName)
{
QString q = QString("UPDATE THINGS SET val=%1 WHERE name=%2").arg(val, name);
// add the reference to your database via the connection name
QSqlDatabase db = QSqlDatabase::database(connName);
QSqlQuery query(db);
query.exec(q);
// handle the query normally, etc
}
Is there a way to dump the generated sql to the Debug log or something? I'm using it in a winforms solution so the mini-profiler idea won't work for me.
I got the same issue and implemented some code after doing some search but having no ready-to-use stuff. There is a package on nuget MiniProfiler.Integrations I would like to share.
Update V2: it supports to work with other database servers, for MySQL it requires to have MiniProfiler.Integrations.MySql
Below are steps to work with SQL Server:
1.Instantiate the connection
var factory = new SqlServerDbConnectionFactory(_connectionString);
using (var connection = ProfiledDbConnectionFactory.New(factory, CustomDbProfiler.Current))
{
// your code
}
2.After all works done, write all commands to a file if you want
File.WriteAllText("SqlScripts.txt", CustomDbProfiler.Current.ProfilerContext.BuildCommands());
Dapper does not currently have an instrumentation point here. This is perhaps due, as you note, to the fact that we (as the authors) use mini-profiler to handle this. However, if it helps, the core parts of mini-profiler are actually designed to be architecture neutral, and I know of other people using it with winforms, wpf, wcf, etc - which would give you access to the profiling / tracing connection wrapper.
In theory, it would be perfectly possible to add some blanket capture-point, but I'm concerned about two things:
(primarily) security: since dapper doesn't have a concept of a context, it would be really really easy for malign code to attach quietly to sniff all sql traffic that goes via dapper; I really don't like the sound of that (this isn't an issue with the "decorator" approach, as the caller owns the connection, hence the logging context)
(secondary) performance: but... in truth, it is hard to say that a simple delegate-check (which would presumably be null in most cases) would have much impact
Of course, the other thing you could do is: steal the connection wrapper code from mini-profiler, and replace the profiler-context stuff with just: Debug.WriteLine etc.
You should consider using SQL profiler located in the menu of SQL Management Studio → Extras → SQL Server Profiler (no Dapper extensions needed - may work with other RDBMS when they got a SQL profiler tool too).
Then, start a new session.
You'll get something like this for example (you see all parameters and the complete SQL string):
exec sp_executesql N'SELECT * FROM Updates WHERE CAST(Product_ID as VARCHAR(50)) = #appId AND (Blocked IS NULL OR Blocked = 0)
AND (Beta IS NULL OR Beta = 0 OR #includeBeta = 1) AND (LangCode IS NULL OR LangCode IN (SELECT * FROM STRING_SPLIT(#langCode, '','')))',N'#appId nvarchar(4000),#includeBeta bit,#langCode nvarchar(4000)',#appId=N'fea5b0a7-1da6-4394-b8c8-05e7cb979161',#includeBeta=0,#langCode=N'de'
Try Dapper.Logging.
You can get it from NuGet. The way it works is you pass your code that creates your actual database connection into a factory that creates wrapped connections. Whenever a wrapped connection is opened or closed or you run a query against it, it will be logged. You can configure the logging message templates and other settings like whether SQL parameters are saved. Elapsed time is also saved.
In my opinion, the only downside is that the documentation is sparse, but I think that's just because it's a new project (as of this writing). I had to dig through the repo for a bit to understand it and to get it configured to my liking, but now it's working great.
From the documentation:
The tool consists of simple decorators for the DbConnection and
DbCommand which track the execution time and write messages to the
ILogger<T>. The ILogger<T> can be handled by any logging framework
(e.g. Serilog). The result is similar to the default EF Core logging
behavior.
The lib declares a helper method for registering the
IDbConnectionFactory in the IoC container. The connection factory is
SQL Provider agnostic. That's why you have to specify the real factory
method:
services.AddDbConnectionFactory(prv => new SqlConnection(conStr));
After registration, the IDbConnectionFactory can be injected into
classes that need a SQL connection.
private readonly IDbConnectionFactory _connectionFactory;
public GetProductsHandler(IDbConnectionFactory connectionFactory)
{
_connectionFactory = connectionFactory;
}
The IDbConnectionFactory.CreateConnection will return a decorated
version that logs the activity.
using (DbConnection db = _connectionFactory.CreateConnection())
{
//...
}
This is not exhaustive and is essentially a bit of hack, but if you have your SQL and you want to initialize your parameters, it's useful for basic debugging. Set up this extension method, then call it anywhere as desired.
public static class DapperExtensions
{
public static string ArgsAsSql(this DynamicParameters args)
{
if (args is null) throw new ArgumentNullException(nameof(args));
var sb = new StringBuilder();
foreach (var name in args.ParameterNames)
{
var pValue = args.Get<dynamic>(name);
var type = pValue.GetType();
if (type == typeof(DateTime))
sb.AppendFormat("DECLARE #{0} DATETIME ='{1}'\n", name, pValue.ToString("yyyy-MM-dd HH:mm:ss.fff"));
else if (type == typeof(bool))
sb.AppendFormat("DECLARE #{0} BIT = {1}\n", name, (bool)pValue ? 1 : 0);
else if (type == typeof(int))
sb.AppendFormat("DECLARE #{0} INT = {1}\n", name, pValue);
else if (type == typeof(List<int>))
sb.AppendFormat("-- REPLACE #{0} IN SQL: ({1})\n", name, string.Join(",", (List<int>)pValue));
else
sb.AppendFormat("DECLARE #{0} NVARCHAR(MAX) = '{1}'\n", name, pValue.ToString());
}
return sb.ToString();
}
}
You can then just use this in the immediate or watch windows to grab the SQL.
Just to add an update here since I see this question still get's quite a few hits - these days I use either Glimpse (seems it's dead now) or Stackify Prefix which both have sql command trace capabilities.
It's not exactly what I was looking for when I asked the original question but solve the same problem.
I want to test one of my Model classes, so i have to insert, update and delete data from my database in order to test if my methods work well.
I am working with a defined Test database where i have already some data.
To test all methos i use two roles, the admin one and the user one. So i get their data using the setUp method like this:
public function setUp() {
parent::setUp();
$this->User = ClassRegistry::init('User');
$admin = $this->User->query("select * from users where admin = 1");
$this->testUser['admin']['id'] = $admin[0]['users']['id'];
$this->testUser['admin']['username'] = $admin[0]['users']['username'];
$this->testUser['admin']['password'] = $admin[0]['users']['password'];
$this->testUser['admin']['verified'] = $admin[0]['users']['verified'];
$this->testUser['admin']['created'] = $admin[0]['users']['created'];
$this->testUser['admin']['nick'] = $admin[0]['users']['nick'];
$this->testUser['admin']['admin'] = $admin[0]['users']['admin'];
$user = $this->User->query("select * from users where admin = 0 and verified = 0");
$this->testUser['user']['id'] = $user[0]['users']['id'];
$this->testUser['user']['username'] = $user[0]['users']['username'];
$this->testUser['user']['password'] = $user[0]['users']['password'];
$this->testUser['user']['verified'] = $user[0]['users']['verified'];
$this->testUser['user']['created'] = $user[0]['users']['created'];
$this->testUser['user']['nick'] = $user[0]['users']['nick'];
$this->testUser['user']['admin'] = $user[0]['users']['admin'];
}
When i want to test methods like the "banAccess" one who moves data from the Users table to the bannedUsers table, then i have a problem because the Test won't run well the next time as the user i selected for the Test won't be in the same table.
It seems that setUP() and tearDown() methods are only executed once after all test methods are called.
This way, if the bannAccess test methods is executed before the testGetUserName method, for example, this last one will fail as the user is not on Users table.
For the moment i am testing the method and deleting the user after it in order to solve this problem but i am sure it have to be a better way to do it:
public function testBanAccess() {
$result = $this->User->banAccess($this->testUser['user']['id'], 'spam', '42');
$expected = true;
$this->assertEquals($expected, $result);
$this->User->query("delete from banUsers where id = ".$this->testUser['user']['id']);
}
Thanks.
Your whole test setup is not good. You should use fixtures for that an have the records present in the fixtures. See http://book.cakephp.org/2.0/en/development/testing.html#fixtures
setUp() and tearDown() are executed only one time while startTest() and endTest() are for each test*() method.
Further you should not use query() because it is potentially unsafe because of SQL injections. The CakePHP ORM will take care of that if you would use it... To see query() present in the test make me think you've used it in the app to and built a pretty unsafe app.
Also why do you have to copy users to another table instead simply flagging them as banned with a simple tinyint field?
I'm trying to write to a local SQLite database using the flash.data.* classes in AIR. I'm opening a synchronous connection in CREATE mode and using the begin() and commit() methods to execute the queries. Everything seems to be executing as expected. The query execute() and connection commit() method's success handler is being called, the connection object's totalChanges property is being incremented, everything looks good except the database file is not being written to. Any ideas what I could be doing wrong?
I don't think it's related to...
the query itself since that was
throwing errors whenever something
didn't match up.
the file mode for the same reason.
file permissions - currently set to 777
Simplified version of the code:
var database:File = new File(File.applicationDirectory.nativePath + "//" + PATH_TO_DB );
var connection:SQLConnection = new SQLConnection();
connection.open( database, SQLMode.CREATE );
connection.begin();
var statement:SQLStatement = new SQLStatement();
statement.sqlConnection = connection;
statement.addEventListener(SQLEvent.RESULT, onQueryResult);
statement.addEventListener(SQLErrorEvent.ERROR, onQueryError);
statement.text = "INSERT INTO myCrazyTable (foo) VALUES ('bar')";
statement.execute();
connection.commit(new Responder(onCommitComplete));
function onQueryResult(event:SQLEvent):void {
trace("Query successful"); // this is getting called
}
function onQueryError(event:SQLErrorEvent):void {
trace("Error in query: " + event.error.message);
}
function onCommitComplete(event:SQLEvent):void {
trace("Commit Success"); // this is getting called
connection.close();
}
// Database isn't getting touched.
Did you check options defined by pragma? For example http://www.sqlite.org/pragma.html#pragma_synchronous can cause this behavior.
The first thing I would ask is, how do you know it isn't being touched? Timestamp? Or are you querying for the item and not finding it?
The main reason I ask is, I'm suspicious of this code:
var database:File = new File(File.applicationDirectory.nativePath + "//" + PATH_TO_DB );
I think the // is incorrect, and I think possibly your DB is ending up somewhere other than where you think it is, most likely at the root filesystem. If my theory is correct, you are writing data to a different db than you think you are, not just "in memory".
I fixed the issue. I ended up rewriting the code for accessing the database. I'm not sure exactly where the problem was, something I must have been overlooking. Thanks for everyone's help.
I have just finished my first site built with zend framework and all works great on my local machine.
Then I uploaded it to the server (godaddy) and all works except any connection my models do with the database. I have made a connetion to the database with regular PDO with the credentials in my application.ini and it worked, and I can interact with the model if it's not returning anything from the database (and again all the models work great on my local machine).
My models looks like this:
class Default_Model_picture extends Zend_Db_Table_Abstract
{
protected $_name = 'pictures';
protected $_primary = 'id';
public function getPicturesByCategory($category)
{
$query = $this->select()->from(array('pictures'), array(
'pictures.id', 'pictures.pic_name', 'pictures.pic_desc',
'pictures.pic_category', 'pictures.pic_date_added',
'pictures.pic_larger', 'pictures.pic_url'));
$query->where('pic_category = ?', $category);
$query->order('pic_date_added ASC');
$result = $this->fetchAll($query);
return $result;
}
}
this is an example for a model, obviously i did not added lots of methods.
i have no idea what to do next.
I am assuming you set up the db connection correctly into $db. Afterwards you must set it as the default adapter for Zend_Db_Table.
Zend_Db_Table::setDefaultAdapter($db);
I am just assuming this is what went wrong. But it is a common problem, so I decided to go ahead and answer anyway.
Since your script works fine on your local machine, the first thing I check is if you have got the database connection params setup correctly in your application.ini
Try to write a test script that uses the pdo functions on itself (without zend framework). see if you get any errors at all
try {
$dbh = new PDO('mysql:host=YOURHOST;dbname=YOURDBNAME', $YOURUSERNAME, $YOURPASSWORD);
foreach($dbh->query('SELECT * from FOO') as $row) {
print_r($row);
}
$dbh = null;
} catch (PDOException $e) {
print "Error!: " . $e->getMessage() . "<br/>";
die();
}