I am experiencing a problem with ADO.NET SqlDataReader. When I run underlying stored procedure directly in SSMS - it returns 1.7 million of records. When I run a related VB.NET code that fills an ADO.NET DataTable - I also get 1.7 million of records.
But when I run a loop to fill a Generic list like this
While i_oDataReader.Read
m_aFullIDList.Add(i_oDataReader.GetInt32(0))
End While
It returns a lot less records and that number can vary. At the end of the loop if I check m_aFullIDList.Count it ca be 100000 or 500000 etc. Any idea why and how to fix it? Thanks!
Thanks to reference pointed by #Tim Schemlter I found the option "CommandBehavior.SequentialAccess" for DataReader creation. That fixed the issue. E.g. instead of
drReader = oCommand.ExecuteReader();
use
drReader = oCommand.ExecuteReader(CommandBehavior.SequentialAccess);
and it works correctly.
Have you tried using the GetInt64() method instead? I realize that with only 1.7 million records, the GetInt32() method should be large enough; I was more just curious.
Also, if you think that SQL cannot keep up with the DataReader, have you tried adding a wait in the loop to allow it to catch up?
Related
My piece of code is as follows:
my_dsn_tns = cx_Oracle.makedsn('xyz', 1521, sid='SAMPLE')
connection = cx_Oracle.connect(user='asdasdasd', password='TIGER', dsn=my_dsn_tns)
cur = connection.cursor()
cur.execute("delete from SPS_CX_ONHAND_QTY")
connection.commit()
When I execute the following code, it does not give me any error but at the same time does not do anything. Do I need to make any changes in my code?
Not python related or the code you have above but generally in SQL, if you are deleting all the rows in a table you can use truncate table SPS_CX_ONHAND_QTY.
This would be almost instantaneous.
I'm trying to get the number of rows returned by a SELECT command.
RowAffected works only for insert, update and so on, but not for select.
PostgreSQL has the PQntuples() function but it doesn't exist in Npgsql and I can't get the answer out of any of the various versions of the Rows property (or Statement.Rows, or... ?).
Is there something I'm missing, or is it not possible at all ?
Thank you.
Nynn
Why not simply count the rows yourself as you're processing them with NpgsqlDataReader? In other words, simply increment a counter every time you call NpgsqlDataReader.Read()...
I've got a big problem with my VB.Net app :
I have a query who works perfectly in SSMS, returning 16 rows.
But when I try to execute it in my VB.Net app, I've got no rows.
Here's my code who works perfecty for filling all other DataTable of my VB.Net app (more than 200 tables). In this case, variable "a_strRequete" contains the query who works well in SQLServer, returning 16 rows.
Dim v_rrqAdapteur As SqlDataAdapter
v_rrqAdapteur = New SqlDataAdapter(a_strRequete, m_cnxSQL)
v_rrqAdapteur.SelectCommand.CommandTimeout = 900
v_rrqAdapteur.SelectCommand.CommandType = CommandType.Text
o_rrqTable = New DataTable
v_rrqAdapteur.Fill(o_rrqTable)
v_rrqAdapteur.Dispose()
But when I execute this in debug, the line "v_rrqAdapteur.Fill(o_rrqTable)" is executed without any errors, but give me no row. Its driving me crazy because there's no logic in this behaviour : if a query returns rows in SSMS, it must also return the same number of rows when called from VB.Net.
The only query who have this problem is using "pivot" instruction in SQLServer code. Perhaps the problem's coming from that ?
Here's the subquery who contains the pivot in my query :
select id_pk, id_fk, [1.00], [1.25], [1.5]
from (
select
id_pk,
id_fk,
NumericField,
h_occ
from [previous subquery]
) As Hpv
pivot (sum(h_occ) for NumericField in(
[1.00], [1.25], [1.5]
)) As Spv
At the beginning, it was a stored procedure I've integrated to my custom DataSet. But when I see it was returning no rows in my app, I've taken the code of this stored procedure to execute it like a full text query in my code (using the code shown here), and it returns always no rows.
I've got only one server and only one DataBase, who contains all stored procedure, views and functions I need, and only one connection to this DataBase.
These's no CRM used in my code.
Thanks to all who will help.
So, I find a solution et now know from where this problem comes : it comes from the "pivot" instruction.
I've replace the pivot instruction by left join on sum subquery and it now works.
So, seems like the "pivot" instruction cannot be used in query executed from VB.Net app.
This is sad because the "pivot" instruction is very useful et quick to execute (much faster than a left join for every value of the variable to agregate).
Does somebody knows something about the use of "pivot" instruction used in queries executed from VB.Net app ?
I'm using Dapper to do a dynamic Query on a SQL string that is quite large. If I paste it into Word, I get the following stats:
Words: 15,433
Characters (with spaces): 103,366
Lines: 1,637
The problem is that when it executes nothing fails but I get an empty IEnumerable result.
So the question is does Dapper have a limit on this size? Or is it failing quietly?
OK, there is no problem with Dapper, just me! I have two versions of the database and one has data, the other doesn't. I was running against the dataless version.
I am looking through some old code and found a piece that i cant' seem to understand the point of....As far as i can tell, it's just a simple insert. but why did they do it this way? would it be okay to rewrite into an insert, or could i potentially break something?
please see below:
Set TextRS = Server.CreateObject("ADODB.RecordSet")
Set TextRS.ActiveConnection = Conn
TextRS.Source = "SELECT IDX,TIMESTAMP,CURRENTFLAG,TEXT FROM " & TextTable & " WHERE 1=2 FOR UPDATE"
TextRS.CursorLocation = 2
TextRS.CursorType = 3
TextRS.Open ,,,3
TextRS.AddNew
TextRS(0).Value = IDX
TextRS(1).Value = Timestamp
TextRS(2).Value = "Y"
TextRS(3).AppendChunk TextPiece
TextRS.Update
TextRS.Close
This part of the source confused me a bit.... where 1 = 2???
Apparently it had a purpose to ensure no match.
Anyway this style of programming is fairly old using ADO technology and people coming from DAO to ADO would often open up a cursor to iterate over the database this way... it does not follow modern best practices on how to do things, you can and should replace it with an insert statement!
It is possible that it was written pre jet4.0/access 2000 in which case it was an attempt to simulate a parameterized stored procedure. Though if the system is at all more modern than that I would strongly recommend using a stored procedure as it has multiple benefits. Cached Execution Plans, Parameters to reduce the chances of SQL injection
I actually used to write code very much like that 12 years ago or so :p Mostly because I just didn't know better, regardless of the tech in use.
Ah, good old classic ASP ;)
The 1 = 2 forces the sql to never return a match. It's basically a way of building up the command (?) so that you can then "conveniently" change the values and then an update will store it.
I've seen it done before, but never did it that way myself. As others have said, a simple paremetised INSERT statement will be better, IMO.
I would rewrite this using parameterized ADO query. The method being used has an unnecessary SELECT, which makes the INSERT slower.
That code seems a bit obscure, but all they are doing is creating an empty instance of a recordset row so the values can be set and the recordset resaved. This is bound to be much slower than doing a straight INSERT.
I would utilize an insert statement. The above mentioned code seems a little bit "lazy"... as in "let ADO do the work for me". However, there is nothing really wrong with it. The where 1=2 part was there to return an "empty table"... um... I mean recordset.