I wrote a dBase procedure and I'm having a hard time converting it to a SQL Server stored procedure.
This is what I have for dbase:
CLOSE ALL
SELECT A
USE DDCS_OLD
SELECT B
USE CROSSELL
DO WHILE .NOT. EOF()
mLAT1 = IND_LAT
mLONG1 = IND_LONG
IF mLAT1 > 0 .AND. mLONG1 < 0
SELECT A
GOTO TOP
DO WHILE .NOT. EOF()
mLAT2 = LAT
mLONG2 = LONG
mPROP_CODE = PROP_CODE
mDISTANCE = 3963.0 * ACOS(SIN(mLAT1/57.2958) * SIN(mLAT2/57.2958) + COS(mLAT1/57.2958) * COS(mLAT2/57.2958) * COS(mLONG2/57.2958 - mLONG1/57.2958))
SELECT B
REPLACE &mPROP_CODE WITH mDISTANCE
SELECT A
SKIP
ENDDO
ENDIF
SELECT B
SKIP
END DO'
I have never written a stored procedure before so I'm not sure how to go about a do while loop while using the two tables ddcs_old and crossell.
That looks like a tremendously complicated calculation that I would consider moving out of the database and into the application that is using this data, if at all possible.
However, I'm guessing that what DO WHILE .NOT. EOF() does in dbase is basically read one row at a time from the table, until it reaches the end. In a SQL Stored Procedure you would achieve this with a cursor:
DECLARE crDDCS_Old CURSOR LOCAL FORWARD_ONLY FOR
SELECT LAT, LONG, PROP_CODE FROM ddcs_old
OPEN crDDCS_Old
FETCH NEXT FROM crDDCS_Old
WHILE ##FETCH_STATUS = 0
BEGIN
-- Do your calculations here
FETCH NEXT FROM crDDCS_Old
END
CLOSE crDDCS_Old
DEALLOCATE crDDCS_Old
As I said, I would strongly recommend reconsidering the best way of implementing this functionality, a direct conversion to a stored procedure is highly unlikely to be the best approach. Cursors are inefficient and, apparently, more lines of code than the dbase equivalent. You'd need in-depth knowledge of what it was doing in dbase and how that data is being used, to come up with the best alternative.
You don't use sql like DBase. If at all possible, you want to execute any operation using a set based operation that update all corresponding rows with a single update command -- I.e., you avoid loops based on a cursor 99.97% of the time. Also, without column definitions for your DBase tables (and hopefully corresponding columns for your SQL tables), I don't know how you expect to get any help as it is not really possible to figure out what your existing code does.
However it also looks a like you you are doing great circle calculations, beginning in Sql 2008, you can use the geography data type, which has build in functions for a number of geographic features, including great circle distances.
You really need to get a little understanding of how SQL works instead of asking for some magic and opaque answer -- the time will be well spent and when you get stuck, S/O is a good source for getting unstuck.
I know this is more of a comment, but it is too long for a comment.
Related
I'm dealing with an Oracle DB, connecting from go via InstantClient (version 11) (https://github.com/mattn/go-oci8). I need to be able to load this object and browse results... t_cursor output parameter.
I have tried many strategies, I know how to map function parameters to go structures but I don't know how to work with t_cursor type since it seems not being implemented in InstantClient
Example of stored procedure
create or replace procedure EXAMPLE(a IN NUMBER, b IN NUMBER, c OUT T_CURSOR) AS BEGIN
[Edit] We have also tried to execute SQL blocks from code to try to handle this third parameter.
i.e.
If you add something like
declare
c t_cursor;
begin
EXAMPLE(:1, :2, c)
end
then I don't know how you can get the block to return a result set that contains the cursor.
declare
c t_cursor;
begin
EXAMPLE(:1, :2, c)
select 1, c from dual
end
The whole block returning the result of that select would be ideal but oracle blocks do not return result sets afaik.
Anyone who can bear a hand on this?
Thank you very much
It can be done with the driver https://github.com/rana/ora instead.
*Rset may be passed to Stmt.Exe when prepared with a stored procedure accepting an OUT SYS_REFCURSOR
The README.me even has that exact example.
Caveats:
It's unclear whether the database/sql interface may be used or you are limited to the lib specific API.
Instant Client gets restricted to versions from 12.1.0.1.0 on.
I'm trying to create SP with dynamic cursor for obtain the result of any Select statement
CREATE PROCEDURE CursorTest (:query IN VARCHAR(5000)) ;
BEGIN
DECLARE :out VARCHAR;
DECLARE :dynamicCursor CURSOR FOR EXEC (:query);
OPEN dynamicCursor;
/* cursor loop */
Cursorloop:
LOOP
FETCH NEXT FROM `enter code here`Cursorloop INTO :out;
End LOOP;
CLOSE dynamicCursor;
END;
I have 2 problems on that, Declare the cursor with the dynamic query and output the result as a row.
Thanks in advance
Since this question is tagged pervasive I'm assuming you want to achieve this in PervasiveSQL.
I don't think what you are trying to do is possible there. The main reason for this is that - to my knowledge - P-SQL has no aggregate functions to combine arbitrary columns or rows into a string (like e.g. PostgreSQL's string_agg).
Secondly, P-SQL does not support querying by column number. The :query argument can be any statement (even an invalid one!), so you don't know how many columns it'll produce.
On a more essential note: what is it exactly that you want to achieve? This stored procedure looks to me like an overly complicated way of just executing :query, and having no means of handling the result. If logging or analysis is your goal, wouldn't you be better off by using an external, more flexible (scripting) language to deal with the result set? Admittedly SQL is a programming language, but it has its limitations.
My ASP Classic application are fetching over 10.000 rows via a triple inner join.
After this, it is going through each line, calculating some stuff and putting data in a multidimensional array with 17 coloumns.
All this are being fetched through a jQuery AJAX when pushing the search button.
Logically this takes a while, 5+ minutes actually.
It all works fine. BUT... While doing this, the whole system freezes, not only for the user doing the calculations, but also for all the other users using the system in other ways.
How can I optimize this?!
A small relevant code snippet:
dim userArray()
sql = "select first.*, second.fullname, second.info, third.inputs from first inner join second on first.userid = second.id inner join third on first.info = third.id where convert(varchar, first.period, 112) between '20020115' and '20120115' order by second.fullname, first.userid"
set rs = conn.execute(sql)
if rs.eof then
response.write("Nothing were found in the period")
else
counter = 0
redim userArray(17, 1000)
do until rs.eof
response.flush
counter = counter + 1
' A LOT of calculations and putting in array...
rs.movenext
loop
for i = 1 to counter
' Doing all the response.writes to the user...
next
end if
Lets analyse this whilst also bearing mind the SQL has an ORDER BY Clause:-
do until rs.eof
response.flush
counter = counter + 1
' A LOT of calculations and putting in array...
rs.movenext
loop
Note the Response.Flush, first thing I would do is get rid of that. You will probably need to increase the ASP Response Buffering Limit (in IIS manager). Flush is sending the generated content so far to the client, it waits for the client to acknowledge receipt of all the packets sent before it completes. That is where I'm gonna guess 90% of the 5+ minutes is being spent.
Now "A LOT calculations". VBScript is not know for its peformance. This code may well take some time. In some cases some calculations can be done much better by SQL than in script so that is one option. Another would be to build some COM compiled component to do complex work (although some accounting needs to made for marshalling which can wipe out benefits). However it may be unavoidable that you need to do these calcs in VBScript.
Now rs.movenext. This loop means you hold the connection and rowset open for pretty much all the time the processing is required. That is while the servers is sending bytes over the network to the client and while VBScript is crunching numbers. A much better approach would be suck up all the rowset quickly and disconnect from the DB, then crunch numbers and finally dump the buffer to the client.
Consider using a disconnected recordset (you specify a client side static cursor) or even the simple GetRows method of the recordset object that dumps the whole rowset into a 2-dimensional array. This will mean that you maintain locks on the various tables for the smallest time possible.
I see you already use response.flush() to flush data to the browser intermittedly during the process, but if you're using AJAX, the call must first complete before your AJAX callback function is called, so I think response.flush is not going to be of any use there.
You might try calling the AJAX url directly, and put in a response.write() in the loop to see what happens (and what the speed is)
To get even more information, you could add a timer before the query, after the query and inside the loop and response.write the time that has passed since starting the script. This will give you a very good idea where the delay is happening: http://www.codefixer.com/codesnippets/vbscript_timer_function.asp
You say the machine freezes, is that the client PC with the browser, or the server where IIS runs? If a bucketload of data is being sent I have seen browsers hang and not update until it's done.
Try to add WITH NOLOCK after each table name in your query to select without locking the database for writing. This might give you some data that was overwritten during the execution of your query, but that's usually not a problem.
Also, indexes. Try changing the clustered index on the fields you use in your WHERE statement, or add some regular indexes if you need your clustered index. Optimizing your indexes will speed things up considerably if you haven't done so.
HTH,
Erik
I'd refactor that code like this:
sql = "select first.*, second.fullname, second.info, third.inputs from first inner join second on first.userid = second.id inner join third on first.info = third.id where convert(varchar, first.period, 112) between '20020115' and '20120115' order by second.fullname, first.userid"
Set rs = conn.Execute(sql)
If NOT rs.EOF Then
aRecords = rs.GetRows() ' retrieve your records and assign them to an array '
End If
rs.Close ' record set is now released, so it shouldn't lock up your database anymore
If IsArray(aRecords) Then ' just a small sanity check '
iCount = UBound(aRecords, 2)
For iCounter = 0 To iCount
' Your calculations and your Response.Writes '
Next
Erase aRecords
Else
' no result found '
End If
Update
You can assign the records to variables in your For loop, e.g.
id = aRecords(0, iCounter)
Then you only need to refer to id when ever you need it. You're right though in that if what you're selecting is dynamic (i.e. column's positions can shift), then this approach can produce problems for when you're trying to assign records to variables further along the line. To assign fullname from your second table, for instance, you'd have to know how many columns are being retrieved from first.*.
I am looking through some old code and found a piece that i cant' seem to understand the point of....As far as i can tell, it's just a simple insert. but why did they do it this way? would it be okay to rewrite into an insert, or could i potentially break something?
please see below:
Set TextRS = Server.CreateObject("ADODB.RecordSet")
Set TextRS.ActiveConnection = Conn
TextRS.Source = "SELECT IDX,TIMESTAMP,CURRENTFLAG,TEXT FROM " & TextTable & " WHERE 1=2 FOR UPDATE"
TextRS.CursorLocation = 2
TextRS.CursorType = 3
TextRS.Open ,,,3
TextRS.AddNew
TextRS(0).Value = IDX
TextRS(1).Value = Timestamp
TextRS(2).Value = "Y"
TextRS(3).AppendChunk TextPiece
TextRS.Update
TextRS.Close
This part of the source confused me a bit.... where 1 = 2???
Apparently it had a purpose to ensure no match.
Anyway this style of programming is fairly old using ADO technology and people coming from DAO to ADO would often open up a cursor to iterate over the database this way... it does not follow modern best practices on how to do things, you can and should replace it with an insert statement!
It is possible that it was written pre jet4.0/access 2000 in which case it was an attempt to simulate a parameterized stored procedure. Though if the system is at all more modern than that I would strongly recommend using a stored procedure as it has multiple benefits. Cached Execution Plans, Parameters to reduce the chances of SQL injection
I actually used to write code very much like that 12 years ago or so :p Mostly because I just didn't know better, regardless of the tech in use.
Ah, good old classic ASP ;)
The 1 = 2 forces the sql to never return a match. It's basically a way of building up the command (?) so that you can then "conveniently" change the values and then an update will store it.
I've seen it done before, but never did it that way myself. As others have said, a simple paremetised INSERT statement will be better, IMO.
I would rewrite this using parameterized ADO query. The method being used has an unnecessary SELECT, which makes the INSERT slower.
That code seems a bit obscure, but all they are doing is creating an empty instance of a recordset row so the values can be set and the recordset resaved. This is bound to be much slower than doing a straight INSERT.
I would utilize an insert statement. The above mentioned code seems a little bit "lazy"... as in "let ADO do the work for me". However, there is nothing really wrong with it. The where 1=2 part was there to return an "empty table"... um... I mean recordset.
I've been battling this one for a while now. I have a stored proc that takes in 3 parameters that are used to filter. If a specific value is passed in, I want to filter on that. If -1 is passed in, give me all.
I've tried it the following two ways:
First way:
SELECT field1, field2...etc
FROM my_view
WHERE
parm1 = CASE WHEN #PARM1= -1 THEN parm1 ELSE #PARM1 END
AND parm2 = CASE WHEN #PARM2 = -1 THEN parm2 ELSE #PARM2 END
AND parm3 = CASE WHEN #PARM3 = -1 THEN parm3 ELSE #PARM3 END
Second Way:
SELECT field1, field2...etc
FROM my_view
WHERE
(#PARM1 = -1 OR parm1 = #PARM1)
AND (#PARM2 = -1 OR parm2 = #PARM2)
AND (#PARM3 = -1 OR parm3 = #PARM3)
I read somewhere that the second way will short circuit and never eval the second part if true. My DBA said it forces a table scan. I have not verified this, but it seems to run slower on some cases.
The main table that this view selects from has somewhere around 1.5 million records, and the view proceeds to join on about 15 other tables to gather a bunch of other information.
Both of these methods are slow...taking me from instant to anywhere from 2-40 seconds, which in my situation is completely unacceptable.
Is there a better way that doesn't involve breaking it down into each separate case of specific vs -1 ?
Any help is appreciated. Thanks.
I read somewhere that the second way will short circuit and never eval the second part if true. My DBA said it forces a table scan.
You read wrong; it will not short circuit. Your DBA is right; it will not play well with the query optimizer and likely force a table scan.
The first option is about as good as it gets. Your options to improve things are dynamic sql or a long stored procedure with every possible combination of filter columns so you get independent query plans. You might also try using the "WITH RECOMPILE" option, but I don't think it will help you.
if you are running SQL Server 2005 or above you can use IFs to make multiple version of the query with the proper WHERE so an index can be used. Each query plan will be placed in the query cache.
also, here is a very comprehensive article on this topic:
Dynamic Search Conditions in T-SQL by Erland Sommarskog
it covers all the issues and methods of trying to write queries with multiple optional search conditions
here is the table of contents:
Introduction
The Case Study: Searching Orders
The Northgale Database
Dynamic SQL
Introduction
Using sp_executesql
Using the CLR
Using EXEC()
When Caching Is Not Really What You Want
Static SQL
Introduction
x = #x OR #x IS NULL
Using IF statements
Umachandar's Bag of Tricks
Using Temp Tables
x = #x AND #x IS NOT NULL
Handling Complex Conditions
Hybrid Solutions – Using both Static and Dynamic SQL
Using Views
Using Inline Table Functions
Conclusion
Feedback and Acknowledgements
Revision History
If you pass in a null value when you want everything, then you can write your where clause as
Where colName = IsNull(#Paramater, ColName)
This is basically same as your first method... it will work as long as the column itself is not nullable... Null values IN the column will mess it up slightly.
The only approach to speed it up is to add an index on the column being filtered on in the Where clause. Is there one already? If not, that will result in a dramatic improvement.
No other way I can think of then doing:
WHERE
(MyCase IS NULL OR MyCase = #MyCaseParameter)
AND ....
The second one is more simpler and readable to ther developers if you ask me.
SQL 2008 and later make some improvements to optimization for things like (MyCase IS NULL OR MyCase = #MyCaseParameter) AND ....
If you can upgrade, and if you add an OPTION (RECOMPILE) to get decent perf for all possible param combinations (this is a situation where there is no single plan that is good for all possible param combinations), you may find that this performs well.
http://blogs.msdn.com/b/bartd/archive/2009/05/03/sometimes-the-simplest-solution-isn-t-the-best-solution-the-all-in-one-search-query.aspx