I'm currently reviewing a program that we're using to populate a custom table since it was experiencing performance issues, and I'm looking for ways to optimize performance. One idea that I came up with was to consolidate loop statements that iterate over similar tables, but I wasn't sure how it would affect the performance and would like to ask for confirmation and/or recommendations if possible. So like the title says, does minimizing LOOP AT-statements improve overall runtime efficiency?
Shown below is a simplified example of the original code with 2 loops that are separated with different WHERE-clause, and then the latter is my idea to consolidate the loop and separate it into multiple ELSEIF-statements as needed.
ORIGINAL CODE:
IF param = 'X'.
LOOP AT lt_intab
INTO wa_intab
WHERE field_one = const_one.
SELECT *
FROM db_table
WHERE field_two = wa_intab-field_two.
MOVE-CORRESPONDING db_table TO wa_outtab.
APPEND wa_outtab TO lt_outtab.
CLEAR wa_outtab.
ENDSELECT.
ENDLOOP.
ENDIF.
LOOP AT lt_intab
INTO wa_intab
WHERE field_one = const_two.
SELECT *
FROM db_table
WHERE field_three = wa_intab-field_three.
MOVE-CORRESPONDING db_table TO wa_outtab.
APPEND wa_outtab TO lt_outtab.
CLEAR wa_outtab.
ENDSELECT.
ENDLOOP.
OPTIMIZED CODE:
LOOP AT lt_intab
INTO wa_intab
IF param = 'X'
AND wa_intab-field_one = const_one.
SELECT *
FROM db_table
WHERE field_two = wa_intab-field_two.
MOVE-CORRESPONDING db_table TO wa_outtab.
APPEND wa_outtab TO lt_outtab.
CLEAR wa_outtab.
ENDSELECT.
ELSEIF wa_intab-field_one = const_two.
SELECT *
FROM db_table
WHERE field_three = wa_intab-field_three.
MOVE-CORRESPONDING db_table TO wa_outtab.
APPEND wa_outtab TO lt_outtab.
CLEAR wa_outtab.
ENDSELECT.
ENDIF.
ENDLOOP.
Is the optimized code a better approach of looping through similar tables?
Assuming field_one isn't a accessible via any sorted or hashed key:
Original code
Param = 'X'
n + n loop iterations
Param = ''
n loop iterations
Optimized code
Always n loop iterations
You can basically already aggregate it to WHERE field_one = const_one OR field_one = const_two.
But I'd expect much more performance by using FOR ALL ENTRIES and also using LOOP AT ASSIGNING instead. So group the keys first and then hit the database with a list of values for db_table~field_two and field_two~field_three.
Would be interesting to know of much db time you spent. Transaction SAT and the ABAP Profiler in ADT can also show you time spent on internal tables itself and database.
According to peterulb, your problem isn't the loop in the first place. Normally, especially with a hana database, you want to access only the data you really need and read it from the database. There are many ways to get the best values, like FOR ALL ENTRIES or, if you want to get the entries from more than one database table, you could use UNION statement to connect to select statements. So you could possibly ignore LOOP AT at all.
Related
I have a solution that includes a LOOP which I would like to spare. So I wonder, whether you know a better way to do this.
My goal is to loop through an internal, alphabetically sorted standard table. This table has two columns: a name and a table, let's call it subtable. For every subtable I want to do some stuff (open an xml page in my xml framework).
Now, every subtable has a corresponding name. I want to group the subtables according to the first letter of this name (meaning, put the pages of these subtables on one main page -one main page for every character-). By grouping of subtables I mean, while looping through the table, I want to deal with the subtables differently according to the first letter of their name.
So far I came up with the following solution:
TYPES: BEGIN OF l_str_tables_extra,
first_letter(1) TYPE c,
name TYPE string,
subtable TYPE REF TO if_table,
END OF l_str_tables_extra.
DATA: ls_tables_extra TYPE l_str_tables_extra.
DATA: lt_tables_extra TYPE TABLE OF l_str_tables_extra.
FIELD-SYMBOLS: <ls_tables> TYPE str_table."Like LINE OF lt_tables.
FIELD-SYMBOLS: <ls_tables_extra> TYPE l_str_tables_extra.
*"--- PROCESSING LOGIC ------------------------------------------------
SORT lt_tables ASCENDING BY name.
"Add first letter column in order to use 'at new' later on
"This is the loop I would like to spare
LOOP AT lt_tables ASSIGNING <ls_tables>.
ls_tables_extra-first_letter = <ls_tables>-name+0(1). "new column
ls_tables_extra-name = <ls_tables>-name.
ls_tables_extra-subtable = <ls_tables>-subtable.
APPEND ls_tables_extra TO lt_tables_extra.
ENDLOOP.
LOOP AT lt_tables_extra ASSIGNING <ls_tables_extra>.
AT NEW first_letter.
"Do something with subtables with same first_letter.
ENDAT.
ENDLOOP.
I wish I could use
AT NEW name+0(1)
instead of
AT NEW first_letter
, but offsets and lengths are not allowed.
You see, I have to inlcude this first loop to add another column to my table which is kind of unnecessary because there is no new info gained.
In addition, I am interested in other solutions because I get into trouble with the framework later on for different reasons. A different way to do this might help me out there, too.
I am happy to hear any thoughts about this! I could not find anything related to this here on stackoverflow, but I might have used not optimal search terms ;)
Maybe the GROUP BY addition on LOOP could help you in this case:
LOOP AT i_tables
INTO DATA(wa_line)
" group lines by condition
GROUP BY (
" substring() because normal offset would be evaluated immediately
name = substring( val = wa_line-name len = 1 )
) INTO DATA(o_group).
" begin of loop over all tables starting with o_group-name(1)
" loop over group object which contains
LOOP AT GROUP o_group
ASSIGNING FIELD-SYMBOL(<fs_table>).
" <fs_table> contains your table
ENDLOOP.
" end of loop
ENDLOOP.
why not using a IF comparison?
data: lf_prev_first_letter(1) type c.
loop at lt_table assigning <ls_table>.
if <ls_table>-name(1) <> lf_prev_first_letter. "=AT NEW
"do something
lf_prev_first_letter = <ls_table>-name(1).
endif.
endloop.
I'm using Google-Appengine-NDB. And I'm tried to get distinct values from database but it's not working.
Now my code is:
query_set = cls.query().order(cls.ls) # Getting ordered queries.
set_of_field = set([data.field for data in query_set]) # And using this loop for differ.`
But the loop is taking too long time (over 12 sec).
Please help me, how can I speed up, or how to get distinct values from ndb?
Try the distinct query,
if your field is indexed you can use this:
https://developers.google.com/appengine/docs/python/ndb/queries#projection
query_set = cls.query(projection=["field"], distinct=True)
set_of_field = [data.field for data in query_set]
But if you have a huge list you can either do this in a taskqueue and store the results somewhere or just keep a distinct data in another model.
I'm using the lsqlite3 lua wrapper and I'm making queries into a database. My DB has ~5million rows and the code I'm using to retrieve rows is akin to:
db = lsqlite3.open('mydb')
local temp = {}
local sql = "SELECT A,B FROM tab where FOO=BAR ORDER BY A DESC LIMIT N"
for row in db:nrows(sql) do temp[row['key']] = row['col1'] end
As you can see I'm trying to get the top N rows sorted in descending order by FOO (I want to get the top rows and then apply the LIMIT not the other way around). I indexed the column A but it doesn't seem to make much of a difference. How can I make this faster?
You need to index the column on which you filter (i.e. with the WHERE clause). THe reason is that ORDER BY comes into play after filtering, not the other way around.
So you probably should create an index on FOO.
Can you post your table schema?
UPDATE
Also you can increase the sqlite cache, e.g.:
PRAGMA cache_size=100000
You can adjust this depending on the memory available and the size of your database.
UPDATE 2
I you want to have a better understanding of how your query is handled by sqlite, you can ask it to provide you with the query plan:
http://www.sqlite.org/eqp.html
UPDATE 3
I did not understand your context properly with my initial answer. If you are to ORDER BY on some large data set, you probably want to use that index, not the previous one, so you can tell sqlite to not use the index on FOO this way:
SELECT a, b FROM foo WHERE +a > 30 ORDER BY b
After much googling and console testing I need some help with arrays in rails. In a method I do a search in the db for all rows matching a certain requirement and put them in a variable. Next I want to call each on that array and loop through it. My problem is that sometimes only one row is matched in the initial search and .each causes a nomethoderror.
I called class on both situations, where there are multiple rows and only one row. When there are multiple rows the variable I dump them into is of the class array. If there is only one row, it is the class of the model.
How can I have an each loop that won't break when there's only one instance of an object in my search? I could hack something together with lots of conditional code, but I feel like I'm not seeing something really simple here.
Thanks!
Requested Code Below
#user = User.new(params[:user])
if #user.save
#scan the invites dbtable and if user email is present, add the new uid to the table
#talentInvites = TalentInvitation.find_by_email(#user.email)
unless #talentInvites.nil?
#talentInvites.each do |tiv|
tiv.update_attribute(:user_id, #user.id)
end
end
....more code...
Use find_all_by_email, it will always return an array, even empty.
#user = User.new(params[:user])
if #user.save
#scan the invites dbtable and if user email is present, add the new uid to the table
#talentInvites = TalentInvitation.find_all_by_email(#user.email)
unless #talentInvites.empty?
#talentInvites.each do |tiv|
tiv.update_attribute(:user_id, #user.id)
end
end
I have been thinking about this problem for long enough but cannot come up with something satisfactory. hope someone can help me out.
I am writing a data entry (database) program that accepts a large number of input (strings, numbers and relationships). The objective of this program is to take all the inputs and run them against a list of conditions, and where a condition is true, it will return a set of outputs (item IDs) that will go into the final report.
My current implementation is as such (simplified): I have a "Trigger" table with the following columns:
1.ITEM_SET (e.g. 001;105;112)
2.TABLE(e.g. [Dimension] INNER JOIN [Product] ON [id])
3.CONDITION (e.g. [Dimension].[width] > 20 AND [Product].[color] = "Red")
Then I will go
triggers = SELECT * FROM Trigger
foreach(trigger in Triggers)
{
items = SELECT trigger.ITEM_SET FROM trigger.TABLE WHERE trigger.CONDITION
if(have items) report.items.add(items)
}
This certainly is a pain to enter every (almost 5000) conditions... I am looking for a simpler way to do this such that it is easily extensible, by the user themselves. Any ideas?
While it does not totally solve all the associated problems i had, I modified the circumstances and http://ncalc.codeplex.com/ was able to fill the holes. Thanks NCalc!