I know, I know... There's no way to safely do this and I should always be using parameterized queries, but here's my situation.
I'm attempting to use Advanced Datagridview (https://github.com/davidegironi/advanceddatagridview) in a VB.Net forms application with a large dataset. The gridview control has Excel like filtering that's quite nifty but the filtering is only done on the client side so I'd need to pull down the entire table to accurately filter. This isn't practical so I need a way to pass the filtering to the server. The control is able to generate a SQL-formatted string of it's current filter, such as:
([Plant] IN ('Argile - Chester', 'Bartlett - Riverton')) AND
([VendorNum] IN ('1015')) AND
([Qty] < 22.12)
So it would be quite easy just to concatenate that onto the query populating the grid view, but that would obviously be wide open to injection attacks. Is there a way I can safely validate the string being supplied to check for any potential issues? Or can anyone think of some other way of solving this problem given my constraints?
Only other option I can think of is doing away with the built-in filtering and just using form controls to set the filter values and pass those as parameters, but I really want to make use of the built-in filtering. Another option would be to try and parse out the filter string the gridview generates and break it into individual fields to pass to parameters. But there could potentially be several fields being filtered on and numerous values. This would get dicey fast.
Thanks in advance.
Related
I believe this question is mostly related to SQL, but I will give context for what we are trying to do in case someone has done it before.
We are trying to implement DataTables JQuery plug-in with server-side processing using SQL Server for our ASP .NET Core MVC app. On top of that, we have dynamic columns, which we were able to generate for DataTables using this github code.
We have been successful in handling paging, entry #, and column searching processes (couldn't figure out global search, and we got column search to work only with SQL views). However, sorting columns are only handled partially.
The issue with column sorting is that we don't know how many and what type of columns could be returned. This means that there may be an NVARCHAR column containing strings that are currency ($3,403.09), or NVARCHARs that are regular numbers, and some of the values in those columns may be NULL. Same with dates with varying formats. We don't know how to handle sorting these NVARCHARs in SQL as the correct type instead of a string. Having said that, client-side DataTables is able to deal with identifying how data should be sorted.
I don't expect an answer, but if anyone has handled a similar situation, I'd greatly appreciate the help.
I want auto search option in textbox and data is fetching from database. I have thousands of data in my database table (almost 8-10000 rows). I know how to achieve this but as I am fetching thousands of data, it will take a lot of time to fetch. How to achieve this without getting slow down? Should I follow any other methodology to achieve this apart from simple fetching methods? I am using Oracle SQL Developer for database.
Besides the obvious solutions involving indexes and caching, if this is web technology and depending on your tool you can sometimes set a minimum length before the server call is made. Here is a jquery UI example: https://api.jqueryui.com/autocomplete/#option-minLength
"The minimum number of characters a user must type before a search is performed. Zero is useful for local data with just a few items, but a higher value should be used when a single character search could match a few thousand items."
It depends on your web interface, but you can use two tecniques:
Paginate your data: if your requirements are to accept empty values and to show all the results load them in block of a predefined size. goggle for example paginates search results. On Oracle pagination is made using the rownum special variable (see this response). Beware: you must first issue a query with a order by and then enclose it in a new one that use rownum. Other databases that use the limit keyword behave in a different way. If you apply the pagination techique to a drop down you end up with an infinite scroll (see this response for example)
Limit you data imposing some filter that limits the number of rows returned; your search display some results only after the user typed at least n chars in the field
You can combine 1 & 2, but unless you find an existing web component (a jquery one for example) it may be a difficult task if you don't have a Javascript knowledge.
I have a table where am storing a startingDate in a DateTime column.
Once i have the startingDate value, am supposed to calculate the
number_of_days,
number_of_weeks
number_of_months and
number_of_years
all from the startingDate to the current date.
If you are going to use these values in two or more places in the application and you do care much about the applications response time, would you rather make the calculations in a view or create computed columns for each so you can query the table directly?
Computed columns are easy to maintain and provide an ideal solution to your problem – I have used such a solution recently. However, be aware the values are calculated when requested (when they are SELECTed), not when the row is INSERTed into the table – so performance might still be an issue. This might be acceptable if you can off-load work from the application server to the database server. Views also don’t exist until they are requested (unless they are materialised) so, again, there will be an overhead at runtime, but, again it’s on the database server, not the application server.
Like nearly everything: It depends.
As #RedX suggest it probably not much of a performance difference either way, so it becomes a question of how will use them. To me this is more of a feel thing.
Using them more than once doesn't wouldn't necessary drive me immediately to either a view or computed columns. If I only use them in a few places or low volume code paths I might calc them in-line in those places or use a CTE. But if the are in wide spread or heavy use I would agree with a view or computed column.
You would also want them in a view or cc if you want them available via ORM tools.
Am I using those "computed columns" individual in places or am I using them in sets? If using them in sets I probably want a view of the table that shows included them all.
When i need them do I usually want them associated with data from a particular other table? If so that would suggest a view.
Am I basing updates on the original table of those computed values? If so then I want computed columns to avoid joining the view in these case.
Calculated columns may seem an easy solution at first, but I have seen companies have trouble with them because when they try to do ETL with CDC for real-time Change Data Capture with tools like Attunity it will not recognize the calculated columns since the values are not there permanently. So there are some issues. Also if the columns will be retrieve many, many times by users, you will save time in the long run by putting that logic in the ETL tool or procedure and write it once to the database instead of calculating it many times for each request.
I have some data (about 1000 rows) that I want to display. I want the user to be able to filter and sort the data dynamicly. Also I have to be able to save and load the data. I am relativly new to databases and need your help deciding which approach is the best/most resonable:
1st variant:
This is my current approach. I am using a ListView/GridView that is bound to a ObservableCollection of RowItems. I can now use the View to sort and filter. Saving and loading is done via Serialisation.
2nd variant:
Use of a database to store the data. The data is then loaded into the same business objects as above.
3rd variant:
Bind the ListView directly to a database. I just found out that this is somehow possible and am still somewhat hazy on the details. Especially on the aspect of sorting and filtering (Could that be done via a database Query?)
4th variant:
Store data in database. Use ObservableCollection of bussines objects for binding. But filter (and sort?) via SQL query.
These are the ideas I have to fullfilling my requirements. I'd like to know which one is the best/easiest/best performed/etc. or if you suggest an other approach.enter code here
Peter, there are couple of points you need to consider:
Where possible never make tight integration between back end data and display. Makes it easier to change either later on. you can use NHibernate or other ORM to make it easier to represent and read/write data from database. That way you have layer/tier in between them (aka n-tier)
you will only need to make call to DB for reading the data, it is 1000 rows it is not too bad, you can read them in one go and keep it in memory. Any filtering/sorting/grouping that you can do it on the in memory DomainModel/Buisness Object without touching the library.
i would use mixture of variant 1) and 2).
I would use business object to represent your data from Database (NHibernate is quite handy). I would then perform all data filtering/sorting on the client without requering DB each time. Any writes to the database must go through some sort of Domain Model layer or ORM so that they are not tighly linked. That allows to me to chnage DB without effecting the GUI Front end
I need to find a relatively robust method of storing variable types data in a single column of a database table. The data may represent a single value or multiple values and may any of a long list of characters (too long to enumerate easily). I'm wondering what approaches might work in this process. I'd toyed with the ideas of adding some form of separator, but I'm worried that any simple separator or combination might occur naturally in the data. I'd also like to avoid XML or snippets since in fact the data could be XML. Arguably I could encode the XML, but that still seems fragile.
I realize this is naturally a bit of an opinion question, but I lack the mojo to make it community.
Edit for Clarification:
Background for the problem: the column will hold data that is then used to make a evaluation based on another column. Functionally it's the test criteria for a decision engine. Other columns hold the evaluation's nature and the source of the value to test. The data doesn't need to be searchable.
Does the data need to be searchable? If not, slap it in a varbinary(MAX) and have a field to assist in deserialization.
Incidentally, though; using the right XML API, there should be no trouble storing XML inside an XML node.
But my guess is there has to be a better way to do this... it seems... ugh!
JSON format, though I agree with djacobson, your question is like asking for the best way to saw a 2x4 in half with a teaspoon.
EDIT: The order in which data are stored in the JSON string is irrelevant; each datum is stored as a key-value pair.
There's not a "good" way to do this. There is a reason that data types exist in SQL.
The only conceivable way I can think of to make it close is to make your column a lookup column, which refers to a GUID or ID in another table, which itself has additional columns indicating which table and row have your data.