All of those disadvantages of stored procedures (database portability etc.) I am now facing. We are going to migrate our VB.Net/ASP/SQL Server application to something like Mono/Postgresql and fully internationalize.
One of the many issues we face is that we have 800-900 stored procedures. What we are thinking of doing is moving the logic in these SP's in to the application code. I have searched around for how people have done this or how it can be done however I have found very little. Most of the info on stored procedures is about what should be in them etc. So I have come to the conclusion that the best way to approach this is to leave the SQL retrieve/update type queries as stored procedures and move the ones with business logic in to the application.
So my question is what is the best approach to take in doing something like this (there may not be a best approach but some suggestions of where to start would be good)?
Thanks
Liam
Since your stored procedures provide an interface to your database, you now need to ensure that all the calls to the stored procedure are going to the same client code. If you currently call the SP from several places in the application, or even multiple applications (possibly multi-modal), they will all need to go through common code. I would start by ensuring that each SP is called from only one place in your client-side library. Then all that logic from the stored proc is going to be encapsulated by that method call. If your stored procs use transactions to ensure integrity for complex operations, those transactions will now need to be initiated and committed in the application-side library.
That refactoring is ultimately necessary, and there are benefits to that even if you aren't able to port all the SPs.
I also encountered same situation regarding migrating an ASP.NET+ SQL Server application to Postgres database.
Instead of migrating SP to your application code it would be lot easier to convert them to equivalent pl/pgsql functions.
All you will need to do it to change the ADO.NET provider and you wont need any change in Application code. There are plenty of providers for postgres (npgsql) which treat SP and postgres functions as equivalent.
Related
I recently completed a SSIS course.
One of the piece of best practice I came away with, was to ALWAYS use stored procedures in data flow tasks in SSIS.
I guess there is an argument around security, however the tutor said that as the stored procedures performed all of the work "native" on the SQL Server there was/is a significant performance boost.
Is there any truth to this or articles that debate the point?
Thanks
Remember - mostly courses are done by clueless people because people with knowledge earn money doing consulting which pays a LOT better than training. Most trainers live in a glass house that never spends 9 months working on a 21tb data warehouse ;)
This is wrong. Point.
It only makes sense when the SQL Statement does not pull data out of the database - for example merging tables etc.
Otherwise it is a question of how smart you set up the SSIS side. SSIS can write data not using SQL, using bulk copy mechanisms. SSIS is a lot more flexible, and if you pull data from a remote database then the argument of not leaving the database (i.e. processing native) is a stupid point to make. When I copy data from SQL Server A to SQL Server B, a SP on B can not process he data from A native.
In general, it is only faster when you take data FROM A and push it TO A and all the processing can be done in a simple SP - which is a degenerate edge case (i.e. a simplistic one).
The advantage of SSIS is the flexibility of processing data in an environment designed for data flow, which in many cases is needed in the project and doing that in stored procedures would turn nightmare.
Old thread, but a pertinent topic.
For a data source connection, I favor SPs over embedded queries when A) the logic is simple enough to be handled in both ways, and B) the support of the SP is easier than working with the package.
I haven't found much, if any, difference in performance for the data source if the SP returns a fairly straighforward result set.
Our shop has a more involved deploy process for packages, which makes SPs a preferred source.
I have not found very many applications for a SP being a data destination, except maybe an occasional logging SP call.
I often use stored procedure for data access purpose but don't know which one is best view or SP.
Stored procedure and views are both compiled and execution plan is saved in database. So please tell me which one is best for data access purpose and why best list down the reason please.
I search google to know which one is best but got no expected answer.
I disagree with Jared Harding when it comes to stored procedures causing application updates to be more difficult. It's much easier to update a stored procedure than to update application code, which might need to be recompiled and require you to kick users out of the system to make an update. If you write all your SQL in stored procedures, you'll probably only have to update the application half as often as you otherwise would. And a stored procedure can be updated with no disruption to the users.
I strongly recommend use of stored procedures over views or SQL written within application code. With parameterized stored procedures that use dynamic SQL built as a string and called with the secure sp_executesql function, you can write a single stored procedure for selecting data that can be used across multiple applications and reports. There is virtually no need to use a view unless you really need to block permissions on underlying tables. It's much better to create your basic SELECT query inside a stored procedure and then use parameter options to change how the results are filtered. For example, each parameter can add a different line to your WHERE clause. By passing no parameters, you get back the full unfiltered recordset. As soon as you need to filter your results by a different field, you only need to add one parameter and one line of code in that procedure, and then have any application that needs to make use of it pass in the parameter. By defaulting your stored procedure parameters to null, you don't have to change any applications which are calling that stored procedure for other purposes.
Views and stored procedures serve entirely different purposes. Views are a convinient way to refer to a complex relational set (such as one that joins across many tables) as a flat table without actually forcing the data to be manifested. You use a view to clean up SQL code. Your stored procedures could call views. Views are often used for permission control. You can grant a database user access to a view without granting them access to the underlying tables. This grants the user column level permissions on the columns in the view which is a far more granular method for permission control than granting access to whole tables.
Stored procedures are used to keep often used functionality together as a unit. To be honest, SPs are falling out of favor among many programmers. While you are correct that SPs have their execution plans cached, dynamic SQL has had execution plan caching since SQL Server 2000 (I believe that's the correct version). The only speed gain you're going to get by going with SPs is by sending less data over the network, and that's going to be extremely minimal. SPs tend to make code more brittle and require changes to the DB to occur when application changes don't really warrant it. For example, if you just wanted to change the conditions for which rows you're selecting. Using SPs, you're going to have to roll changes out to the application and the database code. If you're using dynamic SQL or an ORM tool, you only need to make changes to the application which simplifies deployment. There is absolutely a time and place for SPs, but they don't need to be your only method for interacting with the database.
Also, if you're worried about performance, you can materialize views which reduces the need to repeatedly query the underlying tables. This could greatly enhance your performance if you feel the need to add the extra overhead on inserts/updates that materializing views induces.
To speed up the query you need properly defined indexes on the table. Within a stored procedure you can use paramteres, implement your own logic, however within a view you cannot
Because: Once procedure is compiled it makes its execution plan and use same for every time we call it even when we insert new data in related table as well, untill we make any change in procedure code.
View check for new updated data every time you call it.
You can do whole transaction handling etc with SP.
in my recent subject ,I have to do some queries through dynamic SQL,But I'm curious about
the efficiency in different ways:
1)combine the sql sentences in my server and then send them to the database ,do the query
2)send my variables to database and combine them in a certain procedure and finally do the query
Hope someone can help
BTW(I use .Net and Sqlserver)
Firstly, one of the main things you should do is to parameterise your SQL - whether that be by wrapping it up as a stored procedure in the DB, or by creating the SQL statement in your application code and then firing the whole thing in to the DB. This will mean:
prevention against SQL injection attacks by not directly concatenating user-entered values into a SQL statement
execution plan reuse (subsequent executions of that query, regardless of parameter values, will be able to reuse the original execution plan) (NB. this could be done if not parameterised yourself, via Forced Parameterisation)
Stored procedures do offer some extra advantages:
security ,only need to grant EXECUTE permissions to the stored procedures, you don't need to grant the user direct access to underlying db tables
maintainability, a change to a query does not involve an application code change, you can just change the sproc in the DB
network traffic, not necessarily a major point but you're sending less over the wire especially if the query is pretty large/complex
Personally, I use stored procedures most of the time. Though the times I need to build up SQL dynamically in application code, it is always parameterised.
Best is to use stored procedure and pass parameters from your application, as Stored procedures are precompiled queries and have execution plan ready which saves lot of time.
You can refer this url which has details http://mukund.wordpress.com/2005/10/14/advantages-and-disadvantages-of-stored-procedure/
Happy coding!!
I have a web application where the web server and SQL Server 2008 database sit on different boxes in the same server farm.
If I take a monolithic stored procedure and break it up into several smaller stored procs, thus making the client code responsible for calls to multiple stored procedures instead of just one, and I going to notice a significant performance hit in my web application?
Additional Background Info:
I have a stored procedure with several hundred lines of code containing decision logic, update statements, and finally a select statement that returns a set of data to the client.
I need to insert a piece of functionality into my client code (in this sense, the client code is the ASP web server that is calling the database server) that calls a component DLL. However, the stored procedure is updating a recordset and returning the udpated data in the same call, and my code ideally needs to be called after the decision logic and update statements are called, but before the data is returned to the client.
To get this functionality to work, I'm probably going to have to split the existing stored proc into at least two parts: one stored proc that updates the database and another that retrieves data from the database. I would then insert my new code between these stored proc calls.
When I look at this problem, I can't help but think that, from a code maintenance point of view, it would be much better to isolate all of my update and select statements into thin stored procs and leave the business logic to the client code. That way whenever I need to insert functionality or decision logic into my client code, all I need to do is change the client code instead of modifying a huge stored proc.
Although using thin stored procs might be better from a code maintenance point-of-view, how much performance pain will I experience by increasing the number of trips to the database? The net result to the data is the same, but I'm touching the database more frequently. How does this approach affect performance when the application is scaled up to handle demand?
I'm not one to place performance optimization above everything else, especially when it affects code maintenance, but I don't want to shoot myself in the foot and create headaches when the web application has to scale.
in general, as a rule of thumb you should make roundtrips to SQL server at a minimum.
the "hit" on the server is very expensive, it's actually more expensive to devide the same operation into 3 parts then doing 1 hit and everything else on the server.
regarding maintenance, you can call 1 stored proc from the client, having that proc call another 2 proc's.
I had an application with extreme search logic, thats what I did to implement it.
some benchmarking results...
I had a client a while back that had servers falling and crumbling down, when we checked for the problem it was many roundtrips to SQL server, when we minimized it, the servers got back to normal.
It will affect it. We use a Weblogic server where all the business logic is in the AppServer connected to a DB/2 database. We mostly use entity beans in our project and for most business service calls make several trips to the DB with no visible side effects. (We do tune some queries to be multi-table when needed).
It really depends on your app. You are going to need to benchmark.
A well setup SQL Server on good hardware can process many thousands of transactions per second.
In fact breaking up a large stored procedure can be beneficial, because you can only have one cached query plan per batch. Breaking into several batches means they will each get their own query plan.
You should definitely err on the side of code-maintenance, but benchmark to be sure.
Given that the query plan landscape will chnage, you should alos be prepared to update your indexes, perhaps creating different covering indexes.
In essence, this question is closely related to tight vs. loose coupling.
At the outset: You could always take the monolithic stored procedure and break it up into several smaller stored procs, that are all called by one stored procedure, thus making the client code only responsible for calling one stored procedure.
Unless the client will do something (change the data or provide status to user) I would probably not recommend moving multiple calls to the client, since you would be more tightly coupling the client to the order of operations for the stored procedure without a significant performance increase.
Either way, I would benchmark it and adjust from there.
Lets say on MySQL database (if it matters).
No, you will not be completely safe. As others have mentioned, parameterized queries are always the way to go -- no matter how you're accessing the database.
It's a bit of an urban legend that with procs you're safe. I think the reason people are under this delusion is because most people assume that you'll call the procs with parameterized queries from your code. But if you don't, if for example you do something like the below, you're wide open:
SqlCommand cmd = new SqlCommand("exec #myProc " + paramValue, con);
cmd.ExecuteNonQuery();
Because you're using unfiltered content from the end user. Once again, all they have to do is terminate the line (";"), add their dangerous commands, and boom -- you're hosed.
(As an aside, if you're on the web, don't take unfiltered junk from the query string of the browser -- that makes it absurdly easy to do extremely bad things to your data.)
If you parameterize the queries, you're in much better shape. However, as others here have mentioned, if your proc is still generating dynamic SQL and executing that, there may still be issues.
I should note that I'm not anti-proc. Procs can be very helpful for solving certain problems with data access. But procs are not a "silver-bullet solution to SQL injections.
You are only immune to SQL injections if you consistenly use parameterized queries. You are nearly immune to SQL injections if you use proper escaping everywhere (but there can be, and has been, bugs in the escaping routines, so it's not as foolproof as parameters).
If you call a stored procedure, adding the arguments by concatenation, I can still add a random query at the end of one of the input fields - for example, if you have CALL CheckLogin #username='$username', #password='$password', with the $-things representing directly concatenated variables, nothing stops me from changing the $password variable to read "'; DROP DATABASE; --".
Obviously, if you clean up the input beforehand, this also contributes to preventing SQL injection, but this can potentially filter out data that shouldn't have been cleaned.
It depends what your stored procs do. If they dynamically generate SQL based on their parameters, and then execute that SQL, then you're still vulnerable. Otherwise, you're far more likely to be fine - but I hesitate to sound 100% confident!
nope. If you're constructing SQL that invokes a stored procedure you're still a target.
You should be creating parametized queries on the client side.
No, as you could still use D-SQL in your stored procedures... and validating and restricting your input is a good idea in any case.
Stored Procedures are not a guarantee, because what is actually vulnerable is any dynamic code, and that includes code inside stored procedures and dynamically generated calls to stored procedures.
Parameterized queries and stored procs called with parameters are both invulnerable to injection as long as they don't use arbitrary inputs to generate code. Note that there is plenty of dynamic code which is also not vulnerable to injection (for instance integer parameters in dynamic code).
The benefits of a largely (I'm not sure 100% is really possible) stored procs-based architecture, however, is that injection can even be somewhat defended against (but not perfectly) for dynamic code at the client side because:
Only EXEC permissions are granted to any user context the app is connecting under, so any SELECT, INSERT, UPDATE, DELETE queries will simply fail. Of course, DROP etc should not be allowed anyway. So any injection would have to be in the form of EXEC, so ultimately, only operations which you have defined in your SP layer will even be available (not arbitrary SQL) to inject against.
Amongst the many other benefits of defining your database services as a set of stored procedures (like any abstraction layer in software) are the ability to refactor your database underneath without affecting apps, the ability to better understand and monitor the usage patterns in your database with a profiler, and the ability to selectively optimize within the database without having to deploy new clients.
Additionally, consider using fine grained database access, (also called generally Role Based Access Control) The main user of your database should have exactly the permissions needed to do its job and nothing else. Don't need to create new tables after install? REVOKE that permission. Don't have a legitimate need to run as sysdba? Then don't! A sneaky injection instructing the user to "DROP DATABASE" will be stymied if the user has not been GRANTed that permission. Then all you need to worry about is data-leaking SELECT statements.