A SQL Server Language Extension function is executed in an external process. Does it mean that when such a function is called in a Select clause it creates a new process for every row in the recordset on which it is applied?
I think you are confusing two different features / technologies.
SQLCLR is the ability to run .NET code (most often C# or VB.NET, but sometimes Visual C++ and occasionally F#, though F# is not officially supported) within the SQL Server process. This code can be called as stored procedures, triggers, scalar functions, table-valued functions, aggregate functions, and even user-defined types. This has the ability to execute T-SQL within the calling session, even if there's an active transaction (i.e. true in-process execution).
External Scripts (language extensions) is newer than SQLCLR and is not .NET-specific. Languages that can be used are: R, Python, Java, and most recently C#. This code can only be executed via the sp_execute_external_script stored procedure; there is not option to call these external scripts via a function. Hence, there is no concept of per-row execution. These scripts are executed by a separate service (i.e. external), hence there is no option for in-process T-SQL code execution (meaning: executing T-SQL via an external script will connect as a separate session).
Additional details can be found in my post:
SQLCLR vs SQL Server 2017, Part 8: Is SQLCLR Deprecated in Favor of Python or R (sp_execute_external_script)?
Related
What are the main differences between connecting our RoR application to Snowflake using the ODBC driver vs SQL API?
The main use case is for read only access to run various custom queries against a few tables.
We've prototyped both connections. Both work well. ODBC appears to be faster when running simple queries.
One use case is to execute ~10 queries in one request. ODBC requires us to execute 10 separate SQL statements. While the SQL API allows us to submit the queries together, but then requires an additional API call for each statementHandle to get the results. The API calls are fast, but that's still 11 API calls.
Is ODBC the obvious choice here? What if ~10 queries grows to 50-100? What if the result set is 50-100k+ rows of data? I do see how SQL API partitions the results. That might come in handy. Not sure how ODBC handles that offhand.
Other thoughts on security, performance, etc to think about?
Thanks!
First, it is possible to send a batch of statements with ODBC a single request: Executing a Batch of SQL Statements (Multi-Statement Support)
Other than that there are a few differences that might not matter for your use case. For example, the ODBC driver returns results in the Apache Arrow format. While the REST API will return the first partition of the results in JSON and subsequent partitions in gzipped JSON.
The rest api has some limitations largely based on the fact the REST API cannot maintain a session across requests and a few other quirks like not being able to PUT files into a stage (including unstructured).
The ODBC driver (and other official connectors) will always have access to any command on Snowflake because they can maintain a session. So it will ultimately come down to personal preference and what your app needs to do.
Another thing to consider is the ODBC driver has years of development and the SQL API is relatively new. (Though by all accounts, works great)
Limitations of the SQL API from the docs.
The following commands are not supported:
The PUT command (in Snowflake SQL)
The GET command (in Snowflake SQL)
The CALL command with stored procedures that return a table (stored procedures with the RETURNS TABLE clause).
The following commands and statements are supported only within a request that specifies multiple statements:
Commands that perform explicit transactions, including:
BEGIN
COMMIT
ROLLBACK
Commands that change the context of the session, including:
USE
ALTER SESSION
Statements that set session variables.
Statements that create temporary tables and stages (tables and stages that are available only in the current session).
As it is possible to query SQL Server both with Visual C# and TSQL, I want to know are they all equal in terms of operators and features available, or TSQL has more features which cannot be implemented using Visual C#(kind of administrative operators which do not have equivalent in C#)?
As it is possible to query SQL Server both with Visual C# and TSQL
No, it is not possible to query SQL Server with C# (the "Visual" part was dropped a long time ago and is not used anymore; it's difficult to even find references to "Visual C#" at this point). T-SQL is the only language that queries SQL Server.
There are wrappers that provide a higher-level interface to T-SQL, such as SMO, LINQ to SQL, and Entity Framework. Each of those options merely mask the T-SQL that is submitted to accomplish the operation represented in that particular form. This is similar, in a sense, to SQL Server Management Studio (SSMS), Azure Data Studio, Server Explorer in Visual Studio, or any other GUI allowing you to interact with SQL Server without you needing to remember and type the T-SQL yourself. But, at least in SSMS, there is a "Script" button at the top of all (or most) windows that will show you the T-SQL that will be submitted if you click the "OK" button in whatever window you are in. Even getting a list of databases, or objects (tables, views, stored procedures,etc) to populate Object Explorer / Server Explorer are done via SELECT statements.
You can use SQL Server Profiler or Extended Events to see all of the T-SQL submitted by these GUIs and your app code. LINQ to SQL and Entity Framework use sp_executesql for queries with variables / parameters, and that might show up as an RPC call.
So no, none of those wrapper options implement 100% of the functionality of T-SQL. Nor do they allow you do determine how to structure the query if there are choices (e.g. NOT IN vs NOT EXISTS(), etc). These wrappers provide convenience (to .NET developers), not flexibility.
Can somebody please explain the various scenarios that would make one choose SQLCLR vs Python vs R.
I understand that R is a language and a library specifically designed for statistical analysis and data mining so I understand leveraging that capability when appropriate, but can R (on SQL Server) do more and call additional external libraries like CLR assemblies can?
Is Python meant as an eventual replacement for C# SQLCLR? It seems to me, from what I've read, that Python can simply be embedded inside a stored procedure and then interpreted upon execution as opposed the compiled nature of CLR assemblies, but otherwise the capabilities are the same? Are they?
I'll try to answer your questions:
SQLCLR was introduced in SQL Server 2005, as a way to embed CLR (.NET) in the SQL Server engine. E.g with SQLCLR your .NET code is running in the same memory and process space as SQL Server itself. The way it works (simplified) is that you create an assembly and registers it with SQL Server (CREATE ASSEMBLY). You then create "wrapper" T-SQL stored procedures/functions/triggers etc., against your .NET methods, and it is these procs that you execute at runtime.
R was introduced in SQL Server 2016, and Python in SQL Server 2017 in order to give SQL Server machine learning capabilities. As opposed to .NET, neither R nor Python run embedded in SQL Server, but when you call R/Python code inside SQL Server, calls are made out to the R/Python engine sitting outside SQL Server's memory/process space. This is an important distinction between SQLCLR and R/Python:
SQLCLR code executes in-memory/in-process with SQL Server
R/Python executes outside of SQL Server.
As a side note; I have a series of blog-posts discussing the internals of SQL Server R Services (even if the posts talk about R, everything in there are applicable to Python as well).
As for capabilities; R/Python in SQL Server can do no more, no less than what "standalone" R/Python can do: as mentioned above the actual execution of R/Python happens outside of SQL Server as well.
Personally I do not think Python is a replacement for .NET in SQL Server, I see it as an additional tool in your toolbox. Where I work we use both SQLCLR as well as R/Python (in SQL Server). We have 100's of SQLCLR assemblies in our production databases, doing weird and wonderful things (sending messages to RabbitMQ etc., etc.), and IMHO it'd be very hard to replace that with Python, especially seeing that you'd immediately get a perf degradation - compiled code (SQLCLR) vs. interpreted code (R/Python).
Hope this helps.
Niels
I would like to use existing Delphi code to write SQL Server stored procedures.
In the past I used extended stored procedures, somehow a dll compiled by Delphi wrapped by a SQL Server extended stored procedure.
Extended stored procedures are now deprecated, so somehow I wonder if there is a solution, even in the "trick domain", like some sample CLR code that wraps a normal dll or something like that.
Just to give you an example:
I insert in the db some documents by encrypting them and I would like to create a kind of API based on SQL Server functions / procedures for inserting or reading documents, so other people accessing sql server can call those functions.
Of course an alternative solution is to use webservices but I would like to try the SQL Server way first.
Note: I don't want to use Delphi Prism, my code is for XE2.
Unsafe SQLCLR assemblies can p-invoke native dlls. A better way would be to expose the native DLL services as a COM interface and use COM interop from SQLCLR, or even call the COM APIs directly from SQL via OLE Automation Procedures. An even better way would be to rewrite the Delphi code as CLR code and invoke it directly as SQLCLR procedure. And the best way would be to use SQL Server native encryption features.
Not to mention the fact that CLR in SQL Server is a guaranteed deep performance hit. Keep to the standard CRUD operators and you should be fine. The other way to do it is to use the file system as your encryption mechanism. If you are only trying to prevent casual access to docs this is a fine way to go. Otherwise it might be time to rethink your access protocol.
CLR in SQL Server is a convenient bad idea. Use it sparingly if at all.
I just learned I can't use SMO in ASP.NET without either SQL or a full SMO install on the application server. I'm not allowed to do that, so I can't use SMO.
What other libraries do the same thing as SMO but don't require an MSI installer or COM registrations?
I know I could send DDL to the server inside ADO.NET commands, but that is exactly what I was trying to avoid by using SMO.
What was nice about SMO:
Object oriented API for querying meta-data (columns, data types) that didn't rely on inconsistent COBOL-like DDL.
Didn't require querying undocumented stored procedures, system stored procedures or tables which break every few versions.
Off the top of my head I can think of ADOX and DMO, but both were COM based APIs.
SMO is running T-SQL under the covers. You could prototype in SMO and then watch in profiler to get the T-SQL.
It is probably an EULA violation, but you could redistrib the SMO assemblies side-by-side with your app, nothing to install in that case. I don't think their installer hits the registry. Pretty easy to bust open the SQLServerManagementObjects.msi and find out.