I'm trying to add support for PostgreSQL to application which is currently written for MSSQL. In C# I'd like to have "one code for all" wherever possible.
There are bunch of stored procedures, which are rewritten in Postgres as functions.
In the code, to retrieve sp parameters metadata, there used to be ms specific sp_procedure_params_rowset.
I changed it to:
select p.data_type, p.parameter_name, p.ordinal_position, p.parameter_mode
from information_schema."routines" r
join information_schema.parameters p on r.specific_name=p.specific_name
where r.routine_schema = 'schema' and r.routine_name = 'procedure_name'
This works fine for Postgres, however, for MSSQL the parameter_mode is INOUT for parameters which were set as output.
For example CREATE PROCEDURE [sp_login] #login varchar(50), #uid int output... will return
varchar #login 1 IN
int #uid 2 INOUT
Now the code is failing for MSSQL, because it is expecting the parameter #uid.
Is there a reason why MSSQL would say in information_schema.parameters INOUT when in fact it is an OUT parameter?
Is there a "universal" way to retrieve the metadata from different databases? Or there's no way around it and I'll have to write some conditions in C#?
Related
I'm trying to test a SQL query in SQL Server Management Studio that normally requires a multivalue parameter from the SSRS report its a part of.
I'm not sure to how hard code a multi value parameter in management studio. The report was created by a vendor, I'm just trying to make it runnable for testing outside of SSRS.
For example the parameter in SSRS is a collection of numbers that the user selects - ie "3100, 3102, 3105" would be the selections for the multivalue parameter called #object_code
I've got something like this - but it's not working.
Declare #Object_Code varchar(100)
Set #object_Code = ('3100','3102','3105')
....really long vendor written query I don't thoroughly understand...
IN(#object_code)
You have to use String-Split function to separate comma separated values.
For example-
Declare #Object_Code varchar(100)
Set #Object_Code = '3100,3102,3105'
....really long vendor written query I dont thoroughly understand...
--T.object_code IN (#object_code)
Inner Join dbo.Split(#Object_Code, ',') as S On S.data = T.object_code
Search your database first for any string-split function.
If you want to create string-split function then follow this -
T-SQL split string
If you use SQL Server 2016 you might want to check out the function STRING_SPLIT.
If you use a lower version of SQL Server and you can't or don't want to create a separate function, the following could be an alternative:
declare #object_code varchar(100);
set #object_code = '3100,3102,3105';
select
ltrim(rtrim(x.par.value('.[1]','varchar(max)'))) as object_code
from (
select convert(xml,'<params><param>' + replace(#object_code,',', '</param><param>') + '</param></params>') as c
) tbl
cross apply
c.nodes('/params/param') x(par);
Everybody seems to be getting hung up on splitting a string that doesn't have to be a string. We're just trouble shooting a query here and need a way to feed it values. It's not important how SSRS does it, just that we can reproduce the result.
Declare #Object_Code table (params varchar(20));
INSERT #object_Code
VALUES ('3100'),('3102'),('3105')
....really long vendor written query I don't thoroughly understand...
IN (SELECT params FROM #object_code)
Then spend some quality time getting to know the query.
I'm doing queries in which I want to extract the left-most n characters from a string that has been stripped of all leading and following spaces. An example is:
Select SUBSTRING(LTRIM(RTRIM(somefield)), 0, #n) AS mydata
FROM sometable
It's the only way I can figure to do it on a SQL Server.
I've never written a UDF before, but I think if I was just working on a SQL Server, I could create a user-defined function such as:
CREATE FUNCTION udfLeftTrimmed
(
#inputString nvarchar(50),
#n int
)
RETURNS nvarchar(#n)
AS
BEGIN
RETURN SUBSTRING(LTRIM(RTRIM(#inputString)), 0, #n);
END
I could then do something like:
Select udfLeftTrimmed(somefield,6) AS mydata
FROM sometable
which is at least a little easier to read and understand.
The question is, how do I create the UDF in ColdFusion? All my searches for SQL user-defined function in ColdFusion just gave me how to create ColdFusion functions.
Since there is nothing special or "dynamic" about your UDF you really don't need to create it in CF. You should just create it using MSSQL Manager. UDFs in SQL are like stored procedures. Once created they are a part of the DB/Schema. so create once, use as many times as you like (as #leigh has mentioned).
Keep in mind that using a SQL udf in SQL usually requires the user prepend as in:
<cfquery...>
Select dbo.udfLeftTrimmed(somefield,6) AS mydata
FROM sometable
</cfquery>
Note the "dbo.udf..." that dbo is important and may be why your subsequent try is failing - besides getting a duplicate UDF error by now. :)
NOTE:
To follow up on your comments and Leighs, you can create your UDF in a DB accessible to your user then access it as dbo.dbname.function ... as inthe following code:
<cfquery...>
Select dbo.myspecialDatabase.udfLeftTrimmed(somefield,6) AS mydata
FROM sometable
</cfquery>
Then you need only create it one time.
Can anyone explain the design decision behind preventing table valued parameters from being specified as output parameters to stored procedures?
I can't count the number of times I've started building out a data model hoping to completely lock down my tables to external access (you know...implementation details), grant applications access to the database through stored procedures only (you know... the data interface) and communicate back and forth with TVPs only to have SSMS call me naughty for having the audacity to think that I can use a user-defined table type as the transfer object between my data service and my application.
So someone please provide me a good reason why TVPs were designed to be readonly input parameters.
In the presentation on Optimizing Microsoft SQL Server 2008 Applications Using Table Valued Parameters, XML, and MERGE by Michael Rys he says. (at 32:52)
Note that in SQL Server 2008 table valued parameters are read only.
But as you notice we actually require you to write READONLY. So that
actually then means that at some point in the future maybe if you say
please, please please often enough we might be able to actually make
them writable as well at some point. But at the moment they are read
only.
Here is the connect item you should use to add your "please". Relax restriction that table parameters must be readonly when SPs call each other.
Srini Acharya made a comment on the connect item.
Allowing table valued parameters to be read/write involves quite a bit
of work on the SQL Engine side as well as client protocols. Due to
time/resource constraints as well as other priorirites, we will not be
able to take up this work as part of SQL Server 2008 release. However,
we have investigated this issue and have this firmly in our radar to
address as part of the next release of SQL Server.
Table-valued parameters have the following restrictions(source MSDN):
SQL Server does not maintain statistics on columns of table-valued
parameters.
Table-valued parameters must be passed as input READONLY
parameters to Transact-SQL routines. You cannot perform DML
operations such as UPDATE, DELETE, or INSERT on a table-valued
parameter in the body of a routine.
You cannot use a table-valued parameter as target of a SELECT INTO
or INSERT EXEC statement. A table-valued parameter can be in the
FROM clause of SELECT INTO or in the INSERT EXEC string or stored
procedure.
there are few options to over come this restriction one is
CREATE TYPE RTableType AS TABLE(id INT, NAME VARCHAR )
go
CREATE PROCEDURE Rproc #Rtable RTABLETYPE READONLY,
#id INT
AS
BEGIN
SELECT *
FROM #Rtable
WHERE ID = #id
END
go
DECLARE #Rtable RTABLETYPE
DECLARE #Otable RTABLETYPE
INSERT INTO #Rtable
VALUES (1,'a'),
(2,'b')
INSERT #Otable
EXEC Rproc
#Rtable,
2
SELECT *
FROM #Otable
through this you can get the table values out
With respect to (emphasis added):
So someone please provide me a good reason why TVPs were designed to be readonly input parameters.
I just posted a more detailed answer to this on DBA.StackExchange here:
READONLY parameters and TVP restrictions
But the summary of it goes like this:
According to this blog post ( TSQL Basics II - Parameter Passing Semantics ), a design goal of Stored Procedure OUTPUT parameters is that they merely mimic "by reference" behavior when the Stored Procedure completes successfully! But when there is an error that causes the Stored Procedure to abort, then any changes made to any OUTPUT parameters would not be reflected in the current value of those variables upon control returning to the calling process.
But when TVPs were introduced, they implemented them as truly passing by reference since continuing the "by value" model -- in which a copy of it is made to ensure that changes are lost if the Stored Procedure does not complete successfully -- would not be efficient / scalable, especially if a lot of data is being passed in through TVP.
So there is only one instance of the Table Variable that is the TVP, and any changes made to it within any Stored Procedure (if they were not restricted to being READONLY) would be immediately persisted and would remain, even if the Stored Procedure encountered an error. This violates the design goal stated at the beginning of this summary. And, there is no option for somehow tying changes made to a TVP to a transaction (even something handled automatically, behind the scenes) since table variables are not bound by transactions.
Hence, marking them as READONLY is the only way (at the moment) to maintain the design goal of Stored Procedure parameters such that they do not reflect changes made within the Stored Procedure unless: the parameter is declared as OUTPUT and the Stored Procedure complete successfully.
Be Forewarned. This code will not work. That is the problem
Note that all code was entered directly into post from memory. I may have a type wrong in the example or some similar error. It is just to demonstrate the technique that this would facilitate, which won't work with any version of SQL Server released at the time of this writing. So it doesn't really matter if it currently compiles or not.
I know this question is old by now, but perhaps someone coming across my post here might benefit from understanding why it's a big deal that TVPs can't be directly manipulated by a stored proc and read as output parameters by the calling client.
"How do you..." questions regarding OUTPUT TVPs have littered SQL Server forums for more than half a decade now. Nearly every one of them involves someone attempting some supposed workaround that completely misses the point of the question in the first place.
It is entirely non sequitur that you can "get a result set that matches a table type" by creating a Table Typed variable, Inserting into it and then returning a read from it. When you do that, the result set is still not a message. It is an ad hoc ResultSet that contains arbitrary columns that "just happen to match" a UDTT. What is needed is the ability for the following:
create database [Test]
create schema [Request]
create schema [Response]
create schema [Resources]
create schema [Services]
create schema [Metadata]
create table [Resources].[Foo] ( [Value] [varchar](max) NOT NULL, [CreatedBy] [varchar](max) NOT NULL) ON [PRIMARY]
insert into [Resources].[Foo] values("Bar", "kalanbates");
create type [Request].[Message] AS TABLE([Value] [varchar](max) NOT NULL)
create type [Response].[Message] AS TABLE([Resource] [varchar](max) NOT NULL, [Creator] [varchar](max) NOT NULL, [LastAccessedOn] [datetime] NOT NULL)
create PROCEDURE [Services].[GetResources]
(#request [Request].[Message] READONLY, #response [response].[Message] OUTPUT)
AS
insert into #response
select [Resource].[Value] [Resource]
,[Resource].[CreatedBy] [Creator]
,GETDATE() [LastAccessedOn]
inner join #request as [Request] on [Resource].[Value] = [Request].[Value]
GO
and have an ADO.NET client be able to say:
public IEnumerable<Resource> GetResources(IEnumerable<string> request)
{
using(SqlConnection connection = new SqlConnection("Server=blahdeblah;database=Test;notGoingToFillOutRestOfConnString")
{
connection.Open();
using(SqlCommand command = connection.CreateCommand())
{
command.CommandText = "[Services].[GetResources]"
command.CommandType = CommandType.StoredProcedure;
SqlParameter _request;
_request = command.Parameters.Add(new SqlParameter("#request","[request].[Message]");
_request.Value = CreateRequest(request,_request.TypeName);
_request.SqlDbType = SqlDbType.Structured;
SqlParameter response = new SqlParameter("#response", "[response].[Message]"){Direction = ParameterDirection.Output};
command.Parameters.Add(response);
command.ExecuteNonQuery();
return Materializer.Create<List<ResourceEntity>>(response).AsEnumerable(); //or something to that effect.
//The point is, messages are sent to and received from the database.
//The "result set" contained within response is not dynamic. It has a structure that can be *reliably* anticipated.
}
}
}
private static IEnumerable<SqlDataRecord> CreateRequest(IEnumerable<string> values, string typeName)
{
//Optimally,
//1)Call database stored procedure that executes a select against the information_schema to retrieve type metadata for "typeName", or something similar
//2)Build out SqlDataRecord from returned MetaData
//Suboptimally, hard code "[request].[Message]" metadata into a SqlMetaData collection
//for example purposes.
SqlMetaData[] metaData = new SqlMetaData[1];
metaData[0] = new SqlMetaData("Value", SqlDbType.Varchar);
SqlDataRecord record = new SqlDataRecord(metaData);
foreach(string value in values)
{
record.SetString(0,value);
yield return record;
}
}
The point here is that with this structure, the Database defines [Response].[Message],[Request].[Message], and [Services].[GetResource] as its Service Interface. Calling clients interact with "GetResource" by sending a pre-determined message type and receive their response in a pre-determined message type. Of course it can be approximated with an XML output parameter, you can somewhat infer a pre-determined message type by instituting tribal requirements that retrieval stored procedures must insert its response into a local [Response].[Message] Table Typed variable and then select directly out of it to return its results. But none of those techniques are nearly as elegant as a structure where a stored procedure fills a response "envelope" provided by the client with its payload and sends it back.
Still in 2020, SQL version "Microsoft SQL Azure (RTM) - 12.0.2000.8", I am not able to edit the Table value parameter within the Stored Procedure. So I did the work around by moving the data into Temp table and edited it.
ALTER PROCEDURE [dbo].[SP_APPLY_CHANGESET_MDTST09_MSG_LANG]
#CHANGESET AS [dbo].[MDTSTYPE09_MSG_LANG] READONLY
AS
BEGIN
SELECT * INTO #TCHANGESET FROM #CHANGESET
UPDATE #TCHANGESET SET DTST08_MSG_K = 0 WHERE ....
...............
I'm having a really tough time figuring how to use an xml data column in SQL Server, specifically for use with Entity Framework.
Basically, one of our tables stores "custom metadata" provided by users in the form of XML, so it seemed sensible to store this in an Xml column in the table.
One of the requirements of our application is to support searching of the metadata, however. The users are able to provided an XPath query string, as well as a value to compare the value of the XPath with, to search for elements that contain metadata that matches their query.
I identified the SQL Server xml functions as ideal for this (eg, [xmlcol].exist('/path1/path2[0][text()=''valuetest'''] ), but they're not supported by Entity Framework, irritatingly (or specifically, xml columns aren't supported). As an alternative, I tried creating a UDF that passes the user-provided XPath to the xml functions, but then discovered that the xml functions only allow string literals, so I can't provide variables...
At this point, I was running out of options.
I created a small bit of code that performs a regular expression replace on the result of a IQueryable.ToString(), to inject my XPath filter in, and then send this string to the database manually, but there are problems with this too, such as the result doesn't seem to lazily load the navigational properties, for example.
I kept looking, and stumbled upon the idea of SQLCLR types, and started creating a SQLCLR function that performs the XPath comparison. I thought I was onto a winner at this point, until a colleague pointed out that SQL Server in Azure doesn't support SQLCLR - doh!
What other options do I have? I seem to be running very close to empty...
You could do this in a stored procedure where you build your query dynamically.
SQL Fiddle
MS SQL Server 2008 Schema Setup:
create table YourTable
(
ID int identity primary key,
Name varchar(10) not null,
XMLCol xml
);
go
insert into YourTable values
('Row 1', '<x>1</x>'),
('Row 2', '<x>2</x>'),
('Row 3', '<x>3</x>');
go
create procedure GetIt
#XPath nvarchar(100)
as
begin
declare #SQL nvarchar(max);
set #SQL = N'
select ID, Name
from YourTable
where XMLCol.exist('+quotename(#XPath, '''')+N') = 1';
exec (#SQL);
end
Query 1:
exec GetIt N'*[text() = "2"]'
Results:
| ID | NAME |
--------------
| 2 | Row 2 |
To remain "customisable", the SqlQuery method on DbSet can be used:
var query = #"SET ARITHABORT ON;
select * from [YourTable] where
[xmlcol].exist('/path1/path2[0][text()=''{0}''']";
var numOfResults = 5;
var offsetPage = 1;
var results = Context.YourTable.SqlQuery(String.Format(query,"valuetest"))
.OrderBy(x => x.col)
.Skip(offsetPage * numOfResults)
.Take(numOfResults).ToList();
Note, due to its dynamic nature, this method would also most likely expose some degree of sql injection security holes.
I've mostly worked with SQL Server so far and now I'm moving to Oracle for a new project.
I'm trying to create a proc that will return data to a .net app. The only way I got this to work is by using packages like this:
CREATE OR REPLACE PACKAGE GetAllDepartments AS
TYPE T_CURSOR IS REF CURSOR;
PROCEDURE p_GetAllDepartments
(
cur_Result OUT T_CURSOR
);
END GetAllDepartments;
CREATE OR REPLACE PACKAGE BODY GetAllDepartments AS
PROCEDURE p_GetAllDepartments
(
cur_Result OUT T_CURSOR
)
IS
BEGIN
OPEN cur_Result FOR
SELECT * FROM DEPARTMENTS;
END p_GetAllDepartments;
END GetAllDepartments;
Is this the only way to go with Oracle?, can't I just create the proc and call that directly? Thanks
Assuming you have a supported version of Oracle, you should be able to do something like
CREATE OR REPLACE PROCEDURE get_all_departments( p_result OUT SYS_REFCURSOR )
AS
BEGIN
OPEN p_result
FOR SELECT *
FROM departments;
END get_all_departments;
That said, you are normally better off from an organization standpoint using packages to collect procedures that do related things. In your case, for example, it would generally make sense to have a package that had all the procedures that dealt with departments (i.e. create_department, delete_department, etc.).
And from a general stylistic standpoint, it is more common in Oracle to create a view that implements whatever logic you would put in the stored procedure and to query that view rather than creating a stored procedure that just does a query.
You can do that without a package, e.g. by creating a function that returns result sets.
Those functions can be used like tables, e.g.: SELECT * FROM my_function()
This is especially efficient with pipelined function because the result is not buffered on the server but sent row by row to the client:
http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14251/adfns_packages.htm#i1008519
But the basic technique is still the same i.e. you have to define a type that is used for the return type of the function. You cannot have it return an "anonymous" result set like in PostgreSQL or SQL Server.