I am a rookie/newbie in the postgres data access api. I have worked a bit on oracle, sql server and trying to do what i have done with those dbms
The use is very simple
1) a stored procedure aka function with input params
2) Returning or more ref cursors
3) Using an ent lib wrapper to use the npgsql provider/database with it
4) Doing a data adapter fill and running into the issue with some cursor de-referencing.. it appears though i am inside a tran..
5) I just want to get some simple working sample with the latest npgsql provider..
Here is my function
CREATE OR REPLACE FUNCTION public.geterrorcategories(
v_organizationid integer)
RETURNS refcursor
LANGUAGE 'plpgsql'
AS $BODY$
DECLARE cv_1 refcursor;
BEGIN
open cv_1 for
SELECT errorCategoryId, name, bitFlag
FROM ErrorCategories
ORDER BY name;
RETURN cv_1;
END;
$BODY$;
The code using the enterprise lib api/wrapper is as follows.
/// <summary>
/// Executes GetErrorCategories in case of SQL Server or GetErrorCategories for Oracle
/// </summary>
public static DataTable GetErrorCategoriesAsDataTable(string dbKey ,int? ORGANIZATIONID)
{
DataTable tbl = new DataTable();
Database db = Helper.GetDatabase(dbKey);
using (DbConnection con = db.CreateConnection()){
con.Open();
var tran = con.BeginTransaction();
using (DbCommand cmd = con.CreateCommand()){
cmd.Transaction = tran;
BuildGetErrorCategoriesCommand(db, cmd ,ORGANIZATIONID);
cmd.CommandText = "GetErrorCategories";
try {
Helper.FillDataTable(tbl, db, cmd);
con.Close();
} catch (DALException ) {
throw;
}
}
}
return tbl;
}
The command is built as follows.
private static void BuildGetErrorCategoriesCommand(Database db, DbCommand cmd ,int? ORGANIZATIONID){
Helper.InitializeCommand(cmd, 300, "GetErrorCategories");
db.AddReturnValueParameter(cmd);
db.AddInParameter(cmd, "organizationId", DbType.Int32, ORGANIZATIONID);
db.AddCursorOutParameter(cmd, "CV_1");
}
I am not getting any error. I get only 1 row back which i think is this un_named_portal_1 or something but not the results from my table which my query returns
It is frustrating as i would like to keep my application code the same as much as possible but would like to switch providers at run time. I am using a tweaked 'ent lib' contribution database that was created for npgsql.
Hope this helps to point me to the right areas to look for..
There is absolutely no reason above to declare your PostgreSQL function to return a cursor - you can simply return a table, see the PostgreSQL docs for more info.
Npgsql originally had a feature where it automatically "dereferenced" cursors returned from functions, but this has been removed. For more information about this see this issue (warning, it's long...). Some people are requesting that the feature be returned.
Related
I'm running big dependency scan on legacy db and see that some objects have obsolete ref links, if you run this code in SSMS for View that points to not existing table like in my case, you will get your output on Results tab AND error info in Messages . Like in my case below.
I tried to check all env things I know and output of this stored procedure, but didn't see any indication.
How I can capture this event as I'm running this in looped dynamic SQL script and capture output in my table for further processing?
Updated:
it just text in Message box ,on error, you still have output on
Results tab
this is sp, it loop thru object list I took from sys.object and run this string as my sample to get all dependencies, load all into table. This call to
sql_reference_entities is the only way to get inter database
dependency on column level. So I need stick to this 100$>
--
Select *
From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')
--
----update------
This behavior was fixed in SQL Server 2014 SP3 and SQL Server 2016 SP2:
Starting from Microsoft SQL Server 2012, errors raised by
sys.dm_sql_referenced_entities (such as when an object has undergone a
schema change) cannot be caught in a TRY...CATCH Transact-SQL block.
While this behavior is expected in SQL Server 2012 and above, this
improvement introduces a new column that's called is_incomplete to the
Dynamic Management View (DMV).
KB4038418 - Update adds a new column to DMV sys.dm_sql_referenced_entities in SQL Server 2014 and 2016
----update-------
The tldr is that you can't capture these on the server side, and must use a client program in C#, PowerShell or some other client that can process info messages.
That DMV is doing something strange that I don't fully understand. It's generating errors (which a normal UDF is not allowed to do), and those errors do not trigger a TRY/CATCH block or set ##error. EG
create table tempdb.dbo.foo(id int)
go
create view dbo.v_View_Obs_Table
as
select * from tempdb.dbo.foo
go
drop table tempdb.dbo.foo
go
begin try
Select * From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')
end try
begin catch
select ERROR_MESSAGE(); --<-- not hit
end catch
However these are real errors, as you can see running this from client code:
using System;
using System.Data.SqlClient;
namespace ConsoleApp6
{
class Program
{
static void Main(string[] args)
{
using (var con = new SqlConnection("Server=.;database=AdventureWorks;integrated security=true"))
{
con.Open();
con.FireInfoMessageEventOnUserErrors = true;
con.InfoMessage += (s, a) =>
{
Console.WriteLine($"{a.Message}");
foreach (SqlError e in a.Errors)
{
Console.WriteLine($"{e.Message} Number:{e.Number} Class:{e.Class} State:{e.State} at {e.Procedure}:{e.LineNumber}");
}
};
var cmd = con.CreateCommand();
cmd.CommandText = "Select * From sys.dm_sql_referenced_entities('dbo.v_View_Obs_Table','Object')";
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read() || (rdr.NextResult() && rdr.Read()))
{
Console.WriteLine(rdr[0]);
}
}
Console.ReadKey();
}
}
}
}
outputs
Invalid object name 'tempdb.dbo.foo'.
Invalid object name 'tempdb.dbo.foo'. Number:208 Class:16 State:3 at v_View_Obs_Table:4
0
The dependencies reported for entity "dbo.v_View_Obs_Table" might not include references to all columns. This is either because the entity references an object that does not exist or because of an error in one or more statements in the entity. Before rerunning the query, ensure that there are no errors in the entity and that all objects referenced by the entity exist.
The dependencies reported for entity "dbo.v_View_Obs_Table" might not include references to all columns. This is either because the entity references an object that does not exist or because of an error in one or more statements in the entity. Before rerunning the query, ensure that there are no errors in the entity and that all objects referenced by the entity exist. Number:2020 Class:16 State:1 at :1
I ran into another issue with using a data reader around a sproc with multiple ref cursors coming out. I am getting a not supported exception. Unfortunately, i can see from where it is coming from the source code of npgsql however.. i am not sure if i agree with throwing that exception. The code we have written works with oracle (both fully managed and managed flavors), sql server. Any help appreciated to keep it consistent for an api across some of those key flavors of dbms out there.
sproc body
CREATE OR REPLACE FUNCTION public.getmultipleresultsets (
v_organizationid integer)
RETURNS Setof refcursor
LANGUAGE 'plpgsql'
AS $BODY$
declare public override void AddCursorOutParameter(DbCommand command,
string RefCursorName)
{
NpgsqlParameter parameter = (NpgsqlParameter)CreateParameter(RefCursorName, false);
parameter.NpgsqlDbType = NpgsqlDbType.Refcursor;
parameter.NpgsqlValue = DBNull.Value;
parameter.Direction = ParameterDirection.Output;
command.Parameters.Add(parameter);
}
cv_1 refcursor;
cv_2 refcursor;
BEGIN
open cv_1 for
SELECT a.errorCategoryId, a.name, a.bitFlag
FROM ErrorCategories a
ORDER BY name;
RETURN next cv_1;
open cv_2 for
SELECT *
FROM StgNetworkStats ;
RETURN next cv_2;
END;
$BODY$;
Key Reader code that wraps postgres sql (Entlib implementation of npgsql)
private IDataReader DoExecuteReader(DbCommand command, CommandBehavior cmdBehavior)
{
try
{
var sql = new StringBuilder();
using (var reader = command.ExecuteReader(CommandBehavior.SequentialAccess))
{
while (reader.Read())
{
sql.AppendLine($"FETCH ALL IN \"{ reader.GetString(0) }\";");
}
}
command.CommandText = sql.ToString();
command.CommandType = CommandType.Text;
IDataReader reader2 = command.ExecuteReader(cmdBehavior);
return reader2;
}
catch (Exception)
{
throw;
}
}
The command building code is shown below
Helper.InitializeCommand(cmd, 300, "getmultipleresultsets");
db.AddReturnValueParameter(cmd);
db.AddInParameter(cmd, "organizationId", DbType.Int32, ORGANIZATIONID);
db.AddCursorOutParameter(cmd, "CV_1");
db.AddCursorOutParameter(cmd, "CV_2
The code that adds the refcursor parameter goes something like this
You code above seems to garble the PostgreSQL function with the .NET client code attempting to read its result.
Regardless, your function is declared to return a set of refcursors - this is not the same as two output parameters; you seem to be confusing the name of the cursor (cursors have names, but not ints, for example) with the name of the parameter (int parameters do have names).
Please note that PostgreSQL does not actually have output parameters - a function always returns a single table, and that's it. PostgreSQL does have a function syntax with output parameters, but that is only a way to construct the schema of the output table. This is unlike SQL Server, which apparently can return both a table and a set of named output parameters. To facilitate portability, when reading results, if Npgsql sees any NpgsqlParameter with direction out, it will attempt to find a resultset with the name of the parameter and will simply populate the NpgsqlParameter's Value with the first row's value for that column. This practice has zero added value over simply reading the resultset yourself - it's just there for compatibility.
To sum it up, I'd suggest you read the refcursors with your reader and then fetch their results as appropriate.
I have the following SQL Server 2012 query:
var sql = #"Select question.QuestionUId
FROM Objective,
ObjectiveDetail,
ObjectiveTopic,
Problem,
Question
where objective.examId = 1
and objective.objectiveId = objectiveDetail.objectiveId
and objectiveDetail.ObjectiveDetailId = ObjectiveTopic.ObjectiveDetailId
and objectiveTopic.SubTopicId = Problem.SubTopicId
and problem.ProblemId = question.ProblemId";
var a = db.Database.SqlQuery<string>(sql).ToList();
Can someone help explain to me if it would be a good idea to put this into a
stored procedure and if so then how could I do that and then call it from my C# code. It was
suggested to me that if it is in a stored procedure then it would run more
efficiently as it would not be recompiled often. Is that the case?
Yes, there is. For starters, a stored procedure is precompiled and stored within your database. Being precompiled, the database engine can execute it more efficiently, since no on-the-fly compilation necessary. Also, database optimizations can be added to support a precompiled procedure. A stored procedure also allows business logic to be encapsulated within the database.
If you decide to go the stored procedure route, then consider the following:
First of all, you will need to create a stored procedure that encapsulates your existing SQL query.
CREATE PROCEDURE ListQuestionIds
#ExamId int
AS
BEGIN
SELECT Question.QuestionUId
FROM Objective
INNER JOIN ObjectiveDetail
ON ( Objective.objectiveId = ObjectiveDetail.objectiveId )
INNER JOIN ObjectiveTopic
ON ( ObjectiveDetail.ObjectiveDetailId = ObjectiveTopic.ObjectiveDetailId )
INNER JOIN Problem
ON ( ObjectiveTopic.SubTopicId = Problem.SubTopicId )
INNER JOIN Question
ON ( Problem.ProblemId = Question.ProblemId )
WHERE Objective.examId = #ExamId;
END;
Please make sure that the tables called by your procedure (Objective, Problem, etc,) have all of the relevant primary keys and indexes in place to enhance the performance of your query.
Next, you will need to call that stored procedure from within your C# code. One way--but by no means the only way--is to create a connection to your database using the SqlConnection object and then executing your procedure via the SqlCommand object.
I would recommend that you take a look at How to execute a stored procedure within C# program for some on-topic examples. But a simple example of such might look like:
string connectionString = "your_connection_string";
using (var con = new SqlConnection(connectionString))
{
using (var cmd = new SqlCommand("ListQuestionIds", con)) {
cmd.CommandType = CommandType.StoredProcedure
cmd.Parameters.Add(new SqlParameter("#ExamId", examId))
con.Open();
using (SqlDataReader rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
// Loop through the returned SqlDataReader object (aka. rdr) and
// then evaluate & process the returned question id value(s) here
}
}
}
}
Please note that this sample code does not (intentionally) include any error handling. I leave that up to you to integrate into your application.
Finally, just as an FYI... many of the more modern ORMs (e.g., Entity Framework, NHibernate, etc.) allow you to execute stored procedure-like queries from your C# code without requiring an explicit stored procedure. If you are already using an ORM in your application, then you may want to forgo the stored procedure altogether. Whatever you decide to do, a little research on your end will help you make an informed decision.
I hope this helps you get started. Good luck.
I am retrieving data from SQL Server from a StoredProcedure using Dapper and I'm getting error
Specified cast is not valid.
and details:
Error parsing column 4 (SubTotal=0.00 - Decimal)
On SQL Server side the column SubTotal is decimal(18, 2) NULLABLE and on .NET side it's decimal?. The data being retrieved is 0.00.
I checked this answer: Dapper,decimal to double? Error parsing column X
As per answer, I replaced
il.Emit(OpCodes.Ldtoken, unboxType);
with
il.Emit(OpCodes.Ldtoken, Nullable.GetUnderlyingType(unboxType) ?? unboxType);
on line 2360 and still getting the same error.
Anyone has any ideas about this? Thanks.
Update:
I tried making column non-nullable. Also tried changing column to float (on SQL Server) and double (on .NET side). None of these worked and I was getting the same error. Then I changed column to int and now code works fine. However, I'm working with monetary values and would like to use floating point numbers. Will investigate further...
I'm executing a stored procedure as follows
var transaction = this.db.Query<PaymentTransactions>("usp_PaymentTransactionsGetSingleIfPaid", new { registrationId }, commandType: CommandType.StoredProcedure);
The relevant part of the stored procedure that returns information is below.
SELECT * FROM PaymentTransactions WHERE RegistrationId = #registrationId AND TransactionStatus = 'SUCCESS';
UPDATE 2:
Dapper is working fine. Maybe there was something wrong with my dev environment. All it took was VS restart.
Don't laugh, but I had this exact same problem with Dapper in an ASP.NET MVC project and the solution as in the comment from #erdinger worked also for me:
Close Visual Studio
Start Visual Studio again
The problem was fixed this way...
Seems like this is not Dapper specific, as I just verified the below snippet works as expected.
Try enumerating your column names explictly (instead of select *) so that the procedure returns exactly what should be mapped to PaymentTransactions. Its possible there is another non-decimal column that is misnamed?
This is using Dapper v1.13 on .Net45:
Procedure:
create procedure dbo.Test
as
select [SubTotal] = cast('0.01' as decimal(18,2))
union all
select null;
Linqpad:
void Main()
{
using (IDbConnection cnn = GetOpenConnection())
{
var users = cnn.Query<Sale>("yak.dbo.test", new { }, commandType: CommandType.StoredProcedure);
users.Dump();
}
}
public static readonly string connectionString = "Data Source=.;Initial Catalog=tempdb;Integrated Security=True";
public static IDbConnection GetOpenConnection()
{
var connection = new SqlConnection(connectionString);
connection.Open();
return connection;
}
public class Sale
{
public decimal? SubTotal;
}
Returns:
I have a customer setup that uses stored procedures to return data from its SQL Server database.
Those stored procedures are all built the same way - they take a bunch of input parameters, and they return:
the first is just a single row, single column with the result code (type INT) - 0 for success, some other values otherwise; if the value is 0, then there's a second result set that contains the actual data
the second result set can be anything - any number of columns and rows
I'm trying to create a "generic" way to interface with this system, and my attempt is this:
create a class that takes stored procedure name and input parameters
a return type that contains both an ErrorCode INT property as well as a DataTable results property
However, I'm having trouble getting this to work with ADO.NET and SQL Server 2008 R2.
My code boils down to this:
public MyResultType CallService(string procedureName, MyParameters parameters)
{
MyResultType result = new MyResultType { ErrorCode = 0 };
using (SqlConnection conn = new SqlConnection(_connectionString))
using (SqlCommand cmd = new SqlCommand(procedureName, conn))
{
cmd.CommandType = CommandType.StoredProcedure;
SetupParameters(cmd, parameters);
// open connection, execute the stored procedure
conn.Open();
using (SqlDataReader rdr = cmd.ExecuteReader())
{
// get the first result set - the status code, one row, one column of type INT
while (rdr.Read())
{
result.ErrorCode = rdr.GetInt32(0);
}
// if we got a "0" (success) -> go to the next result set and load it into the table
if(result.ErrorCode == 0 && rdr.NextResult())
{
result.ResultTable = new DataTable();
result.ResultTable.Load(rdr);
int colCount = result.ResultTable.Columns.Count;
int rowCount = result.ResultTable.Rows.Count;
}
rdr.Close();
}
conn.Close();
}
return result;
}
My issue is: the call to the stored procedure works just fine, the error code = 0 is picked up just fine, the data table is created and the number of columns is the expected value - but there are NO ROWS loaded...
I've been trying everything I can think of to get those rows into the DataTable - no luck. And of course - if I execute this very same stored procedure in SQL Server Management Studio, everything works just fine, I get my ErrorCode=0 and my result set of 18 columns and 5 rows - no problem....
What am I missing? Can anyone spot the problem in my code? Why aren't the rows of the second result set loaded (but the columns are being detected, it seems)?
The code as published works fine - there was a difference between how I call it from C# and in SQL Server Management Studio : NULL handling.
I had some input parameters set to int and thus provided a 0 value to the stored procedure, while it really expected to get a NULL if the value isn't defined......
Stupid rookie mistake..... SORRY GUYS! And thanks for your inputs!
The DataTable.Load method implicitly calls the NextResult method, so combining that with your explicit call, it is advancing to a resultset that isn't there. You should remove your own call to NextResult if you want to use DataTable.Load or loop through and fill a datatable yourself.