A stored procedure returns, amongst other things, a varbinary(max) as an OUTPUT. I can't understand how to access this information using Dapper.
Below is some cut-down sample code which illustrates the problem. I supply some parameters to a StoredProcedure and I expect to get back a Photo which is stored as varbinary(max) on the SQL Server.
There is no DbType for varbinary. I tried using DbType.Binary but this causes an exception in Dapper. If I take the photo parameter away, all the other parameters I expect to get back (which are cut out of the sample for the sake of brevity) are working. So the only issue is with retrieving the varbinary data.
What is the correct way to achieve this?
using (var connection = new System.Data.SqlClient.SqlConnection(HelperClassesBJH.HelperMethods.ConString("ProductionLocal")))
{
connection.Open();
DynamicParameters p = new DynamicParameters();
p.Add("#OpID", id, DbType.Int32, ParameterDirection.Input);
p.Add("#StageID", Properties.Settings.Default.StageID, DbType.Int32, ParameterDirection.Input);
p.Add("#Photo", dbType: DbType.Binary, direction: ParameterDirection.Output);
try
{
connection.Execute(sql, p, commandType: CommandType.StoredProcedure);
op.Photo = p.Get<byte[]>("#Photo");
}
catch {}
}
UPDATE:
I found that I have to supply the 'value' parameter in the DynamicParameters constructor. This avoids the exception I was getting. I am unable to understand why I need to supply a value, since the parameter is an output and the value I supply doesn't get used. Here is the revised code:
DynamicParameters p = new DynamicParameters();
MemoryStream b = new MemoryStream();
p.Add("#OpID", id, DbType.Int32, ParameterDirection.Input);
p.Add("#StageID", Properties.Settings.Default.StageID, DbType.Int32, ParameterDirection.Input);
p.Add("#Photo", b, DbType.Binary, direction: ParameterDirection.Output);
try
{
connection.Execute(sql, p, commandType: CommandType.StoredProcedure);
op.Photo = p.Get<byte[]>("#Photo");
}
catch {}
This results in the retrieval of a byte array which contains the expected image data.
You could try:
p.Add("#Photo", dbType: DbType.Binary, direction: ParameterDirection.Output, size: -1);
It seems to work for me locally, and that is how varchar(max) and nvarchar(max) are mapped (except as DbType.AnsiString and DbType.String, respectively), so it would be consistent.
Related
My stored procedure has an output parameter for row count. When the stored procedure is called using Dapper...
var p = new DynamicParameters();
p.Add("#rowCount", dbType: DbType.Int32, direction: ParameterDirection.Output);
var reader = conn.ExecuteReader("SearchStuff", p, commandType: CommandType.StoredProcedure);
the profiler shows the following...
declare #p0 int
set #p0=115
exec SearchStuff #rowCount=#p0 output
select #p0
I'm baffled... how is the row count assigned to the out parameter before the stored procedure has been executed?
The text you see in the trace is a reverse engineered version of the RPC completed event, not what actually got executed. This looks to be an issue with the way SQL Trace/Profiler displays the human readable format.
SQL Server is fast but doesn't yet warp time :-)
I have created a function in postgres that takes a UDTT[] as an importing parameter, and want to eventually insert that data into a Table
Example Udtt
create type udtt_mytype as
(
id uuid,
payload int
);
And then an example Function is something akin to
CREATE OR REPLACE FUNCTION dbo.p_dothething(p_import udtt_mytype[])
RETURNS void
LANGUAGE plpgsql
AS $function$
BEGIN
insert into mytab select * from unnest($1)
RETURN;
END
$function$;
My C# backend presently looks like
public class udtt_mytype
{
[PgName("id")]
public Guid id{ get; set; }
[PgName("payload ")]
public int payload { get; set; }
}
var payload = CreateAndFillUdttMyType();
var conn = new NpgsqlConnection();
conn.Open();
var transaction = conn.BeginTransaction();
conn.MapComposite<udtt_mytype>("udtt_mytype");
var command = new NpgsqlCommand("dbo.p_dothething", conn);
command.CommandType = CommandType.StoredProcedure;
Object[] objArray = new Object[1];
objArray[0] = new NpgsqlParameter { ParameterName = "p_import",
Value = payload , NpgsqlDbType = NpgsqlTypes.NpgsqlDbType.Array |
NpgsqlTypes.NpgsqlDbType.Composite };
command.Parameters.AddRange(objArray);
var result = command.ExecuteScalar();
transaction.Commit();
conn.Close();
While the above works, it is pretty non-performant compared to a similiar UDTT -> SQL StoredProcedure. Prior to our NPGSQL implementation, this was <1 second, but now i seem to be seeing about a 6seconds per 6k rows (whereas the common usages for this end up being much higher numbers than that).
Using some timestamping and returning from the SP, i see that the processing of the data in the function isnt the issue at all..it appears to entirely be transfer time of the payload. In this particular case its a simple array of UDTT_MYTYPE's, and with a single object, execution is instantaneous, but w/ 6k, its up to the 6-7 seconds range. And this performance persists even if i pass it off to an empty function (removing the cost of the unnest/insert).
In reality, udtt_mytype has 12 columns of various types, but we are still talking about a relatively 'shallow' object.
I have attempted to compare this to NPGSqls' documentation on Bulk copy (found here http://www.npgsql.org/doc/copy.html), and that implementation seemed to be even slower than this, which seems contradictive.
Is postgres typically this much slower than MSSQL, or is there something that may be limiting xfer rate of data that im not aware of? Obviously no one can speak for my network connectivity/hardware setup, but anyone that may have converted between the two, was a performance increase seen along this same scale?
I ran into another issue with using a data reader around a sproc with multiple ref cursors coming out. I am getting a not supported exception. Unfortunately, i can see from where it is coming from the source code of npgsql however.. i am not sure if i agree with throwing that exception. The code we have written works with oracle (both fully managed and managed flavors), sql server. Any help appreciated to keep it consistent for an api across some of those key flavors of dbms out there.
sproc body
CREATE OR REPLACE FUNCTION public.getmultipleresultsets (
v_organizationid integer)
RETURNS Setof refcursor
LANGUAGE 'plpgsql'
AS $BODY$
declare public override void AddCursorOutParameter(DbCommand command,
string RefCursorName)
{
NpgsqlParameter parameter = (NpgsqlParameter)CreateParameter(RefCursorName, false);
parameter.NpgsqlDbType = NpgsqlDbType.Refcursor;
parameter.NpgsqlValue = DBNull.Value;
parameter.Direction = ParameterDirection.Output;
command.Parameters.Add(parameter);
}
cv_1 refcursor;
cv_2 refcursor;
BEGIN
open cv_1 for
SELECT a.errorCategoryId, a.name, a.bitFlag
FROM ErrorCategories a
ORDER BY name;
RETURN next cv_1;
open cv_2 for
SELECT *
FROM StgNetworkStats ;
RETURN next cv_2;
END;
$BODY$;
Key Reader code that wraps postgres sql (Entlib implementation of npgsql)
private IDataReader DoExecuteReader(DbCommand command, CommandBehavior cmdBehavior)
{
try
{
var sql = new StringBuilder();
using (var reader = command.ExecuteReader(CommandBehavior.SequentialAccess))
{
while (reader.Read())
{
sql.AppendLine($"FETCH ALL IN \"{ reader.GetString(0) }\";");
}
}
command.CommandText = sql.ToString();
command.CommandType = CommandType.Text;
IDataReader reader2 = command.ExecuteReader(cmdBehavior);
return reader2;
}
catch (Exception)
{
throw;
}
}
The command building code is shown below
Helper.InitializeCommand(cmd, 300, "getmultipleresultsets");
db.AddReturnValueParameter(cmd);
db.AddInParameter(cmd, "organizationId", DbType.Int32, ORGANIZATIONID);
db.AddCursorOutParameter(cmd, "CV_1");
db.AddCursorOutParameter(cmd, "CV_2
The code that adds the refcursor parameter goes something like this
You code above seems to garble the PostgreSQL function with the .NET client code attempting to read its result.
Regardless, your function is declared to return a set of refcursors - this is not the same as two output parameters; you seem to be confusing the name of the cursor (cursors have names, but not ints, for example) with the name of the parameter (int parameters do have names).
Please note that PostgreSQL does not actually have output parameters - a function always returns a single table, and that's it. PostgreSQL does have a function syntax with output parameters, but that is only a way to construct the schema of the output table. This is unlike SQL Server, which apparently can return both a table and a set of named output parameters. To facilitate portability, when reading results, if Npgsql sees any NpgsqlParameter with direction out, it will attempt to find a resultset with the name of the parameter and will simply populate the NpgsqlParameter's Value with the first row's value for that column. This practice has zero added value over simply reading the resultset yourself - it's just there for compatibility.
To sum it up, I'd suggest you read the refcursors with your reader and then fetch their results as appropriate.
I am a rookie/newbie in the postgres data access api. I have worked a bit on oracle, sql server and trying to do what i have done with those dbms
The use is very simple
1) a stored procedure aka function with input params
2) Returning or more ref cursors
3) Using an ent lib wrapper to use the npgsql provider/database with it
4) Doing a data adapter fill and running into the issue with some cursor de-referencing.. it appears though i am inside a tran..
5) I just want to get some simple working sample with the latest npgsql provider..
Here is my function
CREATE OR REPLACE FUNCTION public.geterrorcategories(
v_organizationid integer)
RETURNS refcursor
LANGUAGE 'plpgsql'
AS $BODY$
DECLARE cv_1 refcursor;
BEGIN
open cv_1 for
SELECT errorCategoryId, name, bitFlag
FROM ErrorCategories
ORDER BY name;
RETURN cv_1;
END;
$BODY$;
The code using the enterprise lib api/wrapper is as follows.
/// <summary>
/// Executes GetErrorCategories in case of SQL Server or GetErrorCategories for Oracle
/// </summary>
public static DataTable GetErrorCategoriesAsDataTable(string dbKey ,int? ORGANIZATIONID)
{
DataTable tbl = new DataTable();
Database db = Helper.GetDatabase(dbKey);
using (DbConnection con = db.CreateConnection()){
con.Open();
var tran = con.BeginTransaction();
using (DbCommand cmd = con.CreateCommand()){
cmd.Transaction = tran;
BuildGetErrorCategoriesCommand(db, cmd ,ORGANIZATIONID);
cmd.CommandText = "GetErrorCategories";
try {
Helper.FillDataTable(tbl, db, cmd);
con.Close();
} catch (DALException ) {
throw;
}
}
}
return tbl;
}
The command is built as follows.
private static void BuildGetErrorCategoriesCommand(Database db, DbCommand cmd ,int? ORGANIZATIONID){
Helper.InitializeCommand(cmd, 300, "GetErrorCategories");
db.AddReturnValueParameter(cmd);
db.AddInParameter(cmd, "organizationId", DbType.Int32, ORGANIZATIONID);
db.AddCursorOutParameter(cmd, "CV_1");
}
I am not getting any error. I get only 1 row back which i think is this un_named_portal_1 or something but not the results from my table which my query returns
It is frustrating as i would like to keep my application code the same as much as possible but would like to switch providers at run time. I am using a tweaked 'ent lib' contribution database that was created for npgsql.
Hope this helps to point me to the right areas to look for..
There is absolutely no reason above to declare your PostgreSQL function to return a cursor - you can simply return a table, see the PostgreSQL docs for more info.
Npgsql originally had a feature where it automatically "dereferenced" cursors returned from functions, but this has been removed. For more information about this see this issue (warning, it's long...). Some people are requesting that the feature be returned.
I am retrieving data from SQL Server from a StoredProcedure using Dapper and I'm getting error
Specified cast is not valid.
and details:
Error parsing column 4 (SubTotal=0.00 - Decimal)
On SQL Server side the column SubTotal is decimal(18, 2) NULLABLE and on .NET side it's decimal?. The data being retrieved is 0.00.
I checked this answer: Dapper,decimal to double? Error parsing column X
As per answer, I replaced
il.Emit(OpCodes.Ldtoken, unboxType);
with
il.Emit(OpCodes.Ldtoken, Nullable.GetUnderlyingType(unboxType) ?? unboxType);
on line 2360 and still getting the same error.
Anyone has any ideas about this? Thanks.
Update:
I tried making column non-nullable. Also tried changing column to float (on SQL Server) and double (on .NET side). None of these worked and I was getting the same error. Then I changed column to int and now code works fine. However, I'm working with monetary values and would like to use floating point numbers. Will investigate further...
I'm executing a stored procedure as follows
var transaction = this.db.Query<PaymentTransactions>("usp_PaymentTransactionsGetSingleIfPaid", new { registrationId }, commandType: CommandType.StoredProcedure);
The relevant part of the stored procedure that returns information is below.
SELECT * FROM PaymentTransactions WHERE RegistrationId = #registrationId AND TransactionStatus = 'SUCCESS';
UPDATE 2:
Dapper is working fine. Maybe there was something wrong with my dev environment. All it took was VS restart.
Don't laugh, but I had this exact same problem with Dapper in an ASP.NET MVC project and the solution as in the comment from #erdinger worked also for me:
Close Visual Studio
Start Visual Studio again
The problem was fixed this way...
Seems like this is not Dapper specific, as I just verified the below snippet works as expected.
Try enumerating your column names explictly (instead of select *) so that the procedure returns exactly what should be mapped to PaymentTransactions. Its possible there is another non-decimal column that is misnamed?
This is using Dapper v1.13 on .Net45:
Procedure:
create procedure dbo.Test
as
select [SubTotal] = cast('0.01' as decimal(18,2))
union all
select null;
Linqpad:
void Main()
{
using (IDbConnection cnn = GetOpenConnection())
{
var users = cnn.Query<Sale>("yak.dbo.test", new { }, commandType: CommandType.StoredProcedure);
users.Dump();
}
}
public static readonly string connectionString = "Data Source=.;Initial Catalog=tempdb;Integrated Security=True";
public static IDbConnection GetOpenConnection()
{
var connection = new SqlConnection(connectionString);
connection.Open();
return connection;
}
public class Sale
{
public decimal? SubTotal;
}
Returns: