when I do
SELECT SUM(some_field) FROM some_table
the result is a single record/field with a number in it. Additionally, there will be a message send to the client along the lines of Warning: Null value is eliminated by an aggregate or other SET operation. in case some_field has a NULL value in the table somewhere. Only when they all are NULL (or the table is empty) it will return NULL.
I'm currently in the process of writing my own SqlUserDefinedAggregate and although things work as expected, it does NOT show me this message when one of the values passed turns out to be NULL. The outcome of the function is still correct, but there is no warning. First I assumed I might have to pipe this manually in the Terminate() method, but alas, SQLCLR then throws me an InvalidOperationException saying Data acces is not allowed in this context.
Any hints?
If your aggregate is discarding NULLs then the IsInvariantToNulls property should definitely be set to true else you might get unexpected results sometimes, as stated on the MSDN page for SqlUserDefinedAggregateAttribute.IsInvariantToNulls:
Used by the query processor, this property is true if the aggregate is invariant to nulls. That is, the aggregate of S, {NULL} is the same as aggregate of S. For example, aggregate functions such as MIN and MAX satisfy this property, while COUNT(*) does not.
Incorrectly setting this property can result in incorrect query results. This property is not an optimizer hint; it affects the plan selected and the results returned by the query.
And a UDA is a function so there is no SqlContext.Pipe to use. And even if there was, the Terminate method isn't an appropriate place to handle this since it executes for every group. The warning you are seeing when using SUM, however, is an ANSI warning and is displayed once for the query, not per group.
So, if SQL Server isn't displaying the warning then there likely isn't anything you can do about it. I assume that SQL Server isn't using the IsInvariantToNulls property as a means of knowing if it should display the message or not because it is not guaranteed to be accurately set.
And personally, I find this to be a benefit since, in my opinion, the "Null value is eliminated by an aggregate" warning is entirely not helpful, yet if you want to get rid of it you need to use ISNULL() to inject a value that won't influence the result (e.g. 0 in the case of SUM), or turn off ALL ANSI warnings, in which case you disable some warnings that are sometimes helpful.
Related
IIF is giving me some hell. When I do this everything works:
=IIF (IsNothing(First(Fields!Temperature.Value, "ReportInfoDataSet")),"NULL","GOOD")
But when I want to actually use the value, I end up with #Error for the NULL values, and from what I've read it's because IIF evaluates everything at once, so the .ToString() argument fails for the "GOOD" condition even though it is not used.
This gives #Error when I have a null value:
=IIF(
IsNothing(First(Fields!Temperature.Value, "ReportInfoDataSet")),
"N/A",
First(Fields!Temperature.Value, "ReportInfoDataSet").ToString()
)
So how do I work around the fact that IIF wants to evaluate all terms? There is a TechNet article that shows nesting another IIF statement as the "GOOD" condition that helps with the NULL value, but I still get the error (doing this for example):
=IIF(
IsNothing(First(Fields!Temperature.Value, "ReportInfoDataSet")),
"N/A",
IIF(
IsNothing(First(Fields!Temperature.Value, "ReportInfoDataSet")),
"N/A",
First(Fields!Temperature.Value,"ReportInfoDataSet").ToString() & ChrW("&H00B0") & "F"
)
)
Here is the article that indicates this potential solution, but it seems that I am still missing something, or that something changed invalidating this solution.
One further bit of information: I put a breakpoint in just after I fill the datatable, thinking that I would just intercept the table fill and modify so that NULLs turn to 0 or something, and it seems that datatables are strongly typed and don't allow for nullable data types so there is an exception for the temperature column:
'(ReportInfoRow((new System.Linq.SystemCore_EnumerableDebugView((test.Table).Rows)).Items[0]))
.Temperature' threw an exception of type 'System.Data.StrongTypingException'
Any insights are valued . . . I'm pondering whether I just want to add an extension to the dataset and add a MyTemp parameter that fixes the value for the report.
To not have the .ToString() forced in the lack of short-circuit Boolean evaluation above, replace it with CSTR().
CSTR(First(Fields!Temperature.Value, "ReportInfoDataSet"))
We wrote a function get_timestamp() defined as
CREATE OR REPLACE FUNCTION get_timestamp()
RETURNS integer AS
$$
SELECT (FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 10) - 13885344000)::int;
$$
LANGUAGE SQL;
This was used on INSERT and UPDATE to enter or edit a value in a created and modified field in the database record. However, we found when adding or updating records consecutively it was returning the same value.
On inspecting the function in pgAdmin III we noted that on running the SQL to build the function the key word IMMUTABLE had been injected after the LANGUAGE SQL statement. The documentation states that the default is VOLATILE (If none of these appear, VOLATILE is the default assumption) so I am not sure why IMMUTABLE was injected, however, changing this to STABLE has solved the issue of repeated values.
NOTE: As stated in the accepted answer, IMMUTABLE is never added to a function by pgAdmin or Postgres and must have been added during development.
I am guessing what was happening was that this function was being evaluated and the result was being cached for optimization, as it was marked IMMUTABLE indicating to the Postgres engine that the return value should not change given the same (empty) parameter list. However, when not used within a trigger, when used directly in the INSERT statement, the function would return a distinct value FIVE times before then returning the same value from then on. Is this due to some optimisation algorithm that says something like "If an IMMUTABLE function is used more that 5 times in a session, cache the result for future calls"?
Any clarification on how these keywords should be used in Postgres functions would be appreciated. Is STABLE the correct option for us given that we use this function in triggers, or is there something more to consider, for example the docs say:
(It is inappropriate for AFTER triggers that wish to query rows
modified by the current command.)
But I am not altogether clear on why.
The key word IMMUTABLE is never added automatically by pgAdmin or Postgres. Whoever created or replaced the function did that.
The correct volatility for the given function is VOLATILE (also the default), not STABLE - or it wouldn't make sense to use clock_timestamp() which is VOLATILE in contrast to now() or CURRENT_TIMESTAMP which are STABLE: those return the same timestamp within the same transaction. The manual:
clock_timestamp() returns the actual current time, and therefore its
value changes even within a single SQL command.
The manual warns that function volatility STABLE ...
is inappropriate for AFTER triggers that wish to query rows modified
by the current command.
.. because repeated evaluation of the trigger function can return different results for the same row. So, not STABLE.
You ask:
Do you have an idea as to why the function returned correctly five
times before sticking on the fifth value when set as IMMUTABLE?
The Postgres Wiki:
With 9.2, the planner will use specific plans regarding to the
parameters sent (the query will be planned at execution), except if
the query is executed several times and the planner decides that the
generic plan is not too much more expensive than the specific plans.
Bold emphasis mine. Doesn't seem to make sense for an IMMUTABLE function without input parameters. But the false label is overridden by the VOLATILE function in the body (voids function inlining): a different query plan can still make sense.
Related:
PostgreSQL Stored Procedure Performance
Aside
trunc() is slightly faster than floor() and does the same here, since positive numbers are guaranteed:
SELECT (trunc(EXTRACT(EPOCH FROM clock_timestamp()) * 10) - 13885344000)::int
In a .CFC file, within a CFfunction and with CFargument tags.
<cfscript>
var sp=new storedproc();
sp.setDatasource(variables.datasource);
sp.setProcedure("storedProcedure_INSERT");
sp.addParam(cfsqltype="cf_sql_integer",type="in",value=arguments.one);
sp.addParam(cfsqltype="cf_sql_integer",type="in",value=arguments.two);
sp.addParam(cfsqltype="cf_sql_integer",type="in",value=arguments.three);
sp.addParam(cfsqltype="cf_sql_integer",type="in",value=arguments.four);
sp.addProcResult(name="results",resultset=1);
//writeDump(sp);break; //This dump is reached
var spObj=sp.execute(); //blows up here; this is never reached
writeDump(spObj);break; //This is never reached, either.
var spResults=spObj.getProcResultSets().results;
A shiny nickle to anyone who can tell me why the sp.execute() is blowing up with message
"Cannot find results key in structure.
The specified key, results, does not exist in the structure."
I've used this psuedo-code many, may times in the past, and never had it do this. I'm connected to a MSSQL Server 2012 DB, everything's cricket in CF Admin, and other SPs are working properly. The stack trace doesn't even include any of MY code at all o_O
The error occurred in C:/ColdFusion10/cfusion/CustomTags/com/adobe/coldfusion/base.cfc: line 491
Called from C:/ColdFusion10/cfusion/CustomTags/com/adobe/coldfusion/storedproc.cfc: line 142
Called from //hq-devfs/development$/websites/myProject/cfc/mySOAPWSDLs.cfc: line 123
And SO is blowing up if I try and paste anymore of that. Google has...not been helpful ._.
Short answer: The error means you are trying to retrieve a resultset from the stored procedure, when it does not actually return one. A simple solution is to add a SELECT to the end of your procedure, so it returns a resultset containing the data you need. Then your original code will work:
SELECT ##ROWCOUNT AS NumOfRowsAffected;
Longer answer:
The method you are using, addProcResult(), is the equivalent of <cfprocresult>. It is intended to capture a resultset returned from a stored procedure. (Due to CF's poor choice of attribute names, a lot of people think "resultset" means the storedproc "result" structure, but they are two totally different things). A "resultset" is a query object", in CF parlance.
While all four (4) of the primary sql statements return some result, not all of them return a "query object"
Only SELECT statements generate a "query object"
INSERT/UPDATE/DELETE statements simply return the number of rows affected. They do not generate a "query object".
Since your stored procedure performs an INSERT, it does not generate a "query object". Hence the error when you try and grab the non-existent query here:
sp.addProcResult(name="results",resultset=1);
The simple solution is to add a SELECT statement to the end of your stored procedure, so that it does return a query object. Then your code will work as expected.
As an aside, I suspect you were actually trying to grab the "result" structure, but used the wrong method. The equivalent of <cfstoredproc result=".."> is getPrefix(). Though that would not work here anyway. According to the docs, it does not contain the number of rows affected. Probably because stored procedures can execute multiple statements, each one potentially returning a row count, so there is not just a single value to return.
Trying to run this query in LINQPad 4:
SELECT item_group_id as AccountID, IIF(ISNULL(t_item_group.description),'[blank]',t_item_group.description) AS Name
FROM t_item_group
WHERE active = TRUE
I get, "the isnull function requires 2 argument(s)."
I've tried moving the parens around, changing the "[blank]" to "[blank]" and "[blank]" , but none of it helps...
The queries (I have two similar ones (with IIF(ISNULL)) that LINQPad won't run for this reason, yet they run in actuality (in my Web API app) fine; so, LINQPad is more "picky" than it needs to be, perhaps, but what is it expecting, SQL syntax-wise?
ISNULL is already like a 'if' type statement.
You can just replace
IIF(ISNULL(t_item_group.description),'[blank]',t_item_group.description)
with
ISNULL(t_item_group.description, '[blank]')
The ISNULL uses the first parameter (the 'description'), unless that value is null in which case it will use the second parameter.
As an aside, one of the reasons I don't care for ISNULL is that it is poorly named. You'd assume that given its name it will return a bit - true if the parameter is null, false if not null - which you could use in an 'if' statement like you attempted. But that's not how it works.
The alternative is to use COALESCE. It provides much the same functionality, but the naming makes sense.
co·a·lesce ˌkōəˈles verb
1. come together and form one mass or whole.
To COALESCE two parameters is to force them into one non-nullable result. And the function is actually more powerful, as you can provide multiple parameters - COALESCE(i.description, i.name, '[blank]') is perfectly valid.
I want to write a non-CLR user-defined function in SQL Server 2005. This function takes an input string and returns an output string. If the input string is invalid, then I want to indicate an error to the caller.
My first thought was to use RAISERROR to raise an exception. However, SQL Server does not allow this inside a UDF (though you can raise exceptions in CLR-based UDFs, go figure).
My last resort would be to return a NULL (or some other error-indicator value) from the function if the input value is in error. However, I don't like this option, as it:
Doesn't provide any useful information to the caller
Doesn't allow me to return a NULL in response to valid input (since it's used as an error code).
Is there any caller-friendly way to halt a function on an error in SQL Server?
It seems that SQL Server UDF's are a bit limited in this (and many other) way.
You really can't do a whole lot about it - that's (for now) just the way it is. Either you can define your UDF so that you can signal back an error condition by means of its return value (e.g. returning NULL in case of an error), or then you would almost have to resort to writing a stored procedure instead, which can have a lot more error handling and allows RAISERROR and so forth.
So either design your UDF to not require specific signaling of error conditions, or then you have to re-architect your approach to use stored procedures (which can have multiple OUTPUT parameters and thus can also return error code along with your data payload, if you need that), or managed CLR code for your UDF's.
Sorry I don't have a better idea - for now, I'm afraid, those are your options - take your pick.
Marc
There's a possible solution given in an answer to a duplicate question here, based on this idea:
return cast('Error message here.' as int);
Which throws something like this:
Msg 245, Level 16, State 1, Line 1
Conversion failed when converting the varchar value 'Error message here.' to data type int.
It works OK for scalar-valued UDFs, but not for table-valued ones.