I have a indexed view where I basically need to do this
SELECT ...
CASE
WHEN ISDATE(ColumnName) = 1 THEN CONVERT(datetime, ColumnName, 103)
ELSE NULL
END AS ViewColumn
....
Trying to create the index yields:
Cannot create index on view
'....'. The function
'isdate' yields nondeterministic results. Use a deterministic system
function, or modify the user-defined function to return deterministic
results.
MSDN says
ISDATE is deterministic only if you use it with the CONVERT function,
if the CONVERT style parameter is specified, and style is
not equal to 0, 100, 9, or 109.
here http://msdn.microsoft.com/en-us/library/ms187347.aspx.
But I don't know what that means at all. As far as I can tell, I am using it with a CONVERT function....
Any way to work around this?
It should be, if at all:
SELECT ...
CASE
WHEN ISDATE(ColumnName) = 1 THEN CONVERT(datetime, ColumnName, 103)
ELSE NULL
END
....
but, you are not using ISDATE WITH CONVERT, since there is no expression like
ISDATE(CONVERT(varchar,ColumnName,112))
without the nested convert the return value is dependend on things like language settings, hence it's nondeterministic behaviour. Without "external" knowledge, it's not possible to predict the result one is getting, based on the input alone.
Reference
What are the requirements for Indexed views?
There are several requirements that you must take into consideration when using Indexed views.
1. View definition must always return the same results from the same underlying data.
2. Views cannot use non-deterministic functions.
3. The first index on a View must be a clustered, UNIQUE index.
4. If you use Group By, you must include the new COUNT_BIG(*) in the select list.
5. View definition cannot contain the following
(A) TOP
(B) Text, ntext or image columns
(C)DISTINCT
(d)MIN, MAX, COUNT, STDEV, VARIANCE, AVG
(E)SUM on a nullable expression
(F)A derived table
(G)Rowset function
(H)Another view
(I)UNION
(J)Subqueries, outer joins, self joins
(K)Full-text predicates like CONTAIN or FREETEXT
(L)COMPUTE or COMPUTE BY
(M)Cannot include order by in view definition
Related
I have a handful of scalar-valued functions (which I appreciate may not be best practice, but for now, this will suffice, this is simply a development exercise) which I use within various computed columns with MS SQL tables.
I have a scenario whereby I need to use the result of one of these functions, as a parameter to another within a given computed column. For example;
(case when [DynamicQuantity] IS NULL then ([dbo].[GetFiatValue]([Currency],[Quantity])) ELSE ([dbo].[GetFiatValue]([Currency],([dbo].[getStakedQuantity]([currency])))) end)
If the given [DynamicQuantity] field is null (a column value generated by another SVF), use the [Quantity] field. However, if [DynamicQuantity] is populated, the ELSE will fire, and pass the output of the [dbo].[getStakedQuantity] SVF in place of the [Quantity] value.
Is this achievable? I can't seem to get the syntax to work and am questioning whether this is even possible?
Running it via a normal query works;
SELECT case when [DynamicQuantity] IS NULL then [dbo].[GetFiatValue]([Currency],[Quantity]) ELSE [dbo].[GetFiatValue]([Currency],[dbo].[getStakedQuantity]([currency])) end from dbo.cryptoPortfolio
But when used as a computed column formula below;
case when [DynamicQuantity] IS NULL then [dbo].[GetFiatValue]([Currency],[Quantity]) ELSE [dbo].[GetFiatValue]([Currency],[dbo].[getStakedQuantity]([currency])) end
SSMS will not accept it, with Error validating the formula for column 'FiatValue'.
SELECT House__r.Name, House__r.House_Owner__r.person__r.Email__c, (SELECT Name, Total_Balance__c, Total_Expense__c FROM Expenses__r Where Type__c='Yearly' AND AND CALENDAR_YEAR(CreatedDate) = CALENDAR_YEAR(System.today() ) FROM Member__c
Error=> Unknown parsing error.
Kindly suggest what else can I do.
You have two 'AND' in a row. Also, if you are using query editor, you can't use System.today().
(on mobile, formatting will be poor, sorry)
SOQL supports only "field - operator - value_or_function” style for conditions. You try to make it "function - operator - another function", won't work.
For your particular scenario try with WHERE Created date = THIS_YEAR. It's a special literal, see https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_dateformats.htm
More generic way would be to write what you need as a formula field, you could compare to YEAR(TODAY()). (that could be poor performance to filter by formula, often it results in full table scan, you would have to test, maybe add more filters using indexed columns...).
Or write it as WHERE CreatedDate >= :start AND CreatedDate <= :stop, put right bind variables for your range.
I have a clr aggregate concatenation function, similar to https://gist.github.com/FilipDeVos/5b7b4addea1812067b09. When the number of rows are small, the sequence of concatenated strings follows the input data set. When the number of rows are larger (dozens and more), the sequence seems indeterminate. There is a difference in the execution plan, but I'm not that familiar with the optimizer and what hints to apply (I've tried MAXDOP 1, without success). From a different test than the example below with similar results here's what seems to be the difference in the plan - the separate sorts, then a merge join. The row count where it tipped over here was 60.
yielded expected results:
yielded unexpected results:
Below is the query that demonstrates the issue in the AdventureWorks2014 sample database with the above clr (renamed to TestConcatenate). The intended result is a dataset with a row for each order and a column with a delimited list of products for that order in quantity sequence.
;with cte_ordered_steps AS (
SELECT top 100000 sd.SalesOrderID, SalesOrderDetailID, OrderQty
FROM [Sales].[SalesOrderDetail] sd
--WHERE sd.SalesOrderID IN (53598, 53595)
ORDER BY sd.SalesOrderID, OrderQty
)
select
sd.SalesOrderID,
dbo.TestConcatenate(' QTY: ' + CAST(sd.OrderQty AS VARCHAR(9)) + ': ' + IsNull(p.Name, ''))
FROM [Sales].[SalesOrderDetail] sd
JOIN [Production].[Product] p ON p.ProductID = sd.ProductID
JOIN cte_ordered_steps r ON r.SalesOrderID = sd.SalesOrderID AND r.SalesOrderDetailID = sd.SalesOrderDetailID
where sd.SalesOrderID IN (53598, 53595)
GROUP BY sd.SalesOrderID
When the SalesOrderID is constrained in the cte for 53598, 53595, the sequence is correct (top set), when it's constrained in the main select for 53598, 53595, the sequence is not (botton set).
So what's my question? How can I build the query, with hints or other changes to return consistent (and correct) sequenced concatenated values independent of the number of rows.
Just like a normal query, if there isn't an order by clause, return order isn't guaranteed. If I recall correctly, the SQL 92 spec allows for an order by clause to be passed in to an aggregate via an over clause, SQL Server doesn't implement it. So there's no way to guarantee ordering in your CLR function (unless you implement it yourself by collecting everything in the Accumulate and Merge methods into some sort of collection and then sorting the list in the Terminate method before returning it. But you'll pay a cost in terms of memory grants as now need to serialize the collection.
As to why you're seeing different behavior based on the size of your result set, I notice that a different join operator is being used between the two. A loop join and a merge join walk through the two sets being joined differently and so that might account for the difference you're seeing.
Why not try the aggregate dbo.GROUP_CONCAT_S available at http://groupconcat.codeplex.com. The S is for Sorted output. It does exactly what you want.
While this answer doesn't have a solution, the additional information that Ben and Orlando provided (thanks!) have provided what I need to move on. I'll take the approach that Orlando pointed to, which was also my plan B, i.e. sorting in the clr.
I have a column in SQL Server 2005 that stores a version number as a string that i would like to sort by. I have been unable to find out how to sort this column, although i am guessing it would be some kind of custom function or compare algorithm.
Can anyone point me in the right direction of where to start? I may be googling the wrong stuff.
Cheers
Tris
I'd use separate int columns (e.g. MajorCol + MinorCol if you are tracking major + minor versions) and run something like
order by MajorCol, MinorCol
in my query.
I would look at using a persisted computed column which processes the string into an int or string or something appropriate for sorting in the native SQL Server sort - i.e.
'1.1.1.1' -> '001.001.001.001'
'10.10.10.10' -> '010.010.010.010'
'1.10.1.10' -> '001.010.001.010'
So that you can sort by the computed column and get expected results.
Alternatively, you can use such an operation inline, but it might be very slow. In addition scalar UDFs are extremely slow.
Creating an SQL CLR function is the way to go. They're extremely fast and powerful. It would be quick and effective as you wouldn't have to change any existing code, and you could specify all the information you need right in your SQL statements.
The SQL CLR function could accept an input string, as well as other parameters specifying which piece of information you'd like to extract from the input string. You could then sort on the return values of the function.
Specifically, I'd create a generic function that accepts three parameters: an input string, a regular expression, and a group name. That function would allow you to pass your database field and a regular expression with named groups right in the SQL statement.
The SQL CLR function would create a Regex, test it against the string, and would ultimately return the matched group's value or null if there was no match or the group was not matched (if the group was optional). Ideally, you'd want to pass the same regular expression to each call (perhaps as a variable like #regex), to take advantage of any CLR caching of the compiled regular expression. The end result would be very flexible and fast.
Regular expression options can be specified inline in the pattern like so: "(?imnsx-imnsx:subexpression)". See: msdn.microsoft.com/en-us/library/yd1hzczs.aspx
The code for such a function would look something like this:
[SqlFunction(IsDeterministic=true,IsPrecise=true,DataAccess=DataAccessKind.None,SystemDataAccess=SystemDataAccessKind.None)]
public static SqlString RegexMatchNamedGroup( SqlChars input, SqlString pattern, SqlString group_name )
{
Regex regex = new Regex( pattern.Value );
Match match = regex.Match( new string( input.Value ) );
if (!match.Success) //return null if match failed
return SqlString.Null;
if (group_name.IsNull) //return entire string if matched when no group name is specified
return match.Value;
Group group = match.Groups[group_name.Value];
if (group.Success)
return group.Value; //return matched group value
else
return SqlString.Null; //return null if named group was not matched
}
Your SQL statement could then sort on pieces of your information like so:
declare #regex nvarchar(2000) = '^(?<Major>\d{1,3})\.(?<Minor>\d{1,3})';
select VersionNumber
from YourTable
order by
Cast(RegexMatchNamedGroup( VersionNumber, #regex, 'Major') as int),
Cast(RegexMatchNamedGroup( VersionNumber, #regex, 'Minor') as int)
By definition (at least from what I've seen) sargable means that a query is capable of having the query engine optimize the execution plan that the query uses. I've tried looking up the answers, but there doesn't seem to be a lot on the subject matter. So the question is, what does or doesn't make an SQL query sargable? Any documentation would be greatly appreciated.
For reference: Sargable
The most common thing that will make a query non-sargable is to include a field inside a function in the where clause:
SELECT ... FROM ...
WHERE Year(myDate) = 2008
The SQL optimizer can't use an index on myDate, even if one exists. It will literally have to evaluate this function for every row of the table. Much better to use:
WHERE myDate >= '01-01-2008' AND myDate < '01-01-2009'
Some other examples:
Bad: Select ... WHERE isNull(FullName,'Ed Jones') = 'Ed Jones'
Fixed: Select ... WHERE ((FullName = 'Ed Jones') OR (FullName IS NULL))
Bad: Select ... WHERE SUBSTRING(DealerName,4) = 'Ford'
Fixed: Select ... WHERE DealerName Like 'Ford%'
Bad: Select ... WHERE DateDiff(mm,OrderDate,GetDate()) >= 30
Fixed: Select ... WHERE OrderDate < DateAdd(mm,-30,GetDate())
Don't do this:
WHERE Field LIKE '%blah%'
That causes a table/index scan, because the LIKE value begins with a wildcard character.
Don't do this:
WHERE FUNCTION(Field) = 'BLAH'
That causes a table/index scan.
The database server will have to evaluate FUNCTION() against every row in the table and then compare it to 'BLAH'.
If possible, do it in reverse:
WHERE Field = INVERSE_FUNCTION('BLAH')
This will run INVERSE_FUNCTION() against the parameter once and will still allow use of the index.
In this answer I assume the database has sufficient covering indexes. There are enough questions about this topic.
A lot of the times the sargability of a query is determined by the tipping point of the related indexes. The tipping point defines the difference between seeking and scanning an index while joining one table or result set onto another. One seek is of course much faster than scanning a whole table, but when you have to seek a lot of rows, a scan could make more sense.
So among other things a SQL statement is more sargable when the optimizer expects the number of resulting rows of one table to be less than the tipping point of a possible index on the next table.
You can find a detailed post and example here.
For an operation to be considered sargable, it is not sufficient for it to just be able to use an existing index. In the example above, adding a function call against an indexed column in the where clause, would still most likely take some advantage of the defined index. It will "scan" aka retrieve all values from that column (index) and then eliminate the ones that do not match to the filter value provided. It is still not efficient enough for tables with high number of rows.
What really defines sargability is the query ability to traverse the b-tree index using the binary search method that relies on half-set elimination for the sorted items array. In SQL, it would be displayed on the execution plan as a "index seek".