Linq to SQL rounding of decimal after division - sql-server

I've come across a precision issue when calculating the ration between two units of measures.
The Ratio is stored in a SQL Server database in a UnitOfMeasureTable as: NumberOfDefaultUnits decimal(28,18)
In my linqPad example I have a class to demonstate what is going wrong.
private class Test
{
public decimal saleUom { get; set; }
public decimal piUom { get; set; }
public decimal RatioRight { get; set; }
public decimal RatioWrong { get; set; }
public decimal CalcledAlsoRight { get { return saleUom / piUom; } }
}
void Main()
{
var xx = from uom in UnitsOfMeasures.Where(d=> d.Id == 9)
let buyUom = uom.NumberOfDefaultUnits
let sellUom = UnitsOfMeasures.First(d=> d.Id == 13).NumberOfDefaultUnits
select new Test
{
saleUom = sellUom,
piUom = buyUom,
RatioRight = sellUom / (buyUom * 1m),
RatioWrong = sellUom / buyUom,
};
xx.First().Dump();
}
The results are:
saleUom 453.592370000000000000
piUom 1000000.000000000000000000
RatioRight 0.000453592370000000000
RatioWrong 0.0004535923
CalcledAlsoRight 0.00045359237
It took a while to figure out you have to multiply the divisor by 1m to get the correct result. It gets even weirder if you multiply the sellUom by 1m.
Then the result is:
RatioRight = (sellUom * 1m) / (buyUom)
RatioRight 0.000453
I'm guessing this is something to do with how SQL Server stores the Decimal(28,18) and how Linq converts the divide command.
Update: All values are Decimals
Update 2:
Looks like this is entirely a SQL rounding thing. Removing the .Net from the equation
select top 1 uom.NumberOfDefaultUnits
from UnitOfMeasures uom
where uom.Id = 13
select (select top 1 uom.NumberOfDefaultUnits
from UnitOfMeasures uom
where uom.Id = 13 )
/
(select top 1 uom.NumberOfDefaultUnits
from UnitOfMeasures uom
where uom.Id = 9)
The first query returns: 453.592370000000000000
The second: 0.0004535923

This is definitely due to how SQL Server handles precision & scale in calculations.
If you change #mikeymouses example to use a scale of 6 for #p1 you get consistent results:
DECLARE #p2 Decimal(28,6) = 1000000
DECLARE #p3 Decimal(28,18) = 453.59237
select #p3 / #p2
DECLARE #p1 Decimal(19,18) = 1 -- This is how Linq sends the * 1m
select #p3 / (#p2 *#p1)
select (#p3 *#p1) / #p2
Results:
0.0004535923700000000000
0.00045359237000
0.00045359237000000000
The precision and scale outcomes are documented on MSDN, but the key points are:
For Multiplication:
precision of result = p1 + p2 + 1
Scale of result = s1 + s2
For Division:
Precision of result = p1 - s1 + s2 + max(6, s1 + p2 + 1)
Scale of result = max(6, s1 + p2 + 1)
Where p1 and p2 are the precision of the operands and s1, s2 are the scale of the operands.
It should be remembered that the result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is reduced to prevent the integral part of a result from being truncated.
I think this is what is happening here.

It looks like this is just how SQL deals with these declared decimal sizes being divided by eachother
DECLARE #p2 Decimal(28,18) = 1000000
DECLARE #p3 Decimal(28,18) = 453.59237
select #p3 / #p2
DECLARE #p1 Decimal(19,18) = 1 -- This is how Linq sends the * 1m
select #p3 / (#p2 *#p1)
select (#p3 *#p1) / #p2
Results:
0.0004535923
0.000453
0.00045359

Related

sql server decimal data type issue

Decimal data type in table Length = 9,Precision = 10,Scale = 2,
actual value in #Bvisits = 8.00
declare #Bvisits decimal,#ActualVisit decimal,#Ptax decimal = 1.995;
select #Bvisits = BalanceVisit from PakacgeTb where PackageID = 25306;
set #ActualVisit = #Bvisits - #Ptax;
select #ActualVisit as VIP
result is 6.
But as i use it in real stored procedure it updates result as 4.67 instead of 6 why is the issue occurring even if i use 1.9 instead of 1.995 still same issue occurs.
Try like this :
declare #Bvisits decimal(10,3),#ActualVisit decimal(10,3),#Ptax decimal(10,3) = 1.995;
select #Bvisits = BalanceVisit from PakacgeTb where PackageID = 25306;
set #ActualVisit = #Bvisits - #Ptax;
select #ActualVisit as VIP

Find latitude and longitude with in 2 miles radius from a given lat-long

I have an table with latitude and longitude.
I need to find all latitude and longitude with in 2 miles radius from a given lat-long.
I am fetching data using jquery ajax using C# from sqlserver database.
I need this in sqlserver query.
For example:
If I give 37.774546 and -122.433523, than I need to get all lat-long from the list that
are in 2 miles radius.
Since you're on SQL 2008, you can take advantage of the native geospatial querying capabilities. For instance:
declare #distance_in_meters int = 2000;
declare #point geography = geography::Point( 37.774546, -122.433523, 4326 );
declare #buffer geography = #point.STBuffer( #distance_in_meters );
select *
from dbo.yourtable
where #buffer.STContains( t.Point );
Note that dbo.yourtable.Point will have to have the geography datatype and the actual data will have to have the same SRID as the reference point. But it's pretty straightforward.
You can try this query for Database:
Declare #Miles int=10;
Declare #Latitude varchar(15);
Declare #Longitude varchar(15);
Set #Latitude = 'set input lat here'
Set #Longitude = 'set input long here'
Select Round(((DEGREES(ACOS(SIN(RADIANS(Latitude)) * SIN(RADIANS(#Latitude)) + COS(RADIANS(Latitude)) * COS(RADIANS(#Latitude)) * COS(RADIANS(Longitude - #Longitude))))) * 69.09),0) As Distance,
Latitude as lat,
Longitude as longi
From tablename
Where #Miles > ((DEGREES(ACOS(SIN(RADIANS(Latitude)) * SIN(RADIANS(#Latitude)) + COS(RADIANS(Latitude)) * COS(RADIANS(#Latitude)) * COS(RADIANS(Longitude - #Longitude))))) * 69.09)
Hope this will help you.
Do it that way.
var bounds = map.getBounds();
var ne = bounds.getNorthEast();
var sw = bounds.getSouthWest();
var nw = new google.maps.LatLng(ne.lat(), sw.lng())
var x = google.maps.geometry.spherical.computeDistanceBetween(ne, nw);
var y = google.maps.geometry.spherical.computeDistanceBetween(sw, nw);

Convert SQL Server varbinary(max) into a set of primary keys of type int

Disclaimer: not my code, not my database design!
I have a column of censusblocks(varbinary(max), null) in a MS SQL Server 2008 db table (call it foo for simplicity).
This column is actually a null or 1 to n long list of int. The ints are actually foreign keys to another table (call it censusblock with a pk id of type of int), numbering from 1 to ~9600000.
I want to query to extract the censusblocks list from foo, and use the extracted list of int from each row to look up the corresponding censusblock row. There's a long, boring rest of the query that will be used from there, but it needs to start with the census blocks pulled from the foo table's censusblocks column.
This conversion-and-look-up is currently handled on the middle tier, with a small .NET utility class to convert from List<int> to byte[] (and vice versa), which is then written into/read from the db as varbinary. I would like to do the same thing, purely in SQL.
The desired query would go something along the lines of
SELECT f.id, c.id
FROM foo f
LEFT OUTER JOIN censusblock c ON
c.id IN f.censusblocks --this is where the magic happens
where f.id in (1,2)
Which would result in:
f.id | c.id
1 8437314
1 8438819
1 8439744
1 8441795
1 8442741
1 8444984
1 8445568
1 8445641
1 8447953
2 5860657
2 5866881
2 5866881
2 5866858
2 5862557
2 5870475
2 5868983
2 5865207
2 5863465
2 5867301
2 5864057
2 5862256
NB: the 7-digit results are coincidental. The range is, as stated above, 1-7 digits.
The actual censusblocks column looks like
SELECT TOP 2 censusblocks FROM foo
which results in
censublocks
0x80BE4280C42380C7C080CFC380D37580DC3880DE8080DEC980E7D1
0x596D3159858159856A59749D59938B598DB7597EF7597829598725597A79597370
For further clarification, here's the guts of the .NET utility classes conversion methods:
public static List<int> getIntegersFromBytes(byte[] data)
{
List<int> values = new List<int>();
if (data != null && data.Length > 2)
{
long ids = data.Length / 3;
byte[] oneId = new byte[4];
oneId[0] = 0;
for (long i = 0; i < ids; i++)
{
oneId[0] = 0;
Array.Copy(data, i * 3, oneId, 1, 3);
if (BitConverter.IsLittleEndian)
{ Array.Reverse(oneId); }
values.Add(BitConverter.ToInt32(oneId, 0));
}}
return values;
}
public static byte[] getBytesFromIntegers(List<int> values)
{
byte[] data = null;
if (values != null && values.Count > 0)
{
data = new byte[values.Count * 3];
int count = 0;
byte[] idBytes = null;
foreach (int id in values)
{
idBytes = BitConverter.GetBytes(id);
if (BitConverter.IsLittleEndian)
{ Array.Reverse(idBytes); }
Array.Copy(idBytes, 1, data, count * 3, 3);
count++;
} }
return data;
}
An example of how this might be done. It is unlikely to scale brilliantly.
If you have a numbers table in your database it should be used in place of nums_cte.
This works by converting the binary value to a literal hex string, then reading it in 8-character chunks
-- create test data
DECLARE #foo TABLE
(id int ,
censusblocks varbinary(max)
)
DECLARE #censusblock TABLE
(id int)
INSERT #censusblock (id)
VALUES(1),(2),(1003),(5030),(5031),(2),(6)
INSERT #foo (id,censusblocks)
VALUES (1,0x0000000100000002000003EB),
(2,0x000013A6000013A7)
--query
DECLARE #biMaxLen bigint
SELECT #biMaxLen = MAX(LEN(CONVERT(varchar(max),censusblocks,2))) FROM #foo
;with nums_cte
AS
(
SELECT TOP (#biMaxLen) ((ROW_NUMBER() OVER (ORDER BY a.type) - 1) * 8) AS n
FROM master..spt_values as a
CROSS JOIN master..spt_values as b
)
,binCTE
AS
(
SELECT d.id, CAST(CONVERT(binary(4),SUBSTRING(s,n + 1,8),2) AS int) as cblock
FROM (SELECT Id, CONVERT(varchar(max),censusblocks,2) AS s FROM #foo) AS d
JOIN nums_cte
ON n < LEN(d.s)
)
SELECT *
FROM binCTE as b
LEFT
JOIN #censusblock c
ON c.id = b.cblock
ORDER BY b.id, b.cblock
You could also consider adding your existing .Net conversion methods into the database as an assembly and accessing them through CLR functions.
This is off-topic, but I couldn't resist writing these conversions so they use IEnumerables instead of arrays and Lists. This might not be faster per se, but is more general and would allow you to perform the conversion without loading the whole array at once, which may be helpful if the arrays you are dealing with are large.
Here it is, for what it's worth:
static IEnumerable<int> BytesToInts(IEnumerable<byte> bytes) {
var buff = new byte[4];
using (var en = bytes.GetEnumerator()) {
while (en.MoveNext()) {
buff[0] = en.Current;
if (en.MoveNext()) {
buff[1] = en.Current;
if (en.MoveNext()) {
buff[2] = en.Current;
if (en.MoveNext()) {
buff[3] = en.Current;
if (BitConverter.IsLittleEndian)
Array.Reverse(buff);
yield return BitConverter.ToInt32(buff, 0);
continue;
}
}
}
throw new ArgumentException("Wrong number of bytes.", "bytes");
}
}
}
static IEnumerable<byte> IntsToBytes(IEnumerable<int> ints) {
if (BitConverter.IsLittleEndian)
return ints.SelectMany(
b => {
var buff = BitConverter.GetBytes(b);
Array.Reverse(buff);
return buff;
}
);
return ints.SelectMany(BitConverter.GetBytes);
}
Your code seems to like encoding an int into 3 bytes instead of 4, which would cause problems with values that don't fit into 3 bytes (including negatives) - is that intentional?
BTW, you should be able to adapt this (or your) code for execution in SQL Server CLR. This is not exactly "in SQL", but is "in DBMS".
you can use Convert(int, censusBlock) to convert the varchar value to int value.
the you can join on that column.
Or have i misunderstood the question?

Entity Framework / SQL Server strange decimal division behaviour

I have a table in my SQL server 2008 R2 database which includes two nullable decimal(16,6) columns. Let's call them column1 and column2.
When I try to run a Linq query against the entity generated from this table:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2) / 2m
}
);
I get a System.OverflowException if column1 + column2 >= 15846. The message of the exception is only:
Conversion overflows.
With a bit of trial and error I've managed to make the query work with the following:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2).HasValue ?
(r.Column1 + r.Column2).Value / 2m : 0
}
);
However, I was wondering if anyone could explain what was going wrong with the initial query.
Edit
The first query generates this SQL:
SELECT
1 AS [C1],
([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18)) AS [C2]
FROM [dbo].[Table] AS [Extent1]
With a value of 10000 for both columns, running the query manually in SSMS the result is 10000.0000000000000000000000000 (25 decimal zeros).
The second query has this SQL:
SELECT
1 AS [C1],
CASE WHEN ([Extent1].[Column1] + [Extent1].[Column2] IS NOT NULL)
THEN ([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18))
ELSE cast(0 as decimal(18))
END AS [C2]
FROM [dbo].[Table] AS [Extent1]
Running the query in SSMS returns 10000.00000000000000000000 (20 decimal zeros). Apparently there is a problem when EF tries to convert the first value (with 25 decimal zeros) into a decimal but with the second (with 20 decimal zeros) it works.
In the meantime it turned out that the problem also occurs with non-nullable columns and even a single decimal(16, 6) column. The following ...
Table.Select(r => new Foo
{
Bar = r.Column1 / 2m
}
);
... throws the same conversion exception (with a value of 20000 in the Column1).
Why do those two SQL queries result in two different numbers of digits?
And why can't the first number be converted into a decimal by EF?

Natural (human alpha-numeric) sort in Microsoft SQL 2005

We have a large database on which we have DB side pagination. This is quick, returning a page of 50 rows from millions of records in a small fraction of a second.
Users can define their own sort, basically choosing what column to sort by. Columns are dynamic - some have numeric values, some dates and some text.
While most sort as expected text sorts in a dumb way. Well, I say dumb, it makes sense to computers, but frustrates users.
For instance, sorting by a string record id gives something like:
rec1
rec10
rec14
rec2
rec20
rec3
rec4
...and so on.
I want this to take account of the number, so:
rec1
rec2
rec3
rec4
rec10
rec14
rec20
I can't control the input (otherwise I'd just format in leading 000s) and I can't rely on a single format - some are things like "{alpha code}-{dept code}-{rec id}".
I know a few ways to do this in C#, but can't pull down all the records to sort them, as that would be to slow.
Does anyone know a way to quickly apply a natural sort in Sql server?
We're using:
ROW_NUMBER() over (order by {field name} asc)
And then we're paging by that.
We can add triggers, although we wouldn't. All their input is parametrised and the like, but I can't change the format - if they put in "rec2" and "rec10" they expect them to be returned just like that, and in natural order.
We have valid user input that follows different formats for different clients.
One might go rec1, rec2, rec3, ... rec100, rec101
While another might go: grp1rec1, grp1rec2, ... grp20rec300, grp20rec301
When I say we can't control the input I mean that we can't force users to change these standards - they have a value like grp1rec1 and I can't reformat it as grp01rec001, as that would be changing something used for lookups and linking to external systems.
These formats vary a lot, but are often mixtures of letters and numbers.
Sorting these in C# is easy - just break it up into { "grp", 20, "rec", 301 } and then compare sequence values in turn.
However there may be millions of records and the data is paged, I need the sort to be done on the SQL server.
SQL server sorts by value, not comparison - in C# I can split the values out to compare, but in SQL I need some logic that (very quickly) gets a single value that consistently sorts.
#moebius - your answer might work, but it does feel like an ugly compromise to add a sort-key for all these text values.
order by LEN(value), value
Not perfect, but works well in a lot of cases.
Most of the SQL-based solutions I have seen break when the data gets complex enough (e.g. more than one or two numbers in it). Initially I tried implementing a NaturalSort function in T-SQL that met my requirements (among other things, handles an arbitrary number of numbers within the string), but the performance was way too slow.
Ultimately, I wrote a scalar CLR function in C# to allow for a natural sort, and even with unoptimized code the performance calling it from SQL Server is blindingly fast. It has the following characteristics:
will sort the first 1,000 characters or so correctly (easily modified in code or made into a parameter)
properly sorts decimals, so 123.333 comes before 123.45
because of above, will likely NOT sort things like IP addresses correctly; if you wish different behaviour, modify the code
supports sorting a string with an arbitrary number of numbers within it
will correctly sort numbers up to 25 digits long (easily modified in code or made into a parameter)
The code is here:
using System;
using System.Data.SqlTypes;
using System.Text;
using Microsoft.SqlServer.Server;
public class UDF
{
[SqlFunction(DataAccess = DataAccessKind.None, IsDeterministic=true)]
public static SqlString Naturalize(string val)
{
if (String.IsNullOrEmpty(val))
return val;
while(val.Contains(" "))
val = val.Replace(" ", " ");
const int maxLength = 1000;
const int padLength = 25;
bool inNumber = false;
bool isDecimal = false;
int numStart = 0;
int numLength = 0;
int length = val.Length < maxLength ? val.Length : maxLength;
//TODO: optimize this so that we exit for loop once sb.ToString() >= maxLength
var sb = new StringBuilder();
for (var i = 0; i < length; i++)
{
int charCode = (int)val[i];
if (charCode >= 48 && charCode <= 57)
{
if (!inNumber)
{
numStart = i;
numLength = 1;
inNumber = true;
continue;
}
numLength++;
continue;
}
if (inNumber)
{
sb.Append(PadNumber(val.Substring(numStart, numLength), isDecimal, padLength));
inNumber = false;
}
isDecimal = (charCode == 46);
sb.Append(val[i]);
}
if (inNumber)
sb.Append(PadNumber(val.Substring(numStart, numLength), isDecimal, padLength));
var ret = sb.ToString();
if (ret.Length > maxLength)
return ret.Substring(0, maxLength);
return ret;
}
static string PadNumber(string num, bool isDecimal, int padLength)
{
return isDecimal ? num.PadRight(padLength, '0') : num.PadLeft(padLength, '0');
}
}
To register this so that you can call it from SQL Server, run the following commands in Query Analyzer:
CREATE ASSEMBLY SqlServerClr FROM 'SqlServerClr.dll' --put the full path to DLL here
go
CREATE FUNCTION Naturalize(#val as nvarchar(max)) RETURNS nvarchar(1000)
EXTERNAL NAME SqlServerClr.UDF.Naturalize
go
Then, you can use it like so:
select *
from MyTable
order by dbo.Naturalize(MyTextField)
Note: If you get an error in SQL Server along the lines of Execution of user code in the .NET Framework is disabled. Enable "clr enabled" configuration option., follow the instructions here to enable it. Make sure you consider the security implications before doing so. If you are not the db admin, make sure you discuss this with your admin before making any changes to the server configuration.
Note2: This code does not properly support internationalization (e.g., assumes the decimal marker is ".", is not optimized for speed, etc. Suggestions on improving it are welcome!
Edit: Renamed the function to Naturalize instead of NaturalSort, since it does not do any actual sorting.
I know this is an old question but I just came across it and since it's not got an accepted answer.
I have always used ways similar to this:
SELECT [Column] FROM [Table]
ORDER BY RIGHT(REPLICATE('0', 1000) + LTRIM(RTRIM(CAST([Column] AS VARCHAR(MAX)))), 1000)
The only common times that this has issues is if your column won't cast to a VARCHAR(MAX), or if LEN([Column]) > 1000 (but you can change that 1000 to something else if you want), but you can use this rough idea for what you need.
Also this is much worse performance than normal ORDER BY [Column], but it does give you the result asked for in the OP.
Edit: Just to further clarify, this the above will not work if you have decimal values such as having 1, 1.15 and 1.5, (they will sort as {1, 1.5, 1.15}) as that is not what is asked for in the OP, but that can easily be done by:
SELECT [Column] FROM [Table]
ORDER BY REPLACE(RIGHT(REPLICATE('0', 1000) + LTRIM(RTRIM(CAST([Column] AS VARCHAR(MAX)))) + REPLICATE('0', 100 - CHARINDEX('.', REVERSE(LTRIM(RTRIM(CAST([Column] AS VARCHAR(MAX))))), 1)), 1000), '.', '0')
Result: {1, 1.15, 1.5}
And still all entirely within SQL. This will not sort IP addresses because you're now getting into very specific number combinations as opposed to simple text + number.
RedFilter's answer is great for reasonably sized datasets where indexing is not critical, however if you want an index, several tweaks are required.
First, mark the function as not doing any data access and being deterministic and precise:
[SqlFunction(DataAccess = DataAccessKind.None,
SystemDataAccess = SystemDataAccessKind.None,
IsDeterministic = true, IsPrecise = true)]
Next, MSSQL has a 900 byte limit on the index key size, so if the naturalized value is the only value in the index, it must be at most 450 characters long. If the index includes multiple columns, the return value must be even smaller. Two changes:
CREATE FUNCTION Naturalize(#str AS nvarchar(max)) RETURNS nvarchar(450)
EXTERNAL NAME ClrExtensions.Util.Naturalize
and in the C# code:
const int maxLength = 450;
Finally, you will need to add a computed column to your table, and it must be persisted (because MSSQL cannot prove that Naturalize is deterministic and precise), which means the naturalized value is actually stored in the table but is still maintained automatically:
ALTER TABLE YourTable ADD nameNaturalized AS dbo.Naturalize(name) PERSISTED
You can now create the index!
CREATE INDEX idx_YourTable_n ON YourTable (nameNaturalized)
I've also made a couple of changes to RedFilter's code: using chars for clarity, incorporating duplicate space removal into the main loop, exiting once the result is longer than the limit, setting maximum length without substring etc. Here's the result:
using System.Data.SqlTypes;
using System.Text;
using Microsoft.SqlServer.Server;
public static class Util
{
[SqlFunction(DataAccess = DataAccessKind.None, SystemDataAccess = SystemDataAccessKind.None, IsDeterministic = true, IsPrecise = true)]
public static SqlString Naturalize(string str)
{
if (string.IsNullOrEmpty(str))
return str;
const int maxLength = 450;
const int padLength = 15;
bool isDecimal = false;
bool wasSpace = false;
int numStart = 0;
int numLength = 0;
var sb = new StringBuilder();
for (var i = 0; i < str.Length; i++)
{
char c = str[i];
if (c >= '0' && c <= '9')
{
if (numLength == 0)
numStart = i;
numLength++;
}
else
{
if (numLength > 0)
{
sb.Append(pad(str.Substring(numStart, numLength), isDecimal, padLength));
numLength = 0;
}
if (c != ' ' || !wasSpace)
sb.Append(c);
isDecimal = c == '.';
if (sb.Length > maxLength)
break;
}
wasSpace = c == ' ';
}
if (numLength > 0)
sb.Append(pad(str.Substring(numStart, numLength), isDecimal, padLength));
if (sb.Length > maxLength)
sb.Length = maxLength;
return sb.ToString();
}
private static string pad(string num, bool isDecimal, int padLength)
{
return isDecimal ? num.PadRight(padLength, '0') : num.PadLeft(padLength, '0');
}
}
Here's a solution written for SQL 2000. It can probably be improved for newer SQL versions.
/**
* Returns a string formatted for natural sorting. This function is very useful when having to sort alpha-numeric strings.
*
* #author Alexandre Potvin Latreille (plalx)
* #param {nvarchar(4000)} string The formatted string.
* #param {int} numberLength The length each number should have (including padding). This should be the length of the longest number. Defaults to 10.
* #param {char(50)} sameOrderChars A list of characters that should have the same order. Ex: '.-/'. Defaults to empty string.
*
* #return {nvarchar(4000)} A string for natural sorting.
* Example of use:
*
* SELECT Name FROM TableA ORDER BY Name
* TableA (unordered) TableA (ordered)
* ------------ ------------
* ID Name ID Name
* 1. A1. 1. A1-1.
* 2. A1-1. 2. A1.
* 3. R1 --> 3. R1
* 4. R11 4. R11
* 5. R2 5. R2
*
*
* As we can see, humans would expect A1., A1-1., R1, R2, R11 but that's not how SQL is sorting it.
* We can use this function to fix this.
*
* SELECT Name FROM TableA ORDER BY dbo.udf_NaturalSortFormat(Name, default, '.-')
* TableA (unordered) TableA (ordered)
* ------------ ------------
* ID Name ID Name
* 1. A1. 1. A1.
* 2. A1-1. 2. A1-1.
* 3. R1 --> 3. R1
* 4. R11 4. R2
* 5. R2 5. R11
*/
ALTER FUNCTION [dbo].[udf_NaturalSortFormat](
#string nvarchar(4000),
#numberLength int = 10,
#sameOrderChars char(50) = ''
)
RETURNS varchar(4000)
AS
BEGIN
DECLARE #sortString varchar(4000),
#numStartIndex int,
#numEndIndex int,
#padLength int,
#totalPadLength int,
#i int,
#sameOrderCharsLen int;
SELECT
#totalPadLength = 0,
#string = RTRIM(LTRIM(#string)),
#sortString = #string,
#numStartIndex = PATINDEX('%[0-9]%', #string),
#numEndIndex = 0,
#i = 1,
#sameOrderCharsLen = LEN(#sameOrderChars);
-- Replace all char that have the same order by a space.
WHILE (#i <= #sameOrderCharsLen)
BEGIN
SET #sortString = REPLACE(#sortString, SUBSTRING(#sameOrderChars, #i, 1), ' ');
SET #i = #i + 1;
END
-- Pad numbers with zeros.
WHILE (#numStartIndex <> 0)
BEGIN
SET #numStartIndex = #numStartIndex + #numEndIndex;
SET #numEndIndex = #numStartIndex;
WHILE(PATINDEX('[0-9]', SUBSTRING(#string, #numEndIndex, 1)) = 1)
BEGIN
SET #numEndIndex = #numEndIndex + 1;
END
SET #numEndIndex = #numEndIndex - 1;
SET #padLength = #numberLength - (#numEndIndex + 1 - #numStartIndex);
IF #padLength < 0
BEGIN
SET #padLength = 0;
END
SET #sortString = STUFF(
#sortString,
#numStartIndex + #totalPadLength,
0,
REPLICATE('0', #padLength)
);
SET #totalPadLength = #totalPadLength + #padLength;
SET #numStartIndex = PATINDEX('%[0-9]%', RIGHT(#string, LEN(#string) - #numEndIndex));
END
RETURN #sortString;
END
I know this is a bit old at this point, but in my search for a better solution, I came across this question. I'm currently using a function to order by. It works fine for my purpose of sorting records which are named with mixed alpha numeric ('item 1', 'item 10', 'item 2', etc)
CREATE FUNCTION [dbo].[fnMixSort]
(
#ColValue NVARCHAR(255)
)
RETURNS NVARCHAR(1000)
AS
BEGIN
DECLARE #p1 NVARCHAR(255),
#p2 NVARCHAR(255),
#p3 NVARCHAR(255),
#p4 NVARCHAR(255),
#Index TINYINT
IF #ColValue LIKE '[a-z]%'
SELECT #Index = PATINDEX('%[0-9]%', #ColValue),
#p1 = LEFT(CASE WHEN #Index = 0 THEN #ColValue ELSE LEFT(#ColValue, #Index - 1) END + REPLICATE(' ', 255), 255),
#ColValue = CASE WHEN #Index = 0 THEN '' ELSE SUBSTRING(#ColValue, #Index, 255) END
ELSE
SELECT #p1 = REPLICATE(' ', 255)
SELECT #Index = PATINDEX('%[^0-9]%', #ColValue)
IF #Index = 0
SELECT #p2 = RIGHT(REPLICATE(' ', 255) + #ColValue, 255),
#ColValue = ''
ELSE
SELECT #p2 = RIGHT(REPLICATE(' ', 255) + LEFT(#ColValue, #Index - 1), 255),
#ColValue = SUBSTRING(#ColValue, #Index, 255)
SELECT #Index = PATINDEX('%[0-9,a-z]%', #ColValue)
IF #Index = 0
SELECT #p3 = REPLICATE(' ', 255)
ELSE
SELECT #p3 = LEFT(REPLICATE(' ', 255) + LEFT(#ColValue, #Index - 1), 255),
#ColValue = SUBSTRING(#ColValue, #Index, 255)
IF PATINDEX('%[^0-9]%', #ColValue) = 0
SELECT #p4 = RIGHT(REPLICATE(' ', 255) + #ColValue, 255)
ELSE
SELECT #p4 = LEFT(#ColValue + REPLICATE(' ', 255), 255)
RETURN #p1 + #p2 + #p3 + #p4
END
Then call
select item_name from my_table order by fnMixSort(item_name)
It easily triples the processing time for a simple data read, so it may not be the perfect solution.
Here is an other solution that I like:
http://www.dreamchain.com/sql-and-alpha-numeric-sort-order/
It's not Microsoft SQL, but since I ended up here when I was searching for a solution for Postgres, I thought adding this here would help others.
EDIT: Here is the code, in case the link goes away.
CREATE or REPLACE FUNCTION pad_numbers(text) RETURNS text AS $$
SELECT regexp_replace(regexp_replace(regexp_replace(regexp_replace(($1 collate "C"),
E'(^|\\D)(\\d{1,3}($|\\D))', E'\\1000\\2', 'g'),
E'(^|\\D)(\\d{4,6}($|\\D))', E'\\1000\\2', 'g'),
E'(^|\\D)(\\d{7}($|\\D))', E'\\100\\2', 'g'),
E'(^|\\D)(\\d{8}($|\\D))', E'\\10\\2', 'g');
$$ LANGUAGE SQL;
"C" is the default collation in postgresql; you may specify any collation you desire, or remove the collation statement if you can be certain your table columns will never have a nondeterministic collation assigned.
usage:
SELECT * FROM wtf w
WHERE TRUE
ORDER BY pad_numbers(w.my_alphanumeric_field)
For the following varchar data:
BR1
BR2
External Location
IR1
IR2
IR3
IR4
IR5
IR6
IR7
IR8
IR9
IR10
IR11
IR12
IR13
IR14
IR16
IR17
IR15
VCR
This worked best for me:
ORDER BY substring(fieldName, 1, 1), LEN(fieldName)
If you're having trouble loading the data from the DB to sort in C#, then I'm sure you'll be disappointed with any approach at doing it programmatically in the DB. When the server is going to sort, it's got to calculate the "perceived" order just as you would have -- every time.
I'd suggest that you add an additional column to store the preprocessed sortable string, using some C# method, when the data is first inserted. You might try to convert the numerics into fixed-width ranges, for example, so "xyz1" would turn into "xyz00000001". Then you could use normal SQL Server sorting.
At the risk of tooting my own horn, I wrote a CodeProject article implementing the problem as posed in the CodingHorror article. Feel free to steal from my code.
Simply you sort by
ORDER BY
cast (substring(name,(PATINDEX('%[0-9]%',name)),len(name))as int)
##
I've just read a article somewhere about such a topic. The key point is: you only need the integer value to sort data, while the 'rec' string belongs to the UI. You could split the information in two fields, say alpha and num, sort by alpha and num (separately) and then showing a string composed by alpha + num. You could use a computed column to compose the string, or a view.
Hope it helps
You can use the following code to resolve the problem:
Select *,
substring(Cote,1,len(Cote) - Len(RIGHT(Cote, LEN(Cote) - PATINDEX('%[0-9]%', Cote)+1)))alpha,
CAST(RIGHT(Cote, LEN(Cote) - PATINDEX('%[0-9]%', Cote)+1) AS INT)intv
FROM Documents
left outer join Sites ON Sites.IDSite = Documents.IDSite
Order BY alpha, intv
regards,
rabihkahaleh#hotmail.com
I'm fashionably late to the party as usual. Nevertheless, here is my attempt at an answer that seems to work well (I would say that). It assumes text with digits at the end, like in the original example data.
First a function that won't end up winning a "pretty SQL" competition anytime soon.
CREATE FUNCTION udfAlphaNumericSortHelper (
#string varchar(max)
)
RETURNS #results TABLE (
txt varchar(max),
num float
)
AS
BEGIN
DECLARE #txt varchar(max) = #string
DECLARE #numStr varchar(max) = ''
DECLARE #num float = 0
DECLARE #lastChar varchar(1) = ''
set #lastChar = RIGHT(#txt, 1)
WHILE #lastChar <> '' and #lastChar is not null
BEGIN
IF ISNUMERIC(#lastChar) = 1
BEGIN
set #numStr = #lastChar + #numStr
set #txt = Substring(#txt, 0, len(#txt))
set #lastChar = RIGHT(#txt, 1)
END
ELSE
BEGIN
set #lastChar = null
END
END
SET #num = CAST(#numStr as float)
INSERT INTO #results select #txt, #num
RETURN;
END
Then call it like below:
declare #str nvarchar(250) = 'sox,fox,jen1,Jen0,jen15,jen02,jen0004,fox00,rec1,rec10,jen3,rec14,rec2,rec20,rec3,rec4,zip1,zip1.32,zip1.33,zip1.3,TT0001,TT01,TT002'
SELECT tbl.value --, sorter.txt, sorter.num
FROM STRING_SPLIT(#str, ',') as tbl
CROSS APPLY dbo.udfAlphaNumericSortHelper(value) as sorter
ORDER BY sorter.txt, sorter.num, len(tbl.value)
With results:
fox
fox00
Jen0
jen1
jen02
jen3
jen0004
jen15
rec1
rec2
rec3
rec4
rec10
rec14
rec20
sox
TT01
TT0001
TT002
zip1
zip1.3
zip1.32
zip1.33
I still don't understand (probably because of my poor English).
You could try:
ROW_NUMBER() OVER (ORDER BY dbo.human_sort(field_name) ASC)
But it won't work for millions of records.
That why I suggested to use trigger which fills separate column with human value.
Moreover:
built-in T-SQL functions are really
slow and Microsoft suggest to use
.NET functions instead.
human value is constant so there is no point calculating it each time
when query runs.

Resources