Entity Framework / SQL Server strange decimal division behaviour - sql-server

I have a table in my SQL server 2008 R2 database which includes two nullable decimal(16,6) columns. Let's call them column1 and column2.
When I try to run a Linq query against the entity generated from this table:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2) / 2m
}
);
I get a System.OverflowException if column1 + column2 >= 15846. The message of the exception is only:
Conversion overflows.
With a bit of trial and error I've managed to make the query work with the following:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2).HasValue ?
(r.Column1 + r.Column2).Value / 2m : 0
}
);
However, I was wondering if anyone could explain what was going wrong with the initial query.
Edit
The first query generates this SQL:
SELECT
1 AS [C1],
([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18)) AS [C2]
FROM [dbo].[Table] AS [Extent1]
With a value of 10000 for both columns, running the query manually in SSMS the result is 10000.0000000000000000000000000 (25 decimal zeros).
The second query has this SQL:
SELECT
1 AS [C1],
CASE WHEN ([Extent1].[Column1] + [Extent1].[Column2] IS NOT NULL)
THEN ([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18))
ELSE cast(0 as decimal(18))
END AS [C2]
FROM [dbo].[Table] AS [Extent1]
Running the query in SSMS returns 10000.00000000000000000000 (20 decimal zeros). Apparently there is a problem when EF tries to convert the first value (with 25 decimal zeros) into a decimal but with the second (with 20 decimal zeros) it works.
In the meantime it turned out that the problem also occurs with non-nullable columns and even a single decimal(16, 6) column. The following ...
Table.Select(r => new Foo
{
Bar = r.Column1 / 2m
}
);
... throws the same conversion exception (with a value of 20000 in the Column1).
Why do those two SQL queries result in two different numbers of digits?
And why can't the first number be converted into a decimal by EF?

Related

Sqlite3 calculate the percentages from totals

Question(*):
The total number of cases and deaths as a percentage of the population, for each country (with country, % cases of population, % deaths of population as columns)
I have two tables :
countriesAffected(countriesAndTerritories,geoId,countryterritoryCode,popData2019,continentExp)
victimsCases(dateRep,cases,deaths,geoId)
where primary key(geoid)
I tried to do (*) by this method:
SELECT countriesAndTerritories, (100 *SUM(victimsCases.cases) / popData2019)as "cases" ,(100 * SUM(deaths) / popData2019) as "deaths"
FROM countriesAffected
INNER JOIN victimsCases ON victimsCases.geoId = countriesAffected.geoId
GROUP BY countriesAndTerritories
ORDER BY countriesAndTerritories DESC;
Error: near line 2: near "SELECT countriesAndTerritories": syntax error
But for some reason I get all types of syntax errors, i tried to sort it out but with no results. And not sure where did i went wrong.
If you are getting the error Error: near line 2: near "SELECT countriesAndTerritories": syntax error then the issue is with LINE 1 (perhaps no ; at the end of line 1).
Otherwise your query works albiet probably not as intended (as you may well want decimal places for the percentages).
Consider the following (that shows your SQL with additional SQL added to work as intended (see casesV2 and deathsV2 that utilise CAST to force INTEGER to REAL)).
DROP TABLE If EXISTS victimsCases;
DROP TABLE IF EXISTS countriesAffected;
CREATE TABLE IF NOT EXISTS countriesAffected (countriesAndTerritories TEXT,geoId INTEGER PRIMARY KEY,countryterritoryCode TEXT,popData2019 INTEGER,continentExp TEXT);
CREATE TABLE IF NOT EXISTS victimsCases (dateRep TEXT,cases INTEGER ,deaths INTEGER,geoId INTEGER);
INSERT INTO countriesAffected VALUES
('X',1,'XXX',10000,'?'),('Y',2,'YYY',20000,'?'),('Z',3,'ZZZ',30000,'?')
;
INSERT INTO victimsCases VALUES
('2019-01-01',100,20,1),('2019-01-02',100,25,1),('2019-01-03',100,15,1),
('2019-01-01',30,5,2),('2019-01-02',33,2,2),
('2019-01-01',45,17,3),('2019-01-02',61,4,3),('2019-01-03',75,7,3)
;
SELECT countriesAndTerritories,
(100 *SUM(victimsCases.cases) / popData2019)as "cases", /* ORIGINAL */
(100 * SUM(deaths) / popData2019) as "deaths", /* ORIGINAL */
CAST(SUM(victimsCases.cases) AS FLOAT) / popData2019 * 100 AS "casesV2",
CAST(SUM(victimscases.deaths) AS FLOAT) / popData2019 * 100 as "deathsV2"
FROM countriesAffected
INNER JOIN victimsCases ON victimsCases.geoId = countriesAffected.geoId
GROUP BY countriesAndTerritories
ORDER BY countriesAndTerritories DESC;
DROP TABLE If EXISTS victimsCases;
DROP TABLE IF EXISTS countriesAffected;
The result of the above is :-

SQL Server geography: Reduce size (decimal precision) of WKT text

For my farming app, a stored proc retrieves paddock/field boundaries that are stored as SQL Server geography data type, for display on the user's mobile device.
SQL Server's .ToString() and .STAsText() functions render each vertex as a lat/long pair, to 15 decimal places. From this answer, 15 decimals defines a location to within the width of an atom! To the nearest metre would be good enough for me.
The resulting overly-precise payload is extremely large and too slow for use on larger farms.
From my SQL Server geography data, I would like to produce WKT that is formatted to 4 or 5 decimals. I could find no built-in methods, but my best leads are:
Postgis and Google Cloud "BigQuery" have the ST_SNAPTOGRID function, which would be perfect, and
Regex might be useful, e.g. this answer, but SQL Server doesn't seem to have a regex replace.
I would think this is a common problem: is there a simple solution?
By stepping through #iamdave's excellent answer, and using that same approach, it looks like we need split only on periods.... I think we can ignore all parantheses and commas, and ignore the POLYGON prefix (which means it'll work across other GEOGRAPHY types, such as MULTIPOLYGON.)
i.e. Each time we find a period, grab only the next 4 characters after it, and throw away any numbers after that (until we hit a non-number.)
This works for me (using #iamdave's test data):
DECLARE #wkt NVARCHAR(MAX), #wktShort NVARCHAR(MAX);
DECLARE #decimalPlaces int = 4;
SET #wkt = 'POLYGON((-121.973669 37.365336,-121.97367 37.365336,-121.973642 37.365309,-121.973415 37.365309,-121.973189 37.365309,-121.973002 37.365912,-121.972815 37.366515,-121.972796 37.366532,-121.972776 37.366549,-121.972627 37.366424,-121.972478 37.366299,-121.972422 37.366299,-121.972366 37.366299,-121.972298 37.366356,-121.97223 37.366412,-121.97215 37.366505,-121.97207 37.366598,-121.971908 37.366794,-121.971489 37.367353,-121.971396 37.367484,-121.971285 37.36769,-121.971173 37.367897,-121.971121 37.368072,-121.971068 37.368248,-121.971028 37.36847,-121.970987 37.368692,-121.970987 37.368779,-121.970987 37.368866,-121.970949 37.368923,-121.970912 37.36898,-121.970935 37.36898,-121.970958 37.36898,-121.970975 37.368933,-121.970993 37.368887,-121.971067 37.368807,-121.97114 37.368726,-121.971124 37.368705,-121.971108 37.368685,-121.971136 37.368698,-121.971163 37.368712,-121.97134 37.368531,-121.971516 37.368351,-121.971697 37.368186,-121.971878 37.368021,-121.972085 37.367846,-121.972293 37.36767,-121.972331 37.367629,-121.972369 37.367588,-121.972125 37.367763,-121.97188 37.367938,-121.971612 37.36815,-121.971345 37.368362,-121.971321 37.36835,-121.971297 37.368338,-121.971323 37.368298,-121.97135 37.368259,-121.971569 37.368062,-121.971788 37.367865,-121.971977 37.367716,-121.972166 37.367567,-121.972345 37.367442,-121.972524 37.367317,-121.972605 37.367272,-121.972687 37.367227,-121.972728 37.367227,-121.972769 37.367227,-121.972769 37.367259,-121.972769 37.367291,-121.972612 37.367416,-121.972454 37.367542,-121.972488 37.367558,-121.972521 37.367575,-121.972404 37.367674,-121.972286 37.367773,-121.972194 37.367851,-121.972101 37.367928,-121.972046 37.36799,-121.971991 37.368052,-121.972008 37.368052,-121.972025 37.368052,-121.972143 37.367959,-121.972261 37.367866,-121.972296 37.367866,-121.972276 37.36794,-121.972221 37.36798,-121.972094 37.368097,-121.971966 37.368214,-121.971956 37.368324,-121.971945 37.368433,-121.971907 37.368753,-121.971868 37.369073,-121.97184 37.369578,-121.971812 37.370083,-121.971798 37.370212,-121.971783 37.370342,-121.971542 37.370486,-121.971904 37.370324,-121.972085 37.37028,-121.972266 37.370236,-121.972559 37.370196,-121.972852 37.370155,-121.973019 37.370155,-121.973186 37.370155,-121.973232 37.370136,-121.973279 37.370116,-121.973307 37.370058,-121.973336 37.370001,-121.973363 37.369836,-121.973391 37.369671,-121.973419 37.369227,-121.973446 37.368784,-121.973429 37.368413,-121.973413 37.368041,-121.973361 37.367714,-121.973308 37.367387,-121.973285 37.367339,-121.973262 37.36729,-121.973126 37.3673,-121.972989 37.36731,-121.973066 37.36728,-121.973144 37.367251,-121.973269 37.367237,-121.973393 37.367223,-121.973443 37.367158,-121.973493 37.367093,-121.973518 37.36702,-121.973543 37.366947,-121.973582 37.366618,-121.973622 37.366288,-121.97366 37.365826,-121.973698 37.365363,-121.973669 37.365336))';
-- Split on '.', then get the next N decimals, and find the index of the first non-number.
-- Then recombine the fragments, skipping the unwanted numbers.
WITH points AS (
SELECT value, LEFT(value, #decimalPlaces) AS decimals, PATINDEX('%[^0-9]%', value) AS indx
FROM STRING_SPLIT(#wkt, '.')
)
SELECT #wktShort = STRING_AGG(IIF(indx < #decimalPlaces, '', decimals) + SUBSTRING(value, indx, LEN(value)), '.')
FROM points;
Comparing original vs shortened, we can see that each number is truncated to 4dp:
SELECT #wkt AS Text UNION ALL SELECT #wktShort;
Edit
I believe I may have misunderstood your question and you are looking to transmit the WKT rather than the binary representations of the polygons? If that is the case, my answer below still shows you how to chop off some decimal places (without rounding them). Just don't wrap the stuff(...) FOR XML in a STGeomFromText and you have the amended WKT.
When working with geography data types, it can be handy to maintain a very detailed 'master' version, from which you generate and persist less detailed versions based on your requirements.
An easy way to generate these reduced complexity polygons is to use the helpfully named Reduce function, which I think will actually help you in this situation.
If you would prefer to go down the route of reducing the number of decimal places you will either have to write a custom CLR function or enter the wonderful world of SQL Server string manipulation!
SQL Query
declare #DecimalPlaces int = 4; -- Specify the desired number of lat/long decimals
with g as(
select p.g -- Original polygon, for comparison purposes
,geography::STGeomFromText('POLYGON((' -- stripped apart and then recreated polygon from text, using a custom string split function. You won't be able to use the built in STRING_SPLIT here as it doesn't guarantee sort order.
+ stuff((select ', ' + left(s.item,charindex('.',s.item,0) + #DecimalPlaces) + substring(s.item,charindex(' ',s.item,0),charindex('.',s.item,charindex(' ',s.item,0)) - charindex(' ',s.item,0) + 1 + #DecimalPlaces)
from dbo.fn_StringSplitMax(replace(replace(p.g.STAsText(),'POLYGON ((',''),'))',''),', ',null) as s
for xml path(''), type).value('.', 'NVARCHAR(MAX)') -- STUFF and FOR XML mimics GROUP_CONCAT functionality seen in other SQL languages, to recombine shortened Points back into a Polygon string
,1,2,''
)
+ '))', 4326).MakeValid() as x -- Remember to make the polygon valid again, as you have been messing with the Point data
from(values(geography::STGeomFromText('POLYGON((-121.973669 37.365336,-121.97367 37.365336,-121.973642 37.365309,-121.973415 37.365309,-121.973189 37.365309,-121.973002 37.365912,-121.972815 37.366515,-121.972796 37.366532,-121.972776 37.366549,-121.972627 37.366424,-121.972478 37.366299,-121.972422 37.366299,-121.972366 37.366299,-121.972298 37.366356,-121.97223 37.366412,-121.97215 37.366505,-121.97207 37.366598,-121.971908 37.366794,-121.971489 37.367353,-121.971396 37.367484,-121.971285 37.36769,-121.971173 37.367897,-121.971121 37.368072,-121.971068 37.368248,-121.971028 37.36847,-121.970987 37.368692,-121.970987 37.368779,-121.970987 37.368866,-121.970949 37.368923,-121.970912 37.36898,-121.970935 37.36898,-121.970958 37.36898,-121.970975 37.368933,-121.970993 37.368887,-121.971067 37.368807,-121.97114 37.368726,-121.971124 37.368705,-121.971108 37.368685,-121.971136 37.368698,-121.971163 37.368712,-121.97134 37.368531,-121.971516 37.368351,-121.971697 37.368186,-121.971878 37.368021,-121.972085 37.367846,-121.972293 37.36767,-121.972331 37.367629,-121.972369 37.367588,-121.972125 37.367763,-121.97188 37.367938,-121.971612 37.36815,-121.971345 37.368362,-121.971321 37.36835,-121.971297 37.368338,-121.971323 37.368298,-121.97135 37.368259,-121.971569 37.368062,-121.971788 37.367865,-121.971977 37.367716,-121.972166 37.367567,-121.972345 37.367442,-121.972524 37.367317,-121.972605 37.367272,-121.972687 37.367227,-121.972728 37.367227,-121.972769 37.367227,-121.972769 37.367259,-121.972769 37.367291,-121.972612 37.367416,-121.972454 37.367542,-121.972488 37.367558,-121.972521 37.367575,-121.972404 37.367674,-121.972286 37.367773,-121.972194 37.367851,-121.972101 37.367928,-121.972046 37.36799,-121.971991 37.368052,-121.972008 37.368052,-121.972025 37.368052,-121.972143 37.367959,-121.972261 37.367866,-121.972296 37.367866,-121.972276 37.36794,-121.972221 37.36798,-121.972094 37.368097,-121.971966 37.368214,-121.971956 37.368324,-121.971945 37.368433,-121.971907 37.368753,-121.971868 37.369073,-121.97184 37.369578,-121.971812 37.370083,-121.971798 37.370212,-121.971783 37.370342,-121.971542 37.370486,-121.971904 37.370324,-121.972085 37.37028,-121.972266 37.370236,-121.972559 37.370196,-121.972852 37.370155,-121.973019 37.370155,-121.973186 37.370155,-121.973232 37.370136,-121.973279 37.370116,-121.973307 37.370058,-121.973336 37.370001,-121.973363 37.369836,-121.973391 37.369671,-121.973419 37.369227,-121.973446 37.368784,-121.973429 37.368413,-121.973413 37.368041,-121.973361 37.367714,-121.973308 37.367387,-121.973285 37.367339,-121.973262 37.36729,-121.973126 37.3673,-121.972989 37.36731,-121.973066 37.36728,-121.973144 37.367251,-121.973269 37.367237,-121.973393 37.367223,-121.973443 37.367158,-121.973493 37.367093,-121.973518 37.36702,-121.973543 37.366947,-121.973582 37.366618,-121.973622 37.366288,-121.97366 37.365826,-121.973698 37.365363,-121.973669 37.365336))', 4326))) as p(g)
)
-- select various versions of the polygons into the same column for overlay comparison in SSMS
select 'Original' as l
,g
from g
union all
select 'Short' as l
,x
from g
union all
select 'Original Reduced' as l
,g.Reduce(10)
from g
union all
select 'Short Reduced' as l
,x.Reduce(10)
from g;
Output
What is interesting to note here is the difference in length of the geog binary representation (simple count of characters as displayed). As I mentioned above, simply using the Reduce function may do what you need, so you will want to test various approaches to see how best you cut down on your data transfer.
+------------------+--------------------+------+
| l | g | Len |
+------------------+--------------------+------+
| Original | 0xE6100000010484...| 4290 |
| Short | 0xE6100000010471...| 3840 |
| Original Reduced | 0xE6100000010418...| 834 |
| Short Reduced | 0xE610000001041E...| 1184 |
+------------------+--------------------+------+
Visual Comparison
String Split Function
As polygon data can be bloody huge, you will need a string splitter that can handle more then 4k or 8k characters. In my case I tend to opt for an xml based approach:
create function [dbo].[fn_StringSplitMax]
(
#str nvarchar(max) = ' ' -- String to split.
,#delimiter as nvarchar(max) = ',' -- Delimiting value to split on.
,#num as int = null -- Which value to return.
)
returns table
as
return
with s as
( -- Convert the string to an XML value, replacing the delimiter with XML tags
select convert(xml,'<x>' + replace((select #str for xml path('')),#delimiter,'</x><x>') + '</x>').query('.') as s
)
select rn
,item -- Select the values from the generated XML value by CROSS APPLYing to the XML nodes
from(select row_number() over (order by (select null)) as rn
,n.x.value('.','nvarchar(max)') as item
from s
cross apply s.nodes('x') as n(x)
) a
where rn = #num
or #num is null;

How to assign decimal value to string in SQL Server 2012

I'm converting a formula from Crystal Reports into a Sql View.
The SQL Syntax that produced the decimal output was this:
if {ACCOUNT_INFO.InstitutionIdentifier}="TPOCFBOI" then 1.10 else
if {ACCOUNT_INFO.InstitutionIdentifier}="TPOBCB"then 0.50 else
if {ACCOUNT_INFO.InstitutionIdentifier}="TPOCFB"then 0.75 else
if {INSTITUTION.Region}="WMG" then .01 else
if {vAM_CRG_LOAN_OFFICER_NAME.LoanOfficerName} ="Margaret S. Smith"
then .875
What would the code look like in a Sql View to produce a decimal (3
places)?
I tried
DECLARE
#LO_Individual_Comp DECIMAL(18,3),
BEGIN
SELECT description INTO LO_Individual_Comp
FROM {ACCOUNT_INFO}
WHERE InstitutionIdentifier} = "TPOCFBOI";
DBMS_OUTPUT.PUT_LINE('1.10' || LO_Individual_Comp;
END;
If > Then Sql Equivalent to return a decimal (3 places)
You could use a CASE statement
select
case
when ACCOUNT_INFO.InstitutionIdentifier ='TPOCFBOI' then 1.10
when ACCOUNT_INFO.InstitutionIdentifier ='TPOBCB' then 0.50
when ACCOUNT_INFO.InstitutionIdentifier ='TPOCFB' then 0.75
when INSTITUTION.Region = 'WMG' then .01
when vAM_CRG_LOAN_OFFICER_NAME.LoanOfficerName ="Margaret S. Smith" then .875
END as 'Decimal'
from...
Now you missing detail on how your tables would join. It appears that your trying to compare data from three different tables (ACCOUNT_INFO, INSTITUTION, vAM_CRG_LOAN_OFFICER_NAME).

Visual Studio (2017) SQL cannot display % sign

I am aware that there are differences between Oracle SQL and SQL Server. The query runs fine and displays the result well but the issue arises when I want to display it on a pie chart. I am thinking that it might be some Visual Studio restriction.
Here is my SQL statement:
SELECT
CAST(ROUND(CAST(COUNT(LoanAcNo) AS FLOAT) / 73 * 100, 1) AS VARCHAR) + '%' AS LoanPercentage,
LoanType
FROM
Loan
GROUP BY
LoanType;
This is how I implemented it:
public DataSet ReadLoanByLoanType()
{
SqlConnection myConn = new SqlConnection(DBConnect);
StringBuilder sqlStr = new StringBuilder();
sqlStr.AppendLine("SELECT cast( round( cast ( count(LoanAcNo) as float) / 73 * 100 , 1 ) as varchar ) + '%' as LoanPercentage , LoanType");
sqlStr.AppendLine("FROM Loan");
sqlStr.AppendLine("GROUP BY LoanType");
SqlDataAdapter da = new SqlDataAdapter(sqlStr.ToString(), myConn);
DataSet ds = new DataSet();
da.Fill(ds);
return ds;
}
If you want to download values from a database for inclusion in some reporting tool's pie chart, don't turn the value into a string by adding a '%' onto them
Your reports tool will be expecting some numeric value to chart, not a string
SQL:
SELECT
round(count(LoanAcNo)/ 73.0 * 100.0, 1) as LoanPercentage,
LoanType
FROM Loan
GROUP BY LoanType;
Tip: dividing an integer by a constant number that has a decimal place (e.g. I divided by 73.0 instead of 73) rather than an integer should make SQLS do the calc using floats - saves you having to cast the int to float/makes the SQL shorter and neater

An invalid floating point operation occurred POWER SQL Server

I am have a problem with the code, when I run I get
An invalid floating point operation occurred
My code is the following:
select POWER( ISNULL(value1,0) / NULLIF(value2,0) , 12 / CONVERT(numeric(6,2),value3))
where value1(data type: money), value2(data type: money), value3(data type: int) are columns in my database.
I get that error when value1 = Null, value2 = 164083520.00 and value3 = 177
The weird is that if I replace with these numbers I get right result, but not while running this.
Any ideas?
Executing the following script I'm not getting any errors on SQL Server 2012. AFAICT you won't get any on other versions either. You must be having a calculation somewhere else that results in an invalid floating point operation.
DECLARE #values TABLE(value1 MONEY,value2 MONEY,value3 INT);
INSERT INTO #values(value1,value2,value3)VALUES(NULL,164083520.00,177);
SELECT
result=POWER(ISNULL(value1,0)/NULLIF(value2,0), 12.0/CONVERT(numeric(6,2),value3))
FROM
#values;
Result
0.00

Resources