Is there a way to specify rounding precision when evaluating math operation using SpEL?
For example
ExpressionParser parser = new SpelExpressionParser();
Expression exp = parser.parseExpression("2/3");
exp.getValue(Double.class); //returns 0.0 instead of 0.666666667
Or is this a limitation in SpEL?
Thanks
This works for me:
"2.0/3"
and result is 0.6666666666666666.
Does it make sense?
The point of the getValue(Double.class) is eliminated here since your 2/3 is just an operation between two integers and the result is an integer as well - 0.
And only after that this result is converted to expected double as a 0.0.
You need to explicitly say in the expression that you are going to deal with doubles.
You may consider this as a limitation, but getValue(Double.class) is not a casting operation like in Java. It is a post-conversion. Therefore precision is just lost because your expression evaluates to integer anyway.
This is for searchers as me - who googled "SpEL rounding precision". If you use it already in script/expression like {{car.price}} you can add .doubleValue(). So it will be {{car.price.doubleValue()}} and you can do also {{car.price.doubleValue()+conditioner.price.doubleValue()}}. It gives more precision..
Related
I receive an invalid floating point operation error when I SQRT(X) a value.
Under which reasons does this error occur, and what could I to to the X value to prevent this error?
For some context, the X value is a calculated average sales for stock items.
I have tried SQRT(ABS(x)) with no luck.
Thanks
As far as I know, the only case when SQRT(X) gives the error "An invalid floating point operation occurred" is when X is negative. However, you already fixed this by using the ABS function like this: SQRT(ABS(X)).
So, my guess is that the error does not really come from the SQRT function but from something else nearby. Let's look at the expression you gave:
SQRT(2*50*ABS(X)) / NULLIF(B*0.2/12,0))
This expression obviously has an extra right parenthesis. This makes me think that it is only a part of a larger expression, and that the larger expression is the reason for the error.
For instance, if B is 0 then the NULLIF() becomes NULL. You divide by NULL, thus getting a NULL result. Now, what do you do with this result? Maybe some more calculations that does not handle the NULL well?
There is a lot of guessing here. If my guesses did not point you in the right direction, then it would be helpful to know which values of B and X that give the error, and also the full statement that includes the expression.
There is a T_SQL command that calculates the type of the variables. You can view the type of variable before calculating its square root.
The command is :
SQL_VARIANT_PROPERTY
(See the site https://blog.sqlauthority.com/2013/12/15/sql-server-how-to-identify-datatypes-and-properties-of-variable/)
On the other hand, I'm sorry to tell you that I do not know if it works with the 2018 version, because I do not have the means to check it. But you can find the equivalent command for your SQL Server version.
Hope that can help.
Even though DECIMAL is an exact numeric type (unlike FLOAT, which is approximate), it behaves rather strangely in the following example:
DECLARE #DECIMAL_VALUE1 DECIMAL(20,9) = 504.70 / 0.151562
DECLARE #DECIMAL_VALUE2 DECIMAL(20,0) = 504.70 / 0.151562
DECLARE #INTEGER_VALUE INT = 504.70 / 0.151562
SELECT
#DECIMAL_VALUE1 AS DECIMAL_VALUE1, -- 3329.990366978
#DECIMAL_VALUE2 AS DECIMAL_VALUE2, -- 3330
#INTEGER_VALUE AS INTEGER_VALUE -- 3329
A value other than 3329 causes a bug in our application. Making the variable type an INTEGER solved our issue, but I cannot get my head around as to why it was caused in the first place.
You asked, "Why it was caused in the first place":
To know why you need to understand the nature of each datatype and how it operates within SQL Server.
Integer math truncates decimals (no rounding, same as "FLOOR" function (which is why you get 3329)).
Decimal with 0 places rounds (which is why you get 3330 as 3329.99 rounds up)
Decimal with precision/scale rounds to Nth scale (which is why you get 3329.990366978...).
So this isn't unexpected behavior, it's expected given the datatypes involved. It just may have been unanticipated behavior. The nuances of each datatype can be problematic until one runs into them.
I'll choose to ignore the float comment as it is not germane to the question.
I was performing some simple financial calculations in SQL Server when I discovered some odd behavior. I was trying to convert a string of numbers to a decimal type. While the string did not contain a decimal point, I knew from my specifications that the last 3 positions in the string were supposed to be behind the decimal point.
My first approach was flawed, but went something like this:
select convert(decimal(11,3),89456123/1000) as TotalUnits
This resulted in 89456.000. Performing the division before the cast resulted in the decimal parts being truncated.
So I moved the division operation outside the cast, like this:
select convert(decimal(11,3),89456123)/1000 as TotalUnits
This resulted in an explosion of positions after the decimal point. It returned 89456.12300000
According to my decimal specification, I wanted 11 digits, with 3 of them behind the decimal point. Now I have 13 total digits, with 8 behind the decimal. What happened?
To get what I want, I guess I have to double cast, like this:
select convert(decimal(11,3), convert(decimal(11,3),89456123)/1000)
which gives 89456.123.
It turns out no matter what I divide by, the resulting decimal point explosion is the same. Is the division converting the datatype into a double or something?
My question is this:
Why is this happening, and is there a more elegant way to compensate for it, instead of double-casting to decimal.
EDIT
I found this similar question on SO, but it looks like they are again double-casting.
SQL server does integer arithmetic, to force it to use numeric, you can multiply it by 1.0
No need of using convert twice. This gives 89456.123 with out double convert.
select convert(decimal(11,3),89456123*1.0/1000) as TotalUnits
Why does convert(decimal(11,3),89456123)/1000 end up with 6 decimal places? The rules demand it. numeric division has rather complicated rules about the resulting type.
When you say 1.0 you end up with a numeric with the least scale factors possible to represent this value:
SELECT SQL_VARIANT_PROPERTY(1.11, 'BaseType')
SELECT SQL_VARIANT_PROPERTY(1.11, 'Precision')
SELECT SQL_VARIANT_PROPERTY(1.11, 'Scale')
SELECT SQL_VARIANT_PROPERTY(1.11, 'TotalBytes')
What should you do? I think there is no really elegant solution because of the complicated rules. Any solution I can think of involves rather crazy type inference of intermediate results. I recommend pretty much the same solution that RADAR already gave:
select convert(decimal(11,3), convert(decimal(11, 3), 89456123)/1000) as TotalUnits
The main difference is that I think the *1.0 "trick" used as a short hand for a cast is obfuscating the meaning of the code. If you happen to like it feel free to use it, though.
select convert(decimal(11,3),89456123/CONVERT(decimal(11,3),1000))
I've function in T-SQL:
sum(ar.tothandlingtime)/(60*60*24)
and in my result set I've all 0, because the result of this part of the day. Always is below 0.
I want to continue to work on the results, so I need an accurate result in a form and in a view. How?
It is doing integer division, and thus truncating the decimal.
To get your desired result, try converting one side to a decimal:
CONVERT(decimal(19, 18), SUM(ar.tothandlingtime))/(60*60*24)
Using this lets SQL know to perform decimal-based division.
If you need to, you can also play with the precision and scale of the decimal (read more here: http://msdn.microsoft.com/en-us/library/ms187746.aspx)
Of course, if you don't care about the precision, you can also achieve this by putting .0 after each hard-coded number:
(60.0*60.0*24.0)
For example,
select 5/(60.0*60.0*24.0) -- Result: 0.000057870370
select 5/(60*60*24) -- Result: 0
In my experience, this is generally the quickest way to get it to register as decimal division without explicitly using a CAST or CONVERT. If you were strictly using integer-based column values or aggregate functions, though, you would need to convert it, like in the first example.
You are dividing by an int trying converting that to a decimal. Change it like this
sum(ar.tothandlingtime)/CAST((60*60*24) AS DECIMAL ))
We are stuck with a database that (unfortunately) uses floats instead of decimal values. This makes rounding a bit difficult. Consider the following example (SQL Server T-SQL):
SELECT ROUND(6.925e0, 2) --> returns 6.92
ROUND does round half up, but since floating point numbers cannot accurately represent decimal numbers, the "wrong" result (from the point of view of the end-user) is displayed. I understand why this happens.
I already came up with two possible solutions (both returning a float, which is, unfortunately, also a requirement):
Convert to a decimal data type before rounding: SELECT CONVERT(float, ROUND(CONVERT(decimal(29,14), 6.925e0), 2))
Multiply until the third digit is on the left-hand side of the decimal point (i.e. accurately represented), and then do the rounding: SELECT ROUND(6.925e0 * 1000, -1) / 1000
Which one should I choose? Is there some better solution? (Unfortunately, we cannot change the field types in the database due to some legacy applications accessing the same DB.)
Is there a well-established best practice solution for this (common?) problem?
(Obviously, the common technique "rounding twice" will not help here since 6.925 is already rounded to three decimal places -- as far as this is possible in a float.)
Your first solution seems safer, and also seems like a conceptually closer fit to the problem: convert as soon as possible from float to decimal, do all relevant calculations within the decimal type, and then do a last minute conversion back to float before writing to the DB.
Edit: You'll likely still need to do an extra round (e.g. to 3 decimal places, or whatever's appropriate for your application) immediately after retrieving the float value and converting to decimal, to make sure that you end up with the decimal value that was actually intended. 6.925e0 converted to decimal would again be likely (assuming that the decimal format has > 16 digits of precision) to give something that's very close to, but not exactly equal to, 6.925; an extra round would take care of this.
The second solution doesn't look reliable to me: what if the stored value for 6.925e0 happens to be, due to the usual binary floating-point issues, a tiny amount too small? Then after multiplication by 1000, the result may still be a touch under 6925, so that the rounding step rounds down instead of up. If you know your value always has at most 3 digits after the point, you could fix this by doing an extra round after multiplying by 1000, something like ROUND(ROUND(x * 1000, 0), -1).
(Disclaimer: while I have plenty of experience dealing with float and decimal issues in other contexts, I know next to nothing about SQL.)
Old question, but I am surprised that the normal practice is not mentioned here, so I just add it.
Normally, you would add a small amount that you know is much smaller than the accuracy of the numbers you are working with, e.g. like this:
SELECT ROUND(6.925e0 + 1e-7, 2)
Of course the added amount must be larger than the precision of the floating point type that is used.
Use an arbitrary-precision format such as DECIMAL. That way you can leave it to the language to get it right (or wrong as the case may be).
I managed to round the float column correctly using the following command:
SELECT CONVERT(float, ROUND(ROUND(CONVERT(decimal(38,14),float_column_name),3),2))