I run this example in SQL Server Management Studio:
SELECT CONVERT(REAL, -2101.12) n INTO #t
SELECT * FROM #t
SELECT SUM(n) FROM #t
The first SELECT creates a temp table #t with 1 column n of type real, and it puts 1 row in it with the value -2101.12.
The second SELECT confirms that the table is created with the intended content and the result is:
n
---------
-2101.12
The third SELECT sums the only number that is there, but the result is:
-2101.1201171875
So the question is: Where the 0.0001171875 comes from?
EDIT: I know the lack of precision for the real and float data types, unfortunately I cannot change the database schema because of this. What surprise me though, is that I would expect to see also the extra decimals in the second select since it is supposed to be stored with that lack of precision. Since it does not happens on the second select, then why the sum function picks it up?
You've just discovered real (aka floating point) data is approximate.
Use decimal datatype instead.
The FLOAT and REAL data types are known as approximate data types. The behavior of FLOAT and REAL follows the IEEE 754 specification on approximate numeric data types.
Approximate numeric data types do not store the exact values specified for many numbers; They store an extremely close approximation of the value. For many applications, the tiny difference between the specified value and the stored approximation is not noticeable. At times, though, the difference becomes noticeable. Because of the approximate nature of the FLOAT and REAL data types, do not use these data types when exact numeric behavior is required, such as in financial applications, in operations involving rounding, or in equality checks. Instead, use the integer, decimal, money, or smallmoney data types.
Avoid using FLOAT or REAL columns in WHERE clause search conditions, especially with the = or <> operators. It is best to limit FLOAT and REAL columns with > or < comparisons.
Source of above statement
Related
I am working with a SQL Server database where I need to get data from a column with the type real into another column in another table with the type float. From my understanding, real is essentially just float with less precision (24) compared to float which by default has a precision of 53. Therefore, when casting from real to float I would expect to actually get more precision or at least not lose the precision of the source value. However, some precision actually seems to be lost when doing this:
Why does this happen and is there a way to at least keep the precision of the source values when doing this?
I am confused why SSMS rounds real values when displaying them but does not do the same for float values
The nearest single-precision floating point number (real) to 2.1 is something like 2.0999999. So it makes sense to display it as 2.1.
The nearest double-precision floating point number (float) to 2.1 is quite a long way from
2.09999990000000, which is approximately what you get when you convert 2.0999999 from real to float.
SSMS will display a floats closer to 2.1 as 2.1, eg
select cast(2.1 as float), cast(2.1 as float) - 0.000000000000001
is displayed as
---------------------- ----------------------
2.1 2.1
Here's a paper that reviews algorithms for this conversion and presents a new one: Printing Floating-Point Numbers Quickly and Accurately with
Integers
Just as addition to David Browne answer:
Looks like direct casting doesn't help you. You can get 'correct' (better) results, casting through the character type, like next:
select cast(cast(Valuef as nvarchar(20)) as float)
As an example, select cast(cast(cast(2.1 as real) as nvarchar(20)) as float) displays just 2.1.
I know that similar questions have been asked again in the past, but I think my case is slightly different. I have a column which has Logarithmic values and I'm trying to invert them using the following formula:
SELECT POWER(10,CAST(9.695262723 AS NUMERIC(30,15)))
Let's say the value 9.695262723 is one of the values of that column.
When trying to run this query I get an Arithmetic overflow error for type int, value = 4957500001.400178.
On the other hand, the same query works fine for smaller values e.g. SELECT POWER(10,CAST(8.662644523 AS NUMERIC(30,15)))
How could I overcome that error and calculate the inverse values of the log10 entries I have? Just for information the greater value that exists in the table (in log10 scale) is 12.27256096.
The problem here is your first input parameter (10) which SQL server will, by default, treat as the datatype int.int has a maximum value of 2^31-1 (2,147,483,647), and the number 4,957,500,001 is far larger than this, so you need to use a bigint:
SELECT POWER(CONVERT(bigint,10),CONVERT(numeric(30,15),9.695262723));
Edit: If you need to retain the decimal places, then use a numeric with a large enough scale and precision, instead of bigint.
Recently, I came across an anomaly that while dividing two integers, I am getting only the quotients and reminders are simply ignored.
SELECT 12/8,12%8
The above query gives 1 and 4 respectively, which is similar to Java/C programming. Again applying Java/C programming methods used below calculations to obtain the expected value.
SELECT 12.0/8,12/8.0
The answer is 1.5000 and 1.5000 respectively.
Working on my task I got a situation to obtain percentage value across two counted values (integers) and I stuck up with the results similar to the former query. Then I worked out through the same by multiplying one of the value with 1.0 . This solved my issue.
But later on, going through few scripts, used in my project (developed long back), I noticed in certain cases the decimal values are returned from the query even though two counted values (whole numbers) are divided.
I first noticed this in Netezza. But same holds true in SQL Server as well.
Please advise on what basis the datatypes of returned values are decided.
When dividing both integers, it will perform integer division, which returns an integer. To perform floating point division, you must either cast one or both of the operands to float/decimal/double.
SELECT cast(12 as float)/8
SELECT 12/cast(8 as float)
SELECT cast(12 as float)/cast(8 as float)
SELECT cast(12/8 as float)
Note that the last query is different since the integer division is performed first before casting to float,that is why the decimal value was already lost.
Has anyone encountered the following where when you divide a number in SQL a random number of trailing zeros are appended?...
SELECT 8.95/10 ... results in 0.895000
If you have encountered this, what is the reason for the addition of the zeros?
UPDATE: I am aware that casting the result to FLOAT will remove the 0's
First of all, seeing trailing zeros or anything when querying in SSMS is not because it's something special with the DB engine, but it's always the result of the internal query result formating used for displaying. After all, all numbers are just binary values in some representation that at some point gets translated to strings for displaying.
Anyway, the real reason is because of the datatypes involved, and how SSMS decides to display them. When doing those calculations, SQL Server must decide what datatype the result will be, based on the types of the inputs, and in that particular case it was numeric(7,6). You can easily see the result types by saving the result to a temp table and running sp_columns on that:
SELECT 8.95 AS dividend,10 AS divider,8.95/10 AS result INTO #temp ;
EXEC tempdb..sp_columns '#temp' ;
SELECT * FROM #temp ;
DROP TABLE #temp ;
In my case it returned this (among other uninteresting things for now):
COLUMN_NAME TYPE_NAME PRECISION LENGTH SCALE
dividend numeric 3 5 2
divided int 10 4 0
result numeric 7 9 6
Playing with castings in various places in the division will only change the resulting data types. The interesting fact is the Scale for the result column, note that it's a 6. That's exactly the number of decimal places that SSMS decides to display for the NUMERIC data type, regardless of the actual value. FLOAT don't have this formating from SSMS, which is why the casting eliminates the trailing zeros. Of course, when using the DB from outside SSMS, the formating will depend on the calling application and will not be subject to all this.
As another example of this behavior, just try SELECT CAST(1 AS NUMERIC(18,10)) and see that it shows 1.0000000000.
I'm seeing some strange behavior when rounding in SQL Server 2008. Given the following code:
DECLARE #Value REAL
SELECT #Value = .35
SELECT ROUND(#Value, 1)
I would expect the value to be .4, however it outputs .3. I must assume this is because the value stored is actually less than .35, something like .34999999999999. Is this the case, or am I doing something wrong? Is there a way to ensure this behaves as expected, at least from the visible value?
When you are using floating-point values like REAL and FLOAT (same thing), the SQL Server ROUND() function follows IEEE Standard 754 and uses the "round up" algorithm.
But that means different things for different floating-point values. Some ".5" values end up getting stored as an approximation of ".49999999999", others as ".500000001", etc. It rounds up the value that is actually stored, not the value you gave it to begin with.
http://msdn.microsoft.com/en-us/library/ms187912.aspx
If exact decimal math matters to you, use DECIMAL, not FLOAT or REAL.
It's too bad the data is stored as REAL, but not all hope is lost. Convert the REAL to DECIMAL(10,2) before rounding it. That way the 0.3499999999999 (or whatever inaccurate value is being stored) will be rounded to .35, and then you'll round that to 0.4. You can even convert he result to DECIMAL(10,1) if you want it to be displayed as 0.4:
DECLARE #Value REAL
SELECT #Value = .35
SELECT CONVERT(DECIMAL(10,1), ROUND(CONVERT(DECIMAL(10,2), #Value), 1))