I'm using the mongoid gem and got some float fields the gets their value from complex calculations, at the end I get numbers like 23434.23423432423, I don't really care about more than 2 digits after the floating point, is there a way (instead of find all places in code and do a manual rounding) to define how precise my float is in the document field definition?
something like
field :result, type: Float, accuracy: 2
use after_update hook and do rounding the Float field
Related
I have
price_input = gets.strip.to_i
and I am trying to add it to an array of hashes. The adding to array works, but the .gets.strip.to_i removes any decimals. Is there a way to add it without it removing the decimals?
for further understanding, I have an existing array of hashes where ( price: [an integer] ) and I am adding these integers together with code later, so I need the .to_i because I can't add strings and integers
Storing Input Strings in BigDecimal
Integers are not Floats, and Floats are a bad way to handle financial representations. There are gems for handling money that abstract away the various issues with floating point math, but if you're just looking for something simple:
require "bigdecimal"
print "Enter a price: "
price_input = BigDecimal(gets)
puts "You entered $#{price_input.to_f.round 2}"
You don't even have to chomp or strip your input, as BigDecimal will handle that for you. However, please keep in mind that even though BigDecimal or Real will correctly handle fractional cents, anything involving fractions of a penny can lead to rounding errors of one sort or another when you convert back to a Float with a precision of 2, so you'll need to decide what to do about rounding regardless of your internal representation.
See Also
Float#floor
Float#ceil
Float#round
Float#rationalize
You can use to_f method. It converts String object to Float object. You also don't need strip method
" 0.2 ".to_f
# => 0.2
or in your case
price_input = gets.to_f
Keep in mind that when two integers interact, the result will always be an integer. If one number is an integer and the second is a float, the result will be a float
You can use below, this way if someone enters characters instead of a number, an error will be raised.
price_input = Float(gets)
I am working with a SQL Server database where I need to get data from a column with the type real into another column in another table with the type float. From my understanding, real is essentially just float with less precision (24) compared to float which by default has a precision of 53. Therefore, when casting from real to float I would expect to actually get more precision or at least not lose the precision of the source value. However, some precision actually seems to be lost when doing this:
Why does this happen and is there a way to at least keep the precision of the source values when doing this?
I am confused why SSMS rounds real values when displaying them but does not do the same for float values
The nearest single-precision floating point number (real) to 2.1 is something like 2.0999999. So it makes sense to display it as 2.1.
The nearest double-precision floating point number (float) to 2.1 is quite a long way from
2.09999990000000, which is approximately what you get when you convert 2.0999999 from real to float.
SSMS will display a floats closer to 2.1 as 2.1, eg
select cast(2.1 as float), cast(2.1 as float) - 0.000000000000001
is displayed as
---------------------- ----------------------
2.1 2.1
Here's a paper that reviews algorithms for this conversion and presents a new one: Printing Floating-Point Numbers Quickly and Accurately with
Integers
Just as addition to David Browne answer:
Looks like direct casting doesn't help you. You can get 'correct' (better) results, casting through the character type, like next:
select cast(cast(Valuef as nvarchar(20)) as float)
As an example, select cast(cast(cast(2.1 as real) as nvarchar(20)) as float) displays just 2.1.
I had a table with two columns for coordinates stored in. These columns were REAL datatype, and I noticed that from my application it was only showing 5 decimals for coordinates, and positions were not accurate enough.
I decided to change datatype to FLOAT, so I could use more decimals. It was for my pleasant surprise that when I changed the column data type, the decimals suddenly appeared without me having to store all the coordinates again.
Anyone can tell me why this happens? What happens with the decimal precision on REAL datatype?. Isn´t the data rounded and truncated when inserted? Why when I changed the datatype the precision came up with no loss of data?..
You want to use a Decimal data-type.
Floating point values are caluclated by a value and an exponenent. This allows you have store huge number representations in small amounts of memory. This also means that you don't always get exactly the number you're looking for, just very very close. This is why when you compare floating point values, you compare them within a certain tolerance.
It was for my pleasant surprise that when I changed the column data type, the decimals suddenly appeared without me having to store all the coordinates again.
Be careful, this doesn't mean that the value that was filled in is the accurate value of what you're looking for. If you truncated your original calculation, you need to get those numbers again without cutting off any precision. The values that it autofills when you convert from Real to Float aren't the rest of what you truncated, they are entirely new values which result from adding more precision to the calculation used to populate your Real value.
Here is a good thread that explains the difference in data-types in SQL:
Difference between numeric, float and decimal in SQL Server
Another helpful link:
Bad habits to kick : choosing the wrong data type
I have a problem converting float to string but t-sql returns it with scientific format.
Example:
declare #amount float
set #amount=5223421341.23
print cast(#amount as nchar(20))
returns "5.22342e+009"
Well I tried the function STR but there is the point: I don't know how many decimals could have the float and I don't want to round it.
There is a way to return the float as nchar with the same precision as float is declared?
Thanks a lot.
There is a problem with the decimals places. The below code shows that the decimal value is distorted and I cant find a way around it. If there was a way to determine the precision of the float then the result could be rounded to a correct value.
With an unknown precision, I have yet to find a solution.
Declare #Amount float
Set #Amount=5223421341.23
Select LTrim(RTrim(Str(#Amount, 1000, 1000)))
Produces : 5223421341.2299995000000000
Based on what I am reading floats are only approximations by definition so this may be the best you can get.
https://msdn.microsoft.com/en-us/library/ms173773.aspx
http://www.webopedia.com/TERM/F/floating_point_number.html
Note that most floating-point numbers a computer can represent are
just approximations. One of the challenges in programming with
floating-point values is ensuring that the approximations lead to
reasonable results. If the programmeris not careful, small
discrepancies in the approximations can snowball to the point where
the final results become meaningless.
According to Microsoft, the best method in SQL is likely to use the STR function.
declare #amount float
set #amount=5223421341.23
print str(#amount,20,6)
print convert(VARCHAR(50),rtrim(ltrim(str(#amount,20,6))))
This seems like it would cover most scenarios, but you need to find out the max and min values in your data set and also the max precision.
According to Microsoft SQL Server documentation, (https://msdn.microsoft.com/en-us/library/ms173773.aspx) float is a synonym for float(53), and a float(53) has a precision of 15 digits.
But the 15 digits of precision say nothing about the exponent. The exponent is about 300. Which means that floats are capable of representing numbers containing about 300 zeros in them. That's why scientific notation is indispensable.
If scientific notation was not a problem for you, 21 characters would be enough to hold 15 digits with a period among them, plus the trailing e+999.
But since scientific notation is an issue for you, then you have to have a text field large enough to hold numbers of the form
0.0000000000000000000000000000000...00000000000000000000000000723237435435644
(that's almost 300 zeros followed by 15 digits)
and
377733453453453000000000000000000...00000000000000000000000000000000000000.0
(that's 15 digits followed by almost 300 zeros.)
So clearly, you are going to need some pretty huge text fields for what you are trying to do.
So, bottom line is, instead of looking for a solution to the problem as stated, I would recommend revising what you are trying to accomplish here.
I have a postgresql database and am using it for a project which handles monetary value. My question is:
How do I limit the number of decimal digits to two for a Double Precision or float8 type column in a postgresql table?
Simply use Decimal (Numeric) type instead, as documented in the manual, e.g. cost decimal(10,2).
The first number (10) defines the total length of the number (all digits, including those after the decimal point), while the second number (2) tells how many of them are after the decimal point. The above declaration gives you 8 digits in front of the point. Of course, you can increase that - the limitations of the Numeric type are much, much higher.
In my opinion there is no need to use floating point when you handle currencies or other monetary-related values. I have no idea about the performance comparison between the two types, but Decimal has one advantage - it is an exact number (which usually is not the case with floating point). I also suggest to read an interesting, but short discussion on the topic.