float sql server conversion issue - sql-server

i have this statement :
Select STORE_NAME,STORE_LATITUDE,STORE_LONGTITUDE
from stores
where STORE_LATITUDE=24.6669863852163
and STORE_LONGTITUDE=46.69189453125
this statement should return one row and i am pretty sure that i put the correct latitude and longitude as they are in the Database Table but it returning null value and not the store that i am looking for but i can't find my mistake STORE_LATITUDE,STORE_LONGITUDE are declared as float

Floating point values are inherently approximations. It's really hard to get them to compare equal; the old adage about "close counts in hand-grenades and horseshoes" also applies to floating point computations.
What I'm trying to say is, you can't reliably test for equality between floating point numbers, ever.
You need a query something like this:
Select STORE_NAME,STORE_LATITUDE,STORE_LONGTITUDE
from stores
where STORE_LATITUDE BETWEEN 24.6669863852163 - 0.0001
AND 24.6669863852163 + 0.0001
AND STORE_LONGTITUDE BETWEEN 46.69189453125 - 0.0001
AND 46.69189453125 + 0.0001
By the way, 0.0001 degrees of latitude is equivalent to about 11 meters on the ground. It's unlikely that your store locations are anywhere near that accurate, because that's where the spherical assumption for the shape of the earth starts to fall apart.
The precision of a latitude datum like 24.6669863852163 is around 11 nanometers, or the size of a molecule of complex carbohydrate or some such thing. That's cool for molecular biology, but obviously overkill for a store finder.

Floats don't hold the precision correctly, if you need the exact precission then you have to with the decimal data Type .
in the above case you can re-define the values as Decimal(38,20) to store the longitude and latitude values.
If you are using SQL Server 2008 or later version, SQL Server supports SPATIAL data types, which has the functionality to hold the longitudes and latitudes and also does a lot of calculations on top of them.
You can find more about the spatial data types from here http://msdn.microsoft.com/en-us/library/bb933790.aspx

Try using cast() or convert()
Select STORE_NAME,STORE_LATITUDE,STORE_LONGTITUDE
from stores
where Convert(decimal(9,2),STORE_LATITUDE)=24.66
and Convert(decimal(9,2),STORE_LONGTITUDE)=46.69

Your code works fine in Sql Server 2008: sqlfiddle

Related

Entity framework Core is adding digits when converting float (db data type) to double in c#

I have a SQL table which has two columns Latitude and Longitude with values 29.47731 and -98.46272 respectively. Column datatype is float in SQL server.
When we retrieve the record using EF core like _context.Table.ToListAsync(), it retrieves record but when it is converted to c# equivalent double? datatype, it adds extra digits like -98.4627199999999.
How can i avoid this? It should return same value as there in database.
SQL Float and C# Double are imprecise data types, they store very close approximations of the number, but often not the exact number. If you need to maintain the exact values, you should use SQL Numeric data type which will allow you to specify the number of significant digits; and while the C# Decimal data type is still imprecise, it will provide a higher precision approximation than Double does.
I you find out that your data isn't stable and you are dealing with decimal values, it is propably a 'precise notation' issue.
Try taking a look at these:
SQL:
https://learn.microsoft.com/en-us/sql/t-sql/data-types/decimal-and-numeric-transact-sql?view=sql-server-ver15
C#
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types
Those should guide you trough using the most precise notation for your situation :-)

How to round Double's to exact bit-representation?

I'm doing geography calculations, and ultimately end up with a latitude and longitude to store in a Geography::Point object.
Both latitude and longitude can have 7 digits at most (which also gives precision up to 11 mm, which is plenty).
The problem is: if the value of a field cannot be stored correctly in a Double, MS SQL rounds towards the nearest number that can, but does so by adding a bunch of digits.
=> e.g. 5.9395772 is stored as 5.9395771999999996
The problem this creates, is that [Position].ToString() then exceeds the maximum amount of characters is allowed for that column (and no, I can't increase that limit).
Since we're dealing with Latitude, Longitude, Altitude and Accuracy, there's space for exactly 11 characters for Latitude and Longitude each:
String.Format(CultureInfo.InvariantCulture, "{0:##0.0######}", num)
I've tried simply Math.Round()ing to 6 digits, but then other numbers (e.g. 6.098163 to 6.0981629999999996) get the same problem.
How do I Math.Round towards the nearest 7-digit valid bit representation?
EDIT/ADD
Public Function ToString_LatLon(ByVal num As Double) As String
num = Math.Round(num, 7, MidpointRounding.AwayFromZero)
Return String.Format(CultureInfo.InvariantCulture, "{0:##0.0######}", num)
End Function 'IN = 5.9395772, OUT = 5.9395772
The above code receives a Double and correctly returns the String representation. I've checked it, this is correct also for troubling numbers.
It's stored in SQL Server through the framework we use. I think the problem occurs when storing the value
When I retrieve the value, I get an error in VB, saying the value is wider than the framework allows (max of 50 characters).
If I run a query in SSMS, I find e.g. POINT (X.0981629999999996 XX.664725 NULL 15602.707) (51 characters, anonimized).
EDIT 2
I've done some more research and some calculations. It seems that the stored value 5.9395772 is converted to binary and returned as 5.9395771999999996, which is stored as a double inside the database (in a binary Geography::Point object, not to worry.) Convert the binary 0 10000000001 0111110000100010000010000110100010000100010011011101 back to decimal, and you get 5.93957719999999955717839839053340256214141845703125, but abbreviated at 16 decimals - whereas I would like it abbreviated at 7 decimals.
Solutions:
Round the value down/up to the nearest value where everything from the 8th decimal onward is 0 (or enough zeroes before another nonzero digit is found)
Query for only so many decimals.
Query the actual (hexadecimal) value, and convert that (instead of the string representation)
Keep the string representation, but round the values before storing and after retrieving to the required amount of decimals.
Discussions:
Both in office and here (at #RobertBaron's answer): this is quite tricky, might have a huge decrease in precision, and is basically a lot of work.
Perhaps this is possible, I don't know.
This would be the cleanest solution, as my colleagues and I agree, however this is a lot of work in developing and testing.
Instead of caring about the value in memory to be equal to the value in the database, we don't care about the value in the database (too much).
In the end, after quite some whiteboard bit-calculations and a lengthy discussion, we've gone with option 4. After we retrieve the [Position].ToString() (for which we've increased the string limit) from the database, we convert that as we're already doing, and as additional step before using it anywhere we round the value to the required amount of decimals. When returning the value to the database, we once again round the value to the amount of decimals, and don't care what the database really does with it.
Essentially, this is option 2, but then on the program-side instead of database-side.
This is only a partial answer.
If by valid bit representation you mean exact bit representation, then this is possible. The decimal numbers that have exact bit representation are 1/2, 1/4, 3/4, 1/8, 3/8, 5/8, 7/8, 1/16, 3/16, ...
The challenge is to characterize among these powers of two, those whose base 10 representation has 7 digits or less, and then to round any base 10 number to the closest of these numbers.
I am posting this in the hope that it may get you one step further toward a solution.
If you cannot change the data type into a DECIMAL for whatever reasons, you have to cast it into a DECIMAL every time you need the value. It's that simple. And you can either do it on the SQL Server side or in VB.NET, but you need a DECIMAL. DOUBLEs are imprecise.
By the way, it is not the SQL Server that rounds towards the nearest number it recognizes by adding a bunch of digits - it's the processor that does it. That's also why you may get slightly different DOUBLE values after restoring your database on another server.
And never ever even think of using them as an ID: I know an application that uses FLOAT values containing the timestamp (<creation day since whatever>.<time as fractals of the day>) as part of the primary key (of nearly every table!). Every 10000th record or so cannot be addressed directly by its ID because the value differs somewhat on the client that sends the query and the server by some nanoseconds although the number looks exactly the same in SSMS on the client and the server.

Real to Float conversion with no loss of data

I had a table with two columns for coordinates stored in. These columns were REAL datatype, and I noticed that from my application it was only showing 5 decimals for coordinates, and positions were not accurate enough.
I decided to change datatype to FLOAT, so I could use more decimals. It was for my pleasant surprise that when I changed the column data type, the decimals suddenly appeared without me having to store all the coordinates again.
Anyone can tell me why this happens? What happens with the decimal precision on REAL datatype?. IsnĀ“t the data rounded and truncated when inserted? Why when I changed the datatype the precision came up with no loss of data?..
You want to use a Decimal data-type.
Floating point values are caluclated by a value and an exponenent. This allows you have store huge number representations in small amounts of memory. This also means that you don't always get exactly the number you're looking for, just very very close. This is why when you compare floating point values, you compare them within a certain tolerance.
It was for my pleasant surprise that when I changed the column data type, the decimals suddenly appeared without me having to store all the coordinates again.
Be careful, this doesn't mean that the value that was filled in is the accurate value of what you're looking for. If you truncated your original calculation, you need to get those numbers again without cutting off any precision. The values that it autofills when you convert from Real to Float aren't the rest of what you truncated, they are entirely new values which result from adding more precision to the calculation used to populate your Real value.
Here is a good thread that explains the difference in data-types in SQL:
Difference between numeric, float and decimal in SQL Server
Another helpful link:
Bad habits to kick : choosing the wrong data type

SQL Server 2008 MAX and MIN function not working as expected with negative numbers

I've noticed that the MAX/MIN aggregate functions for SQL Server 2008 do not work as expected with negative numbers.
I was working with latitude and longitude values (many are negative #s) and I'm getting results that appear to only be looking at the absolute value.
SELECT
MAX(g.Geo_Lat) AS MaxLat, MAX(g.Geo_Long) AS MaxLong,
MIN(g.Geo_Lat) AS MinLat, MIN(g.Geo_Long) AS MinLong
FROM Geolocations g
Here are results of a query:
MaxLat MaxLong MinLat MinLong
38.3346412 -85.7667496 38.1579234 -85.5289429
note the results for maxlong and minlong are incorrect.
Is there some workaround for this (other than a special UDF)?
Data types and collation determine order.
Geographic data, stored for instance as a geography type, could sort differently than float values - but in this case, they would not. Gopegraphic types are not sortable, only the latitudes and longitudes are, as you display. But those output as float values.
What data type are you using that causes this to occur? Affter some testing, I eventually figured it out. It would work as expected for geographic data, or any numeric type that holds negative numbers.
You are storing your latitudes and longitudes as text data - aren't you?
Cast them as floats. That will fix it.
Max and Min aggregates have worked as expected for me in SQL Server 2008. Can you provide the column types for Geolocations table?
It does depend on the locations in your table. As you get more and more entries, it should approach
MaxLat MaxLong MinLat MinLong
90 180 -90 -180
south pole far west of north pole far east of
prime meridian prime meridian
Prime meridian 0 degrees goes through London, England.
-180 and 180 are the same line opposite 0.
Try adding these locations to your table if you only have a few rows:
insert into Geolocations (Geo_Lat, GeoLong) values(-180,-90)
insert into Geolocations (Geo_Lat, GeoLong) values(0,0)
insert into Geolocations (Geo_Lat, GeoLong) values(180,90)

C# Float to Real in SQL Server

I am attempting to use Entity Framework and have a contact database that has Longitude and Latitude data from Google Maps.
The guide says that this should be stored as float.
I have created my POCO entity which includes Longitude as a float and Latitude as a float.
I have just noticed that in the database, these both come up as real.
There is still a lot of work to be done before I can get any data back from Google, I am very far away from testing and I was just wondering if anyone can tell me if this is going to be a problem later on?
Nope, that should be fine. Note that you may not get the exact same value back as you received from Google Maps, as that would have been expressed in decimal-formatted text. However, this is effectively a physical quantity, and thus more appropriate as a float/double/real/(whatever binary floating point type you like) than as a decimal. (Decimals are more appropriate for man-made quantities - particularly currency.)
If you look at the documentation for float and real you'll see that real is effectively float(24) - a 32-bit floating binary point type, just like float in C#.
EDIT: As noted in comments, if you want more than the significant 7-8 digits of accuracy provided by float, then you probably want double instead in C#, which would mean float(53) in SQL Server.
This link:
http://msdn.microsoft.com/en-us/library/aa258876(v=sql.80).aspx
explains that, in SQL Server, real is a synonym for float(24), using 4 bytes of data. In .NET a Single precision floating point number also uses 4 bytes, so these are pretty much equivalent:
http://msdn.microsoft.com/en-us/library/47zceaw7(v=vs.71).aspx

Resources