I have a .NET Core 6 application and SQL Server database which has a table with a column defined as type Geography.
My goal is to use a Linq query to retrieve a list of rows where the Geography Distance < 4000 meters from a point.
The following T-SQL works fine:
-- SPATIAL WITHIN DISTANCE
DECLARE #Origin GEOGRAPHY,
-- distance defined in meters
#Distance INTEGER = 40000;
-- center point
SET #Origin = GEOGRAPHY::STGeomFromText('POINT(-93.5663 44.9033)', 4326);
-- return all rows from events in 40km radius
SELECT * FROM dbo.GeographyAreas WHERE #Origin.STDistance(Geography) <= #Distance;
I would prefer to implement this code in my application data layer and use LINQ syntax rather than a stored procedure using the above code. But there seems to be a mismatch between SQL Server Geography column type and the spatial libraries Microsoft offers:
Microsoft.EntityFrameworkCore.SqlServer.NetTopologySuite
NetTopologySuite.Core
Microsoft.Spatial
Microsoft.SqlServer.Types
NetTopologySuite supports the Geometry type, but not Geography from what I see.
Microsoft.Spatial Geography type will not map to the SQL Server Geography type.
Microsoft.SqlServer.Types appears to be for .NET (classic) not .NET Core.
I am looking for something more like this:
public class GeographyAreas()
{
public <SomeGeographyType> Geography { get; set; }
}
var origin = <SomeGeographyType>(-93.123, 44.456);
var dist = 40000;
List<GeographyAreas> results = _context.GeographyAreas.Where(x => x.Geography.Distance(origin) < dist).ToList();
where <SomeGeographyType> is a placeholder for any of these:
Microsoft.EntityFrameworkCore.SqlServer.NetTopologySuite
Microsoft.Spatial
When I use this approach I get an error:
The Geography property of GeographicAreas.Geography could not be mapped because the database provider does not support this type
Is it possible to do this in Entity Framework Core or does it make sense to implement a stored procedure with the T-SQL code above?
I solved this myself. As it turns out there is a video where the fact that the Geography aspect of NetTopologySuite is described.
It is intentionally left out (but the requirement solved in an admittedly roundabout way).
According to this video here when creating the model in Entity Framework, the data type to use is Geometry (even for a Geography SQL Server data type). The decision by the NetTopologySuite team was made to not add Geography-specific features because, in fact, the code used to work with Geography and Geometry is identical. This is not to say that a globe-based Geography value is the same as a Cartesian-based Geometry one; just that the code to operate on them is the same.
I was in fact able to get this to work using LINQ query. If interested I will post the full solution here.
I have a table of x,y coordinates - let's say they represent map locations of cell phone pings. The table also has a personID to represent the owner of the phone.
I also have a function (which could be changed to something else to suit this solution) that accepts a set of points as a table variable and generates some aggregations of them (average and stdev in both directions). The example shows the table type and a function that could do this.
create type tblCoordinate AS table (x float, y float)
go
create function PointsAggregations(#pts tblCoordinate readonly)
returns table as
return
select avg(x) AvgX, avg(y) avgY, stdev(x) stdevx, stdev(y) stdevy
from #pts
Question is how would I structure this so I could get a table of these aggregations by PersonID. That is, I would like to call this function and pass it the unique set of points for every PersonID in the table, and end up with a table of structure.
PersonID, avgX, avgY, stdevX, stdevY.
Feels like a weird reverse-cross apply kinda thing, but I can't quite figure out if it's possible. Note: the actual problem is much more complicated than the simple aggregations shown here - I am trying to encapsulate an algorithm that creates a "confidence ellipse" of the points - lots of math involved that aren't applicable to the posted question - so simply performing the aggregations within a single query and getting rid of the function isn't a solution.
I have the following query that checks whether is point (T.latitude, T.longitude) is inside a POLYGON
query = """
SELECT id
FROM T
WHERE ST_Intersects(ST_Point(T.latitude, T.longitude), 'POLYGON(({points}))')
"""
But it works slow, how can I speed up it if I have the following index:
(latitude, longitude)?
The query is slow because it must compute the formula for every possible pair of points. So it makes the postgress server do a lot of math, and it forces it to scan through your whole location table. How can we optimize this? Maybe we can eliminate the points that are too far north or too far south or too far east or west?
1) Add a geometry column of type Geometry(Point) and fill it:
ALTER TABLE T add COLUMN geom geometry(Point);
UPDATE T SET geom = ST_Point(T.latitude, T.longitude);
2) Create a spatial index:
CREATE INDEX t_gix ON t USING GIST (geom);
3) Use ST_DWithin instead of ST_Intersect:
WHERE ST_DWithin('POLYGON(({points}))', geom, 0)
You want actually find the points which are within a polygon, so ST_DWithin() is what you need. From the documentation:
This function call will automatically include a bounding box
comparison that will make use of any indexes that are available
PS:
If you for some reason cannot make the points 1 and 2, so at least use ST_Dwithin instead of ST_Intersect:
WHERE ST_DWithin('POLYGON(({points}))', ST_Point(T.latitude, T.longitude), 0)
The last parameter is the tolerance.
You can easly speed up your spatial queries that adding t1.geom&&t2.geom condition to your scripts
This condition;
required spatial indexies so your spatial columns must have spatial indexies
returns approximate result (but with st_ Operators gives exact result)
Here is a example at my database and query timings;
select p.id,k.id, p.poly&&k.poly as intersects
from parcel p , enterance k
where st_contains(p.poly,k.poly) and p.poly&&k.poly
--without && 10.4 sec
--with && 1.6 sec
select count(*) from parcel --34797
select count(*) from enterance --70715
https://postgis.net/docs/overlaps_geometry_box2df.html
I have a Django model with a geodjango PointField:
geopoint = gis_models.PointField(srid=4326, verbose_name='location', geography=True, null=True)
I want to query the model for all points North of a certain latitude.
It ought to be easy, just a > on the latitude component of the point, but I can't find any example of how to do that among the endless examples of distance queries, bounding boxes etc.
Thanks to helpful people of #postgis IRC I have found a way to do this...
Firstly: the way to query against the latitude of a point in PostGIS is via the ST_y function.
As far as I can tell none of the GeoDjango queryset lookups map directly to that function though :(
There is a strictly_above lookup (AFAICT 'above' means 'north of') but it doesn't work for me because my field has geography=True and:
ValueError: PostGIS geography does not support the "strictly_above" lookup.
Thanks to help on #postgis IRC I learned I can cast my geographic field to geometry and thus use ST_y. As a raw PostGIS SQL query it looks like:
SELECT COUNT(id) FROM place WHERE st_y(geopoint::geometry) > 51.508129
The answer:
In Django ORM it looks like:
Place.objects.extra(where=["st_y(geopoint::geometry) > %s"], params=['51.508129']).count()
Further thoughts:
It's a shame we can't use the existing strictly_above ORM lookup, because of two limitations:
No way in the ORM to cast geography to geometry (though there is an 'accepted' ticket for adding this to geodjango)
If we use extra(select={... to add a cast-to-geometry version of the point field to the queryset we can't filter on it (and "won't fix" in Django), except by adding another extra(where=... clause, in which case we're no better off than above
Out of curiosity I wanted to try the strictly_above query anyway. In PostGIS this is the |>> operator and the raw SQL, comparing my point field to another arbitrary point, looks like:
SELECT COUNT(id) FROM place WHERE geopoint::geometry |>> ST_GeomFromText('Point(-0.128005 51.508129)', 4326);
It returns the same answer as the other query so I'm pretty sure the meaning is the same.
I'm designing a table in SQL Server 2008 that will store a list of users and a Google Maps co-ordinate (longitude & latitude).
Will I need two fields, or can it be done with 1?
What's the best (or most common) data-type to use for storing this kind of data?
Fair Warning! Before taking the advice to use the GEOGRAPHY type, make sure you are not planning on using Linq or Entity Framework to access the data because it's not supported (as of November 2010) and you will be sad!
Update Jul 2017
For those reading this answer now, it is obsolete as it refers to backdated technology stack. See comments for more details.
Take a look at the new Spatial data-types that were introduced in SQL Server 2008. They are designed for this kind of task and make indexing and querying much easier and more efficient.
More information:
MS TechNet: SQL Server 2008 Spatial Data Types,
MSDN: Working with Spatial Data (Database Engine).
I don't know the answer for SQL Server but...
In MySQL save it as FLOAT( 10, 6 )
This is the official recommendation from the Google developer documentation.
CREATE TABLE `coords` (
`lat` FLOAT( 10, 6 ) NOT NULL ,
`lng` FLOAT( 10, 6 ) NOT NULL ,
) ENGINE = MYISAM ;
The way I do it: I store the latitude and longitude and then I have a third column which is a automatic derived geography type of the 1st two columns. The table looks like this:
CREATE TABLE [dbo].[Geopoint]
(
[GeopointId] BIGINT NOT NULL PRIMARY KEY IDENTITY,
[Latitude] float NOT NULL,
[Longitude] float NOT NULL,
[ts] ROWVERSION NOT NULL,
[GeographyPoint] AS ([geography]::STGeomFromText(((('POINT('+CONVERT([varchar](20),[Longitude]))+' ')+CONVERT([varchar](20),[Latitude]))+')',(4326)))
)
This gives you the flexibility of spatial queries on the geoPoint column and you can also retrieve the latitude and longitude values as you need them for display or extracting for csv purposes.
I hate to be a contrarian to those who said "here is a new type, let's use it". The new SQL Server 2008 spatial types have some pros to it - namely efficiency, however you can't blindly say always use that type. It really depends on some bigger picture issues.
As an example, integration. This type has an equivilent type in .Net - but what about interop? What about supporting or extending older versions of .Net? What about exposing this type across the service layer to other platforms? What about normalization of data - maybe you are interested in lat or long as standalone pieces of information. Perhaps you've already written complex business logic to handle long/lat.
I'm not saying that you shouldn't use the spatial type - in many cases you should. I'm just saying you should ask some more critical questions before going down that path. For me to answer your question most accurately I would need to know more about your specific situation.
Storing long/lat separately or in a spatial type are both viable solutions, and one may be preferable to the other depending on your own circumstances.
What you want to do is store the Latitude and Longitude as the new SQL2008 Spatial type -> GEOGRAPHY.
Here's a screen shot of a table, which I have.
alt text http://img20.imageshack.us/img20/6839/zipcodetable.png
In this table, we have two fields that store geography data.
Boundary: this is the polygon that is the zip code boundary
CentrePoint: this is the Latitude / Longitude point that represents the visual middle point of this polygon.
The main reason why you want to save it to the database as a GEOGRAPHY type is so you can then leverage all the SPATIAL methods off it -> eg. Point in Poly, Distance between two points, etc.
BTW, we also use Google's Maps API to retrieve lat/long data and store that in our Sql 2008 DB -- so this method does work.
SQL Server has support for spatial related information. You can see more at http://www.microsoft.com/sqlserver/2008/en/us/spatial-data.aspx.
Alternativly you can store the information as two basic fields, usually a float is the standard data type reported by most devices and is accurate enough for within an inch or two - more than adequate for Google Maps.
NOTE: This is a recent answer based on recent SQL server, .NET stack updates
latitute and longitude from google Maps should be stored as Point(note capital P) data in SQL server under geography data type.
Assuming your current data is stored in a table Sample as varchar under columns lat and lon, below query will help you convert to geography
alter table Sample add latlong geography
go
update Sample set latlong= geography::Point(lat,lon,4326)
go
PS: Next time when you do a select on this table with geography data, apart from Results and Messages tab, you will also get Spatial results tab like below for visualization
If you are using Entity Framework 5 < you can use DbGeography. Example from MSDN:
public class University
{
public int UniversityID { get; set; }
public string Name { get; set; }
public DbGeography Location { get; set; }
}
public partial class UniversityContext : DbContext
{
public DbSet<University> Universities { get; set; }
}
using (var context = new UniversityContext ())
{
context.Universities.Add(new University()
{
Name = "Graphic Design Institute",
Location = DbGeography.FromText("POINT(-122.336106 47.605049)"),
});
context. Universities.Add(new University()
{
Name = "School of Fine Art",
Location = DbGeography.FromText("POINT(-122.335197 47.646711)"),
});
context.SaveChanges();
var myLocation = DbGeography.FromText("POINT(-122.296623 47.640405)");
var university = (from u in context.Universities
orderby u.Location.Distance(myLocation)
select u).FirstOrDefault();
Console.WriteLine(
"The closest University to you is: {0}.",
university.Name);
}
https://msdn.microsoft.com/en-us/library/hh859721(v=vs.113).aspx
Something I struggled with then I started using DbGeography was the coordinateSystemId. See the answer below for an excellent explanation and source for the code below.
public class GeoHelper
{
public const int SridGoogleMaps = 4326;
public const int SridCustomMap = 3857;
public static DbGeography FromLatLng(double lat, double lng)
{
return DbGeography.PointFromText(
"POINT("
+ lng.ToString() + " "
+ lat.ToString() + ")",
SridGoogleMaps);
}
}
https://stackoverflow.com/a/25563269/3850405
If you are just going to substitute it into a URL I suppose one field would do - so you can form a URL like
http://maps.google.co.uk/maps?q=12.345678,12.345678&z=6
but as it is two pieces of data I would store them in separate fields
Store both as float, and use unique key words on them.i.em
create table coordinates(
coord_uid counter primary key,
latitude float,
longitude float,
constraint la_long unique(latitude, longitude)
);