I have a DEM dataset and some polygons that represents parcels. For each parcels I would like to compute the maximum/minimum altitude and the average slope. Based on the PostGIS documentation and several example on Internet, two functions could be used to compute this data. The first one is ST_SummaryStatsAgg and the other is ST_DumpAsPolygons.
So, I've created a trigger that computes, before a new parcel is inserted, some statistics, but I am confused about the results. Here is my code:
--First cut the raster based on the parcel shape
SELECT INTO __rasterClip
ST_Union(ST_Clip(foncier.reunion_mnt.rast, NEW.geom, -9999, TRUE))
FROM foncier.reunion_mnt
WHERE NEW.geom && foncier.reunion_mnt.rast;
--Compute slope with ST_DumpAsPolygons OR ST_SummaryStatsAgg
SELECT INTO __slope1 (ST_DumpAsPolygons(ST_Slope(__rasterClip, 1, '32BF', 'DEGREES', 1.0))).val;
SELECT INTO __slope2 (ST_SummaryStatsAgg(ST_Slope(__rasterClip, 1, '32BF', 'DEGREES', 1.0), 1, TRUE, 1)).max;
RAISE NOTICE 'Slope1 %', MAX(__slope1 );
RAISE NOTICE 'Slope2 %', __slope2;
--Compute min/max altitude
SELECT INTO __rasterStats (ST_SummaryStatsAgg(__rasterClip, 1, TRUE, 1)).*;
SELECT INTO __polyDump (ST_DumpAsPolygons(__rasterClip, 1, TRUE)).*;
RAISE NOTICE 'Stat % - %', __rasterStats.min, __rasterStats.max;
RAISE NOTICE 'Poly % - %', Min( __polyDump.val ), Max( __polyDump.val );
The results of the RAISE NOTICE:
NOTICE: Slope1 5.14276456832886
NOTICE: Slope2 51.9147148132324
NOTICE: Stat 222.76 - 251.22
NOTICE: Poly 225.929992675781 - 225.929992675781
There is clearly something wrong. The slope between the two functions is not the same and the min and max altitude for the ST_DumpAsPolygons is the same.
So could you please help me and tell me:
What is the most effective way to compute the min/max altitude and the average slope for a parcel based on a raster DEM?
For my general knowledge is it best to use ST_SummaryStatsAgg or ST_DumpAsPolygons. In which case is it best to use on or the other?
In a trigger how to declare the variable type of these two functions (ST_SummaryStatsAgg, ST_DumpAsPolygons). My first attempt was to declare them using their return type (summarystats and geomval). But I was getting errors so I switch to Record. Is it correct?
Thanks for your help!
Related
We currently use the Geography type to calculate distance between a current location and the coordinates in our tsql table. Our code is based on this sqlauthority.com example.
Is there a faster way to retrieve the distance between two points? These calls will be done by a mobile phone app, so they should ideally be very fast.
After testing it with a distance I know, looping 100 times per batch and running the batch 15 times to make sure the 10 runs the client statistics stores in SSMS are cycled past initial query plan generation so it doesn't skew the results. Here are the averages of the remaining. The calculation method seems to be twice as fast as the geography option.
With a difference in distance returned of 0.0000000020044.
Calculation script used (returned miles: 41.9013152732833)
set nocount on;
declare
#lat1 float = 45.489614
,#lon1 float = -122.650021
,#lat2 float = 44.94404
,#lon2 float = -123.025739
select 3959.1825574 * acos(sin(#lat1/57.295779513082323) * sin(#lat2/57.295779513082323) + cos(#lat1/57.295779513082323) * cos(#lat2/57.295779513082323) * cos((#lon2-#lon1)/57.295779513082323)) distance_in_miles
GO 100
Geography script used (returned miles: 41.9013152752877)
set nocount on;
declare
#g geography = geography::Point(45.489614, -122.650021, 4326)
,#h geography = geography::Point(44.94404, -123.025739, 4326)
select #h.STDistance(#g) / 1609.344 distance_in_miles -- 1609.344 is meters in mile. STDistance = meters.
GO 100
Fair warning, doing it in a non-system function will still have unpredictable performance. I would recommend doing it inline for calculation.
Here's a raw calculation example.
Working example of inline syntax for miles. It is the easiest, most accurate and shortest syntax I could find.
adjusted for accuracy
if object_id('tempdb..#LatLongInfo','U') is not null
begin
drop table #LatLongInfo;
end;
create table #LatLongInfo (
lat1 float,
lon1 float,
lat2 float,
lon2 float
);
insert into #LatLongInfo
values (21, -76, 23, -72);
select
3959.1825574 * acos(sin(lat1/57.295779513082323) * sin(lat2/57.295779513082323) + cos(lat1/57.295779513082323) * cos(lat2/57.295779513082323) * cos((lon2-lon1)/57.295779513082323)) distance_in_miles
from #LatLongInfo;
Hope this helps. I used something like this to find the doctors within a given range for patients back when sql2000 was released, it's been a while. Google was a newborn, no maps, nothing but a search box and one button. You have me all nostalgic now...I remember reading this when I coded that the first time.
Consider the overly simplistic example: POINT(0 0) and LINESTRING (1 -10, 1 10)
The closest point on the line to the POINT would be 1, 0.
How would one determine this in TSQL? My simple, not entirely accurate, approach was to make a linestring (POINT POINT) and extend out the X coord of one coords until the two linestrings intersected.
So:
linestring (0 0, 0.25 0) (no intersect)
linestring (0 0, 0.5 0) (no intersect)
linestring (0 0, 0.75 0) (no intersect)
linestring (0 0, 1 0) (intersection - so 1 0 is the point closest to POINT
This quasi worked, but doesn't seem to the most bestest/more performant way of accomplishing this.
For example, one inefficiency is that I move it one direction (positive increments), and if there was no match (after x attempts), then I would start over, but with negative increments.
To optimize, I tried moving in larger steps, then when intersected (probably went past the point), I backed off 1 increment and started from there with a smaller increment. I did this a couple of times - instead of going in tiny tiny increments so as not to overshoot by too much.
One acceptable assumption based on my processing that the POINT will be next to (left/right) of the LINESTRING.
Another acceptable assumption is that the LINESTRING will be fairly "perpendicular" to the POINT.
I think you can do this mathematically rather than with a brute-force iterative algorithm.
There is a post to get closest point to a line that describes the method.
I converted this method to SQL which returns the correct value (1,0). Your 'trivial' example is actually a bit of an edge case (vertical line with infinite slope) so it seems robust.
I also tested the source code with this example: https://www.desmos.com/calculator/iz07az84f5 and using the input for the line of (-1,2) (3,0) and a point at (2,2) got the correct answer (1.4, 0.8).
SQL code (also in SQL Fiddle at http://sqlfiddle.com/#!6/d87aa/15)
DECLARE #x int, #y int, #x1 int, #y1 int, #x2 int, #y2 int
DECLARE #atb2 float, #atp_dot_atb float
DECLARE #t float
--SELECT #x=0, #y=0
--SELECT #x1=1, #y1=10, #x2=1, #y2=-10
SELECT #x=2, #y=2
SELECT #x1=-1, #y1=2, #x2=3, #y2=0
SELECT #atb2 = SQUARE(#x2-#x1) + SQUARE(#y2-#y1) -- Basically finding the squared magnitude of a_to_b
SELECT #atp_dot_atb = (#x-#x1)*(#x2-#x1) + (#y-#y1)*(#y2-#y1) -- The dot product of a_to_p and a_to_b
SELECT #t = #atp_dot_atb / #atb2 -- The normalized "distance" from a to your closest point
SELECT #x1 + (#x2-#x1)*#t, #y1 + (#y2-#y1)*#t --Add the distance to A, moving towards B
I am trying to perform spatial statistics using postgis. Once in a while I have ST_Clip crushes and halt the query. I figure that this occurs when polygon barely intersects with raster. Please see the sample below.
SELECT ST_Summary(
ST_Clip(
ST_AddBand(
ST_MakeEmptyRaster(16, 16, 0, 0, 1, 1, 0, 0),
ARRAY[
ROW(1, '8BUI'::text, 0, 255),
ROW(2, '8BUI'::text, 0, 255),
ROW(3, '8BUI'::text, 0, 255)
]::addbandarg[]
)
-- this works
--, ST_GeomFromText('POLYGON((15.999999 15.999999, 15.999999 17, 17 17, 17 15.999999, 15.999999 15.999999))')
-- this fails
, ST_GeomFromText('POLYGON((15.9999999 15.9999999, 15.9999999 17, 17 17, 17 15.9999999, 15.9999999 15.9999999))')
)
);
With the above query I am getting following error.
psql:demo_clip_fail_barelyintersects.sql:16: ERROR: RASTER_clip: Could not get band from working raster
CONTEXT: PL/pgSQL function st_clip(raster,integer[],geometry,double precision[],boolean) line 8 at RETURN
I am hoping to getting no record returned instead, or some kind of empty raster. In my production code, the geometry/raster pair was found by ST_Intersects(r.rast, p.geom) between table of polygons and raster. One way I thought about making bounding box for raster which is slightly smaller than the extent of raster, but this is pretty ugly...
My version of postgres and postgis are
PostgreSQL 9.6.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.1,
64-bit
POSTGIS="2.3.1 r15264" GEOS="3.6.0-CAPI-1.10.0 r0" PROJ="Rel.
4.9.3, 15 August 2016" GDAL="GDAL 2.1.2, released 20 16/10/24" LIBXML="2.9.4" LIBJSON="0.12.1" RASTER
Thanks!
My tentative solution, wrap with begin/exception/end block and let the exception part to return empty raster. Performance suffers (~ two times). It will create false negative, but not sure what to look for...
-- function to work around bug in st_clip (fails when polygon barely intersects with raster)
-- not sure how much damage this has on performance
create or replace function st_clip_fuzzy(
rast raster, nband integer[],
geom geometry,
nodataval double precision[] DEFAULT NULL, crop boolean DEFAULT TRUE
)
returns raster
as $$
declare
rec record;
g geometry;
begin
return st_clip($1, $2, $3, $4, $5);
exception
when others then
select st_intersection(st_envelope(rast), geom) into g;
raise warning 'st_clip_fuzzy: intersection %', st_astext(g);
raise warning 'st_clip_fuzzy: area intersection %', st_area(g);
raise warning 'st_clip_fuzzy: area pixel %', abs(ST_ScaleX(rast) * ST_ScaleY(rast));
raise warning 'st_clip_fuzzy: area ratio %', st_area(g) / abs(ST_ScaleX(rast) * ST_ScaleY(rast));
return ST_MakeEmptyRaster(0, 0, ST_UpperLeftX(rast), ST_UpperLeftY(rast), ST_ScaleX(rast), ST_ScaleY(rast), ST_SkewX(rast), ST_SkewY(rast), ST_SRID(rast));
end;
$$ language 'plpgsql' immutable;
CREATE OR REPLACE FUNCTION st_clip_fuzzy(
rast raster, nband integer,
geom geometry,
nodataval double precision, crop boolean DEFAULT TRUE
)
-- four more interfaces with different set of arguments
I have many polygons and many points and want to find the point such that when a line is drawn between that point and the polygon it does not intersect with any other polygon. So basically I need a point that is very close to the polygon and no other polygon is between them.
I tried the following query and it gives me all the points whether or not they are being intersected by a polygon or not.
SELECT P.POINTLOC from pointTable P WHERE NOT MDSYS.SDO_OVERLAPBDYINTERSECT(P.POINTLOC," +
"MDSYS.SDO_GEOMETRY(2003,null,null,MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1)," +
"MDSYS.SDO_ORDINATE_ARRAY(4, 226, 150, 254, 164, 240, 191, 212, 176,4,226))) = 'TRUE'";
Then I tried this query and it gives some correct points but miss a few correct points:
SELECT P.POINTLOC from pointTable P WHERE MDSYS.SDO_WITHIN_DISTANCE(P.POINTLOC," +
"MDSYS.SDO_GEOMETRY(2003,null,null,MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1)," +
"MDSYS.SDO_ORDINATE_ARRAY(4, 226, 150, 254, 164, 240, 191, 212, 176,4,226)),'distance = 40') = 'TRUE'";
Can some one point out which Oracle spatial operator would be best suited for this situation?
It sounds like you're looking for a Nearest Neighbour implementation - Oracle provides the Spatial operator SDO_NN.
You can find more details here:
http://docs.oracle.com/database/121/SPATL/sdo_operat.htm#SPATL1032
This will only get you so far, in that it'll find the nearest point to a polygon, but it won't guarantee that there are no polygons between your point and your target polygon. I suspect if you want to ensure this, you'll have have to get creative.
My approach would be:
use SDO_NN to get the closest point or points
use SDO_GEOM.SDO_CENTROID to find the polygon centre of gravity
create an in-query/in-memory SDO_GEOMETRY line that joins the two points
use this as the basis of a NOT EXISTS clause to exclude points where a polygon intersects that line
Something like the following untested / not-quite finished example, perhaps:
SELECT *
FROM points pnt
WHERE sdo_nn(pnt.point, sdo_geometry(your polygon here)) = 'TRUE' -- fill this with your polygon
AND NOT EXISTS (
SELECT 1
FROM polygons plg
WHERE sdo_geom.sdo_intersection(
plg.polygon
, sdo_geometry(
2002
, NULL -- oracle srid
, NULL -- point
, sdo_elem_info_array(1, 2, 1) -- line
, sdo_ordinate_array(
, sdo_geom.sdo_centroid(
sdo_geometry(your polygon here) -- fill this with your polygon
, 0.05
).x
, sdo_geom.sdo_centroid(
sdo_geometry(your polygon here) -- fill this with your polygon
, 0.05 -- tolerance
).t
, pnt.point.sdo_point.x
, pnt.point.sdo_point.y
) -- line between point and polygon centroid
)
, 0.05 -- tolerance
) = 'TRUE'
)
Depending on your dataset/performance, you might want to do some of this in PL/SQL using collections or loops.
The example above is a bit rough and ready, but I hope you get the gist.
I have an application which receives GPS data from a mobile device as well as receiving co-ordinate data it also provides signal strength from the GSM network.
I am trying to plot the points on a map to display areas of good signal strength and areas of poor signal strength.
When I have a few points it all works well, the points are retrieved from the database and a square is built around the point with the top left corner 0.5km from the point. I then display the square shapes on the VE map using colour coding for signal strength.
The problem is that there may be thousands and thousands of readings and I need a way to average out those readings that are less than 0.5km from each other or I need to build the square (or circle perhaps) in SQL Server and average out the intersections.
I have no idea where to begin with this so any pointers to decent articles or some tips would be much appreciated.
Thanks.
One simple and somewhat inaccurate way to do this would be to decrease the granularity of your data. It might not even be inaccurate, depending on how accurate your x, y measurements are.
let's say we have the following data:
x y signal_strenth
10.2 5.1 10
10.1 5.3 12
10.3 5.5 8
If we floor the x and y values, we get:
x y signal_strenth
10 5 10
10 5 12
10 5 9
Then we can average those values by the floored x and y to show that we have average signal strength in the rectangle (10, 5) to (11, 6).
Here's the SQL:
select
floor(x) as rectangle_xmin,
floor(y) as rectangle_ymin,
floor(x) + 1 as rectangle_xmax,
floor(y) + 1 as rectangle_ymax,
avg(signal_strength) as signal_strength
from table
group by floor(x), floor(y);
Now, admittedly, you'd ideally want to group data points by distance from point to point, and this groups them by a maximum distance that varies from 1 and to square_root(2) =~1.44, flooring them into rectangular blocks. So it's less than ideal. But it may work well enough for you, especially if the flooring/grouping is less than the error in your measurement of position.
If floor() is not granular enough, you can use floor( x * someweight) / someweight to adjust it to the granularity you want. And of course you can use ceil() or round() to do the same thing.
The whole point is to collapse a bunch of nearby measurements to one "measurement", and then take the average of the collapsed values.
You might want to look into Delaunay Triangulation where you can plot X,Y,Z coordinates into a graph. It might be possible, not knowing exactly what you have for points, to use X,Y for the location and then plot the Z as signal strength and create a spike graph. I've only seen c++ examples CodePlex sample but it might be something you can write a SQL function for.
SELECT
geography::STPointFromText('POINT(' + CONVERT(varchar, AvgSignalReadings.rect_lngmin / 100) + ' ' + CONVERT(varchar, AvgSignalReadings.rect_latmin / 100) + ')', 4326) as Location,
AvgSignalReadings.lat / 100 as Latitude,
AvgSignalReadings.lng / 100 as Longitude,
AvgSignalReadings.SignalStrength
FROM
(
SELECT
FLOOR(l.Latitude * 100) as lat,
FLOOR(l.Longitude * 100) as lng,
AVG(l.SignalStrength) as SignalStrength,
COUNT(*) as NumberOfReadings
FROM SignalLog l
WHERE l.SignalStrength IS NOT NULL AND l.SignalStrength <> 0 AND l.Location IS NOT NULL
AND l.[Timestamp] > DATEADD(month, -1, GETDATE())
GROUP BY FLOOR(l.Latitude * 100), FLOOR(l.Longitude * 100))
AS AvgSignalReadings