I have 2 tables, one containing a bunch of polygons like this
CREATE TABLE [dbo].[GeoPolygons](
[OID] [int] IDENTITY(1,1) NOT NULL,
[Version] [int] NULL,
[entity_id] [varchar](200) NULL,
[Geometry] [geometry] NULL
)
and one containing a bunch of nodes (places) like this
CREATE TABLE [dbo].[GeoPoints](
[point_id] [int] NOT NULL,
[point_name] [varchar](40) NULL,
[latitude] [decimal](8, 6) NULL,
[longitude] [decimal](9, 6) NULL,
[point_geometry] [geography] NULL
)
These tables came from disparate sources, so one has a geography field type and the other has a geometry field type.
What I want to do - which is very possible within my GIS, but I want to do it in T-SQL for a variety of reasons - is find out which nodes are inside which polygons.
Is the first step to match the geo-field types? geometry-to-geometry or geography-to-geography?
Or can that be done on the fly?
My ideal output would be a 3 field table
CREATE TABLE [dbo].[Points2Polygons](
[match_id] [int] IDENTITY(1,1) NOT NULL,
[entity_id] [varchar](200) NOT NULL,
[point_id] [int] NOT NULL
)
and be able to update that on the fly (or perhaps daily) when new records are added to each table.
I found this post but it seems to deal with a single point and a single polygon, as well as WKT definitions. I don't have WKT, and I have thousands of polys and thousands of points. I want to do it at a larger scale.
How do I do it in a T-SQL query?
Server is running SQL-Server Web 64-bit V15.0.4138.2 on Server 2019 Datacenter.
TIA
From the comments above, here's a proposal to convert the Geometry column in your GeoPolygons table. As with anything like this where you have data one way and you want it to look a different way on an ongoing basis, the high level steps are:
Start writing the data in both formats
Convert the old format into the new format
Convert all read paths to the new format
Drop the old format
I'll be focusing on "Convert the old format into the new format". Create a new column in your table (I'll call it Polygon).
alter table [dbo].[GeoPolygons] add
Polygon geography null;
Note that this is a prerequisite for the "Start writing the data in both formats" phase and so should already be done by the time you're ready to convert data.
The most straightforward method to do that looks like this:
update [dbo].[GeoPolygons]
set [Polygon] = geography::STGeomFromText(
[Geometry].STAsText(),
[Geometry].STSrid
)
where [Geometry] is not null
and [Polygon] is null;
I'm making the assumption that the SRID on your Geometry column is set properly. If not, you'll have to find the SRID that is appropriate given the WKT that was used to create t
Related
We are using SQL Server 2014 (not SQL Azure) and I have defined a simple table to store images (.jpg) and
CREATE TABLE [dbo].[Graphic](
[GraphicID] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY,
[FileName] [varchar](100) NOT NULL,
[FileDescription] [nvarchar](200) NULL,
[Image] [varbinary](max) NULL
)
max size of an image stored is 1MB, this validation is taken care on the front-end. I just inserted 15 images and the table size is 5544 KB currently. There is a primary key placed on GraphicID column. No other indexes placed.
But when I retrieve one or more images using the below (simple SELECT) query, it is taking longer time like 25 - 30 seconds.
select * from [Graphic]
where [GraphicID] = 53
Is there a faster mechanism to query images in SQL Server in less than 5 seconds ?
Is there any alternate SAVE & RETRIEVE mechanism for images in SQL Server 2014 for better performance ?
Please help.
Thanks
Bhanu
CREATE TABLE [dbo].[Graphic](
[GraphicID] [int] IDENTITY(1,1) NOT NULL,
[FileName] [varchar](100) NOT NULL,
[FileDescription] [nvarchar](200) NULL,
[Image] [varbinary](max) NULL
)
If that is indeed your schema (and you seem extremely unsure of it), the problem is that you never added indices to your table. Add a clustered index over GraphicID and it should fix this particular access pattern.
Another note would be that if you know the maximum size of your varbinary (and you said you do), you should use it rather than using max. That way the table layout will store the image inside the row (up to a certain size) rather than appending it at the end, making row retrieval (select *) a lot faster for larger, fragmented tables.
I have the following SQLtable
[rel_id] [int] NOT NULL,
[rel_type_id] [int] NOT NULL,
[left_object_id] [int] NOT NULL,
[left_object_type] [int] NOT NULL,
[right_object_id] [int] NOT NULL,
[right_object_type] [int] NOT NULL,
[template] [char](1) NOT NULL
Basically this table controls relationships between different objects. I’ need to construct some SQL that given left_object_id can work out recursively all the objects that has relationships to it.
I’m only interested in object where both the left & right object types are 5
In simple terms the SQL would need to do the following
1. Input a left object id
2. Search the table and return all right object ids where associated left object matches the input
3. For each of the right object id found go back repeat from step 1
4. Repeat until no further values are found
Sounds complex, but I’m sure there is some SQL guru out there who can solve this.
Many Thanks.
It's difficult without test data but something like this should do the trick:
I assume a tree, having one parent element left side and a children list right side. This query should come back with entries having their parent of type 5 and at least one child of type 5.
SELECT *
FROM #tbl AS tbl
WHERE (SELECT leftObj.rel_type_id FROM #tbl AS leftObj WHERE leftObj.right_object_id=tbl.rel_id)=5
AND EXISTS(SELECT rightObjList.rel_type_id
FROM #tbl AS rightObjList
WHERE rightObjList.left_object_id=tbl.rel_id
AND rightObjList.rel_type_id=5)
<table><tbody><tr><th>rel_id</th><th>rel_type_id</th><th>left_object_id</th><th>left_object_type</th><th>right_object_id</th><th>right_object_type</th></tr><tr><td>52</td><td>44</td><td>4</td><td>5</td><td>1</td><td>5</td></tr><tr><td>53</td><td>44</td><td>4</td><td>5</td><td>3</td><td>5</td></tr><tr><td>54</td><td>44</td><td>5</td><td>5</td><td>4</td><td>5</td></tr><tr><td>55</td><td>44</td><td>5</td><td>5</td><td>6</td><td>5</td></tr></tbody></table>
With the table specified below, how could I modify the locking behavior in SQL Server 2012 to block statements and transactions attempting to select data pertaining to a specific UserId column?
I have been attempting to come up with a stored procedure that will successfully change an address for a specific user. In order to do this, the existing record is marked as deleted by setting DeletedOn to the current date. Afterward, the new record is inserted. I do not want any queries to be able to see that no valid address is present for the given user in the table between the deletion mark and the insertion.
Queries related to a different user's address should be able to complete, so long as that user's address is not in the process of being modified.
CREATE TABLE [Address]
(
[Id] BIGINT NOT NULL,
[UserId] FOREIGN KEY REFERENCES [User]([Id]) NOT NULL,
[House] CHARACTER VARYING(255) NOT NULL,
[Street] CHARACTER VARYING(255) NOT NULL,
[City] CHARACTER VARYING (255) NOT NULL,
[State] CHARACTER VARYING(255) NOT NULL,
[Zip] CHARACTER VARYING(15) NOT NULL,
[CreatedOn] DATETIMEOFFSET NOT NULL,
[DeletedOn] DATETIMEOFFSET NULL,
UNIQUE([UserId], [DeletedOn]),
CHECK(([DeletedOn] IS NULL) OR ([CreatedOn] <= [DeletedOn])),
PRIMARY KEY([Id])
);
Using a history table solved this issue. It seems that UNIQUE constraints cause lots of lock escalations when they are defined as composites.
The history table now tracks all of the old versions of a particular record and history inserts are combined with live table updates in a repeatable read transaction.
What do you know, I was approaching the whole problem the wrong way!
I'm rather new to SQL Server, (did learn SQL back in late 1980's, DB2 if I recall) Today I'm integrating my database layer into SQL, to begin with, SQL Server.
To begin with. As I do today, I will generate in runtime every databases objects, tables objects and indexes programmatically as I do with almost every visual and data object in my projects. That is, I use the visual designing tools very limited.
Every column in my project has a external description file's (every user has profile which contains these files), just as I do with database key's and for visual objects as for effect's as positioning, length, picture-mask, font size, etc. etc. i.e. dynamic forms. Almost every window, grids, filters is created in runtime just as far most of my database connections.
I did build a small test "machine" to create tables in this environment and did well, very easy to create tables within program (I use delphi and ADO)
The problem I encounter is when I flag a column as "autoincrement" or as Identity in SQL Server or if I describe a column as primary key, then SQL Server Management Studio creates automatically a index or key.
That would be ok if I could manage the name it gives this index or key.
Example of this situations:
AdoCommand.CommandText := Str_SQL;
TRY
AdoCommand.Execute;
FINALLY
NotiFy_TimeOut ('Table was created', wait_for_one_sec);
END;
My database engine creates this SQL script which I pass into the string Str_SQL above:
CREATE TABLE GENGITFL
(
NR INT NOT NULL IDENTITY,
GJM CHAR(3) NOT NULL,
HEITI VARCHAR(25) NOT NULL,
KAUPG REAL NULL,
SOLUG REAL NULL,
TOLLG REAL NULL,
DUMMY VARCHAR(20) NULL,
UNIQUE (GJM),
PRIMARY KEY (GJM)
)
SQL Server creates these two indexes automatically :
PK__GENGITFL__C51F17260A463F49
UQ__GENGITFL__C51F17277FA3E6E6
I don't want to use these names for these files, I would prefer names as:
IDX_GENGITFL_GJM
IDX_GENGITFL_NR
The reason should be obvious in light of my intro, the runtime engine can't create these names automatically and I consider it not a option to look for what index files are available within system database. If my external description say there should be index, I would like just to create names for the index automatically by using the prefix, IDX_ next the table name and last the field name or name's with underscore between, as IDX_GENGITFL_GJM etc.
Hope someone understand my poor english and presentation.. I'm rather rusty in english.
Thanks
Edit: After help from marc_s my SQL "script" is like this:
CREATE TABLE GENGITFL
(
NR INT NOT NULL IDENTITY,
GJM CHAR(3) NOT NULL,
HEITI VARCHAR(25) NOT NULL,
KAUPG REAL NULL,
SOLUG REAL NULL,
TOLLG REAL NULL,
DUMMY VARCHAR(20) NULL,
CONSTRAINT IDX_GENGITFL_NR UNIQUE (NR),
CONSTRAINT IDX_GENGITFL_GJM PRIMARY KEY (GJM),
)
CREATE INDEX IDX_GENGITFL_HEITI ON GENGITFL (HEITI)
Thanks again.
If you don't want the system default names - then just specify your own! :
CREATE TABLE GENGITFL
(
NR INT NOT NULL IDENTITY,
GJM CHAR(3) NOT NULL,
HEITI VARCHAR(25) NOT NULL,
KAUPG REAL NULL,
SOLUG REAL NULL,
TOLLG REAL NULL,
DUMMY VARCHAR(20) NULL,
CONSTRAINT IDX_GENGITFL_NR UNIQUE (GJM),
CONSTRAINT IDX_GENGITFL_GJM PRIMARY KEY (GJM)
)
See those CONSTRAINT (yourownnamehere) before the UNIQUE and the PRIMARY KEY ?
Now, your constraints are named as you defined.
I need to store data in a SQL Server 2008 database from various data sources with different data types. Data types allowed are: Bit, Numeric (1, 2 or 4 bytes), Real and String. There is going to be a value, a timestamp, a FK to the item of which the value belongs and some other information for the data stored.
The most important points are the read performance and the size of the data. There might be a couple thousand items and each item may have millions of values.
I have 5 possible options:
Separate tables for each data type (ValueBit, ValueTinyInt, ValueSmallInt, etc... tables)
Separate tables with inheritance (Value table as base table, ValueBit table just for storing the Bit value, etc...)
Single value table for all data types, with separate fields for each data type (Value table, with ValueBit BIT, ValueTinyInt TINYINT etc...)
Single table and single value field using sql_variant
Single table and single value field using UDT
With case 2, a PK is a must, and,
1000 item * 10 000 000 data each > Int32.Max, and,
1000 item * 10 000 000 data each * 8 byte BigInt PK is huge
Other than that, I am considering 1 or 3 with no PK. Will they differ in size?
I do not have experience with 4 or 5 and I do not think that they will perform well in this scenario.
Which way shall I go?
Your question is hard to answer as you seem to use a relational database system for something it is not designed for. The data you want to keep in the database seems to be too unstructured for getting much benefit from a relational database system. Database designs with mostly fields like "parameter type" and "parameter value" that try to cover very generic situations are mostly considered to be bad designs. Maybe you should consider using a "non relational database" like BigTable. If you really want to use a relational database system, I'd strongly recommend to read Beginning Database Design by Clare Churcher. It's an easy read, but gets you on the right track with respect to RDBS.
What are usage scenarios? Start with samples of queries and calculate necessary indexes.
Consider data partitioning as mentioned before. Try to understand your data / relations more. I believe the decision should be based on business meaning/usages of the data.
I think it's a great question - This situation is fairly common, though it is awkward to make tables to support it.
In terms of performance, having a table like indicated in #3 potentially wastes a huge amount of storage and RAM because for each row you allocate space for a value of every type, but only use one. If you use the new sparse table feature of 2008 it could help, but there are other issues too: it's a little hard to constrain/normalize, because you want only only one of the multiple values to be populated for each row - having two values in two columns would be an error, but the design doesn't reflect that. I'd cross that off.
So, if it were me I'd be looking at option 1 or 2 or 4, and the decision would be driven by this: do I typically need to make one query returning rows that have a mix of values of different types in the same result set? Or will I almost always ask for the rows by item and by type. I ask because if the values are different types it implies to me some difference in the source or the use of that data (you are unlikely, for example, to compare a string and a real, or a string and a bit.) This is relevant because having different tables per type might actually be a significant performance/scalability advantage, if partitioning the data that way makes queries faster. Partitioning data into smaller sets of more closely related data can give a performance advantage.
It's like having all the data in one massive (albeit sorted) set or having it partitioned into smaller, related sets. The smaller sets favor some types of queries, and if those are the queries you will need, it's a win.
Details:
CREATE TABLE [dbo].[items](
[itemid] [int] IDENTITY(1,1) NOT NULL,
[item] [varchar](100) NOT NULL,
CONSTRAINT [PK_items] PRIMARY KEY CLUSTERED
(
[itemid] ASC
)
)
/* This table has the problem of allowing two values
in the same row, plus allocates but does not use a
lot of space in memory and on disk (bad): */
CREATE TABLE [dbo].[vals](
[itemid] [int] NOT NULL,
[datestamp] [datetime] NOT NULL,
[valueBit] [bit] NULL,
[valueNumericA] [numeric](2, 0) NULL,
[valueNumericB] [numeric](8, 2) NULL,
[valueReal] [real] NULL,
[valueString] [varchar](100) NULL,
CONSTRAINT [PK_vals] PRIMARY KEY CLUSTERED
(
[itemid] ASC,
[datestamp] ASC
)
)
ALTER TABLE [dbo].[vals] WITH CHECK
ADD CONSTRAINT [FK_vals_items] FOREIGN KEY([itemid])
REFERENCES [dbo].[items] ([itemid])
GO
ALTER TABLE [dbo].[vals] CHECK CONSTRAINT [FK_vals_items]
GO
/* This is probably better, though casting is required
all the time. If you search with the variant as criteria,
that could get dicey as you have to be careful with types,
casting and indexing. Also everything is "mixed" in one
giant set */
CREATE TABLE [dbo].[allvals](
[itemid] [int] NOT NULL,
[datestamp] [datetime] NOT NULL,
[value] [sql_variant] NOT NULL
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[allvals] WITH CHECK
ADD CONSTRAINT [FK_allvals_items] FOREIGN KEY([itemid])
REFERENCES [dbo].[items] ([itemid])
GO
ALTER TABLE [dbo].[allvals] CHECK CONSTRAINT [FK_allvals_items]
GO
/* This would be an alternative, but you trade multiple
queries and joins for the casting issue. OTOH the implied
partitioning might be an advantage */
CREATE TABLE [dbo].[valsBits](
[itemid] [int] NOT NULL,
[datestamp] [datetime] NOT NULL,
[val] [bit] NOT NULL
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[valsBits] WITH CHECK
ADD CONSTRAINT [FK_valsBits_items] FOREIGN KEY([itemid])
REFERENCES [dbo].[items] ([itemid])
GO
ALTER TABLE [dbo].[valsBits] CHECK CONSTRAINT [FK_valsBits_items]
GO
CREATE TABLE [dbo].[valsNumericA](
[itemid] [int] NOT NULL,
[datestamp] [datetime] NOT NULL,
[val] numeric( 2, 0 ) NOT NULL
) ON [PRIMARY]
GO
... FK constraint ...
CREATE TABLE [dbo].[valsNumericB](
[itemid] [int] NOT NULL,
[datestamp] [datetime] NOT NULL,
[val] numeric ( 8, 2 ) NOT NULL
) ON [PRIMARY]
GO
... FK constraint ...
etc...