How to remove space when concatenating data from different rows into one column using xml? - sql-server

I am trying to combine data from different rows into one column, and this is working with just one minor problem.
declare #RitID int = 16
select ...,
( select distinct
ISNULL(LTRIM(RTRIM(r2.LotNr)), LTRIM(RTRIM(r.LotNr))) + '+' as 'data()'
from tblExtraBestemming eb2
inner join tblRit r2 on eb2.RitID = r2.RitID
where eb2.BestemmingID = eb.BestemmingID
and eb2.BestemmingTypeID = eb.BestemmingTypeID
and ( (eb.CombinedChildExtraBestemmingID is null and eb2.RitID = #RitID)
or
(eb.CombinedChildExtraBestemmingID is not null and eb2.RitID in (select r4.RitID from tblRit r4 where r4.MasterRitID = #RitID) )
)
for XML PATH('')
) as LotNr
from tblExtraBestemming eb
where ...
this returns the correct data for the column LotNr, like this
GTT18196
GTT18197
GTT18198+ GTT18199
Now my only problem is the space after the + sign in the third row from the result, how can I get rid of this ?
I expect this result
GTT18196
GTT18197
GTT18198+GTT18199
PS, actually there is also a + at the end of each row, but that is removed by the client. I thought I better mentions this already.
EDIT
I checked the data, there are no spaces at the end or the beginning of the data
EDIT
Query updated as suggested by #Larnu
EDIT
if I check the data in the table, this is the result
select '/' + r.LotNr + '/' from tblRit r where r.RitID in (50798, 50799)
COLUMN1
-------
/GTT18198/
/GTT18199/
So it appears to me there are no characters before or after the data

Just remove AS 'data()' from your query (it is not required in the above case).
And if trailing + is a problem, move it to the beginning and use STUFF function to chop off the first character from the result.

Related

SQL Server geography: Reduce size (decimal precision) of WKT text

For my farming app, a stored proc retrieves paddock/field boundaries that are stored as SQL Server geography data type, for display on the user's mobile device.
SQL Server's .ToString() and .STAsText() functions render each vertex as a lat/long pair, to 15 decimal places. From this answer, 15 decimals defines a location to within the width of an atom! To the nearest metre would be good enough for me.
The resulting overly-precise payload is extremely large and too slow for use on larger farms.
From my SQL Server geography data, I would like to produce WKT that is formatted to 4 or 5 decimals. I could find no built-in methods, but my best leads are:
Postgis and Google Cloud "BigQuery" have the ST_SNAPTOGRID function, which would be perfect, and
Regex might be useful, e.g. this answer, but SQL Server doesn't seem to have a regex replace.
I would think this is a common problem: is there a simple solution?
By stepping through #iamdave's excellent answer, and using that same approach, it looks like we need split only on periods.... I think we can ignore all parantheses and commas, and ignore the POLYGON prefix (which means it'll work across other GEOGRAPHY types, such as MULTIPOLYGON.)
i.e. Each time we find a period, grab only the next 4 characters after it, and throw away any numbers after that (until we hit a non-number.)
This works for me (using #iamdave's test data):
DECLARE #wkt NVARCHAR(MAX), #wktShort NVARCHAR(MAX);
DECLARE #decimalPlaces int = 4;
SET #wkt = 'POLYGON((-121.973669 37.365336,-121.97367 37.365336,-121.973642 37.365309,-121.973415 37.365309,-121.973189 37.365309,-121.973002 37.365912,-121.972815 37.366515,-121.972796 37.366532,-121.972776 37.366549,-121.972627 37.366424,-121.972478 37.366299,-121.972422 37.366299,-121.972366 37.366299,-121.972298 37.366356,-121.97223 37.366412,-121.97215 37.366505,-121.97207 37.366598,-121.971908 37.366794,-121.971489 37.367353,-121.971396 37.367484,-121.971285 37.36769,-121.971173 37.367897,-121.971121 37.368072,-121.971068 37.368248,-121.971028 37.36847,-121.970987 37.368692,-121.970987 37.368779,-121.970987 37.368866,-121.970949 37.368923,-121.970912 37.36898,-121.970935 37.36898,-121.970958 37.36898,-121.970975 37.368933,-121.970993 37.368887,-121.971067 37.368807,-121.97114 37.368726,-121.971124 37.368705,-121.971108 37.368685,-121.971136 37.368698,-121.971163 37.368712,-121.97134 37.368531,-121.971516 37.368351,-121.971697 37.368186,-121.971878 37.368021,-121.972085 37.367846,-121.972293 37.36767,-121.972331 37.367629,-121.972369 37.367588,-121.972125 37.367763,-121.97188 37.367938,-121.971612 37.36815,-121.971345 37.368362,-121.971321 37.36835,-121.971297 37.368338,-121.971323 37.368298,-121.97135 37.368259,-121.971569 37.368062,-121.971788 37.367865,-121.971977 37.367716,-121.972166 37.367567,-121.972345 37.367442,-121.972524 37.367317,-121.972605 37.367272,-121.972687 37.367227,-121.972728 37.367227,-121.972769 37.367227,-121.972769 37.367259,-121.972769 37.367291,-121.972612 37.367416,-121.972454 37.367542,-121.972488 37.367558,-121.972521 37.367575,-121.972404 37.367674,-121.972286 37.367773,-121.972194 37.367851,-121.972101 37.367928,-121.972046 37.36799,-121.971991 37.368052,-121.972008 37.368052,-121.972025 37.368052,-121.972143 37.367959,-121.972261 37.367866,-121.972296 37.367866,-121.972276 37.36794,-121.972221 37.36798,-121.972094 37.368097,-121.971966 37.368214,-121.971956 37.368324,-121.971945 37.368433,-121.971907 37.368753,-121.971868 37.369073,-121.97184 37.369578,-121.971812 37.370083,-121.971798 37.370212,-121.971783 37.370342,-121.971542 37.370486,-121.971904 37.370324,-121.972085 37.37028,-121.972266 37.370236,-121.972559 37.370196,-121.972852 37.370155,-121.973019 37.370155,-121.973186 37.370155,-121.973232 37.370136,-121.973279 37.370116,-121.973307 37.370058,-121.973336 37.370001,-121.973363 37.369836,-121.973391 37.369671,-121.973419 37.369227,-121.973446 37.368784,-121.973429 37.368413,-121.973413 37.368041,-121.973361 37.367714,-121.973308 37.367387,-121.973285 37.367339,-121.973262 37.36729,-121.973126 37.3673,-121.972989 37.36731,-121.973066 37.36728,-121.973144 37.367251,-121.973269 37.367237,-121.973393 37.367223,-121.973443 37.367158,-121.973493 37.367093,-121.973518 37.36702,-121.973543 37.366947,-121.973582 37.366618,-121.973622 37.366288,-121.97366 37.365826,-121.973698 37.365363,-121.973669 37.365336))';
-- Split on '.', then get the next N decimals, and find the index of the first non-number.
-- Then recombine the fragments, skipping the unwanted numbers.
WITH points AS (
SELECT value, LEFT(value, #decimalPlaces) AS decimals, PATINDEX('%[^0-9]%', value) AS indx
FROM STRING_SPLIT(#wkt, '.')
)
SELECT #wktShort = STRING_AGG(IIF(indx < #decimalPlaces, '', decimals) + SUBSTRING(value, indx, LEN(value)), '.')
FROM points;
Comparing original vs shortened, we can see that each number is truncated to 4dp:
SELECT #wkt AS Text UNION ALL SELECT #wktShort;
Edit
I believe I may have misunderstood your question and you are looking to transmit the WKT rather than the binary representations of the polygons? If that is the case, my answer below still shows you how to chop off some decimal places (without rounding them). Just don't wrap the stuff(...) FOR XML in a STGeomFromText and you have the amended WKT.
When working with geography data types, it can be handy to maintain a very detailed 'master' version, from which you generate and persist less detailed versions based on your requirements.
An easy way to generate these reduced complexity polygons is to use the helpfully named Reduce function, which I think will actually help you in this situation.
If you would prefer to go down the route of reducing the number of decimal places you will either have to write a custom CLR function or enter the wonderful world of SQL Server string manipulation!
SQL Query
declare #DecimalPlaces int = 4; -- Specify the desired number of lat/long decimals
with g as(
select p.g -- Original polygon, for comparison purposes
,geography::STGeomFromText('POLYGON((' -- stripped apart and then recreated polygon from text, using a custom string split function. You won't be able to use the built in STRING_SPLIT here as it doesn't guarantee sort order.
+ stuff((select ', ' + left(s.item,charindex('.',s.item,0) + #DecimalPlaces) + substring(s.item,charindex(' ',s.item,0),charindex('.',s.item,charindex(' ',s.item,0)) - charindex(' ',s.item,0) + 1 + #DecimalPlaces)
from dbo.fn_StringSplitMax(replace(replace(p.g.STAsText(),'POLYGON ((',''),'))',''),', ',null) as s
for xml path(''), type).value('.', 'NVARCHAR(MAX)') -- STUFF and FOR XML mimics GROUP_CONCAT functionality seen in other SQL languages, to recombine shortened Points back into a Polygon string
,1,2,''
)
+ '))', 4326).MakeValid() as x -- Remember to make the polygon valid again, as you have been messing with the Point data
from(values(geography::STGeomFromText('POLYGON((-121.973669 37.365336,-121.97367 37.365336,-121.973642 37.365309,-121.973415 37.365309,-121.973189 37.365309,-121.973002 37.365912,-121.972815 37.366515,-121.972796 37.366532,-121.972776 37.366549,-121.972627 37.366424,-121.972478 37.366299,-121.972422 37.366299,-121.972366 37.366299,-121.972298 37.366356,-121.97223 37.366412,-121.97215 37.366505,-121.97207 37.366598,-121.971908 37.366794,-121.971489 37.367353,-121.971396 37.367484,-121.971285 37.36769,-121.971173 37.367897,-121.971121 37.368072,-121.971068 37.368248,-121.971028 37.36847,-121.970987 37.368692,-121.970987 37.368779,-121.970987 37.368866,-121.970949 37.368923,-121.970912 37.36898,-121.970935 37.36898,-121.970958 37.36898,-121.970975 37.368933,-121.970993 37.368887,-121.971067 37.368807,-121.97114 37.368726,-121.971124 37.368705,-121.971108 37.368685,-121.971136 37.368698,-121.971163 37.368712,-121.97134 37.368531,-121.971516 37.368351,-121.971697 37.368186,-121.971878 37.368021,-121.972085 37.367846,-121.972293 37.36767,-121.972331 37.367629,-121.972369 37.367588,-121.972125 37.367763,-121.97188 37.367938,-121.971612 37.36815,-121.971345 37.368362,-121.971321 37.36835,-121.971297 37.368338,-121.971323 37.368298,-121.97135 37.368259,-121.971569 37.368062,-121.971788 37.367865,-121.971977 37.367716,-121.972166 37.367567,-121.972345 37.367442,-121.972524 37.367317,-121.972605 37.367272,-121.972687 37.367227,-121.972728 37.367227,-121.972769 37.367227,-121.972769 37.367259,-121.972769 37.367291,-121.972612 37.367416,-121.972454 37.367542,-121.972488 37.367558,-121.972521 37.367575,-121.972404 37.367674,-121.972286 37.367773,-121.972194 37.367851,-121.972101 37.367928,-121.972046 37.36799,-121.971991 37.368052,-121.972008 37.368052,-121.972025 37.368052,-121.972143 37.367959,-121.972261 37.367866,-121.972296 37.367866,-121.972276 37.36794,-121.972221 37.36798,-121.972094 37.368097,-121.971966 37.368214,-121.971956 37.368324,-121.971945 37.368433,-121.971907 37.368753,-121.971868 37.369073,-121.97184 37.369578,-121.971812 37.370083,-121.971798 37.370212,-121.971783 37.370342,-121.971542 37.370486,-121.971904 37.370324,-121.972085 37.37028,-121.972266 37.370236,-121.972559 37.370196,-121.972852 37.370155,-121.973019 37.370155,-121.973186 37.370155,-121.973232 37.370136,-121.973279 37.370116,-121.973307 37.370058,-121.973336 37.370001,-121.973363 37.369836,-121.973391 37.369671,-121.973419 37.369227,-121.973446 37.368784,-121.973429 37.368413,-121.973413 37.368041,-121.973361 37.367714,-121.973308 37.367387,-121.973285 37.367339,-121.973262 37.36729,-121.973126 37.3673,-121.972989 37.36731,-121.973066 37.36728,-121.973144 37.367251,-121.973269 37.367237,-121.973393 37.367223,-121.973443 37.367158,-121.973493 37.367093,-121.973518 37.36702,-121.973543 37.366947,-121.973582 37.366618,-121.973622 37.366288,-121.97366 37.365826,-121.973698 37.365363,-121.973669 37.365336))', 4326))) as p(g)
)
-- select various versions of the polygons into the same column for overlay comparison in SSMS
select 'Original' as l
,g
from g
union all
select 'Short' as l
,x
from g
union all
select 'Original Reduced' as l
,g.Reduce(10)
from g
union all
select 'Short Reduced' as l
,x.Reduce(10)
from g;
Output
What is interesting to note here is the difference in length of the geog binary representation (simple count of characters as displayed). As I mentioned above, simply using the Reduce function may do what you need, so you will want to test various approaches to see how best you cut down on your data transfer.
+------------------+--------------------+------+
| l | g | Len |
+------------------+--------------------+------+
| Original | 0xE6100000010484...| 4290 |
| Short | 0xE6100000010471...| 3840 |
| Original Reduced | 0xE6100000010418...| 834 |
| Short Reduced | 0xE610000001041E...| 1184 |
+------------------+--------------------+------+
Visual Comparison
String Split Function
As polygon data can be bloody huge, you will need a string splitter that can handle more then 4k or 8k characters. In my case I tend to opt for an xml based approach:
create function [dbo].[fn_StringSplitMax]
(
#str nvarchar(max) = ' ' -- String to split.
,#delimiter as nvarchar(max) = ',' -- Delimiting value to split on.
,#num as int = null -- Which value to return.
)
returns table
as
return
with s as
( -- Convert the string to an XML value, replacing the delimiter with XML tags
select convert(xml,'<x>' + replace((select #str for xml path('')),#delimiter,'</x><x>') + '</x>').query('.') as s
)
select rn
,item -- Select the values from the generated XML value by CROSS APPLYing to the XML nodes
from(select row_number() over (order by (select null)) as rn
,n.x.value('.','nvarchar(max)') as item
from s
cross apply s.nodes('x') as n(x)
) a
where rn = #num
or #num is null;

Query fails on "converting character string to smalldatetime data type"

I've been tasked with fixing some SQL code that doesn't work. The query reads from a view against a predicate. The query right now looks like so.
SELECT TOP (100) Beginn
FROM V_LLAMA_Seminare
//Removal of the following line makes the query successful, keeping it breaks it
where Beginn > (select cast (getdate() as smalldatetime))
order by Beginn desc
When I run the above query, I am greeted with the following error.
Msg 295, Level 16, State 3, Line 1
Conversion failed when converting character string to smalldatetime data type.
I decided to remove the WHERE clause, and now it runs returning 100 rows.
At first, I thought that behind the scenes, SQL Server was somehow including my predicate when bringing back the View . But then I investigated how the View was being created, especially the Beginn field, and at no point does it return a String.
Long story short, the column that becomes the Beginn field is a BIGINT timestamp like 201604201369.... The original user transforms this BIGINT to a smalldatetime using the following magic.
....
CASE WHEN ma.datum_dt = 0
THEN null
ELSE CONVERT(smalldatetime, SUBSTRING(CAST(ma.datum_dt AS varchar(max)),0,5) + '-' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),5,2) + '-' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),7,2) + ' ' +
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),9,2) +':'+
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),11,2) +':' +
RIGHT(CAST(ma.datum_dt AS varchar(max)),2)) END AS Beginn
...
My last attempt at finding the problem was to query the view and run the function ISDATE over the Beginn column and see if it returned a 0 which it never did.
So my question is two fold, "Why does a predicate break something" and two "Where on earth is this string error coming from when the Beginn value is being formed from a BIGINT".
Any help is greatly appreciated.
This problem is culture related...
Try this and then change the first SET LANGUAGE to GERMAN
SET LANGUAGE ENGLISH;
DECLARE #bi BIGINT=20160428001600;
SELECT CASE WHEN #bi = 0
THEN null
ELSE CONVERT(datetime, SUBSTRING(CAST(#bi AS varchar(max)),0,5) + '-' +
SUBSTRING(CAST(#bi AS varchar(max)),5,2) + '-' +
SUBSTRING(CAST(#bi AS varchar(max)),7,2) + ' ' +
SUBSTRING(CAST(#bi AS varchar(max)),9,2) +':'+
SUBSTRING(CAST(#bi AS varchar(max)),11,2) +':' +
RIGHT(CAST(#bi AS varchar(max)),2)) END AS Beginn
It is a very bad habit to think, that date values look the same everywhere (Oh no, my small application will never go international ...)
Try to stick to culture independent formats like ODBC or ISO
EDIT
A very easy solution for you actually was to replace the blank with a "T"
SUBSTRING(CAST(ma.datum_dt AS varchar(max)),7,2) + 'T' +
Then it's ISO 8601 and will convert...
The solution was found after looking through #Shnugo's comment. When I took my query which contained the Bigint->Datetime conversion logic, and put it into a CTE with "TOP 100000000" to avoid any implicit conversion actions, my query worked. Here is what my view looks like now with some unimportant parts omitted.
---Important part---
CREATE VIEW [dbo].[V_SomeView] AS
WITH CTE AS (
SELECT TOP 1000000000 ma.id AS MA_ID,
---Important part---
vko.extkey AS ID_VKO,
vko.text AS Verkaufsorganisation,
fi.f7000 AS MDM_Nr,
vf.f7105 AS SAPKdnr,
CASE WHEN ma.datum_dt = 0 --Conversion logic
CASE WHEN ma.endedatum_dt = 0 --Conversion logic
CONVERT(NVARCHAR(MAX),art.text) AS Art,
.....
FROM [ucrm].[dbo].[CRM_MA] ma,
[ucrm].[dbo].[CRM_fi] fi,
[ucrm].[dbo].[CRM_vf] vf,
[ucrm].[dbo].[CRM_ka] vko,
[ucrm].[dbo].[CRM_ka] art,
[ucrm].[dbo].[CRM_ka] kat
where ma.loskz = 0
and fi.loskz = 0
and vf.loskz = 0
and fi.F7029 = 0
and vf.F7023 = 0
...
GROUP BY ma.id,
vko.extkey,
vko.text,
fi.f7000 ,
vf.f7105,
ma.datum_dt,
ma.endedatum_dt,
....
)
select * FROM CTE;

Concatenation Statement

I am using a concatenation statement to return Last Name, First Name Middle initial with a period (.). Ex: “Brown, John R.”. The statement is as follows:
IF([Middle Name] IS NULL )
THEN (trim([Last Name])+', '+trim([First Name]))
ELSE (trim([Last Name])+', '+trim([First Name])+' '+substring(nullif([Middle Name],' '),1,1)+'.'))
However, the statement is randomly omitting several of the names in the returned result. If I do not use the "nullif" option, it still returns periods where there is no middle initial.
Any help would be appreciated.
Thanks
How about replace the whole thing with:
SELECT trim([Last Name])+', '+trim([First Name]) + Isnull(' ' + LEFT([Middle NAME],1) + '.','') AS FullName
Are all the first and last names filled in (NOT Null)?

Missing data row in NOT IN clause

I just realised that my Auto-Price Calculation doesn't fill the Prices for ListID 4.
I inserted the Prices from the SELECT shown below.
For bugg research I executed the SELECT without the WHERE part and it shows me the example data row.
I can't find the error though, why it is not shown in the complete select (it has no entry with ListID = 4).
Someone can see my mistake?
Edit: Just tried the subselect alone, it shows no rows for the requested article. Why is the NOT IN clause unaffected by this fact?
Most likely, it's because of how you are combining the artikelnummer and the auspraegungID.
I do not think you have accounted for that fact that '100' + '0' is idential to '10' + '00'.
Instead of trying to merge two fields into one, perhaps try the following?
SELECT
*
FROM
#allArticles AS allArticles
WHERE
Artikelnummer = 'IT-810260'
AND NOT EXISTS (SELECT *
FROM KHKPreisListenArtikel
WHERE ListeID = 4
AND Artikelnummer = allArticles.Artikelnummer
AND Auspraegung = allArticles.Auspraegung
)
If that still doesn't work, then you must have corresponding records in that other table, find them like this...
SELECT
*
FROM
#allArticles AS allArticles
INNER JOIN
KHKPreisListenArtikel AS Preis
ON Preis.ListeID = 4
AND Preis.Artikelnummer = allArticles.Artikelnummer
AND Preis.Auspraegung = allArticles.Auspraegung
WHERe
allArticles.Artikelnummer = 'IT-810260'
PLEASE ALSO NOTE
Please don't include images of code, please copy the code, so that we can copy it too.
Especially when the tables/fields are in another language...
EDIT
Here is a query that will show the cause of your original query to fail.
SELECT
*
FROM
#allArticles AS allArticles
INNER JOIN
KHKPreisListenArtikel AS Preis
ON Preis.ListeID = 4
AND Preis.Artikelnummer + CONVERT(VARCHAR(50), Preis.Auspraegung)
=
allArticles.Artikelnummer + CONVERT(VARCHAR(50), allArticles.Auspraegung)
WHERE
allArticles.Artikelnummer = 'IT-810260'

Oracle split text into multiple rows

Inside a varchar2 column I have text values like :
aaaaaa. fgdfg.
bbbbbbbbbbbbbb ccccccccc
dddddd ddd dddddddddddd,
asdasdasdll
sssss
if i do select column from table where id=... i get the whole text in a single row, normally.
But i would like to get the result in multiple rows, 5 for the example above.
I have to use just one select statement, and the delimiters will be new line or carriage return (chr(10), chr(13) in oracle)
Thank you!
Like this, maybe (but it all depends on the version of oracle you are using):
WITH yourtable AS (SELECT REPLACE('aaaaaa. fgdfg.' ||chr(10)||
'bbbbbbbbbbbbbb ccccccccc ' ||chr(13)||
'dddddd ddd dddddddddddd,' ||chr(10)||
'asdasdasdll ' ||chr(13)||
'sssss '||chr(10),chr(13),chr(10)) AS astr FROM DUAL)
SELECT REGEXP_SUBSTR ( astr, '[^' ||chr(10)||']+', 1, LEVEL) data FROM yourtable
CONNECT BY LEVEL <= LENGTH(astr) - LENGTH(REPLACE(astr, chr(10))) + 1
see: Comma Separated values in Oracle
The answer by Kevin Burton contains a bug if your data contains empty lines.
The adaptation below, based on the solution invented here, works. Check that post for an explanation on the issue and the solution.
WITH yourtable AS (SELECT REPLACE('aaaaaa. fgdfg.' ||chr(10)||
'bbbbbbbbbbbbbb ccccccccc ' ||chr(13)||
chr(13)||
'dddddd ddd dddddddddddd,' ||chr(10)||
'asdasdasdll ' ||chr(13)||
'sssss '||chr(10),chr(13),chr(10)) AS astr FROM DUAL)
SELECT REGEXP_SUBSTR ( astr, '([^' ||chr(10)||']*)('||chr(10)||'|$)', 1, LEVEL, null, 1) data FROM yourtable
CONNECT BY LEVEL <= LENGTH(astr) - LENGTH(REPLACE(astr, chr(10))) + 1;

Resources