Insert Substring Values into SQL Temp Table Using While Loop? - sql-server

I have a table called accountNumbers. An example of values in the table are:
01-005-000-000-001-000
01-005-311-097-000
001-005-105-545
What I want to do is split the column (accountNum) at the dash, and then insert that value into a temp table, #test. When printing out #test, it should look like:
01
005
000
001
311
097
and so on. I cannot use store procedures or functions. I can get the first value, but any while loop I try just prints that first row over and over again.
WHILE ##ROWCOUNT > (select count(*) from dbo.accountNumbers
BEGIN
insert into #test (split, accountNum)
select SUBSTRING(accountNum, 1, CHARINDEX('-', accountNum) -1), accountNum
from dbo.accountNumbers
END

The restriction of no functions or procedures seems a little strange but you don't have to use a function to do this. A VERY minor tweak to the XML function found here http://sqlperformance.com/2012/07/t-sql-queries/split-strings can be utilized so you don't need a function to do this.
if OBJECT_ID('tempdb..#something') is not null
drop table #something
create table #something
(
AccountNumbers varchar(100)
)
insert #something
select '01-005-000-000-001-000' union all
select '01-005-311-097-000' union all
select '001-005-105-545'
select *
from #something s
cross apply
(
SELECT Item = y.i.value('(./text())[1]', 'nvarchar(4000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(s.AccountNumbers, '-', '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
)MySplit

Related

Substring is slow with while loop in SQL Server

One of my table column stores ~650,000 characters (each value of the column contains entire table). I know its bad design however, Client will not be able to change it.
I am tasked to convert the column into multiple columns.
I chose to use dbo.DelimitedSplit8K function
Unfortunately, it can only handle 8k characters at max.
So I decided to split the column into 81 8k batches using while loop and store the same in a variable table (temp or normal table made no improvement)
DECLARE #tab1 table ( serialnumber int, etext nvarchar(1000))
declare #scriptquan int = (select MAX(len (errortext)/8000) from mytable)
DECLARE #Counter INT
DECLARE #A bigint = 1
DECLARE #B bigint = 8000
SET #Counter=1
WHILE ( #Counter <= #scriptquan + 1)
BEGIN
insert into #tab1 select ItemNumber, Item from dbo.mytable cross apply dbo.DelimitedSplit8K(substring(errortext, #A, #B), CHAR(13)+CHAR(10))
SET #A = #A + 8000
SET #B = #B + 8000
SET #Counter = #Counter + 1
END
This followed by using below code
declare #tab2 table (Item nvarchar(max),itemnumber int, Colseq varchar(10)) -- declare table variable
;with cte as (
select [etext] ,ItemNumber, Item from #tab1 -- insert table name
cross apply dbo.DelimitedSplit8K(etext,' ')) -- insert table columns name that contains text
insert into #tab2 Select Item,itemnumber, 'a'+ cast (ItemNumber as varchar) colseq
from cte -- insert values to table variable
;WITH Tbl(item, colseq) AS(
select item, colseq from #tab2
),
CteRn AS(
SELECT item, colseq,
Rn = ROW_NUMBER() OVER(PARTITION BY colseq ORDER BY colseq)
FROM Tbl
)
SELECT
a1 Time,a2 Number,a3 Type,a4 Remarks
FROM CteRn r
PIVOT(
MAX(item)
FOR colseq IN(a1,a2,a3,a4)
)p
where a3 = 'error'
gives the desired output. However, just the loop takes 15 minutes to complete and overall query completes by 27 minutes. Is there any way I can make it faster? Total row count in my table is 2. So I don't think Index can help.
Client uses Azure SQL Database so I can't choose PowerShell or Python to accomplish this either.
Please let me know if more information is needed. I tried my best to mention everything I could.

How to obtain a string equals to each first letter of words in a sentence IN SQL

I have words separated with a space in a column like
apple orange banana I need the first letters as the result will be something like :
aob
First, split your text. I recommend some function:
CREATE FUNCTION Split(#text nvarchar(MAX),#separator nvarchar(MAX))
RETURNS TABLE AS RETURN
WITH Indexed AS
(
SELECT 1 N, CAST(1 AS bigint) S, CHARINDEX(#separator, #text, 1) E WHERE #text IS NOT NULL
UNION ALL
SELECT N+1, E+DATALENGTH(#separator)/2, CHARINDEX(#separator, #text, E+DATALENGTH(#separator)/2) FROM Indexed WHERE E>S
), Token AS
(
SELECT N, SUBSTRING(#text, S, CASE WHEN E=0 THEN DATALENGTH(#text)/2 ELSE E-S END) T FROM Indexed
)
SELECT * FROM Token
If you are using SQL 2016 and greater, use STRING_SPLIT instead.
Then, you can select first character of every word and join. See following example:
DECLARE #Sample TABLE (T nvarchar(100));
INSERT #Sample VALUES (N'apple orange banana'),(N'dog cat');
SELECT (SELECT SUBSTRING(T,1,1) [*] FROM Split(T,N' ') FOR XML PATH(''))
FROM #Sample
Result:
(no column name)
------
aob
dc
If you declare REGEX function in your DB (not native with SQL SERVER).
Using regexp_replace
select regexp_replace('apple orange banana','(\\w)(\\w* ?)','$1')
return
aob
I think the shortest will be this:
Here a mockup-table with two rows to simulate your issue:
DECLARE #mockup TABLE(ID INT IDENTITY,YourWords VARCHAR(100));
INSERT INTO #mockup VALUES('apple orange banana'),('one two three');
--That is the query:
SELECT m.ID
,REPLACE(Casted.query('for $w in /x return substring($w,1,1)').value('.','varchar(max)'),' ','')
FROM #mockup m
CROSS APPLY(SELECT CAST('<x>' + REPLACE(m.YourWords,' ','</x><x>') + '</x>' AS XML)) A(Casted);
The idea behind:
The string apple orange banana is tranformed to <x>apple</x><x>orange</x><x>banana</x> and is casted to XML, which allows to use XQuery.
Now we use .query() on the XML with a simple FLWOR statement. It tells the engine: run through each value of /x and return just the first letter. Calling value() on this with a . as XPath will return the values in one.
We need a final REPLACE() to get rid of blanks, which would otherwise appear as a o b instead of aob.
Just another option using a little XML. You could also use ParseName() provided you trap any periods in the string.
Example
Declare #YourTable table(ID int,LastName varchar(50),FirstName varchar(50))
Insert Into #YourTable values
(1,'Waston','Mary Jane')
Select A.ID
,NewValue = upper(
concat(
xmlData.value('/x[1]','varchar(1)')
,xmlData.value('/x[2]','varchar(1)')
,xmlData.value('/x[3]','varchar(1)')
,xmlData.value('/x[4]','varchar(1)')
,'.'
,LastName
)
)
From #YourTable A
Cross Apply ( values (convert(xml,'<x>' + replace(A.FirstName,' ','</x><x>')+'</x>' )) ) B(xmlData)
Returns
ID NewValue
1 MJ.WASTON
EDIT - Added ParseName() option
Select A.ID
,NewValue = upper(concat(Pos1,Pos2,Pos3,Pos4,'.',LastName))
From #YourTable A
Cross Apply (
Select Pos1 = left(parsename(tStr,4),1)
,Pos2 = left(parsename(tStr,3),1)
,Pos3 = left(parsename(tStr,2),1)
,Pos4 = left(parsename(tStr,1),1)
From ( values(replace(FirstName,' ','.'))) B1(tStr)
) B

Select rows with any member of list of substrings in string

In a Micrososft SQL Server table I have a column with a string.
Example:
'Servernamexyz.server.operationunit.otherstuff.icouldnt.predict.domain.domain2.domain3'
I also have a dynamic list of substrings
Example:
('icouldnt', 'stuff', 'banana')
I don't care for string manipulation. The substrings could also be called:
('%icouldnt%', '%stuff%', '%banana%')
What's the best way to find all rows where the string contains one of the substrings?
Solutions that are not possible:
multiple OR Statements in the WHERE clause, the list is dynamic
external Code to do a "for each", its a multi value parameter from the reportbuilder, so nothing useful here
changing the database, its the database of a tool a costumer is using and we can't change it, even if we would like... so much
I really cant believe how hard such a simple problem can turn out. It would need a "LIKE IN" command to do it in a way that looks ok. Right now I cant think of anything but a messy temp table.
One option is to use CHARINDEX
DECLARE #tab TABLE (Col1 NVARCHAR(200))
INSERT INTO #tab (Col1)
VALUES (N'Servernamexyz.server.operationunit.otherstuff.icouldnt.predict.domain.domain2.domain3' )
;WITH cteX
AS(
SELECT 'icouldnt' Strings
UNION ALL
SELECT 'stuff'
UNION ALL
SELECT 'banana'
)
SELECT
T.*, X.Strings
FROM #tab T
CROSS APPLY (SELECT X.Strings FROM cteX X) X
WHERE CHARINDEX(X.Strings, T.Col1) > 1
Output
EDIT - using an unknown dynamic string variable - #substrings
DECLARE #tab TABLE (Col1 NVARCHAR(200))
INSERT INTO #tab (Col1)
VALUES (N'Servernamexyz.server.operationunit.otherstuff.icouldnt.predict.domain.domain2.domain3' )
DECLARE #substrings NVARCHAR(200) = 'icouldnt,stuff,banana'
SELECT
T.*, X.Strings
FROM #tab T
CROSS APPLY
( --dynamically split the string
SELECT Strings = y.i.value('(./text())[1]', 'nvarchar(4000)')
FROM
(
SELECT x = CONVERT(XML, '<i>'
+ REPLACE(#substrings, ',', '</i><i>')
+ '</i>').query('.')
) AS a CROSS APPLY x.nodes('i') AS y(i)
) X
WHERE CHARINDEX(X.Strings, T.Col1) > 1

SQL Server group by count eliminate duplicates [duplicate]

How do I get:
id Name Value
1 A 4
1 B 8
2 C 9
to
id Column
1 A:4, B:8
2 C:9
No CURSOR, WHILE loop, or User-Defined Function needed.
Just need to be creative with FOR XML and PATH.
[Note: This solution only works on SQL 2005 and later. Original question didn't specify the version in use.]
CREATE TABLE #YourTable ([ID] INT, [Name] CHAR(1), [Value] INT)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'A',4)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'B',8)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (2,'C',9)
SELECT
[ID],
STUFF((
SELECT ', ' + [Name] + ':' + CAST([Value] AS VARCHAR(MAX))
FROM #YourTable
WHERE (ID = Results.ID)
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)')
,1,2,'') AS NameValues
FROM #YourTable Results
GROUP BY ID
DROP TABLE #YourTable
If it is SQL Server 2017 or SQL Server Vnext, SQL Azure you can use STRING_AGG as below:
SELECT id, STRING_AGG(CONCAT(name, ':', [value]), ', ')
FROM #YourTable
GROUP BY id
using XML path will not perfectly concatenate as you might expect... it will replace "&" with "&" and will also mess with <" and ">
...maybe a few other things, not sure...but you can try this
I came across a workaround for this... you need to replace:
FOR XML PATH('')
)
with:
FOR XML PATH(''),TYPE
).value('(./text())[1]','VARCHAR(MAX)')
...or NVARCHAR(MAX) if thats what youre using.
why the hell doesn't SQL have a concatenate aggregate function? this is a PITA.
I ran into a couple of problems when I tried converting Kevin Fairchild's suggestion to work with strings containing spaces and special XML characters (&, <, >) which were encoded.
The final version of my code (which doesn't answer the original question but may be useful to someone) looks like this:
CREATE TABLE #YourTable ([ID] INT, [Name] VARCHAR(MAX), [Value] INT)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'Oranges & Lemons',4)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'1 < 2',8)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (2,'C',9)
SELECT [ID],
STUFF((
SELECT ', ' + CAST([Name] AS VARCHAR(MAX))
FROM #YourTable WHERE (ID = Results.ID)
FOR XML PATH(''),TYPE
/* Use .value to uncomment XML entities e.g. > < etc*/
).value('.','VARCHAR(MAX)')
,1,2,'') as NameValues
FROM #YourTable Results
GROUP BY ID
DROP TABLE #YourTable
Rather than using a space as a delimiter and replacing all the spaces with commas, it just pre-pends a comma and space to each value then uses STUFF to remove the first two characters.
The XML encoding is taken care of automatically by using the TYPE directive.
Another option using Sql Server 2005 and above
---- test data
declare #t table (OUTPUTID int, SCHME varchar(10), DESCR varchar(10))
insert #t select 1125439 ,'CKT','Approved'
insert #t select 1125439 ,'RENO','Approved'
insert #t select 1134691 ,'CKT','Approved'
insert #t select 1134691 ,'RENO','Approved'
insert #t select 1134691 ,'pn','Approved'
---- actual query
;with cte(outputid,combined,rn)
as
(
select outputid, SCHME + ' ('+DESCR+')', rn=ROW_NUMBER() over (PARTITION by outputid order by schme, descr)
from #t
)
,cte2(outputid,finalstatus,rn)
as
(
select OUTPUTID, convert(varchar(max),combined), 1 from cte where rn=1
union all
select cte2.outputid, convert(varchar(max),cte2.finalstatus+', '+cte.combined), cte2.rn+1
from cte2
inner join cte on cte.OUTPUTID = cte2.outputid and cte.rn=cte2.rn+1
)
select outputid, MAX(finalstatus) from cte2 group by outputid
Install the SQLCLR Aggregates from http://groupconcat.codeplex.com
Then you can write code like this to get the result you asked for:
CREATE TABLE foo
(
id INT,
name CHAR(1),
Value CHAR(1)
);
INSERT INTO dbo.foo
(id, name, Value)
VALUES (1, 'A', '4'),
(1, 'B', '8'),
(2, 'C', '9');
SELECT id,
dbo.GROUP_CONCAT(name + ':' + Value) AS [Column]
FROM dbo.foo
GROUP BY id;
Eight years later... Microsoft SQL Server vNext Database Engine has finally enhanced Transact-SQL to directly support grouped string concatenation. The Community Technical Preview version 1.0 added the STRING_AGG function and CTP 1.1 added the WITHIN GROUP clause for the STRING_AGG function.
Reference: https://msdn.microsoft.com/en-us/library/mt775028.aspx
SQL Server 2005 and later allow you to create your own custom aggregate functions, including for things like concatenation- see the sample at the bottom of the linked article.
This is just an addition to Kevin Fairchild's post (very clever by the way). I would have added it as a comment, but I don't have enough points yet :)
I was using this idea for a view I was working on, however the items I was concatinating contained spaces. So I modified the code slightly to not use spaces as delimiters.
Again thanks for the cool workaround Kevin!
CREATE TABLE #YourTable ( [ID] INT, [Name] CHAR(1), [Value] INT )
INSERT INTO #YourTable ([ID], [Name], [Value]) VALUES (1, 'A', 4)
INSERT INTO #YourTable ([ID], [Name], [Value]) VALUES (1, 'B', 8)
INSERT INTO #YourTable ([ID], [Name], [Value]) VALUES (2, 'C', 9)
SELECT [ID],
REPLACE(REPLACE(REPLACE(
(SELECT [Name] + ':' + CAST([Value] AS VARCHAR(MAX)) as A
FROM #YourTable
WHERE ( ID = Results.ID )
FOR XML PATH (''))
, '</A><A>', ', ')
,'<A>','')
,'</A>','') AS NameValues
FROM #YourTable Results
GROUP BY ID
DROP TABLE #YourTable
An example would be
In Oracle you can use LISTAGG aggregate function.
Original records
name type
------------
name1 type1
name2 type2
name2 type3
Sql
SELECT name, LISTAGG(type, '; ') WITHIN GROUP(ORDER BY name)
FROM table
GROUP BY name
Result in
name type
------------
name1 type1
name2 type2; type3
This kind of question is asked here very often, and the solution is going to depend a lot on the underlying requirements:
https://stackoverflow.com/search?q=sql+pivot
and
https://stackoverflow.com/search?q=sql+concatenate
Typically, there is no SQL-only way to do this without either dynamic sql, a user-defined function, or a cursor.
Just to add to what Cade said, this is usually a front-end display thing and should therefore be handled there. I know that sometimes it's easier to write something 100% in SQL for things like file export or other "SQL only" solutions, but most of the times this concatenation should be handled in your display layer.
Don't need a cursor... a while loop is sufficient.
------------------------------
-- Setup
------------------------------
DECLARE #Source TABLE
(
id int,
Name varchar(30),
Value int
)
DECLARE #Target TABLE
(
id int,
Result varchar(max)
)
INSERT INTO #Source(id, Name, Value) SELECT 1, 'A', 4
INSERT INTO #Source(id, Name, Value) SELECT 1, 'B', 8
INSERT INTO #Source(id, Name, Value) SELECT 2, 'C', 9
------------------------------
-- Technique
------------------------------
INSERT INTO #Target (id)
SELECT id
FROM #Source
GROUP BY id
DECLARE #id int, #Result varchar(max)
SET #id = (SELECT MIN(id) FROM #Target)
WHILE #id is not null
BEGIN
SET #Result = null
SELECT #Result =
CASE
WHEN #Result is null
THEN ''
ELSE #Result + ', '
END + s.Name + ':' + convert(varchar(30),s.Value)
FROM #Source s
WHERE id = #id
UPDATE #Target
SET Result = #Result
WHERE id = #id
SET #id = (SELECT MIN(id) FROM #Target WHERE #id < id)
END
SELECT *
FROM #Target
Let's get very simple:
SELECT stuff(
(
select ', ' + x from (SELECT 'xxx' x union select 'yyyy') tb
FOR XML PATH('')
)
, 1, 2, '')
Replace this line:
select ', ' + x from (SELECT 'xxx' x union select 'yyyy') tb
With your query.
You can improve performance significant the following way if group by contains mostly one item:
SELECT
[ID],
CASE WHEN MAX( [Name]) = MIN( [Name]) THEN
MAX( [Name]) NameValues
ELSE
STUFF((
SELECT ', ' + [Name] + ':' + CAST([Value] AS VARCHAR(MAX))
FROM #YourTable
WHERE (ID = Results.ID)
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)')
,1,2,'') AS NameValues
END
FROM #YourTable Results
GROUP BY ID
didn't see any cross apply answers, also no need for xml extraction. Here is a slightly different version of what Kevin Fairchild wrote. It's faster and easier to use in more complex queries:
select T.ID
,MAX(X.cl) NameValues
from #YourTable T
CROSS APPLY
(select STUFF((
SELECT ', ' + [Name] + ':' + CAST([Value] AS VARCHAR(MAX))
FROM #YourTable
WHERE (ID = T.ID)
FOR XML PATH(''))
,1,2,'') [cl]) X
GROUP BY T.ID
Using the Stuff and for xml path operator to concatenate rows to string :Group By two columns -->
CREATE TABLE #YourTable ([ID] INT, [Name] CHAR(1), [Value] INT)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'A',4)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'B',8)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (1,'B',5)
INSERT INTO #YourTable ([ID],[Name],[Value]) VALUES (2,'C',9)
-- retrieve each unique id and name columns and concatonate the values into one column
SELECT
[ID],
STUFF((
SELECT ', ' + [Name] + ':' + CAST([Value] AS VARCHAR(MAX)) -- CONCATONATES EACH APPLICATION : VALUE SET
FROM #YourTable
WHERE (ID = Results.ID and Name = results.[name] )
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)')
,1,2,'') AS NameValues
FROM #YourTable Results
GROUP BY ID
SELECT
[ID],[Name] , --these are acting as the group by clause
STUFF((
SELECT ', '+ CAST([Value] AS VARCHAR(MAX)) -- CONCATONATES THE VALUES FOR EACH ID NAME COMBINATION
FROM #YourTable
WHERE (ID = Results.ID and Name = results.[name] )
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)')
,1,2,'') AS NameValues
FROM #YourTable Results
GROUP BY ID, name
DROP TABLE #YourTable
Using Replace Function and FOR JSON PATH
SELECT T3.DEPT, REPLACE(REPLACE(T3.ENAME,'{"ENAME":"',''),'"}','') AS ENAME_LIST
FROM (
SELECT DEPT, (SELECT ENAME AS [ENAME]
FROM EMPLOYEE T2
WHERE T2.DEPT=T1.DEPT
FOR JSON PATH,WITHOUT_ARRAY_WRAPPER) ENAME
FROM EMPLOYEE T1
GROUP BY DEPT) T3
For sample data and more ways click here
If you have clr enabled you could use the Group_Concat library from GitHub
Another example without the garbage: ",TYPE).value('(./text())[1]','VARCHAR(MAX)')"
WITH t AS (
SELECT 1 n, 1 g, 1 v
UNION ALL
SELECT 2 n, 1 g, 2 v
UNION ALL
SELECT 3 n, 2 g, 3 v
)
SELECT g
, STUFF (
(
SELECT ', ' + CAST(v AS VARCHAR(MAX))
FROM t sub_t
WHERE sub_t.g = main_t.g
FOR XML PATH('')
)
, 1, 2, ''
) cg
FROM t main_t
GROUP BY g
Input-output is
************************* -> *********************
* n * g * v * * g * cg *
* - * - * - * * - * - *
* 1 * 1 * 1 * * 1 * 1, 2 *
* 2 * 1 * 2 * * 2 * 3 *
* 3 * 2 * 3 * *********************
*************************
I used this approach which may be easier to grasp. Get a root element, then concat to choices any item with the same ID but not the 'official' name
Declare #IdxList as Table(id int, choices varchar(max),AisName varchar(255))
Insert into #IdxLIst(id,choices,AisName)
Select IdxId,''''+Max(Title)+'''',Max(Title) From [dbo].[dta_Alias]
where IdxId is not null group by IdxId
Update #IdxLIst
set choices=choices +','''+Title+''''
From #IdxLIst JOIN [dta_Alias] ON id=IdxId And Title <> AisName
where IdxId is not null
Select * from #IdxList where choices like '%,%'
For all my healthcare folks out there:
SELECT
s.NOTE_ID
,STUFF ((
SELECT
[note_text] + ' '
FROM
HNO_NOTE_TEXT s1
WHERE
(s1.NOTE_ID = s.NOTE_ID)
ORDER BY [line] ASC
FOR XML PATH(''),TYPE).value('(./text())[1]','VARCHAR(MAX)')
,
1,
2,
'') AS NOTE_TEXT_CONCATINATED
FROM
HNO_NOTE_TEXT s
GROUP BY NOTE_ID

Creating contra values in the same column SQL

Aim to write a statement to produce a table with a numeric column that contains contra values, e.g.
Ref Value
a 100
b 75
c 50
c -50
b -75
a -100
I am new to SQL but aware it works row by row, so the only way I could think of doing this is write an initial SELECT statement into a temporary table and INSERT into my temporary table with the contra values, i.e.
SELECT
[Ref],
[Value]
INTO #Temp
FROM
mytable
INSERT INTO #Temp ([Ref], [Value])
SELECT
[Ref],
0 - [Value]
FROM
mytable
While this 'does the job' I fear it is 'messy' (could possibly cause problems when used for its intended purpose) and wondered if anyone would be able to provide a better solution.
Use Union ALL to combine the original and negative values. Then insert into temp table. Try this.
SELECT [Ref],
[Value]
INTO #Temp
FROM (SELECT [Ref],
[Value]
FROM mytable
UNION ALL
SELECT [Ref],
[Value] * -1
FROM mytable) a
If you just want to view the result remove the into #temp table
SELECT [Ref],
[Value]
FROM (SELECT [Ref],
[Value]
FROM mytable
UNION ALL
SELECT [Ref],
[Value] * -1
FROM mytable) a

Resources