I am attempting to Pivot a table passed in as a UDT.. Column 1 will have duplicate values and Column 2 will have different values (example below). I am hoping for the Column Name to be set as Phone Number and the two Values under the Phone Number column.
DECLARE #query AS NVARCHAR(MAX) = '';
--Temp Table to act as UDT for ease of testing
DECLARE #udt TABLE (DatabaseFieldName nvarchar(50), Value nvarchar(50))
INSERT INTO #udt VALUES('PhoneNumber','01234567890')
INSERT INTO #udt VALUES('PhoneNumber','09876543210')
--Preview of table before Pivot
select * from #udt
CREATE TABLE #temp
(
DatabaseFieldName nvarchar(50),
Value nvarchar(50)
)
INSERT INTO #temp
SELECT DatabaseFieldName, Value
FROM #udt
SELECT #cols = #cols + QUOTENAME(DatabaseFieldName) + ',' FROM (select distinct DatabaseFieldName from #temp) as temp
SELECT #cols = substring(#cols, 0, len(#cols)) -- Trims ',' at the end
SET #query =
'
SELECT * FROM
(
SELECT DatabaseFieldName, Value
FROM #temp
) AS SRC
PIVOT
(
MIN(Value) for DatabaseFieldName in (' + #cols + ')
) AS PivotTable';
execute(#query)
DROP TABLE #temp
Example of current & desired results
Data as it comes in:
DatabaseFieldName | Value
--------------------------------
PhoneNumber | 01234567890
PhoneNumber | 09876543210
Outcome I am hoping to get:
PhoneNumber
------------
01234567890
09876543210
What I am currently getting:
PhoneNumber
------------
01234567890
At the moment the second number is being ignored due to the use of distinct in the select statement, however an error is thrown if distinct is not used.
Are all the values that you want in the pivoted column PhoneNumbers? If so, you can just select the phone numbers themselves with an alias. E.g
SELECT Value As PhoneNumber
FROM (either #UDT or #temp)
WHERE DatabaseFieldName = 'PhoneNumber'
Unless there is a requirement I'm missing here, this would do everything you want it to.
It is not due to DISTINCT, it is because of using MIN(Value) , it gets the first one.
Let's say you had another set of rows
INSERT INTO #udt VALUES('Name','Hello')
INSERT INTO #udt VALUES('Name','Test')
Now the result set of your dynamic SQL will look like this.
Name PhoneNumber
Hello 01234567890
Having said that PIVOT can be used only with aggregate functions, you will get error if MIN is removed. You might have to think of other options for getting desired results such as using case if needed for multiple fields
SELECT CASE WHEN DatabaseFieldName = 'PhoneNumber' THEN Value END AS PhoneNumber,
CASE WHEN DatabaseFieldName = 'Name' THEN Value END AS Name
FROM #udt
Related
One of my table column stores ~650,000 characters (each value of the column contains entire table). I know its bad design however, Client will not be able to change it.
I am tasked to convert the column into multiple columns.
I chose to use dbo.DelimitedSplit8K function
Unfortunately, it can only handle 8k characters at max.
So I decided to split the column into 81 8k batches using while loop and store the same in a variable table (temp or normal table made no improvement)
DECLARE #tab1 table ( serialnumber int, etext nvarchar(1000))
declare #scriptquan int = (select MAX(len (errortext)/8000) from mytable)
DECLARE #Counter INT
DECLARE #A bigint = 1
DECLARE #B bigint = 8000
SET #Counter=1
WHILE ( #Counter <= #scriptquan + 1)
BEGIN
insert into #tab1 select ItemNumber, Item from dbo.mytable cross apply dbo.DelimitedSplit8K(substring(errortext, #A, #B), CHAR(13)+CHAR(10))
SET #A = #A + 8000
SET #B = #B + 8000
SET #Counter = #Counter + 1
END
This followed by using below code
declare #tab2 table (Item nvarchar(max),itemnumber int, Colseq varchar(10)) -- declare table variable
;with cte as (
select [etext] ,ItemNumber, Item from #tab1 -- insert table name
cross apply dbo.DelimitedSplit8K(etext,' ')) -- insert table columns name that contains text
insert into #tab2 Select Item,itemnumber, 'a'+ cast (ItemNumber as varchar) colseq
from cte -- insert values to table variable
;WITH Tbl(item, colseq) AS(
select item, colseq from #tab2
),
CteRn AS(
SELECT item, colseq,
Rn = ROW_NUMBER() OVER(PARTITION BY colseq ORDER BY colseq)
FROM Tbl
)
SELECT
a1 Time,a2 Number,a3 Type,a4 Remarks
FROM CteRn r
PIVOT(
MAX(item)
FOR colseq IN(a1,a2,a3,a4)
)p
where a3 = 'error'
gives the desired output. However, just the loop takes 15 minutes to complete and overall query completes by 27 minutes. Is there any way I can make it faster? Total row count in my table is 2. So I don't think Index can help.
Client uses Azure SQL Database so I can't choose PowerShell or Python to accomplish this either.
Please let me know if more information is needed. I tried my best to mention everything I could.
I want to be able to grab all of the records in a table into a comma delimited list that I can then use to insert into a table on another database. Due to permission restrictions on the customer's server I cannot access any of the options when right-clicking on the database name, and all of the solutions I've found so far involve having permission to do so (e.g. Tasks > Export Data...)
I have tried using COALESCE to do this, however the problem is that my table could have any number of columns. Columns can be added/deleted at any time through the UI by the users and therefore I cannot hard code the columns in my select statement.
Here is what I have written so far, using a simple CTE statement where there are three columns (RowCode, RowOrd, RowText) and concatenating them into a variable that I print out. I just want to find a way to grab these column names dynamically instead of hard coding them. I'll also need to account for various types of column names by casting them each as varchar in the variable.
DECLARE #listStr VARCHAR(MAX)
;WITH tableData AS
(
SELECT *
FROM tableRows
)
SELECT
#listStr = ISNULL(#listStr + 'select ','select ') + '''' + RowCode + ''',''' + cast(RowOrd as varchar) + ''',''' + RowText + '''' + Char(13)
FROM
tableData
PRINT #listStr
The tableRows table contains the following records
RowCode RowOrd RowText
-----------------------
RowA 1 Row A
RowB 2 Row B
And the variable #listStr is currently printing this, which is correct
select 'RowA','1.00','Row A'
select 'RowB','2.00','Row B'
Thanks in advance!
With a bit of XML you can dynamically gather and "stringify" your values
Declare #tableRows table (RowCode varchar(50), RowOrd int, RowText varchar(50))
Insert Into #tableRows values
('RowA',1,'Row A'),
('RowB',2,'Row B')
Declare #listStr VARCHAR(MAX) = ''
Select #listStr = #listStr + C.String + char(13)
From #tableRows A
Cross Apply (Select XMLData = cast((Select A.* for XML RAW) as xml)) B
Cross Apply (
Select String = 'select '+Stuff((Select ',' +Value
From (
Select Value = ''''+attr.value('.','varchar(max)')+''''
From B.XMLData.nodes('/row') as A(r)
Cross Apply A.r.nodes('./#*') AS B(attr)
) X
For XML Path ('')),1,1,'')
) C
Select #listStr
Returns
select 'RowA','1','Row A'
select 'RowB','2','Row B'
Problem Statement :
when #a has a single word(Ex. 'name1') OR comma separated string (Example 'name1,name2,name3') then the query should return the manager names of employees with name1 and name2 and name3
when #a has an empty string then return the manager names of all the employees in the emp_master table
I have defined a stored procedure where I pass a variable.
This variable can be a comma separated string, a single word or an empty string.
If the string is comma separated then I split that string and get values based on the return table of split statement
else
I get the related value of the non comma separated data using normal subquery
I have tried to achieve this in the following way
Declare #a varchar(50)= ''
select emp.Name from
emp_master emp
where
(LEN(#a)=0 AND emp.Name in
(
SELECT DISTINCT [Name] FROM
[dbo].[Emp_Master] WHERE [EmpId] IN
(
SELECT
DISTINCT [MGR_ID]
FROM [dbo].[Emp_Master]
)
)
)
OR
emp.Name in (Select * from [dbo].[SplitString](#a, ','))
Details for the above sample:
[dbo].[SplitString] - custom written function : returns a table of split values. So
Select * from [dbo].SplitString
will return
SplitTable
----------
name1
name2
name3
and
Select * from [dbo].[SplitString](',','name1')
will return
SplitTable
----------
name1
[dbo].[Emp_Master] contains data for all the employees
[MGR_ID] is the column which has the employeeID of the employee manager
#a is the input variable
The Database is MS SQL 2008
My current solution(the above insane query) solves my purpose but it is very slow, it would be helpful to get an optimized and faster working solution for the problem
Emp_master Table has 400 000 rows, 30 columns
There are 18 000 managers in that table
CREATE NONCLUSTERED INDEX ix ON dbo.Emp_Master ([MGR_ID])
GO
DECLARE #a VARCHAR(50) = ''
DECLARE #t TABLE (val VARCHAR(50) PRIMARY KEY WITH(IGNORE_DUP_KEY=ON))
INSERT INTO #t
SELECT item = t.c.value('.', 'INT')
FROM (
SELECT txml = CAST('<r>' + REPLACE(#a, ',', '</r><r>') + '</r>' AS XML)
) r
CROSS APPLY txml.nodes('/r') t(c)
SELECT /*DISTINCT*/ [Name]
FROM dbo.Emp_Master e1
WHERE (
#a = ''
AND
e1.[EmpId] IN (SELECT DISTINCT MGR_ID FROM dbo.Emp_Master)
)
OR (
#a != ''
AND
e.Name IN (SELECT * FROM #t)
)
OPTION(RECOMPILE)
TRY THIS
CREATE NONCLUSTERED INDEX IX_MGR_ID_Emp_Master ON dbo.Emp_Master ([MGR_ID])
GO
Create Procedure searchname (#a varchar(255))
as
IF (#a = '')
BEGIN
EXEC Searchname1 #a
END
ELSE
BEGIN
EXEC Searchname2 #a
END
GO
Create Procedure Searchname1 (#a varchar(255))
AS
SELECT DISTINCT [Name] FROM
[dbo].[Emp_Master] m1 WHERE
exists
(
SELECT
*
FROM [dbo].[Emp_Master] m2
WHERE
m1.[EmpId]= m2.[MGR_ID]
)
GO
Create Procedure Searchname2 (#a varchar(max))
AS
Select #a = ' SELECT '''+replace( #a,',',''' Union ALL SELECT ''')+' '''
Create table #names (name varchar(255))
insert into #names
EXEC ( #a )
select emp.Name from
emp_master emp
WHERE
emp.Name in( Select name FRom #names)
option (recompile)
IF YOU ARE ALREADY DEALING WITH SQL INJECTION AT APPLICATION LEVEL
THEN
ALTER procedure [dbo].[Searchname2] (#a varchar(max))
AS
select #a = ''''+replace ( #a,',',''',''')+''''
DECLARE #sql NVARCHAR(MAX) = N'
select distinct emp.Name from
emp_master emp
WHERE
emp.Name in( '+#a+')'
EXEC (#sql)
The problem that I need to resolve is data transfer from one table with many dynamic fields into other structured key value table.
The first table comes from a data export from another system, and has the following structure ( it can have any column name and data):
[UserID],[FirstName],[LastName],[Email],[How was your day],[Would you like to receive weekly newsletter],[Confirm that you are 18+] ...
The second table is where I want to put the data, and it has the following structure:
[UserID uniqueidentifier],[QuestionText nvarchar(500)],[Question Answer nvarchar(max)]
I saw many examples showing how to unpivot table, but my problem is that I dont know what columns the Table 1 will have. Can I somehow dynamically unpivot the first table,so no matter what columns it has, it is converted into a key-value structure and import the data into the second table.
I will really appreciate your help with this.
You can't pivot or unpivot in one query without knowing the columns.
What you can do, assuming you have privileges, is query sys.columns to get the field names of your source table then build an unpivot query dynamically.
--Source table
create table MyTable (
id int,
Field1 nvarchar(10),
Field2 nvarchar(10),
Field3 nvarchar(10)
);
insert into MyTable (id, Field1, Field2, Field3) values ( 1, 'aaa', 'bbb', 'ccc' );
insert into MyTable (id, Field1, Field2, Field3) values ( 2, 'eee', 'fff', 'ggg' );
insert into MyTable (id, Field1, Field2, Field3) values ( 3, 'hhh', 'iii', 'jjj' );
--key/value table
create table MyValuesTable (
id int,
[field] sysname,
[value] nvarchar(10)
);
declare #columnString nvarchar(max)
--This recursive CTE examines the source table's columns excluding
--the 'id' column explicitly and builds a string of column names
--like so: '[Field1], [Field2], [Field3]'.
;with columnNames as (
select column_id, name
from sys.columns
where object_id = object_id('MyTable','U')
and name <> 'id'
),
columnString (id, string) as (
select
2, cast('' as nvarchar(max))
union all
select
b.id + 1, b.string + case when b.string = '' then '' else ', ' end + '[' + a.name + ']'
from
columnNames a
join columnString b on b.id = a.column_id
)
select top 1 #columnString = string from columnString order by id desc
--Now I build a query around the column names which unpivots the source and inserts into the key/value table.
declare #sql nvarchar(max)
set #sql = '
insert MyValuestable
select id, field, value
from
(select * from MyTable) b
unpivot
(value for field in (' + #columnString + ')) as unpvt'
--Query's ready to run.
exec (#sql)
select * from MyValuesTable
In case you're getting your source data from a stored procedure, you can use OPENROWSET to get the data into a table, then examine that table's column names. This link shows how to do that part.
https://stackoverflow.com/a/1228165/300242
Final note: If you use a temporary table, remember that you get the column names from tempdb.sys.columns like so:
select column_id, name
from tempdb.sys.columns
where object_id = object_id('tempdb..#MyTable','U')
-- Pivot table with one row and four columns
SELECT 'Values' tValues,
ID,Name,ValueID,Value FROM (
Select ID,Name,ValueID,Value FROM Table WHERE OptionID = 1000000
) AS SourceTable
PIVOT (
COUNT(tValues)
FOR tValues IN ( ID,Attribute,ValueID,Value )
) AS PivotTable;
I'm going off the example at Microsoft.com: http://msdn.microsoft.com/en-us/library/ms177410.aspx
But there are a few things about Pivot i don't really understand, so don't be surprised when you see it in the code above, such as COUNT(tValues), I have no idea what this is for, by judging from the example on microsoft, it seems to be always some sort of numeric value, so i figured i'd try it to see if it would return something, but all it returns is an error. Anyhow, if someone out there can share why this query doesn't work, and possibly explain what the numeric value above the FOR is used for?
The Table containts an x amount of rows, with four columns, so it looks like this:
ID | Name | ValueID | Value
100 | Color | 10000 | Black
101 | Size | 10005 | Large
The output should be like this:
Name_100 | Color | Name_101 | Size |
10000 | Black | 10005 | Large |
Something like this maybe.
This will only work if the name column is unique. If not then you might want to append an id on it.
So first some test data:
CREATE TABLE tblValues
(
ID INT,
Name VARCHAR(100),
ValueID INT,
Value VARCHAR(100)
)
INSERT INTO tblValues
VALUES
(100,'Color',10000,'Black'),
(101,'Size',10005,'Large')
Then you need to get the columns to pivot on:
DECLARE #cols VARCHAR(MAX)
;WITH CTE AS
(
SELECT
'Name_'+CAST(tbl.ID AS VARCHAR(100)) AS Name,
'Name_'+CAST(tbl.ID AS VARCHAR(100)) AS Sort,
tbl.ID
FROM
tblValues AS tbl
UNION ALL
SELECT
tbl.Name,
'Value_'+CAST(tbl.ID AS VARCHAR(100)) AS Sort,
tbl.ID
FROM
tblValues AS tbl
)
SELECT
#cols = COALESCE(#cols + ','+QUOTENAME(Name),
QUOTENAME(Name))
FROM
CTE
ORDER BY
CTE.ID,
CTE.Sort
Then declaring and executing the dynamic sql like this:
DECLARE #query NVARCHAR(4000)=
N'SELECT
*
FROM
(
SELECT
''Name_''+CAST(tbl.ID AS VARCHAR(100)) AS pivotName,
CAST(tbl.ValueID AS VARCHAR(100)) AS name
FROM
tblValues AS tbl
UNION ALL
SELECT
tbl.Name AS pivotName,
tbl.Value AS name
FROM
tblValues AS tbl
) AS p
PIVOT
(
MAX(name)
FOR pivotName IN ('+#cols+')
) AS pvt'
EXECUTE(#query)
Then in my case I will drop the table I have created
DROP TABLE tblValues
Edit
Or in you case it should be something like this:
First the columns:
DECLARE #cols VARCHAR(MAX)
;WITH CTE AS
(
SELECT
'Name_'+CAST(tbl.ID AS VARCHAR(100)) AS Name,
'Name_'+CAST(tbl.ID AS VARCHAR(100)) AS Sort,
tbl.ID
FROM
[Table] AS tbl
WHERE
tbl.OptionID = 1000000
UNION ALL
SELECT
tbl.Name,
'Value_'+CAST(tbl.ID AS VARCHAR(100)) AS Sort,
tbl.ID
FROM
[Table] AS tbl
WHERE
tbl.OptionID = 1000000
)
SELECT
#cols = COALESCE(#cols + ','+QUOTENAME(Name),
QUOTENAME(Name))
FROM
CTE
ORDER BY
CTE.ID,
CTE.Sort
Then the dynamic sql.
DECLARE #query NVARCHAR(4000)=
N'SELECT
*
FROM
(
SELECT
''Name_''+CAST(tbl.ID AS VARCHAR(100)) AS pivotName,
CAST(tbl.ValueID AS VARCHAR(100)) AS name
FROM
[Table] AS tbl
WHERE
tbl.OptionID = 1000000
UNION ALL
SELECT
tbl.Name AS pivotName,
tbl.Value AS name
FROM
[Table] AS tbl
WHERE
tbl.OptionID = 1000000
) AS p
PIVOT
(
MAX(name)
FOR pivotName IN ('+#cols+')
) AS pvt'
EXECUTE(#query)
You do not need to create the table or drop the table. That was just because I did not have your table in my database and that if someone else want's to run the example.
If you want to use Pivot tables with a variable number of columns, then I'd suggest using something along the lines of;
DECLARE #cols VARCHAR(4000)
DECLARE #query VARCHAR(8000)
SELECT #cols = STUFF(( SELECT DISTINCT
'],[' + Name
FROM Table
ORDER BY '],[' + Name
FOR XML PATH('')
), 1, 2, '') + ']'
SET #query =
'SELECT * FROM
(
SELECT col1, col2, col3, whateverColYourInterestedIn, Name, Value
FROM Table
)t
PIVOT (MAX(Value) FOR Name
IN ('+#cols+')) AS pvt'
EXECUTE (#query)
That is probably not quite right, but it should hopefully be a starting point for you.
For more info, check out links such as this or this.