I have a products table with two attribute column, and a json column. I'd like to be able to delimit the json column and insert extra rows retaining the attributes. Sample data looks like:
ID Name Attributes
1 Nikon {"4e7a":["jpg","bmp","nef"],"604e":["en"]}
2 Canon {"4e7a":["jpg","bmp"],"604e":["en","jp","de"]}
3 Olympus {"902c":["yes"], "4e7a":["jpg","bmp"]}
I understand OPENJSON can convert JSON objects into rows, and key values into cells but how do I apply it on a single column that contains JSON data?
My goal is to have an output like:
ID Name key value
1 Nikon 902c NULL
1 Nikon 4e7a ["jpg","bmp","nef"]
1 Nikon 604e ["en"]
2 Canon 902c NULL
2 Canon 4e7a ["jpg","bmp"]
2 Canon 604e ["en","jp","de"]
3 Olympus 902c ["yes"]
3 Olympus 4e7a ["jpg","bmp"]
3 Olympus 604e NULL
Is there a way I can query this products table like? Or is there a way to reproduce my goal data set?
SELECT
ID,
Name,
OPENJSON(Attributes)
FROM products
Thanks!
Here is something that will at least start you in the right direction.
SELECT P.ID, P.[Name], AttsData.[key], AttsData.[Value]
FROM products P CROSS APPLY OPENJSON (P.Attributes) AS AttsData
The one thing that has me stuck a bit right now is the missing values (value is null in result)...
I was thinking of maybe doing some sort of outer/full join back to this, but even that is giving me headaches. Are you certain you need that? Or, could you do an existence check with the output from the SQL above?
I am going to keep at this. If I find a solution that matches your output exactly, I will add to this answer.
Until then... good luck!
You can get the rows with NULL value fields by creating a list of possible keys and using CROSS APPLY to associate each key to each row from the original dataset, and then left-joining in the parsed JSON.
Here's a working example you should be able to execute as-is:
-- Throw together a quick and dirty CTE containing your example data
WITH OriginalValues AS (
SELECT *
FROM (
VALUES ( 1, 'Nikon', '{"4e7a":["jpg","bmp","nef"],"604e":["en"]}' ),
( 2, 'Canon', '{"4e7a":["jpg","bmp"],"604e":["en","jp","de"]}' ),
( 3, 'Olympus', '{"902c":["yes"], "4e7a":["jpg","bmp"]}' )
) AS T ( ID, Name, Attributes )
),
-- Build a separate dataset that includes all possible 'key' values from the JSON.
PossibleKeys AS (
SELECT DISTINCT A.[key]
FROM OriginalValues CROSS APPLY OPENJSON( OriginalValues.Attributes ) AS A
),
-- Get the existing keys and values from the JSON, associated with the record ID
ValuesWithKeys AS (
SELECT OriginalValues.ID, Atts.[key], Atts.Value
FROM OriginalValues CROSS APPLY OPENJSON( OriginalValues.Attributes ) AS Atts
)
-- Join each possible 'key' value with every record in the original dataset, and
-- then left join the parsed JSON values for each ID and key
SELECT OriginalValues.ID, OriginalValues.Name, KeyList.[key], ValuesWithKeys.Value
FROM OriginalValues
CROSS APPLY PossibleKeys AS KeyList
LEFT JOIN ValuesWithKeys
ON OriginalValues.ID = ValuesWithKeys.ID
AND KeyList.[key] = ValuesWithKeys.[key]
ORDER BY ID, [key];
If you need to include some pre-determined key values where some of them might not exist in ANY of the JSON values stored in Attributes, you could construct a CTE (like I did to emulate your original dataset) or a temp table to provide those values instead of doing the DISTINCT selection in the PossibleKeys CTE above. If you already know what your possible key values are without having to query them out of the JSON, that would most likely be a less costly approach.
Related
I'm new to Pivots so I'm not exactly sure how I would need to do this.
I have the following data:
I need to pivot/unpivot the above data.
I need the following Columns:
GLLink
AccType
Budget
Budget Value
I've tried developing the code, but I'm afraid I am doing it completely wrong:
select
*
from
(
select
*
from Budgets
) x
unpivot
(
GLLink for Budget in ([AccType])
) p
order by GLLink, Budget
The results I would require should look like this:
Assuming the data you showed us are in a table named 'yourtable', here is the cross apply way:
select t.GLLink,t.Acctype,u.*
from yourtable t
cross apply
(
values
('Budget01',Budget01),
('Budget02',Budget02),
('Budget03',Budget03),
('Budget04',Budget04),
('Budget05',Budget05),
('Budget06',Budget06),
('Budget07',Budget07)
)u(Budget,[Budget Value])
The way it works: cross apply "sees" one row each time. For this row, it uses the values() operator to create a table, so we make the pairs as you need them. In the end, we give a name to this table (u for unpivoted), and also name the columns.
Converting columns to rows is unpivoting (what you need), pivoting is the reverse.
I am trying to figure out a way to check if their is repeated values in rows that are shared.
Example:
HMOID Name Addon10 Addon15 Addon20
RFFF Blah img path1 img path2 img path1
For my example, I would like to check if any of the addons for RFFF have any repeated value. In my example above, 'RFFF' has two images that are the same in Addon10 and Addon20 (The images have a path. so currently, they look like
http://oc2-reatest.regalmed.local/ocupgrade52/images/NDL_SCAN_SR.PNG).
I would like to be able to do this for multiple rows. I thought the following would give me an idea how to begin:
select * from HlthPlan
Group By HMO1A, HMONM
Having COUNT(*) > 1
However, it throughs the following error:
Msg 8120, Level 16, State 1, Line 1
Column 'HlthPlan.HMOID' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.*
I am fairly new to SQL and any suggestions would be appreciated.
Don't include * for your select query. Only include the columns that you are using in GROUP BY
SELECT HMO1A, HMONM, COUNT(*) from HlthPlan
GROUP BY HMO1A, HMONM
HAVING COUNT(*) > 1;
With only three columns to check, assuming non-null values across a single row:
select * from HlthPlan
where Addon10 in (Addon15, Addon20) or Addon15 = Addon20
You can also use cross apply to pivot the values for grouping:
select HMOID, addon
from HlthPlan cross apply (
select addon
from (values (Addon01), (Addon02), (Addon03), ... (Addon20)) as pvt(addon)
) a
where addon is not null
group by HMOID, addon
having count(*) > 1;
http://rextester.com/QWIW87618
You'll get multiple rows for each HMOID where the are different groups of columns having the same value. By the way, reporting on the names of specific columns involved would add another degree of difficulty to the query.
One way you can check for this is using UNPIVOT to compare your results:
create table #hmo (hmoid varchar(6), name varchar(25), Addon10 varchar(25),
Addon15 varchar(25), addon20 varchar(25));
insert into #hmo
values ('RFFF', 'Blah','img path1', 'img path2', 'img path1');
select hmoid, name, addval, addcount = count(adds)
FROM #hmo
UNPIVOT
(
addval FOR adds IN
(addon10, addon15, addon20)
) as unpvt
group by hmoid, name, addval having count(*) > 1
Will give results:
hmoid name addval addcount
RFFF Blah img path1 2
This way you can check against every row in the table and will see any row that has any two or more columns with the same value.
This does have the potential to get tedious if you have a lot of columns; the easiest way to correct for that is to build your query dynamically using info from sys.tables and sys.columns.
I am trying to build a temp table with grouped data from multiple tables (in an SP), I am successful in building the data set however I have a requirement that each grouped row have a unique id. I know there are ways to generate unique ids for each row, However the problem I have is that I need the id for a given row to be the same on each run regardless of the number of rows returned.
Example:
1st run:
ID Column A Column B
1 apple 15
2 orange 10
3 grape 11
2nd run:
ID Column A Column B
3 grape 11
The reason I want this is because i am sending this data up to SOLR and when I do a delta I need to have the ID back for the same row as its trying to re-index
Any way I can do this?
Not sure if this will help, not entirely confident of your wider picture, but ...
As your new data is assembled, log each [column a] value in a table of your own.
Give that table an IDENTITY column to do the numbering for you.
Now you can join any new data sets to your lookup table and you'll have a persistent number for each column A.
You just need to ensure that each time you query new data, you add new values to the lookup table.
create table dbo.myRef(
idx int identity(1,1)
,[A] nvarchar(100)
)
General draft as below ...
--- just simulating some input data here
with cte as (
select 'apple' as [A], 15 as [B]
UNION
select 'orange' as [A], 10 as [B]
UNION
select 'banana' as [A], 4 as [B]
)
select * into #temp from cte;
-- Put any new values into the lookup table
-- and they will be assigned a new index number by the identity column
insert into dbo.myRef([A])
select distinct [A]
from #temp where [A] not in (select [A] from dbo.myRef)
-- now pull your original data for output, joining to the lookup table to get a ref number.
select T.*,R.idx
from #temp T
inner join
oer.myRef R
on T.[A] = R.[A]
Sorry for the late reply, i was stuck with something else, however i solved my own issue.
I built 2 temp tables one with all the data from the various tables (#master) and another temp table (#final) to house all the grouped data with an empty column for ID
Next i did a concat(column1, '-',column2,'-', column3) on 3 columns from the #master and updated the #final table based on the type
this helped me to get the same concat ids on each run
In the database on which I am attempting to create a FullText Search I need to construct a table with its column names coming from one column in a previous table. In my current implementation attempt the FullText indexing is completed on the first table Data and the search for the phrase is done there, then the second table with the search results is made.
The schema for the database is
**Players**
Id
PlayerName
Blacklisted
...
**Details**
Id
Name -> FirstName, LastName, Team, Substitute, ...
...
**Data**
Id
DetailId
PlayerId
Content
DetailId in the table Data relates to Id in Details, and PlayerId relates to Id in Players. If there are 1k rows in Players and 20 rows in Details, then there are 20k rows in Data.
WITH RankedPlayers AS
(
SELECT PlayerID, SUM(KT.[RANK]) AS Rnk
FROM Data c
INNER JOIN FREETEXTTABLE(dbo.Data, Content, '"Some phrase like team name and player name"')
AS KT ON c. DataID = KT.[KEY]
GROUP BY c.PlayerID
)
…
Then a table is made by selecting the rows in one column. Similar to a pivot.
…
SELECT rc.Rnk,
c.PlayerID,
PlayerName,
TeamID,
…
(SELECT Content FROM dbo.Data data WHERE DetailID = 1 AND data.PlayerID = c.PlayerID) AS [TeamName],
…
FROM dbo.Players c
JOIN RankedPlayers rc ON c. PlayerID = rc. PlayerID
ORDER BY rc.Rnk DESC
I can return a ranked table with this implementation, the aim however is to be able to produce results from weighted columns, so say the column Playername contributes to the rank more than say TeamName.
I have tried making a schema bound view with a pivot, but then I cannot index it because of the pivot. I have tried making a view of that view, but it seems the metadata is inherited, plus that feels like a clunky method.
I then tried to do it as a straight query using sub queries in the select statement, but cannot due to indexing not liking sub queries.
I then tried to join multiple times, again the index on the view doesn't like self-referencing joins.
How to do this?
I have come across this article http://developmentnow.com/2006/08/07/weighted-columns-in-sql-server-2005-full-text-search/ , and other articles here on weighted columns, however nothing as far as I can find addresses weighting columns when the columns were initially row data.
A simple solution that works really well. Put weight on the rows containing the required IDs in another table, left join that table to the table to which the full text search had been applied, and multiply the rank by the weight. Continue as previously implemented.
In code that comes out as
DECLARE #Weight TABLE
(
DetailID INT,
[Weight] FLOAT
);
INSERT INTO #Weight VALUES
(1, 0.80),
(2, 0.80),
(3, 0.50);
WITH RankedPlayers AS
(
SELECT PlayerID, SUM(KT.[RANK] * ISNULL(cw.[Weight], 0.10)) AS Rnk
FROM Data c
INNER JOIN FREETEXTTABLE(dbo.Data, Content, 'Karl Kognition C404') AS KT ON c.DataID = KT.[KEY]
LEFT JOIN #Weight cw ON c.DetailID = cw.DetailID
GROUP BY c.PlayerID
)
SELECT rc.Rnk,
...
I'm using a temporary table here for evidence of concept. I am considering adding a column Weights to the table Details to avoid an unnecessary table and left join.
i had a table depends on more than one tables and i get this final
ScrrenShoot have a look in picture
i need to choose from values if firstdate duplicated in specific criteria
for ex . i need one row for 18.2.2016 / max value ( get the greater one ) / min value (get the less one )
You need to provide us with better information, but here is what I think you're looking for.
You need a separate query for each min/max value you want to find. Where you see "MyTable" you need to replace it with the object name shown in the screenshot.
Query 1 "Max"
SELECT MyTable.FirstOfDate, Max(MyTable.MaxValue) AS MaxOfMaxValue
FROM MyTable
GROUP BY MyTable.FirstOfDate;
Query 2 "Min"
SELECT MyTable.FirstOfDate, Min(MyTable.MinValue) AS MinOfMinValue
FROM MyTable
GROUP BY MyTable.FirstOfDate;
Query 3 "Merge"
SELECT DISTINCT MyTable.FirstOfDate, Max.MaxOfMaxValue, Min.MinOfMinValue
FROM (MyTable
INNER JOIN [Max] ON MyTable.FirstOfDate = Max.FirstOfDate)
INNER JOIN [Min] ON MyTable.FirstOfDate = Min.FirstOfDate
GROUP BY MyTable.FirstOfDate, Max.MaxOfMaxValue, Min.MinOfMinValue;