Datalake analytic join - analytics

I have 2 table. I want classified URL who is in table [Activite_Site] I've try the query below, but it doesn't work... Anyone have idea.
Thank you in advance
Table [Categorie]
URL CAT
http//www.site.com/business B2B
http//www.site.com/office B2B
http//www.site.com/job B2B
http//www.site.com/home B2C
Table [Actvite_Site]
URL
http//www.site.com/business/page2/test.html
http//www.site.com/business/page3/pagetest/tot.html
http//www.site.com/office/all/tot.html
http//www.site.com/home/holiday/paris.html
http//www.site.com/home/private/moncompte.html
I would like OUTPUT :
URL_SITE CATEGORIE
http//www.site.com/business/page2/test.html B2B
http//www.site.com/business/page3/pagetest/tot.html B2B
http//www.site.com/office/all/tot.html B2B
http//www.site.com/home/holiday/paris.html B2C
http//www.site.com/home/private/moncompte.html B2C
http//www.site.com/test/pte.html Null
My query :
SELECT A.URL AS URL_SITE
C.CAT AS CATEGORIE
FROM Actvite_Site as A
LEFT Categorie as C ON C.URL==A.URL.PadLeft(C.URL.Lenght)

RE error E_CSC_USER_JOINCOLUMNSEXPECTEDONEACHSIDEOFCONDITION, U-SQL does not currently support derived columns in join conditions.
One way to achieve this might be to find the matched URLs, then the unmatched and UNION them together.
#category = SELECT *
FROM (
VALUES
( "http//www.site.com/business", "B2B" ),
( "http//www.site.com/office", "B2B" ),
( "http//www.site.com/job", "B2B" ),
( "http//www.site.com/home", "B2C" )
) AS x(url, cat);
#siteActivity = SELECT *
FROM (
VALUES
( "http//www.site.com/business/page2/test.html" ),
( "http//www.site.com/business/page3/pagetest/tot.html" ),
( "http//www.site.com/office/all/tot.html" ),
( "http//www.site.com/home/holiday/paris.html" ),
( "http//www.site.com/home/private/moncompte.html" ),
( "http//www.site.com/test/pte.html" )
) AS x(url);
// Find matched URLs
#working =
SELECT sa.url,
c.cat
FROM #siteActivity AS sa
CROSS JOIN
#category AS c
WHERE sa.url.Substring(0, c.url.Length) == c.url;
// Combine the matched and unmatched URLs
#output =
SELECT url,
cat
FROM #working
UNION ALL
SELECT url,
(string) null AS cat
FROM #siteActivity AS sa
ANTISEMIJOIN
#working AS w
ON sa.url == w.url;
OUTPUT #output TO "/output/output.csv"
USING Outputters.Csv(quoting:false);
I am wondering if there is a more efficient way though.

Related

How to set permission for multiple users in SSRS Report using SQL Query?

I am working on a SQL Server Reporting Services (SSRS) report project in which I have to give access to 150 different users who have different roles. Is there any way to provide user access via SQL query which simplifies my work?
I would setup an Active Directory group per role then associate the roles to the report folders. Then it's a matter of managing the AD groups' membership.
In SSRS, new roles and adjustments to existing roles must be performed in SQL Server Management studio, SSMS. After opening up SSMS, click on "Connect" and select "Reporting Services…"
Enter your Server Name and login information and then click Connect.
After connecting to the Report Server, open the Security Folder right click on "Roles" and click on "New Role…"
I created 2 new roles ("Browser Group", "Functional Owner") with the same permission as "Browser".
Then on the report manager click on the down arrow for a folder and select "Security"
Then click "New Role Assignment"
Then enter the Active Directory group or an email address and check the new role you created.
You can then query the server for the permissions and show these in a report.
;WITH
catalog_type_description
AS
(
SELECT tbl.* FROM (VALUES
( 1, 'Folder')
, ( 2, 'Report')
, ( 3, 'Resource')
, ( 4, 'Linked Report')
, ( 5, 'Data Source')
, ( 6, 'Report Model')
, ( 8, 'Shared Dataset')
, ( 9, 'Report Part')
) tbl ([TypeID], [TypeDescription])
WHERE
TypeID = 1
)
,
nonreport_folders
AS
(
SELECT tbl.* FROM (VALUES
( 'Images')
, ( 'SharedDataSets')
, ( 'Data Sources')
, ( '')
) tbl ([FolderName])
)
,
reporting_role_names -- added roles to the report server
AS
(
SELECT tbl.* FROM (VALUES
( 'Browser Group')
, ( 'Functional Owner')
) tbl ([RoleName])
)
,
user_list
AS
(
SELECT
usr.UserID
, usr.UserName
, UserNameFormat =
CASE
WHEN CHARINDEX('\', usr.UserName) > 0 THEN UPPER(SUBSTRING(usr.UserName ,CHARINDEX('\', usr.UserName) + 1, LEN(usr.UserName)))
ELSE usr.UserName
END
FROM
dbo.Users AS usr
)
,
reporting_roles
AS
(
SELECT
cat.Name
, rol.RoleName
, usr.UserNameFormat
, ReportingRoleName = rpt.RoleName
FROM
dbo.[Catalog] AS cat
INNER JOIN catalog_type_description AS tpd ON cat.[Type] = tpd.TypeID
LEFT JOIN dbo.PolicyUserRole AS urol ON urol.PolicyID = cat.PolicyID
LEFT JOIN dbo.Roles AS rol ON urol.RoleID = rol.RoleID
LEFT JOIN reporting_role_names AS rpt ON rpt.RoleName = rol.RoleName
LEFT JOIN dbo.Policies AS pol ON urol.PolicyID = pol.PolicyID
LEFT JOIN user_list AS usr ON urol.UserID = usr.UserID
LEFT JOIN nonreport_folders AS nrf ON nrf.FolderName = cat.Name
WHERE
1=1
AND nrf.FolderName IS NULL
)
SELECT DISTINCT
FolderName = rpt.Name
, rpt.RoleName
, UserNameFormat = STUFF((SELECT '; ' + rol.UserNameFormat FROM reporting_roles rol WHERE rol.RoleName = rpt.RoleName AND rol.Name = rpt.Name FOR XML PATH('')),1,1,'')
, ReportingRoleName
FROM
reporting_roles AS rpt
References:
https://social.technet.microsoft.com/wiki/contents/articles/40150.ssrs-how-to-setup-the-folder-security-roles-in-sql-server-reporting-services.aspx
https://code.msdn.microsoft.com/SQL-Server-Reporting-SSRS-50c4d06b
http://bhushan.extreme-advice.com/user-roles-and-permissions-in-ssrs/
https://www.mssqltips.com/sqlservertip/2793/sql-server-reporting-services-2012-permissions/

How to filter JSON array in Postgres

I currently have this LEFT JOIN which is part of a bigger select
LEFT JOIN (
SELECT
tags_components.component_id,
array_to_json(array_agg(tags.*)) as tags
FROM tags_components
LEFT JOIN tags ON tags.id = tags_components.tag_id AND tags_components.component_name = 'contact'
GROUP BY tags_components.component_id
) AS tags ON tags.component_id = contact.id
Which works as expected if component has all tags assigned. However tags array is always of size COUNT(tags.*) so for component without any tags is filled with null. Is there a way how filter those nulls out? I tried different things like using json_strip_nulls or having FILTER on the array but I didn't achieve the right result (JSON array containing only non-nulls)
If I understood everything correctly then the issue you face is in the line:
...
array_to_json(array_agg(tags.*)) as tags
...
Maybe you used FILTER in a wrong way, but this does work to eliminta NULL results like:
SELECT array_to_json(
-- FILTER is applied to this specific 'array_agg'
array_agg( t.* ) FILTER ( WHERE t.tag IS NOT NULL )
)
FROM ( VALUES
( 'a1' ),
( 'b1' ),
( null ),
( 'c1' ),
( null ),
( 'd1' )
) t( tag );
-- Resolves to:
array_to_json
-------------------------------------------------------
[{"tag":"a1"},{"tag":"b1"},{"tag":"c1"},{"tag":"d1"}]
(1 row)
Alternatively you can use jsonb_agg (read more at Postgres Aggregate Functions) instead of array_to_json + array_agg to provide same result like:
SELECT jsonb_agg( t.* ) FILTER ( WHERE t.tag IS NOT NULL )
FROM ( VALUES
( 'a1' ),
( 'b1' ),
( null ),
( 'c1' ),
( null ),
( 'd1' )
) t( tag );
array_remove function is going to be your best bet now:
array_to_json(array_remove(array_agg(tags.*), null)) as tags

SQL Server Query for required result

I am using SQL Server with my application.
The Table data is as following :
And I want result in following format:
I have tried with split function but its not working properly.
Is it possible to get such a result.
Please suggest.
Thank you.
Try this. I did not manage to get a single Not Req, it is like this ("Not Req/Not Req").
drop table if exists dbo.TableB;
create table dbo.TableB (
OldSPC varchar(100)
, old_freq varchar(100)
, NewSPC varchar(100)
, new_freq varchar(100)
);
insert into dbo.TableB(OldSPC, old_freq, NewSPC, new_freq)
values ('ADH,BAP', '7,7', 'ADH,BAP', '7,7')
, ('Not Req', 'Not Req', 'ADH,BAP', '7,7')
, ('BAP,EXT,ADL', '35,7,42', 'BAP,EXT,BAP,ADL', '21,7,35,42');
select
tt1.OldSPCOldFreq
, tt2.NewSPCNewFreq
from (
select
t.OldSPC, t.old_freq, t.NewSPC, t.new_freq
, STRING_AGG(t1.value + '/' + t2.value, ',') OldSPCOldFreq
from dbo.TableB t
cross apply (
select
ROW_NUMBER () over (order by t.OldSPC) as Rbr
, ss.value
from string_split (t.OldSPC, ',') ss
) t1
cross apply (
select
ROW_NUMBER () over (order by t.old_freq) as Rbr
, ss.value
from string_split (t.old_freq, ',') ss
) t2
where t1.Rbr = t2.Rbr
group by t.OldSPC, t.old_freq, t.NewSPC, t.new_freq
) tt1
inner join (
select
t.OldSPC, t.old_freq, t.NewSPC, t.new_freq
, STRING_AGG(t3.value + '/' + t4.value, ',') NewSPCNewFreq
from dbo.TableB t
cross apply (
select
ROW_NUMBER () over (order by t.NewSPC) as Rbr
, ss.value
from string_split (t.NewSPC, ',') ss
) t3
cross apply (
select
ROW_NUMBER () over (order by t.new_freq) as Rbr
, ss.value
from string_split (t.new_freq, ',') ss
) t4
where t3.Rbr = t4.Rbr
group by t.OldSPC, t.old_freq, t.NewSPC, t.new_freq
) tt2 on tt1.OldSPC = tt2.OldSPC
and tt1.old_freq = tt2.old_freq
and tt1.NewSPC = tt2.NewSPC
and tt1.new_freq = tt2.new_freq
As mentioned in comments, it might be easier for you to do it on front end, but it could be done in SQL Server as well.
Partial Rextester Demo
I didn't replicate your whole scenario but got it for 2 columns. To do it first of all, you need a unique identifier for each row. I am using a sequence number (1,2,3...).
Now refer to this answer, which uses recursive subquery to split csv to rows. Then I used XML PATH to change columns back to csv.
This is the query which is doing it for OLD SPC and OLD FREQ.
;with tmp(SEQ,OldSPCItem,OldSPC,OLD_FREQ_item,OLD_FREQ) as (
select SEQ, LEFT(OldSPC, CHARINDEX(',',OldSPC+',')-1),
STUFF(OldSPC, 1, CHARINDEX(',',OldSPC+','), ''),
LEFT(OLD_FREQ, CHARINDEX(',',OLD_FREQ+',')-1),
STUFF(OLD_FREQ, 1, CHARINDEX(',',OLD_FREQ+','), '')
from table1
union all
select SEQ, LEFT(OldSPC, CHARINDEX(',',OldSPC+',')-1),
STUFF(OldSPC, 1, CHARINDEX(',',OldSPC+','), ''),
LEFT(OLD_FREQ, CHARINDEX(',',OLD_FREQ+',')-1),
STUFF(OLD_FREQ, 1, CHARINDEX(',',OLD_FREQ+','), '')
from tmp
where OldSPC > ''
)
select seq,STUFF( (SELECT ',' + CONCAT(OldSPCItem,'/',OLD_FREQ_item) FROM TMP I
WHERE I.seq = O.seq FOR XML PATH('')),1,1,'') OLD_SPC_OLD_FREQ
from tmp O
GROUP BY seq
;
It will give you this output
+-----+------------------+
| seq | OLD_SPC_OLD_FREQ |
+-----+------------------+
| 1 | ADH/7,BAP/9 |
| 2 | NOT REQ/NOT REQ |
+-----+------------------+
What do you have to do now
- Find a way to generate a sequence number to uniquely identify each row. If you can use any column, use that instead of SEQ.
Similarly add logic for NEW_SPC and NEW_FREQ. (just copy paste LEFT and STUFF like in OLD_FREQ and change it for NEW_SPC and NEW_FREQ.
Replace multiple NOT REQ/ with '', so you will get only one NOT REQ. You can do it with replace function.
If you face any issue/error while doing so, add it to the Rexterster Demo and share the URL, we will check that.

Update latest record with data from an older record

I have a table with newspaper subscribers:
Subscribers:
==============
ID INT,
Status,
Address,
IndexAddress,
StartDate,
EndDate,
SubscriberID,
PaperID
IndexAddress is a reference to my internal Address table where I keep "correct" addresses (you woulnd't believe how many people don't know where they live). Address is the address supplied by the customer.
Each time a subscriber ends his subscription I save the data and when he renews his subscription I want to re-fetch the old IndexAddress from the old subscrption line in my table.
The data in the database can look like this:
1 1 MyLocalAddress 13455 20160101 20160501 100 5
8 1 MyLocalAddress 13455 20160820 20161201 100 5
14 1 MyLocalAddress 13455 20161228 20170107 100 5
18 0 MyLocalAddress NULL 20170109 NULL 100 5
So ID 1, has status 1, a local address, pointing to address 13455 in my internal system, started 160101 and ended 160501 with customer number 100 and paper number 5.
The last row, ID 18 has just arrived in the database, I want to make sure I automatically find the IndexAddress number so I don't have to match it by hand, but I also want to make absolutlely sure that I fetch the information from the row with ID 14 since the older information in the database MIGHT be wrong (in this case it isn't but it might).
Here is my SQL to fix this:
UPDATE s SET
Status = s2.Status,
IndexAddress = s2.IndexAddress
FROM dbo.Subscribers s
JOIN dbo.Subscribers s2 ON s2.SubscriberID = s.SubscriberID
WHERE 1 = 1
AND s.Status <> s2.Status
AND s2.Status = 1
AND s2.ID IN
(
SELECT
MAX(s3.ID)
FROM dbo.Subscribers s3
WHERE 1 = 1
AND s3.SubscriberID = s.SubscriberID
AND s3.PaperID = s.PaperID
AND s3.Status = 1
AND s3.ID <> s.ID
)
-- Make sure it's the same customer. Customer number is checked in
-- join above.
AND s.PaperID = s2.PaperID
AND s.Address = s2.Address
This works, but I wanted to know if the subquery approach was the best solution or is there a better approach?
I would like to deepen my understand of MS SQL and thus my questions.
I think your query is way over complicated:
with toupdate as (
select s.*,
lag(address) over (partition by subscriberid, paperid order by id) as prev_address,
lag(status) over (partition by subscriberid, paperid order by id) as prev_status
from dbo.Subscribers s
)
update toupdate
set address = prev_address,
status = prev_status
where address is null;
This is not the answer you're looking for but it's not really suitable for a comment. I don't really agree with the design of the tables as you have redundant data. You shouldn't have to repeat data for address and indexaddress in Subscribers or do updates like you are doing.
I would suggest a design something like the below that would avoid you having to do updates like the one you are doing. The below code is re-runnable, so you can run and modify if required to test it.
-- user level information with 1 row per user - address should be linked here
CREATE TABLE #user
(
id INT ,
name NVARCHAR(20) ,
indexAddress INT
)
-- all subscriptions - with calculated status compared to current date
CREATE TABLE #subscription
(
id INT ,
startDate DATETIME ,
endDate DATETIME ,
staus AS CASE WHEN endDate < GETDATE() THEN 1
ELSE 0
END
)
-- table to link users with their subscriptions
CREATE TABLE #userSubscription
(
userId INT ,
subscriptionId INT
)
INSERT INTO #user
( id, name, indexAddress )
VALUES ( 1, N'bob', 13455 ),
( 2, 'dave', 55332 )
INSERT INTO #subscription
( id, startDate, endDate )
VALUES ( 1, '20160101', '20160201' ),
( 8, '20160820', '20161201' ),
( 14, '20161228', '20170107' ),
( 18, '20170109', NULL ),
( 55, '20170101', NULL );
INSERT INTO #userSubscription
( userId, subscriptionId )
VALUES ( 1, 1 ) ,
( 1, 8 ) ,
( 1, 14 ) ,
( 1, 18 ) ,
( 2, 55 );
-- show active users
SELECT u.name ,
u.indexAddress ,
us.userId ,
us.subscriptionId ,
s.startDate ,
s.endDate ,
s.staus
FROM #user u
INNER JOIN #userSubscription us ON u.id = us.userId
INNER JOIN #subscription s ON s.id = us.subscriptionId
WHERE s.staus = 0 -- active
-- show inactive users
SELECT u.name ,
u.indexAddress ,
us.userId ,
us.subscriptionId ,
s.startDate ,
s.endDate ,
s.staus
FROM #user u
INNER JOIN #userSubscription us ON u.id = us.userId
INNER JOIN #subscription s ON s.id = us.subscriptionId
WHERE s.staus = 1 -- inactive
-- tidy up
DROP TABLE #subscription
DROP TABLE #user
DROP TABLE #userSubscription

SQL Server - IN clause with multiple fields

Is it possible to include in a IN clause multiple fields? Something like the following:
select * from user
where code, userType in ( select code, userType from userType )
I'm using ms sql server 2008
I know this can be achieved with joins and exists, I just wanted to know if it could just be done with the IN clause.
Not the way you have posted. You can only return a single field or type for IN to work.
From MSDN (IN):
test_expression [ NOT ] IN
( subquery | expression [ ,...n ]
)
subquery - Is a subquery that has a result set of one column.
This column must have the same data type as test_expression.
expression[ ,... n ] - Is a list of expressions to test for a match.
All expressions must be of the same type as
test_expression.
Instead of IN, you could use a JOIN using the two fields:
SELECT U.*
FROM user U
INNER JOIN userType UT
ON U.code = UT.code
AND U.userType = UT.userType
You could use a form like this:
select * from user u
where exists (select 1 from userType ut
where u.code = ut.code
and u.userType = ut.userType)
Only with something horrific, like
select * from user
where (code + userType) in ( select code + userType from userType )
Then you have to manage nulls and concatenating numbers rather than adding them, and casting, and a code of 12 and a usertype of 3 vs a code of 1 and a usertype of 23, and...
..which means you start heading into perhaps something like:
--if your SQLS supports CONCAT
select * from user
where CONCAT(code, CHAR(9), userType) in ( select CONCAT(code, CHAR(9), userType) from ... )
--if no concat
select * from user
where COALESCE(code, 'no code') + CHAR(9) + userType in (
select COALESCE(code, 'no code') + CHAR(9) + userType from ...
)
CONCAT will do a string concatenation of most things, and won't zip the whole output to NULL if one element is NULL. If you don't have CONCAT then you'll string concat using + but anything that might be null will need a COALESCE/ISNULL around it.. And in either case you'll need something like CHAR(9) (a tab) between the fields to prevent them mixing.. The thing between the fields should be southing that is not naturally present in the data..
Tis a shame SQLS doesn't support this, that Oracle does:
where (code, userType) in ( select code, userType from userType )
but it's probably not worth switching DB for; I'd use EXISTS or a JOIN to achieve a multi column filter
So there ya go: a solution that doesn't use joins or exists.. and a bunch of reasons why you shouldn't use it ;)
How about this instead:
SELECT user.* FROM user JOIN userType on user.code = userType.code AND user.userType = userType.userType
You can either use joins
SELECT * FROM user U
INNER JOIN userType UT on U.code = UT.code
AND U.userType = UT.userType
I had to do something very similar but EXISTS didn't work in my situation. Here is what worked for me:
UPDATE tempFinalTbl
SET BillStatus = 'Non-Compliant'
WHERE ENTCustomerNo IN ( SELECT DISTINCT CustNmbr
FROM tempDetailTbl dtl
WHERE dtl.[Billing Status] = 'NEEDS FURTHER REVIEW'
AND dtl.CustNmbr = ENTCustomerNo
AND dtl.[Service] = [Service])
AND [Service] IN ( SELECT DISTINCT [Service]
FROM tempDetailTbl dtl
WHERE dtl.[Billing Status] = 'NEEDS FURTHER REVIEW'
AND dtl.CustNmbr = ENTCustomerNo
AND dtl.[Service] = [Service])
EDIT: Now that I look, this is very close to #v1v3kn's answer
I don't think that query is quite portable,it would be safer to use something like
select * from user
where code in ( select code from userType ) and userType in (select userType from userType)
select * from user
where (code, userType) in ( select code, userType from userType );

Resources