In my query I have to join tables from db that is not under my control. It is driving me mad as sometimes this db is not accessible (please don't ask me why) and this breaks my query. Fields I'm joining are not fundamental for my operations and I want my app to work normally even if these fields are not accessible at a time.
Here's the data structure that I do not own:
[DBOutOfControl].[dbo].[Table1]:
[Field1]
[Field2]
[DBOutOfControl].[dbo].[Table2]:
[Field1]
[Field2]
[Field3]
And here is my table:
[DBInMyControl].[dbo].[Table3]:
[Field1]
My original query looks something like that:
SELECT [Table3].[MyID],
[ForeignDataQry].[A],
[ForeignDataQry].[B]
FROM [DBInMyControl].[dbo].[Table3]
LEFT JOIN
(SELECT [Table1].[Field1] AS [MyID],
[Table1].[Field2] AS [A],
[SubQry].[Field2] AS [B]
FROM [DBOutOfControl].[dbo].[Table1]
LEFT JOIN
(SELECT [Table2].[Field1],
[Table2].[Field2]
FROM [DBOutOfControl].[dbo].[Table2]
WHERE [Table2].[Field3] = 'Where') AS [SubQry] ON [Table1].[Field1] = [SubQry].[Field1]) AS [ForeignDataQry] ON [Table3].[MyID]=[ForeignDataQry].[MyID]
How can i bullet-proof this query so when [ForeignDataQry] generates an error the result would be:
[MyID] [A] [B]
1 NULL NULL
Otherwise
[MyID] [A] [B]
1 Va1 Val2
Is there something that could be done server side?
Just specify the expected result of COUNT, the three names, and you can check tables beforehand. A minor rewrite can allow this to check for objects other than tables, utilize EXISTS if desired, skip or add more checks, etc.:
IF 0 = ( -- Specify how many records you expect to come.
SELECT COUNT(C.[name]) AS [COUNT]
FROM sys.objects AS O
LEFT JOIN sys.schemas AS S ON S.schema_id = O.schema_id
LEFT JOIN sys.columns AS C ON C.object_id = O.object_id
WHERE O.[name] = 'tablename'
AND S.[name] = 'schemaname'
AND C.[name] = 'columnname'
)
SELECT 1 AS A -- Do some code.
ELSE
SELECT 2 AS B -- Do some other code.
I'd wrap the problematic query in dynamic code in order to be able to catch the compilation error (that we cannot catch in the same scope) like this:
begin try
declare #sql varchar(4000) =
'SELECT [Table3].[MyID],
[ForeignDataQry].[A],
[ForeignDataQry].[B]
FROM [DBInMyControl].[dbo].[Table3]
LEFT JOIN
(SELECT [Table1].[Field1] AS [MyID],
[Table1].[Field2] AS [A],
[SubQry].[Field2] AS [B]
FROM [DBOutOfControl].[dbo].[Table1]
LEFT JOIN
(SELECT [Table2].[Field1],
[Table2].[Field2]
FROM [DBOutOfControl].[dbo].[Table2]
WHERE [Table2].[Field3] = ''Where'') AS [SubQry] ON [Table1].[Field1] = [SubQry].[Field1]) AS [ForeignDataQry] ON [Table3].[MyID]=[ForeignDataQry].[MyID]'
exec(#sql)
end try
begin catch
SELECT [Table3].[MyID],
cast(null as ... )as [A],
cast(null as ...) as [B]
FROM [DBInMyControl].[dbo].[Table3]
end catch
Here I use cast(null as ... )as [A] to get the same type as [ForeignDataQry].[A] has, for example if [ForeignDataQry].[A] is int there should be int: cast(null as int )as [A]
I dealt with this problem in a different way that I originally wanted but anyway:
I created a [Table4] that keeps copy of the records from foreign tables - fields matches [ForeignDataQry] + timestamp. I created procedure:
CREATE PROCEDURE [dbo].[CopyForeignData]
AS
DECLARE #Timestamp datetime
SET #Timestamp = getdate()
BEGIN
INSERT INTO [DBInMyControl].[dbo].[Table4] ([MyID], [A], [B], [Timestamp])
SELECT [Table1].[Field1] AS [MyID],
[Table1].[Field2] AS [A],
[SubQry].[Field2] AS [B],
#Timestamp
FROM [DBOutOfControl].[dbo].[Table1]
LEFT JOIN
(SELECT [Table2].[Field1],
[Table2].[Field2]
FROM [DBOutOfControl].[dbo].[Table2]
WHERE [Table2].[Field3] = 'Where') AS [SubQry] ON [Table1].[Field1] = [SubQry].[Field1]
DELETE FROM [DBInMyControl].[dbo].[Table4] WHERE [Timestamp] <> #Timestamp
END
I will call this every time when I start my app and handle error there and modify my main LEFT JOIN to refer [Table4]
Related
I need help with creating a stored procedure. What I am trying to do is validate count between source and target table, upon executing stored procedure I need print that would say "there is miscount btwn source and target table. However I am not getting print message that I need, please see my code below and if you could let me know where error is in my code? FYI, this is my first attempt to automate some of my scripts as QA (for ETL). Please go easy on me :)
CREATE PROCEDURE ##accountDB_count
AS
BEGIN TRY
DROP TABLE IF EXISTS ##error_count;
CREATE TABLE ##error_count
(
[object] varchar(255),
source_count integer
);
WIT cte AS
(
SELECT
'Data Migration source count' as [object]
,count(distinct s.account_id) as count_all
FROM MariaDBMigration.infoarmor_secure.subscribers s
INNER JOIN MariaDBMigration.infoarmor_secure.subscribers_accounts sa ON s.account_id = sa.id
INNER JOIN MariaDBMigration.infoarmor_secure.subscribers_accounts_plan sap ON sa.account_type
= sap.id
INNER JOIN migration.dbo.map_subscriber_four_fields map on map.subscriber_id = s.id
WHERE s.customerid in (3444,3497,4662,4663,3958,3959,4549,4550,4655)
AND s.active = 1 --OR ( (DATEADD(DAY, 90, coalesce(canceled_date, getdate() ))) >
GETDATE() ) )
AND S.is_primary =1
)
,cte1 AS
(
SELECT 'ACCOUNT_target count' AS [object], COUNT(*) AS count_all
FROM account.dbo.account
)
,cte2 AS
(
SELECT * FROM cte
UNION
SELECT * FROM cte1
UNION
SELECT
'differance' AS [object],
(SELECT count_all FROM cte) - (SELECT count_all FROM cte1)
)
INSERT INTO ##error_count
SELECT * FROM CTE2
END TRY
BEGIN CATCH
DECLARE #COUNT_ALL INT = 0;
WHILE #COUNT_ALL != (SELECT * FROM ##error_count WHERE [object] = 'differance')
begin
PRINT 'MISSCOUNT'
end
END CATCH
I am trying to insert all records from T1 into T2 that are not currently in T2
I have tried in a loop as I am generating a code from a stored proc as the identifier of T2
declare #Part VARCHAR(255),
#GenValue VARCHAR(255),
#x INT
set #x = (select count(*) from T1)
WHILE #x >=0
BEGIN
EXEC [dbo].[usp_GenInd] #GenValue OUT,#GencCode = 'TKM', #GencIncrement = 1
set #Part = #GencValue
INSERT INTO dbo.T2
SELECT #Part AS [part],
[Prod_Code] + Column_Header AS [identifier],
[part_rev] = NULL,
'!' AS [u_version],
a.[Descr] AS [descr],
GETDATE() AS [last_updated],
'ME' AS [last_upd_user],
'EA' AS [basic_unit],
[source] = NULL,
'MAIN' AS [level_1],
'GROUP' AS [level_2],
'ME' AS [user_created],
'20' AS [status],
[Prod_Code] AS [master_part],
[drawing_no] = NULL
FROM [dbo].T1 a
LEFT JOIN dbo.T2 b
ON a.Prod_Code + a.Column_Header = b.part
WHERE b.part is null
END
I keep getting error saying primary key violation on T2 which is the #part variable I am generating from the stored proc.
really slow as well, I thought an insert on left join on null was quicker than a cursor.
only have 67 rows in T1
Thanks for helping in advance
Nope - go back to the cursor if you must continue to use this stored procedure to generate primary key values. The logic error you added to this script is the insert statement. It does not select a specific row from T1 - it selects all rows in T1 that do not exist in T2 (assuming that logic is correct - I'm not going to evaluate it). Presumably you must call the procedure usp_GenInd to generate a PK value for each row in T1. In addition, you never decrement #x - so you have an endless loop.
And notice the wording - "not exists". Generally I find it easier to understand undocumented logic when the query matches (as close as possible) the intent of the code. Your left join logic is the same as not exists - just more difficult to figure out. And you also have a potential problem with your concatenation logic to check for existence. 'AA' + 'B' = 'A' + 'AB' - but the columns contain different values. Be careful about assumptions.
I would try something like:
;WITH cte AS (
SELECT your needed data
FROM [dbo].T1
EXCEPT
SELECT already existing data
FROM [dbo].T2
)
INSERT INTO dbo.T2
SELECT *
FROM cte
Your JOIN logic is flawed.
In your INSERT you have this:
INSERT INTO dbo.T2
SELECT #Part AS [part],
[Prod_Code] + Column_Header AS [identifier],
Inserting #Part into [part]
But when you do your JOIN to rule out existing rows, you have this:
LEFT JOIN dbo.T2 b
ON a.Prod_Code + a.Column_Header = b.part
To rule out existing rows, you should be joining on #part=b.part.
I have two fields with an email address in #data table that I am trying to join on. I want it to join on rep email address and if that doesn't work, I want it to join on email. I tried running the following query:
select a.*
from #data a
join #email b on b.email=coalesce(a.rep_email_address,a.email)
where a.rep_email_address<>a.email
This however doesn't work, because in the case where a.rep_email_adress is not null but doesn't match with b.email, it will drop the record instead of taking the a.email field.
This is the work-around I found:
select a.*
from #data a
join #email b on a.email=b.email
except
select a.*
from #data a
join #email b on a.rep_email_address=b.email
union
select a.*
from #data a
join #email b on a.rep_email_address=b.email
where a.rep_email_address<>a.email
This however, is far from optimal, so I am wondering- any way to write this to perform better/look cleaner or simpler? Just to clarify- this query works (the second query), I am wondering if there is a better way to write it.
Thank you!
This should be much simpler. However, I also recommend you check the execution plan on this query to help you analyze if this is more optimal. [Or just compare the resulting execution times on your tables]
SELECT a.*
FROM #data a
JOIN #email b
ON (a.rep_email_address = b.email
OR a.email = b.email )
WHERE a.rep_email_address<>a.email;
# Not sure why or IF you need this where clause specifically.
Try it like this...
SELECT
a.*,
SomeColumn = ISNULL(e1.SomeColumn, e2.SomeColumn)
from
#data a
LEFT JOIN #email e1
ON e1.email = a.rep_email_address
LEFT JOIN #email e2
ON e2.email = a.email
AND e1.email IS NULL
WHERE
a.rep_email_address <> a.email
AND (
e1.email IS NOT NULL
OR
e2.email IS NOT NULL
);
HTH, Jason
SQL sever uses "Three-Valued Logic" in boolean evaluations:
NULL <> 2 --> Unknown (in your case in the where clause it will essentially become false)
NULL <> NULL --> Unknown (same as above)
a <> b --> true
In your case your original query should be:
select a.*
from #data a
join #email b on b.email=coalesce(a.rep_email_address,a.email)
where ISNULL( a.rep_email_address, '' ) <> ISNULL( a.email, '' )
If you care about performance, then try to avoid the use of functions in join predicates or WHERE conditions as this prevents SQL Server from using indexes on columns that are passed into the function.
SELECT a.*
FROM #data AS a
INNER JOIN #email AS b ON b.email = a.rep_email_address OR b.email = a.email
WHERE a.rep_email_address <> a.email OR ( a.rep_email_address IS NULL OR a.email IS NULL )
Summary:
Do not use NULLs to denote empty strings as this requires a lot of extra code to then check for NULLs
I have a variable in my report which holds 2 possible values: 'monthly' and 'daily'. How can I put this variable (lets call it #reportModel). I think it should be somewhere in GROUP BY clause, but don't know how should it look like.
DECLARE #reportModel VARCHAR(10)
SET #reportModel = 'monthly'
SELECT P.product, SUM(O.price * O.quantity), O.orderDate
FROM Products AS P
INNER JOIN Orders AS O ON P.ID = O.ID
And what now?
How about a stored procedure to handle this, something like.....
CREATE PROCEDURE rpt_GetData
#reportModel VARCHAR(10)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #Sql NVARCHAR(MAX);
IF (#reportModel = 'daily')
BEGIN
SET #Sql = N' SELECT P.product
, SUM(O.price * O.quantity) AS Total
, O.orderDate
FROM Products AS P
INNER JOIN Orders AS O ON P.ID = O.ID
GROUP BY P.product , O.orderDate'
Exec sp_executesql #Sql
END
ELSE IF (#reportModel = 'monthly')
BEGIN
SET #Sql = N' SELECT P.product
, SUM(O.price * O.quantity) AS Total
, MONTH(O.orderDate) AS [Month]
FROM Products AS P
INNER JOIN Orders AS O ON P.ID = O.ID
GROUP BY P.product, MONTH(O.orderDate)'
Exec sp_executesql #Sql
END
END
Adding this as an answer because I mentioned it in the comments in #M.Ali's answer.
So I would suggest you change thinking slightly with one of these options.
Two reports - Simply make a report for daily and another for monthly. Now you have no worries with complex SQL etc.
Make 2 stored procedures, one with the GROUP BY daily and one monthly. Then in your SSRS dataset, create an expression for you SQL that chooses the procedure based on parameter:
=IIf(Parameters!reportModel.Value = "monthly", "GetMonthlyData", "GetDailyData")
I would put it in the Group On Expression of the table or chart rather than doing it in the query.
=IIF(Parameters!reportModel.Value = "monthly", Month(Fields!orderDate.Value), Fields!orderDate.Value)
If you'd rather do it in the query and don't want to wait for DBAs to get around to deploying a Stored Procedure (not to mention maintaining it whenever there's a change), you could use your parameter in a CASE like:
SELECT P.product, SUM(O.price * O.quantity),
CASE WHEN #reportModel = 'monthly' THEN CAST(MONTH(O.orderDate) AS VARCHAR(12))
ELSE CAST(O.orderDateAS VARCHAR(12)) END AS DT
FROM WORKFLOW_SHARED.MAIN.VW_CLAIMSOVERPAYMENT
WHERE DATECOMPLETED > '7/1/2015'
GROUP BY P.product,
CASE WHEN #reportModel = 'monthly' THEN CAST(MONTH(O.orderDate) AS VARCHAR(12))
ELSE CAST(O.orderDateAS VARCHAR(12)) END
This way you don't have to maintain two separate reports.
How about something simple like this
select
P.product
,Total = sum(O.price * O.quantity)
, O.orderDate
from
Products as P
inner join Orders as O on P.ID = O.ID
where
#reportModel = 'daily'
union all
select
P.product
,Total = sum(O.price * O.quantity)
,[Month] = MONTH(O.orderDate)
from
Products as P
inner JOIN Orders as O on P.ID = O.ID
group by
P.product
,[Month] = MONTH(O.orderDate)
where
#reportModel = 'monthly'
I wanted to get the table name. I have the column name and when I try to look up at the Sys.Columns table I get the matching name of the column. How will I get the table name to which the required column is associated
SELECT OBJECT_SCHEMA_NAME(object_id) AS TableSchemaName,
OBJECT_NAME(object_id) AS TableName
FROM sys.columns
WHERE name = 'YourColumnName'
I hope this helps:
select t.name from sys.columns c
inner join sys.tables t
on c.object_id = t.object_id
where c.name = 'insert column name here'
select OBJECT_NAME(object_id) as TableName from sys.Columns where name='columnNamehere'
Try this
declare #columnName As varchar(50) = 'ParentColumnName'
select t.name from sys.tables t
join sys.columns c
on c.object_id = t.object_id
and c.name = #columnName
select name as 'TableName' from sys.tables where object_id=
(select object_id from sys.columns where name='UserName')
I know that this is an old question, but the answers listed to date do not get at what the parent table name is for a view's columns, nor if a column is aliased to have a new name with respect to the column name in the parent table.
Unfortunately, (at least in 2008R2) it seems that even with registering your Views to a Schema, the referencing_minor_id of sys.dm_sql_referenced_entities (or the equivalent column from sys.SQL_Modules) is always set to zero. However, you can retrieve all referred-to tables (and parent views), along with which fields of those tables are queried with sys.dm_sql_referenced_entities (or sys.SQL_Modules). However, it does not capture the oorder of those bindings, so the following won't quite work to link view columns directly to table columns, but it'll provide an approximation:
DECLARE #obj_name nvarchar(50) = 'Table_or_View_name_here';
with obj_id as (
select object_id, name, OBJECT_SCHEMA_NAME(object_id)+'.'+name as qualified_name from sys.all_objects as o
where o.name = #obj_name
),
tv_columns as ( -- table or view
select o.name as Obj_Name, c.* from sys.columns as c join
obj_id as o on
c.object_id=o.object_id
),
sql_referenced_entities as (
SELECT
o.name as referencing_name,
o.object_id,
COALESCE(NULLIF(referencing_minor_id,0),ROW_NUMBER() OVER (ORDER BY (select 'NO_SORT'))) as referencing_minor_id,
referenced_server_name,
referenced_database_name,
referenced_schema_name,
referenced_entity_name,
referenced_minor_name,
referenced_id,
referenced_minor_id,
referenced_class,
referenced_class_desc,
is_caller_dependent,
is_ambiguous
FROM obj_id as o, sys.dm_sql_referenced_entities((select qualified_name from obj_id), 'OBJECT') where referenced_minor_id<>0
)
select
c.object_id as object_id,
o.name as object_name,
c.column_id,
c.name as column_name,
c2.object_id as parent_table_object_id,
o2.name as parent_table_name,
c2.column_id as parent_column_id,
c2.name as parent_column_name
-- ,c.*,
-- ,c2.*
from sys.columns as c join
obj_id as o on
c.object_id=o.object_id left outer join
(sql_referenced_entities as s join
sys.all_objects as o2 on
s.referenced_id=o2.object_id and s.referenced_class=1 join
sys.columns as c2 on
s.referenced_id=c2.object_id and s.referenced_minor_id=c2.column_id
) on
c.object_id=s.object_id and c.column_id=s.referencing_minor_id
To get the true aliases used, as well as any calculations involving the combinations of multiple fields, you would have to parse the output of either OBJECT_DEFINITION(OBJECT_ID('schema.view')) (or potentially exec sp_helptext 'schema.view'), as follows
Starting with OBJECT_DEFINITION(OBJECT_ID('schema.view'))
Mark what is enclosed in single quotes as superseeding the removal and other rules to follow
Remove blocks between /* */ comments
Remove any text after --, up to the next linebreak sequence (see EXEC SP_HELPTEXT 'sp_helptext' for their end-of-line code)
Look up table/subselect aliases from the FROM clause
Parse out the SELECT clause and break on commas, rather than end-of-line.
Reduce any contiguous whitespace to a single space character. Force whitespace before [ and after ] when a dot/period (or whitespace) don't already appear.
We'll put the above into a stored procedure that we'll call usp_helptext_for_view
See which table_alias.field_name are aliased or simply appear as field_name. See the below code snippet to see how to seperate the field alias from the definition.
Link the view to the table as appropriate.
drop table #s
create table #s (id bigint identity(1,1) primary key, text nvarchar(max))
insert into #s (text) exec usp_helptext_for_view #qualified_viewname
with s as (select
id,
text,
az=(select
MIN(x*(case x when 0 then null else 1 end)) FROM (VALUES
(charindex(#viewfieldname,text COLLATE Latin1_General_CI_AS)),
(charindex('['+#viewfieldname+']',text COLLATE Latin1_General_CI_AS)),
(charindex('AS ['+#viewfieldname+']',text COLLATE Latin1_General_CI_AS)),
(charindex('as '+#viewfieldname,text COLLATE Latin1_General_CI_AS))
) AS value(x)
),
NULLIF(charindex('=',text),0)) as eq --oh, the irony of how the two different styles are applied
FROM #s
)
SELECT
#viewfieldname as ViewField,
CASE eq WHEN NULL
THEN IIF(az IS NULL, NULL, LEFT(text, az-2))
ELSE RIGHT(text,LENGTH(text)-eq) -- alternately ELSE CASE az WHEN NULL THEN NULL WHEN <eq THEN RIGHT(text,LENGTH(text)-eq) ELSE NULL END
END as ViewFieldDefinition,
id as sortPosition
FROM s
WHERE text like '%'+#viewfieldname+'%' -- you should be able to eliminate this clause without affecting the results.
ORDER BY id, #viewfieldname