the relationship between table is following this
1 job may contain 0-M subjob = 1 : M
0-M subjob may have 0-M contract = M :M
the table I design are
Job :JobID
Subjob:SubjobID
Contract:ContractID
Subjob_Contract:SubjobID,ContractID
The problem I have face is
when we want to view Job and contract.....incase Job doesnt have Subjob so how could contract link with Job
I would eliminate the distinction between a job and a SubJob from the table structure. You could use the SubJob table as a link to to other Jobs, then you only have a Job_Contract reference.
SubJob would then contain a link between all Jobs and their SubJobs.
Subjobs:
parent_job_id -- Reference to parent Job
job_id -- Previously, your SubJobId
Example:
Select * from subjobs where subjobs.parent_job = {jobid};
Return a set of all subjobs which contain "subjobid"s that are actually job id's.
This way you can reference a contract from any job.
Related
I am new to Snowflake and am trying to create my first task.
CREATE TASK task_update_table
WAREHOUSE = "TEST"
SCHEDULE = 'USING CRON 0 5 * * * America/Los_Angeles'
AS
INSERT INTO "TEST"."WEB"."SOME_TABLE" (ID,VALUE1,VALUE2,VALUE3)
WITH CTE AS
(SELECT
ID
,VALUE1
,VALUE2
,VALUE3
FROM OTHER_TABLE
WHERE ID NOT IN (SELECT ID FROM "TEST"."WEB"."SOME_TABLE")
)
SELECT
ID,VALUE1,VALUE2,VALUE3
FROM CTE
I got a message that the task was created successfully
"Task task_update_table successfully created"
I then try to run show tasks in schema SHOW TASKS IN "TEST"."WEB" and get 0 rows as a result. What am I doing wrong? why is the task not showing?
I did all of this under sysadmin and was using the same warehouse, db and schema.
There are some limitations around show commands that might be blocking you,
particularly "SHOW commands only return objects for which the current user’s current role has been granted the necessary access privileges".
https://docs.snowflake.com/en/sql-reference/sql/show.html#general-usage-notes
I suspect the task was created by a different role (therefore owned by a different role), or perhaps it was created in different database or schema.
To find it, I'd recommend running the following using a role such as ACCOUNTADMIN.
show tasks in account;
SELECT *
FROM (
SELECT *
FROM TABLE(RESULT_SCAN(LAST_QUERY_ID())))
WHERE "name" = 'TASK_UPDATE_TABLE';
While testing and learning in Snowflake, it is critical you set your session "context" correctly, using commands like this:
USE ROLE my_role_here;
USE WAREHOUSE my_warehouse_here;
USE DATABASE my_database_here;
USE SCHEMA my_schema_here;
Doing those four commands, or setting defaults for them for your user will help you tremendously when learning.
I hope this helps...Rich
I am looking through the SSIS DB Project Catalog to find the Begin and End datetime for every SSIS package called from a Master package. Is there any query to conduct this?
After looking at [internal].[executions], and SSISDB.catalog.operation_messages, etc
Running this query does not work, if I there is a master package calling Child Packages.
select start_time,end_time,*
from catalog.executions cc order by cc.start_time desc
I am trying to find the begin and end time for every child pacakge.
The CATALOG.EXECUTABLE_STATISTICS DMV logs execution statistics for components within a package, including execute package tasks. The START_TIME and END_TIME columns store the time that the component began and completed execution. The EXECUTION_DURATION column holds the time that an executable, in this case the child packages, took to execute in milliseconds. This can of course be converted to seconds, minutes, etc. depending on what you need. While this has a column for the execution path of the component within the parent package, it doesn't have a column for the direct name of component, thus CATALOG.EXECUTABLES is included for the EXECUTABLE_NAME, and this DMV can be omitted if you only want to view the execution path (EXECUTION_PATH column) instead. CATALOG.EXECUTIONS has columns for the folder and project name, and you can join to this to apply filters for the specific project and folder that the package is located in. You can also apply a filter on the EXECUTION_ID column to only view details for a specific execution. Executing a package at the basic logging level with allow execution details to be logged for the components.
SELECT
EX.FOLDER_NAME,
EX.PROJECT_NAME,
E.EXECUTABLE_NAME,
EX.PACKAGE_NAME,
ES.START_TIME AS ComponentStartTime,
ES.END_TIME AS ComponentEndTime,
EX.start_time AS PackageStartTime,
EX.end_time AS PackageEndTime,
ES.EXECUTION_DURATION AS ComponentExecutionTimeInMilliseconds
FROM SSISDB.CATALOG.EXECUTIONS EX
INNER JOIN SSISDB.CATALOG.EXECUTABLES E on EX.EXECUTION_ID = E.EXECUTION_ID
INNER JOIN SSISDB.CATALOG.EXECUTABLE_STATISTICS ES on E.EXECUTABLE_ID = ES.EXECUTABLE_ID AND EX.EXECUTION_ID = ES.EXECUTION_ID
--PACKAGE_NAME- parent package
WHERE E.PACKAGE_NAME = 'Package Name.dtsx' AND EX.PROJECT_NAME = 'Project Name'
AND EX.FOLDER_NAME = 'Folder Name'
I have a two tables in SQL Server, in which one is the source for a MERGE operation into another.
The source table has 30Mil Records
The Target table has 180Mil Records. Both tables have 227 columns.
I do have SSIS, but I'm told in this case, a MERGE statement is the better option. Below is a shortened version of it:
;WITH MySource as (
SELECT * FROM [STAGE].[dbo].[STAGE_TABLE]
)
MERGE [EDW].[dbo].[TARGET_TABLE] AS MyTarget
USING MySource
ON MySource.[ID_FIELD] = MyTarget.[ID_FIELD]
AND MySource.[LoadDate] >= MyTarget.[LoadDate]
WHEN MATCHED THEN UPDATE SET
<<Target Column>> = MySource.<<Source Colums>> --227 columns
WHEN NOT MATCHED THEN INSERT
(
[ID_FIELD],
[LoadDate],
<<225 Other Columns>>
)
VALUES (
MySource.[ID_FIELD],
MySource.[LoadDate],
MySource.<<225 other columns>>
);
The only changes I made to the script above is truncating the list of columns to keep the code block here short.
My Problem is that I am getting hung on the execution. The profile screen shows a CXPACKET suspension with the error: cwaitpipenewrow, node=2.
How do I troubleshoot this? Thank you.
Seems like CXPACKET and suspended state means that some threads which have completed are logging that other thread's state which have not completed yet.
Please check here. The query need to update upto 1 Billion values in the table. hence it would be slow running queries.
https://dba.stackexchange.com/questions/96346/cxpacket-suspended-and-null-wait-type
https://www.sqlshack.com/troubleshooting-the-cxpacket-wait-type-in-sql-server/
Hope these articles might help you debug.
We have different SSIS package that we use in daily tasks (updates, ETL...) and we have a kind of complicated structure, where a package calls different other packages. And there are primarily about 10 principal jobs that call secondary ones. So these 10 jobs are always on success even if a step fails so it wouldn't block other executions. Although we would like to retrieve the steps (and their status) that are related to these jobs via a SQL Query but we couldn't join between the steps and their calling jobs and at the same time retrieve the status (The step status in this case and not the jobs).
I searched a lot on the net and i always find a script that joins the steps and calling jobs without the status or steps and status without knowing which job is calling...
(for example this link and this one )
so to sum it all up, we are trying to do a Query where we can join the jobs, their Status and their parent job.
Any help in this matter would be really appreciated and thanks in advance.
EDIT
Thanks to the link in #BaconBits comment i was able to create a query joining three tables (msdb.dbo.sysjobsteps, msdb.dbo.sysjobs, msdb.dbo.sysjobhistory) that retrieves something like the following:
Job_name1 Step_name1 Job1_status
Job_name1 Step_name2 Job1_status
Job_name1 Step_name3 Job1_status
Job_name2 Step_name1 Job2_status
Job_name2 Step_name2 Job2_status
But I still couldn't retrieve the step status (which is what i need in this case since the job outcome is always on success even if a step fails)
Query:
select j.name, s.step_name,
CASE WHEN s.last_run_outcome=0 THEN 'Failed'
WHEN s.last_run_outcome=1 THEN 'Success'
WHEN s.last_run_outcome=2 THEN 'Retry'
WHEN s.last_run_outcome=3 THEN 'Canceled'
END
,h.run_date, s.output_file_name
from msdb.dbo.sysjobsteps s
inner join msdb.dbo.sysjobs j on s.job_id=j.job_id
inner join msdb.dbo.sysjobhistory h
on h.job_id=j.job_id or s.step_id=h.step_id
--where j.name like '%Dem%'
order by h.run_date, j.name
Thank you #BaconBits and anyone for any further help.
I am not much strong in SQL, so looking for some help.
First I am looking for suggestion for the best way to implement this logic in SQL and then some sample code to implement.
My portal is going to connect Students and Training Providers.
Students: Select what courses (multiple) they want, type of delivery (online, class room), Industry(domain) to which the course to be targeted more, Location Preference.
Training Providers: Select what courses offering (so one record for each course), offering locations, type of delivery for each course, industries (multiple) it is targeting.
When student login:
I would like to create SP which in turn create view to store the matched records of the Training Providers data which matches that student needs of that StudentID, CourseID passed to SP
I have created the following sp ( but not included create view part as I am not sure how to do this)
set ANSI_NULLS ON
set QUOTED_IDENTIFIER ON
go
ALTER PROCEDURE [dbo].[sp_TPsMatched2StuCourse]
-- Add the parameters for the stored procedure here
#StuID int,
#CourseID int
AS
BEGIN
Select TP.MemID,TP.PastExp,SN.DeliveryType,SN.LocPref,SN.Industry,SC.CourseID from
tbl_TrainingProvider as TP , tbl_StuCourses as SC, tbl_StuNeeds SN
where SN.CourseID = #CourseID and SN.StuID = #StuID and
SN.DeliveryType in (TP.DeliveryMode) and
SN.LocPref IN (TP.LocOffering) and
SN.Industry IN (TP.Industries)
END
--- exec sp_ELsMatched2EntProp 1, 1
Why I need to put the data is as follows:
Assume the data is stored in that dynamic view and that would be bind to datagrid. Student then select interested TPs. Then only contact details would be shared to each other and this cannot be reveresed. So I would put this interested data in another table later. Every time data changes, hence the matches. Student can change some of his/her needs or new TPs join etc so view to be temparory.
when I executed this using above command, I am not getting data though it matches few records. What is wrong I am doing.
Any help would be greatly appreciated.
You are not getting expected results because you filter out too many records in WHERE( I'm talking about this part : SN.DeliveryType in (TP.DeliveryMode) and
SN.LocPref IN (TP.LocOffering) and SN.Industry IN (TP.Industries)). I'd recommend to use JOIN ... ON instead of specifying all tables in FROM and join condition in WHERE. I'm not sure what you want exactly, but I believe you are looking for
FROM tbl_StuNeeds SN
LEFT JOIN tbl_TrainingProvider as TP ON (TP.DeliveryMode = SN.DeliveryType AND
SN.LocPref = TP.LocOffering AND TP.Industries = SN.Industry)
WHERE SN.CourseID = #CourseID and SN.StuID = #StuID
Also, there is no join conditions in your code for tbl_StuCourses as SC which results in cross-join.
Finally, why do you need a stored procedure at all? From what I see in your example, a table-valued function will work better:
CREATE FUNCTION [dbo].getTPsMatched2StuCourse(#StuID INT,#CourseID INT)
RETURNS TABLE AS
RETURN
Select .... ;