sql server profiler 2014 "failed to open a table" - sql-server

I'm trying to replay SQL Server 2014 Profiler trace that I saved to a DB table. When I open I get "Failed to open a table" error message. There is nothing in the windows logs.
I googled and this error used to happen when upgrading a SQL Server 2000 system to a 64 bit system. That doesn't apply here. I'm running on Windows Server 2012 with a fresh install of SQL Server 2014.
The trace was a TSQL_replay template. I saved it to a table using the following code. The code produced a table with the definition shown.
SELECT *
INTO myTrace
FROM ::fn_trace_gettable(N'c:\Logs\sql_trace_events.trc', default)
CREATE TABLE [dbo].[myTrace]
(
[TextData] [ntext] NULL,
[BinaryData] [image] NULL,
[DatabaseID] [int] NULL,
[TransactionID] [bigint] NULL,
[LineNumber] [int] NULL,
[NTUserName] [nvarchar](256) NULL,
[NTDomainName] [nvarchar](256) NULL,
[HostName] [nvarchar](256) NULL,
[ClientProcessID] [int] NULL,
[ApplicationName] [nvarchar](256) NULL,
[LoginName] [nvarchar](256) NULL,
[SPID] [int] NULL,
[Duration] [bigint] NULL,
[StartTime] [datetime] NULL,
[EndTime] [datetime] NULL,
[Reads] [bigint] NULL,
[Writes] [bigint] NULL,
[CPU] [int] NULL,
[Permissions] [bigint] NULL,
[Severity] [int] NULL,
[EventSubClass] [int] NULL,
[ObjectID] [int] NULL,
[Success] [int] NULL,
[IndexID] [int] NULL,
[IntegerData] [int] NULL,
[ServerName] [nvarchar](256) NULL,
[EventClass] [int] NULL,
[ObjectType] [int] NULL,
[NestLevel] [int] NULL,
[State] [int] NULL,
[Error] [int] NULL,
[Mode] [int] NULL,
[Handle] [int] NULL,
[ObjectName] [nvarchar](256) NULL,
[DatabaseName] [nvarchar](256) NULL,
[FileName] [nvarchar](256) NULL,
[OwnerName] [nvarchar](256) NULL,
[RoleName] [nvarchar](256) NULL,
[TargetUserName] [nvarchar](256) NULL,
[DBUserName] [nvarchar](256) NULL,
[LoginSid] [image] NULL,
[TargetLoginName] [nvarchar](256) NULL,
[TargetLoginSid] [image] NULL,
[ColumnPermissions] [int] NULL,
[LinkedServerName] [nvarchar](256) NULL,
[ProviderName] [nvarchar](256) NULL,
[MethodName] [nvarchar](256) NULL,
[RowCounts] [bigint] NULL,
[RequestID] [int] NULL,
[XactSequence] [bigint] NULL,
[EventSequence] [bigint] NULL,
[BigintData1] [bigint] NULL,
[BigintData2] [bigint] NULL,
[GUID] [uniqueidentifier] NULL,
[IntegerData2] [int] NULL,
[ObjectID2] [bigint] NULL,
[Type] [int] NULL,
[OwnerID] [int] NULL,
[ParentName] [nvarchar](256) NULL,
[IsSystem] [int] NULL,
[Offset] [int] NULL,
[SourceDatabaseID] [int] NULL,
[SqlHandle] [image] NULL,
[SessionLoginName] [nvarchar](256) NULL,
[PlanHandle] [image] NULL,
[GroupID] [int] NULL
)

I tried the same thing and I did not run into any issues. Have you tried with a new trace and save to a different named table?

You have to wait...the 'replay' is grayed out for about 1 minute until it fully loads the script.

Had the same issue and it turned out I was trying to open a trace recorded in Profiler 2014 with Profiler 2008 on a diffrent SQL instance in order to reply the trace. Upgrading profiler to 2014 on a replay instance solved the problem.

You have to create table of specific structure first. Try to export trace into a table from profiler and look what it created. Then just insert subset of columns into the table. Here is what I used for SQL 2012-2017:
------- Trace created with Replay template
USE [testdb]
GO
/****** Object: Table [dbo].[TraceTable] Script Date: 29-Oct-18 17:37:07 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[TraceTableSQL1]
(
[RowNumber] [int] IDENTITY ( 0 , 1 ) NOT NULL ,
[EventClass] [int] NULL ,
[BinaryData] [image] NULL ,
[DatabaseID] [int] NULL ,
[NTUserName] [nvarchar] ( 128 ) NULL ,
[NTDomainName] [nvarchar] ( 128 ) NULL ,
[HostName] [nvarchar] ( 128 ) NULL ,
[ClientProcessID] [int] NULL ,
[ApplicationName] [nvarchar] ( 128 ) NULL ,
[LoginName] [nvarchar] ( 128 ) NULL ,
[SPID] [int] NULL ,
[StartTime] [datetime] NULL ,
[EndTime] [datetime] NULL ,
[Error] [int] NULL ,
[DatabaseName] [nvarchar] ( 128 ) NULL ,
[RowCounts] [bigint] NULL ,
[RequestID] [int] NULL ,
[EventSequence] [bigint] NULL ,
[IsSystem] [int] NULL ,
[ServerName] [nvarchar] ( 128 ) NULL ,
[TextData] [ntext] NULL ,
[EventSubClass] [int] NULL ,
[Handle] [int] NULL ,
PRIMARY KEY CLUSTERED
(
[RowNumber] ASC
)
WITH ( PAD_INDEX = OFF , STATISTICS_NORECOMPUTE = OFF , IGNORE_DUP_KEY = OFF , ALLOW_ROW_LOCKS = ON , ALLOW_PAGE_LOCKS = ON ) ON [PRIMARY]
)
ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
INSERT [TraceTableSQL1]
SELECT
[EventClass] ,
[BinaryData] ,
[DatabaseID] ,
[NTUserName] ,
[NTDomainName] ,
[HostName] ,
[ClientProcessID] ,
[ApplicationName] ,
[LoginName] ,
[SPID] ,
[StartTime] ,
[EndTime] ,
[Error] ,
[DatabaseName] ,
[RowCounts] ,
[RequestID] ,
[EventSequence] ,
[IsSystem] ,
[ServerName] ,
[TextData] ,
[EventSubClass] ,
[Handle]
FROM sys.fn_trace_gettable ( N'd:\temp\profiler.trc' , DEFAULT )

Related

The wait operation timed out

I have 2 tables,each contains 4-500k records
CREATE TABLE [dbo].[User][UserId] [int] IDENTITY(1,1) NOT NULL,
[Password] [nvarchar](max) NULL,
[RoleId] [int] NOT NULL,
[Name] [nvarchar](max) NULL,
[Address] [nvarchar](max) NULL,
[Email] [nvarchar](max) NULL,
[Landline] [nvarchar](max) NULL,
[MobileNumberCode] [int] NULL,
[MobileNumber] [nvarchar](max) NULL,
[DateOfBirth] [datetime] NULL,
[MarriageDate] [datetime] NULL,
[CreatedDate] [datetime] NOT NULL,
[UpdatedDate] [datetime] NOT NULL,
[Status] [nvarchar](max) NOT NULL,
[BranchId] [int] NULL,
[UserTitle] [nvarchar](50) NULL,
[MiddleName] [nvarchar](50) NULL,
[LastName] [nvarchar](50) NULL,
[HouseNumber] [nvarchar](50) NULL,
[BuildingNumber] [nvarchar](50) NULL,
[RoadNumber] [nvarchar](50) NULL,
[BlockNumber] [nvarchar](50) NULL,
[City] [nvarchar](50) NULL,
[NearBranchId] [int] NULL,
[MobileIsValid] [bit] NULL,
[EmailIsValid] [bit] NULL,
[Gender] [nvarchar](50) NULL,
[SourceId] [int] NULL)
CREATE TABLE [dbo].[PurchaseOrder]
[PurchaseOrderId] [int] NOT NULL,
[BranchId] [int] NOT NULL,
[PurchaseDate] [datetime] NOT NULL,
[Amount] [decimal](18, 3) NOT NULL,
[UserId] [int] NOT NULL,
[Status] [nvarchar](max) NULL,
[sbs_no] [int] NOT NULL)
And I have stored procedure to get data from these tables using join.
CREATE PROC Sp_SearchCustomer (#FromDate datetime = null,
#ToDate datetime = null,
#RegFromDate datetime = null,
#RegToDate datetime = null)
AS
BEGIN
select a.UserId,a.Name,b.PurchaseOrderId,b.Amount from dbo.[User] a left join PurchaseOrder b on a.UserId=b.UserId
where
((a.CreatedDate >= ''' + cast(#RegFromDate as varchar) + ''')
AND (a.CreatedDate <= ''' + cast(#RegToDate as varchar) + '''))
and ((b.PurchaseDate >= ''' + cast(#FromDate as varchar) + ''')
AND (b.PurchaseDate <= ''' + cast(#ToDate as varchar) + '''))
END
When executing this procedure with date, its getting "The wait operation timed out" exception. Please help to solve this issue.
Your date in your tables and in your Procedure are both saved as varchar. This is perfect and there is no need to convert them to varchar.
Beside, varchar is surrounded by quotes and won't be executed. This is just becoming a string:
where ((a.CreatedDate >= 'cast(#RegFromDate as varchar)')...
There are also way too many useless parenthesis since you are using AND.
Try this instead:
CREATE PROC Sp_SearchCustomer (
#FromDate datetime = null,
#ToDate datetime = null,
#RegFromDate datetime = null,
#RegToDate datetime = null
)
AS
BEGIN
SELECT a.UserId
,a.Name
,b.PurchaseOrderId
,b.Amount
FROM dbo.[User] a
LEFT JOIN PurchaseOrder b
ON a.UserId = b.UserId
WHERE
a.CreatedDate >= #RegFromDate
AND a.CreatedDate <= #RegToDate
AND b.PurchaseDate >= #FromDate
AND b.PurchaseDate <= #ToDate
END
Once the query has been improved, you can test it again.
You should also look at Statistics and Indexes and make sure that Statistics are up-to-date and Indexes are not fragmented.
For Statistics, you can use: exec sp_updatestats
For Indexes on these 2 tables, look at the Fragmentation % and choose to REBUILD or REORGANIZE them.

Improve performance of select query in SQL Server database

select
a.COUNTY_FIPS
,COUNT(e.PROPERTY_ID) as house_count
,AVG(cast(e.AVM_FINAL_VALUE as bigint)) as avg_avm
,max(cast(e.AVM_FINAL_VALUE as bigint)) as max_avm
,min(cast(e.AVM_FINAL_VALUE as bigint)) as min_avm
from
RAW_Equity e
left join
(SELECT
SA_PROPERTY_ID, MM_FIPS_STATE_CODE, MM_FIPS_MUNI_CODE,
CASE
WHEN MM_FIPS_STATE_CODE < 10
THEN '0' + CAST(MM_FIPS_STATE_CODE as VARCHAR)
ELSE CAST(MM_FIPS_STATE_CODE as VARCHAR)
END
+ CASE
WHEN MM_FIPS_MUNI_CODE < 10
THEN '00' + CAST(MM_FIPS_MUNI_CODE as VARchar)
WHEN MM_FIPS_MUNI_CODE < 100
THEN '0' + CAST(MM_FIPS_MUNI_CODE as VARchar)
ELSE CAST(MM_FIPS_MUNI_CODE as VARchar)
END AS COUNTY_FIPS
FROM
RAW_Address) a ON a.SA_PROPERTY_ID = e.PROPERTY_ID
where
AVM_CONFIDENCE_SCORE >= 70
group by
a.COUNTY_FIPS
Is there any way I can improve the performance of this query? Schema for both the tables are shown below. I am was thinking about creating non clustered index on AVM_CONFIDENCE_SCORE but I think it will only going to increase the query time. Any help will be greatly appreciated.
RAWADDRESS table:
CREATE TABLE [dbo].[RAW_Address]
(
[SA_PROPERTY_ID] [int] NOT NULL,
[SA_SCM_ID] [int] NOT NULL,
[MM_STATE_CODE] [varchar](2) NOT NULL,
[MM_MUNI_NAME] [varchar](24) NOT NULL,
[MM_FIPS_STATE_CODE] [tinyint] NOT NULL,
[MM_FIPS_MUNI_CODE] [smallint] NOT NULL,
[MM_FIPS_COUNTY_NAME] [varchar](35) NOT NULL,
[SA_SITE_HOUSE_NBR] [varchar](20) NULL,
[SA_SITE_FRACTION] [varchar](10) NULL,
[SA_SITE_DIR] [varchar](2) NULL,
[SA_SITE_STREET_NAME] [varchar](40) NULL,
[SA_SITE_SUF] [varchar](4) NULL,
[SA_SITE_POST_DIR] [varchar](2) NULL,
[SA_SITE_UNIT_PRE] [varchar](10) NULL,
[SA_SITE_UNIT_VAL] [varchar](6) NULL,
[SA_SITE_CITY] [varchar](30) NULL,
[SA_SITE_STATE] [varchar](2) NOT NULL,
[SA_SITE_ZIP] [int] NULL,
[SA_SITE_PLUS_4] [smallint] NULL,
[SA_SITE_CRRT] [varchar](4) NULL,
[SA_MAIL_HOUSE_NBR] [varchar](20) NULL,
[SA_MAIL_FRACTION] [varchar](10) NULL,
[SA_MAIL_DIR] [varchar](2) NULL,
[SA_MAIL_STREET_NAME] [varchar](50) NULL,
[SA_MAIL_SUF] [varchar](4) NULL,
[SA_MAIL_POST_DIR] [varchar](2) NULL,
[SA_MAIL_UNIT_PRE] [varchar](10) NULL,
[SA_MAIL_UNIT_VAL] [varchar](6) NULL,
[SA_MAIL_CITY] [varchar](50) NULL,
[SA_MAIL_STATE] [varchar](2) NULL,
[SA_MAIL_ZIP] [int] NULL,
[SA_MAIL_PLUS_4] [smallint] NULL,
[SA_MAIL_CRRT] [varchar](4) NULL,
[SA_SITE_MAIL_SAME] [varchar](1) NULL
) ON [PRIMARY]
RAW Equity table:
CREATE TABLE [dbo].[RAW_Equity]
(
[PROPERTY_ID] [int] NOT NULL,
[SCM_ID] [int] NOT NULL,
[MM_STATE_CODE] [varchar](2) NOT NULL,
[MM_MUNI_NAME] [varchar](24) NOT NULL,
[MM_FIPS_STATE_CODE] [int] NOT NULL,
[MM_FIPS_MUNI_CODE] [int] NOT NULL,
[MM_FIPS_COUNTY_NAME] [varchar](35) NOT NULL,
[AVM_FINAL_VALUE] [int] NULL,
[AVM_LOW_VALUE] [int] NULL,
[AVM_HIGH_VALUE] [int] NULL,
[AVM_CONFIDENCE_SCORE] [int] NULL,
[FINAL_VALUE] [float] NULL,
[FIRST_POSITION_SR_UNIQUE_ID] [int] NULL,
[FIRST_POSITION_LOAN_DATE] [int] NULL,
[FIRST_POSITION_DOC_NBR] [varchar](20) NULL,
[FIRST_POSITION_LOAN_VAL] [int] NULL,
[FIRST_POSITION_LENDER_CODE] [int] NULL,
[FIRST_POSITION_LNDR_LAST_NAME] [varchar](50) NULL,
[FIRST_POSITION_LNDR_FIRST_NAME] [varchar](50) NULL,
[FIRST_POSITION_LENDER_TYPE] [varchar](1) NULL,
[FIRST_POSITION_LOAN_TYPE] [varchar](1) NULL,
[FIRST_POSITION_INTEREST_RATE_TYPE] [varchar](1) NULL,
[FIRST_POSITION_ESTIMATED_INTEREST_RATE] [float] NULL,
[FIRST_POSITION_LNDR_CREDIT_LINE] [varchar](1) NULL,
[FIRST_POSITION_MODELED_MORTGAGE_TYPE] [varchar](1) NULL,
[SECOND_POSITION_SR_UNIQUE_ID] [int] NULL,
[SECOND_POSITION_LOAN_DATE] [int] NULL,
[SECOND_POSITION_DOC_NBR] [varchar](20) NULL,
[SECOND_POSITION_LOAN_VAL] [int] NULL,
[SECOND_POSITION_LENDER_CODE] [int] NULL,
[SECOND_POSITION_LNDR_LAST_NAME] [varchar](50) NULL,
[SECOND_POSITION_LNDR_FIRST_NAME] [varchar](50) NULL,
[SECOND_POSITION_LENDER_TYPE] [varchar](1) NULL,
[SECOND_POSITION_LOAN_TYPE] [varchar](1) NULL,
[SECOND_POSITION_INTEREST_RATE_TYPE] [varchar](1) NULL,
[SECOND_POSITION_ESTIMATED_INTEREST_RATE] [float] NULL,
[SECOND_POSITION_LNDR_CREDIT_LINE] [varchar](1) NULL,
[SECOND_POSITION_MODELED_MORTGAGE_TYPE] [varchar](1) NULL,
[THIRD_POSITION_SR_UNIQUE_ID] [int] NULL,
[THIRD_POSITION_LOAN_DATE] [int] NULL,
[THIRD_POSITION_DOC_NBR] [varchar](20) NULL,
[THIRD_POSITION_LOAN_VAL] [int] NULL,
[THIRD_POSITION_LENDER_CODE] [int] NULL,
[THIRD_POSITION_LNDR_LAST_NAME] [varchar](50) NULL,
[THIRD_POSITION_LNDR_FIRST_NAME] [varchar](50) NULL,
[THIRD_POSITION_LENDER_TYPE] [varchar](1) NULL,
[THIRD_POSITION_LOAN_TYPE] [varchar](1) NULL,
[THIRD_POSITION_INTEREST_RATE_TYPE] [varchar](1) NULL,
[THIRD_POSITION_ESTIMATED_INTEREST_RATE] [float] NULL,
[THIRD_POSITION_LNDR_CREDIT_LINE] [varchar](1) NULL,
[THIRD_POSITION_MODELED_MORTGAGE_TYPE] [varchar](1) NULL,
[TOTAL_OUTSTANDING_LOANS] [bigint] NULL,
[LTV] [int] NULL,
[AVAILABLE_EQUITY] [int] NULL,
[LENDABLE_EQUITY] [int] NULL,
[PROCESS_ID] [int] NOT NULL,
[FILLER] [varchar](4) NULL,
CONSTRAINT [PK_RAW_Equity] PRIMARY KEY CLUSTERED
(
[PROPERTY_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
You could try this way :-
(willing to know, how much this one, helpful to you)
Declare #result Table
(
RowId Int Identity(1,1) Primary Key
,COUNTY_FIPS Varchar(100)
,MM_FIPS_STATE_CODE Int
,MM_FIPS_MUNI_CODE Int
,house_count Int
,avg_avm Int
,max_avm Int
,min_avm Int
)
Insert Into #result(MM_FIPS_STATE_CODE,MM_FIPS_MUNI_CODE,house_count,avg_avm,max_avm,min_avm)
Select a.MM_FIPS_STATE_CODE
,a.MM_FIPS_MUNI_CODE
,Count(e.PROPERTY_ID) as house_count
,Avg(Cast(e.AVM_FINAL_VALUE as bigint)) as avg_avm
,Max(Cast(e.AVM_FINAL_VALUE as bigint)) as max_avm
,Min(Cast(e.AVM_FINAL_VALUE as bigint)) as min_avm
From RAW_Equity As e With (Nolock)
Left Join RAW_Address As a With (Nolock) On e.PROPERTY_ID = a.SA_PROPERTY_ID
Where e.AVM_CONFIDENCE_SCORE >= 70
Group by a.MM_FIPS_STATE_CODE
,a.MM_FIPS_MUNI_CODE
Update r
Set r.COUNTY_FIPS = REPLICATE('0',2-LEN(RTRIM(r.MM_FIPS_STATE_CODE))) + RTRIM(r.MM_FIPS_STATE_CODE) + REPLICATE('0',3-LEN(RTRIM(r.MM_FIPS_MUNI_CODE))) + RTRIM(r.MM_FIPS_MUNI_CODE)
From #result As r
Select r.COUNTY_FIPS
,r.house_count
,r.avg_avm
,r.max_avm
,r.min_avm
From #result As r
1st try without any index, and after that create clustered index as mentioned and try AGAIN the above same query
CREATE INDEX IX_RAW_Address_SA_PROPERTY_ID ON RAW_Address(SA_PROPERTY_ID)
I would put an index on:
Table: RAW_Equity
Columns: PROPERTY_ID, AVM_CONFIDENCE_SCORE
and
Table: RAW_Address
Columns: SA_PROPERTY_ID
Include: MM_FIPS_STATE_CODE,MM_FIPS_MUNI_CODE
Your query can be simplified, which will likely make it faster:
select
REPLICATE('0',2-LEN(RTRIM(a.MM_FIPS_STATE_CODE)))
+ RTRIM(a.MM_FIPS_STATE_CODE)
+ REPLICATE('0',3-LEN(RTRIM(a.MM_FIPS_MUNI_CODE)))
+ RTRIM(a.MM_FIPS_MUNI_CODE)
AS COUNTY_FIPS
,COUNT(e.PROPERTY_ID) as house_count
,AVG(cast(e.AVM_FINAL_VALUE as bigint)) as avg_avm
,max(cast(e.AVM_FINAL_VALUE as bigint)) as max_avm
,min(cast(e.AVM_FINAL_VALUE as bigint)) as min_avm
from
RAW_Equity e
left join RAW_Address a
ON a.SA_PROPERTY_ID = e.PROPERTY_ID
where
e.AVM_CONFIDENCE_SCORE >= 70
group by
a.MM_FIPS_STATE_CODE, a.MM_FIPS_MUNI_CODE
If RAW_Address doesn't already have a CLUSTERED INDEX with SA_PROPERTY_ID as the first key of the index, then this may help:
CREATE INDEX IX_RAW_Address_SA_PROPERTY_ID ON RAW_Address(SA_PROPERTY_ID)
INCLUDE (MM_FIPS_STATE_CODE, MM_FIPS_MUNI_CODE)

Violation of Primary Key error on Identity column

This is maddening! Code in question has been running for over 5 years.
Here's the scoop....
I am doing an INSERT...SELECT into a table with a primary key that is an identity column. I do not specify the key when I insert - SQL Server generates it as expected.
I am doing the insert in a stored procedure that I call in a loop (for loop in SSIS, actually). The stored procedure will insert rows in batches (configurable). It might insert 1000 rows at a time or it might insert 50,000 - doesn't matter. It will work for a random number of calls (inserting thousands of rows) and then it will fail, out of the blue, with a
Violation of primary key / duplicate
error. If I check the identity seed - it is correct. If I kick off the process again it will work fine, for a while.
The values being inserted are coming from 2 tables that I join together, as if that matters.
The bulk of my code is below:
WHILE #pk <= #max_pk
BEGIN
INSERT INTO tbl_claim_line (fk_batch_control_group, fk_claim, fk_provider, service_from_date, service_to_date, allowed, net_paid, COB, flex_1, flex_2, flex_3, flex_4)
SELECT
#fk_batch_control_group
, c.pk_claim
, p.pk_provider
, i.date_of_service_from
, i.date_of_service_to
, i.allowed_amount
, i.net_paid_amount
, i.cob_amount
, i.claimline_flex_1
, i.claimline_flex_2
, i.claimline_flex_3
, i.claimline_flex_4
FROM
tbl_import i
INNER JOIN
tbl_import__claim c ON i.claim_number = c.claim_number
LEFT JOIN
tbl_import__provider p ON ISNULL(i.provider_type,'') = ISNULL(p.provider_type,'')
AND ISNULL(i.provider_specialty,'') = ISNULL(p.provider_specialty,'')
AND ISNULL(i.provider_zip_code,'') = ISNULL(p.provider_zip_code,'')
WHERE
pk_import = #pk
UPDATE tbl_import
SET fk_claim_line = SCOPE_IDENTITY()
WHERE pk_import = #pk
SET #pk += 1
END
--TABLE DEFINITIONS...
CREATE TABLE [dbo].[tbl_claim_line](
[fk_batch_control_group] [int] NOT NULL,
[fk_claim] [int] NOT NULL,
[fk_provider] [int] NULL,
[service_from_date] [date] NULL,
[service_to_date] [date] NULL,
[allowed] [money] NULL,
[net_paid] [money] NULL,
[COB] [money] NULL,
[flex_1] [varchar](200) NULL,
[flex_2] [varchar](200) NULL,
[flex_3] [varchar](200) NULL,
[flex_4] [varchar](200) NULL,
[pk_claim_line] [int] IDENTITY(1,1) NOT NULL,
[insert_date] [datetime] NOT NULL,
CONSTRAINT [PK_tbl_claim_line] PRIMARY KEY NONCLUSTERED
(
[pk_claim_line] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[tbl_claim_line] WITH CHECK
ADD CONSTRAINT [FK_tbl_claim_line_tbl_batch_control_group]
FOREIGN KEY([fk_batch_control_group])
REFERENCES [dbo].[tbl_batch_control_group] ([pk_batch_control_group])
GO
ALTER TABLE [dbo].[tbl_claim_line] CHECK CONSTRAINT [FK_tbl_claim_line_tbl_batch_control_group]
GO
ALTER TABLE [dbo].[tbl_claim_line] WITH CHECK
ADD CONSTRAINT [FK_tbl_claim_line_tbl_claim]
FOREIGN KEY([fk_claim])
REFERENCES [dbo].[tbl_claim] ([pk_claim])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[tbl_claim_line] CHECK CONSTRAINT [FK_tbl_claim_line_tbl_claim]
GO
ALTER TABLE [dbo].[tbl_claim_line] WITH CHECK
ADD CONSTRAINT [FK_tbl_claim_line_tbl_provider]
FOREIGN KEY([fk_provider])
REFERENCES [dbo].[tbl_provider] ([pk_provider])
GO
ALTER TABLE [dbo].[tbl_claim_line] CHECK CONSTRAINT [FK_tbl_claim_line_tbl_provider]
GO
ALTER TABLE [dbo].[tbl_claim_line] ADD CONSTRAINT [DF_tbl_claim_line__insert_date] DEFAULT (getdate()) FOR [insert_date]
GO
----second table
CREATE TABLE [dbo].[tbl_import](
[fk_claim_line] [int] NULL,
[member_id] [varchar](50) NULL,
[member_card_id] [varchar](50) NULL,
[member_first_name] [varchar](50) NULL,
[member_last_name] [varchar](50) NULL,
[member_dob] [varchar](50) NULL,
[member_gender] [varchar](50) NULL,
[member_subscriber_relationship_code] [varchar](50) NULL,
[member_address_line_1] [varchar](100) NULL,
[member_address_line_2] [varchar](100) NULL,
[member_city] [varchar](50) NULL,
[member_state] [varchar](50) NULL,
[member_zip] [varchar](50) NULL,
[member_phone] [varchar](50) NULL,
[member_email] [varchar](50) NULL,
[subscriber_id] [varchar](50) NULL,
[group_line_of_business] [varchar](50) NULL,
[group_product] [varchar](50) NULL,
[group_employer] [varchar](50) NULL,
[provider_first_name] [varchar](50) NULL,
[provider_last_or_full_name] [varchar](200) NULL,
[provider_type] [varchar](200) NULL,
[provider_specialty] [varchar](400) NULL,
[provider_zip_code] [varchar](50) NULL,
[provider_tax_id] [varchar](50) NULL,
[medical_code_1] [varchar](10) NULL,
[medical_code_1_description] [varchar](500) NULL,
[medical_code_2] [varchar](10) NULL,
[medical_code_2_description] [varchar](500) NULL,
[medical_code_3] [varchar](10) NULL,
[medical_code_3_description] [varchar](500) NULL,
[medical_code_4] [varchar](10) NULL,
[medical_code_4_description] [varchar](500) NULL,
[medical_code_5] [varchar](10) NULL,
[medical_code_5_description] [varchar](500) NULL,
[medical_code_6] [varchar](10) NULL,
[medical_code_6_description] [varchar](500) NULL,
[medical_code_7] [varchar](10) NULL,
[medical_code_7_description] [varchar](500) NULL,
[medical_code_8] [varchar](10) NULL,
[medical_code_8_description] [varchar](500) NULL,
[medical_code_9] [varchar](10) NULL,
[medical_code_9_description] [varchar](500) NULL,
[medical_code_10] [varchar](10) NULL,
[medical_code_10_description] [varchar](500) NULL,
[medical_code_11] [varchar](10) NULL,
[medical_code_11_description] [varchar](500) NULL,
[medical_code_12] [varchar](10) NULL,
[medical_code_12_description] [varchar](500) NULL,
[medical_code_13] [varchar](10) NULL,
[medical_code_13_description] [varchar](500) NULL,
[medical_code_14] [varchar](10) NULL,
[medical_code_14_description] [varchar](500) NULL,
[medical_code_15] [varchar](10) NULL,
[medical_code_15_description] [varchar](500) NULL,
[medical_code_16] [varchar](10) NULL,
[medical_code_16_description] [varchar](500) NULL,
[date_of_service_from] [varchar](50) NULL,
[date_of_service_to] [varchar](50) NULL,
[claim_number] [varchar](50) NULL,
[claim_line_number] [varchar](50) NULL,
[original_claim_number] [varchar](50) NULL,
[allowed_amount] [varchar](50) NULL,
[net_paid_amount] [varchar](50) NULL,
[cob_amount] [varchar](50) NULL,
[date_paid] [varchar](50) NULL,
[member_flex_1] [varchar](200) NULL,
[member_flex_2] [varchar](200) NULL,
[member_flex_3] [varchar](200) NULL,
[member_flex_4] [varchar](200) NULL,
[claim_flex_1] [varchar](200) NULL,
[claim_flex_2] [varchar](200) NULL,
[claim_flex_3] [varchar](200) NULL,
[claim_flex_4] [varchar](200) NULL,
[claimline_flex_1] [varchar](200) NULL,
[claimline_flex_2] [varchar](200) NULL,
[claimline_flex_3] [varchar](200) NULL,
[claimline_flex_4] [varchar](200) NULL,
[pk_import] [int] IDENTITY(1,1) NOT NULL,
CONSTRAINT [PK_tbl_import] PRIMARY KEY NONCLUSTERED
(
[pk_import] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
I ran into this and much like user3170349, it was a seed issue on the column. I'm adding some additional info, however.
First, you can run this to figure out if you have a seed problem:
DBCC CHECKIDENT ('TABLE_NAME_GOES_HERE', NORESEED);
This will give you information which will read something like this:
Checking identity information: current identity value 'XXXX', current column value 'YYYY'.
If YYYY is larger than XXXX, then you have a problem and need to RESEED the table to get things going again. You can do so with the following command:
DBCC CHECKIDENT ('TABLE_NAME_GOES_HERE', RESEED, ZZZZZ);
Where ZZZZ is the reseed value. That value should be at least one higher than YYYY. YMMV, so pick a value that is appropriate for your situation.
"Code in question has been running for over 5 years."
"It might insert 1000 records at a time or it might insert 50,000 "
Is it possible you have finally overflowed the integer type of the primary key?
Did it wrap around and is now starting over? That would cause duplicate primary keys.

Using exec() function when inserting to a table

When you have 2 tables table like this
CREATE TABLE #BranchType
(
[External_BranchTypeID] [uniqueidentifier] DEFAULT newsequentialid() NOT NULL,
[BranchTypeID] [smallint] identity(1,1) ,
[BranchTypeDescription] [nvarchar](20) NOT NULL,
[DateCreated] [datetime] NOT NULL,
[UserCreated] [nvarchar](20) NOT NULL,
[DateModified] [datetime] NULL,
[UserModified] [nvarchar](20) NULL,
[IsDeleted] [bit] NOT NULL,
)
CREATE TABLE BranchSubType
(
[External_BranchSubTypeID] [uniqueidentifier] DEFAULT newsequentialid() NOT NULL,
[BranchSubTypeID] [smallint] identity(1,1) ,
[BranchTypeID] [uniqueidentifier] NOT NULL,
[BranchSubTypeDescription] [nvarchar](30) NOT NULL,
[FinancialSystemTypeId] [smallint] NOT NULL,
[DateCreated] [datetime] NOT NULL,
[UserCreated] [nvarchar](20) NOT NULL,
[DateModified] [datetime] NULL,
[UserModified] [nvarchar](20) NULL,
[IsDeleted] [bit] NOT NULL,
)
How can you do an insert like the one below in SQL Server? I am trying to return the guid value
DECLARE #SQLCmd VARCHAR(max)
set #SQLCmd = 'SELECT External_BranchTypeID FROM #BranchType WHERE BranchTypeID =1'
INSERT INTO BranchSubType (BranchTypeID, BranchSubTypeDescription, BranchSubTypeId, DateCreated, UserCreated,IsDeleted)
VALUES ( exec(#SQLCmd), 'Normal',1, getdate(), 'System',0) --FROM #BranchType A WHERE A.BranchTypeID = 1
In this case you don't need to use EXEC
INSERT INTO BranchSubType
(BranchTypeID,
BranchSubTypeDescription,
BranchSubTypeId,
DateCreated,
UserCreated,
IsDeleted)
SELECT External_BranchTypeID,
'Normal',
1,
getdate(),
'System',
0
FROM #BranchType WHERE BranchTypeID =1

A typical performance case of SQL Optimization

I have an engineering practice of SQL Optimization problem, which I think is a typical case ,and will help a lot of guys.
SQL SERVER 2005,
Firstly, create the main table. This is a person info table.
CREATE TABLE [dbo].[OLAPAgentDim](
[RoleID] [varchar](50) NULL CONSTRAINT [DF_OLAPAgentDim_RoleID] DEFAULT ((1)),
[OLAPKey] [bigint] IDENTITY(1,1) NOT NULL,
[FatherKey] [bigint] NULL,
[FatherKeyValue] [nvarchar](100) NULL,
[System] [varchar](6) NULL,
[Level] [int] NULL,
[IfLeaf] [real] NULL,
[IfDel] [real] NULL CONSTRAINT [DF_OLAPAgentDim_IfDel] DEFAULT ((0)),
[SourceKey] [varchar](50) NULL,
[MainDemoName] [nvarchar](100) NULL,
[FastCode] [varchar](50) NULL,
[TagValue] [varchar](50) NULL,
[Script] [nvarchar](max) NULL,
[Birthday] [datetime] NULL,
[EarlyStartTime] [datetime] NULL,
[StartTime] [datetime] NULL,
[EndTime] [datetime] NULL,
[EditTime] [datetime] NULL,
[BecomesTime] [datetime] NULL,
[ContractTime] [datetime] NULL,
[ContractEndTime] [datetime] NULL,
[XMLIcon] [nvarchar](max) NULL,
[PassKey] [varchar](50) NULL CONSTRAINT [DF_OLAPAgentDim_PassKey] DEFAULT ('N3pkY3RHaeZXA9mGJdfm8A=='),
[Address] [nvarchar](100) NULL,
[HomeTel] [varchar](50) NULL,
[Mobile] [varchar](50) NULL,
[Email] [varchar](100) NULL,
[IDCard] [varchar](50) NULL,
[IDSecu] [varchar](50) NULL,
[IDEndowment] [varchar](50) NULL,
[IDAccumulation] [varchar](50) NULL,
[ContactPerson] [nvarchar](100) NULL,
[ContactPersonTel] [varchar](50) NULL,
[Others1] [varchar](50) NULL,
[SexKey] [varchar](2) NULL CONSTRAINT [DF_OLAPAgentDim_SexKey] DEFAULT ((1)),
[SexKeyValue] [nvarchar](100) NULL,
[MarrageKey] [varchar](2) NULL CONSTRAINT [DF_OLAPAgentDim_MarrageKey] DEFAULT ((1)),
[MarrageKeyValue] [nvarchar](100) NULL,
[Nation] [nvarchar](50) NULL,
[Race] [nvarchar](50) NULL,
[PartyMemberKey] [varchar](2) NULL CONSTRAINT [DF_OLAPAgentDim_PartyMemberKey] DEFAULT ((1)),
[PartyMemberKeyValue] [nvarchar](100) NULL,
[RegionKey] [bigint] NULL CONSTRAINT [DF_OLAPAgentDim_RegionKey] DEFAULT ((1)),
[RegionKeyValue] [nvarchar](100) NULL,
[LeaveResonKey] [bigint] NULL CONSTRAINT [DF_OLAPAgentDim_LeaveResonKey] DEFAULT ((1)),
[LeaveResonKeyValue] [nvarchar](100) NULL,
[RoleStr] [varchar](max) NULL,
[RoleStrValue] [nvarchar](max) NULL,
[LeaderKey] [bigint] NULL CONSTRAINT [DF_OLAPAgentDim_LeaderKey] DEFAULT ((1)),
[LeaderKeyValue] [nvarchar](100) NULL,
[FastCode2] [varchar](50) NULL,
[FastCode3] [varchar](50) NULL,
[FastCode4] [varchar](50) NULL,
[FastCode5] [varchar](50) NULL,
[OtherAddress] [nvarchar](100) NULL,
[ShowOrder] [int] NULL,
[RaceKey] [bigint] NULL DEFAULT ((1)),
[RaceKeyValue] [nvarchar](100) NULL,
[DepartLevelKey] [bigint] NULL DEFAULT ((1)),
[DepartLevelKeyValue] [nvarchar](100) NULL,
[forumname] [nvarchar](100) NULL,
[IfCloseKey] [bigint] NULL DEFAULT ((1)),
[IfCloseKeyValue] [nvarchar](100) NULL,
[InsureStartTime] [datetime] NULL,
[AccumulationStartTime] [datetime] NULL,
[Rate] [varchar](50) NULL,
[DirectLeaderKey] [bigint] NULL CONSTRAINT [DF_OLAPAgentDim_DirectLeaderKey] DEFAULT ((1)),
[DirectLeaderAttriKey] [bigint] NULL CONSTRAINT [DF_OLAPAgentDim_DirectLeaderAttriKey] DEFAULT ((1)),
[DirectLeaderKeyValue] [nvarchar](100) NULL,
[DirectLeaderSourceKey] [varchar](50) NULL,
[DirectLeaderPartName] [nvarchar](100) NULL,
[DirectLeaderPositionName] [nvarchar](100) NULL,
[NOTSync] [int] NULL,
[FatherPath] [nvarchar](max) NULL,
[SaleDiscount] [real] NULL,
CONSTRAINT [PK_OLAPAgent Dim] PRIMARY KEY CLUSTERED
(
[OLAPKey] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Secondly, insert abount 10,000 record into the table. I think 10,000 record is not a very big number to SQL SERVER. You can see this is a father and children dimention table in fact. The records with ifleaf=0 means the person's department structure node, the records with ifleaf=1 means the person. You can define father-children relationship using FahterKey column. For Example:
OLAPKey IfLeaf FatherKey DepartLevelKey MainDemoName
2 0 0 1 IBM Company
3 0 2 2 Sales Depart
4 0 2 2 Service Depart
5 0 3 3 Sales Team1
6 1 5 NULL John Smith
7 1 4 NULL Mary
......
DepartLevelKey Column means the depart node's level.
So in this table, we can save the whole HR tree info.
Thirdly, we see the problem SQL:
create table #t
(
TableID int IDENTITY(1,1),
OLAPKey bigint,
MainDemoName nvarchar(max)
)
declare #t4 table
(
TableID int IDENTITY(1,1),
MainDemoName nvarchar(max),
OLAPKeystr varchar(100)
)
declare #agentkey bigint
set #agentkey ='2'
--Part A
--DepartLevelKey=2, to get #agentkey node's all level=2 department
;WITH Result AS(
SELECT OLAPKey,DepartLevelKey,maindemoname FROM OLAPAgentDim WHERE OLAPKey =#agentkey
UNION ALL
SELECT a.OLAPKey,a.DepartLevelKey,a.maindemoname FROM OLAPAgentDim AS a,Result AS b WHERE a.FatherKey = b.OLAPKey
)
insert #t select OLAPKey,maindemoname from Result where DepartLevelKey=4
--Part B
;with One as
(
select *,convert(varchar(50),OLAPKey) as Re from #t
)
insert #t4 select maindemoname,stuff((select ','+Re from One where One.maindemoname=#t.maindemoname for xml path('')),1,1,'') as Two
from #t
group by maindemoname
drop table #t
The SQL above is divided into Part A and Part B.
Part A SQL get all the childrens below a root node(and filtered those belong to the specified DepartLevelKey). For example, to get all persons in Sales Department's child-department with level=3.
Part B SQL change the rows to column, For example:
Change:
TableID OLAPKey MainDemoName
1 6 Sales Team1
2 10 Sales Team1
3 12 Sales Team1
to:
TableID MainDemoName OLAPKeystr
1 Sales Team1 6,10,12
Thus we get each goal department's persons, for further processing(omited here).
The Problem:
The Part A is very slow, cost about 5 minutes. The Part B is slow too.
I wonder how to optimize it basing the table struc existed.
yours,
Ivan
Try:
(i) Adding this index to OLAPAgentDim:
create index IX_OLAPAgentDim_FatherKey on OLAPAgentDim (FatherKey) include (DepartLevelKey, MainDemoName)
(ii) Changing MainDemoName in #t from nvarchar(max) to nvarchar(100). This matches the column definition in OLAPAgentDim.
(iii) Between Part A and Part B, i.e. after Part A and before Part B, adding this index to #t:
create clustered index IX on #t (MainDemoName)

Resources