I am stuck and I can not figure out. Does anyone have any ideas? I would greatly appreciate any help.
I have a table in SQL Server 2008 with the following defintion:
I can insert/delete/update without any error. However, when i run a select statement that filter out ONE single item, i got the following error:
Msg 511, Level 16, State 1, Line 1
Cannot create a row of size 8104 which is greater than the allowable maximum row size of 8060.
If I remove the where condition a15.MEMBER_PARENT_LEVEL_ID4 in (91329) it ran fine. If I add 5 or 10 criterias it ran fine, including the one that is throwing the error if you add it by itself.
[MSTR_PROD_HIER_ID] [int] NOT NULL,
[MSTR_PROD_HIER_DESC] [varchar](50) NULL,
[MEMBER_GROUP_LEVEL_ID1] [int] NULL,
[MEMBER_GROUP_LEVEL_DESC1] [varchar](60) NULL,
[MEMBERKEY_SORT1] [int] NULL,
[MEMBER_GROUP_LEVEL_ID2] [int] NULL,
[MEMBER_GROUP_LEVEL_DESC2] [varchar](60) NULL,
[MEMBERKEY_SORT2] [int] NULL,
[MEMBER_GROUP_LEVEL_ID3] [int] NULL,
[MEMBER_GROUP_LEVEL_DESC3] [varchar](60) NULL,
[MEMBERKEY_SORT3] [int] NULL,
[MEMBER_LEVEL_ID4] [int] NULL,
[MEMBER_LEVEL_DESC4] [varchar](60) NULL,
[MEMBER_LEVEL_DESC4_1] [varchar](60) NULL,
[MEMBER_PARENT_LEVEL_ID1] [int] NOT NULL,
[MEMBER_PARENT_LEVEL_DESC1] [varchar](60) NULL,
[SORT1] [int] NULL,
[MEMBER_PARENT_LEVEL_ID2] [int] NULL,
[MEMBER_PARENT_LEVEL_DESC2] [varchar](60) NULL,
[SORT2] [int] NULL,
[MEMBER_PARENT_LEVEL_ID3] [int] NULL,
[MEMBER_PARENT_LEVEL_DESC3] [varchar](60) NULL,
[SORT3] [int] NULL,
[MEMBER_PARENT_LEVEL_ID4] [int] NULL,
[MEMBER_PARENT_LEVEL_DESC4] [varchar](60) NULL,
[SORT4] [int] NULL,
[CLASS_DESC] [varchar](60) NULL,
[CLASS_DESC1] [varchar](50) NULL,
[TOTAL_BRAND_IND] [int] NOT NULL
}
{
select a11.PlanFormat PlanFormat,
max(a18.FormatName) FormatName,
a15.MEMBER_PARENT_LEVEL_ID2 MEMBER_PARENT_LEVEL_ID2,
max(a15.MEMBER_PARENT_LEVEL_DESC2) MEMBER_PARENT_LEVEL_DESC2,
a15.MEMBER_PARENT_LEVEL_ID3 MEMBER_PARENT_LEVEL_ID3,
max(a15.MEMBER_PARENT_LEVEL_DESC3) MEMBER_PARENT_LEVEL_DESC3,
a15.MEMBER_PARENT_LEVEL_ID4 MEMBER_PARENT_LEVEL_ID4,
max(a15.MEMBER_PARENT_LEVEL_DESC4) MEMBER_PARENT_LEVEL_DESC4,
max(a15.CLASS_DESC) CLASS_DESC,
max(a15.CLASS_DESC1) CLASS_DESC1,
a16.MSTR_SEASON_ID MSTR_SEASON_ID,
max(a16.MSTR_SEASON_DESC) MSTR_SEASON_DESC,
max(a16.MSTR_SEASON_DESC1) MSTR_SEASON_DESC1,
a17.MSTR_STORE_ID MSTR_STORE_ID,
max(a17.MSTR_STORE_DESC1) MSTR_STORE_DESC1,
a13.MSTR_C_MNTH_ID MSTR_C_MNTH_ID,
max(a13.MSTR_C_MNTH_DESC) MSTR_C_MNTH_DESC,
a13.MSTR_CALENDAR_GRP_ID MSTR_CALENDAR_GRP_ID,
max(a13.MSTR_CALENDAR_GRP_DESC) MSTR_CALENDAR_GRP_DESC,
a12.YEAR_NBR YEAR_NBR,
a11.PlanNo PlanNo,
max(a18.PlanName) PlanName,
sum((a11.Value1821 / 1000.0)) ACT_PROJ_SLS_DOL
from A_MSTR_PLAN_FCT a11
join A_MSTR_CALENDAR_MONTH_XREF a12
on (a11.Member2 = a12.Member2)
join A_MSTR_CALENDAR_MONTH a13
on (a12.DIM_TIME_ID = a13.DIM_TIME_ID and
a12.YEAR_NBR = a13.YEAR_NBR)
join A_MSTR_PRODUCT_LEVEL_3 a14
on (a11.MEMBER0 = a14.P_MEMBER3)
join A_MSTR_PRODUCT_LEVEL_3_PARENT a15
on (a14.MEMBER_LEVEL_ID4 = a15.MEMBER_LEVEL_ID4 and
a14.MSTR_PROD_HIER_ID = a15.MSTR_PROD_HIER_ID)
join A_MSTR_SEASONALITY a16
on (a11.MEMBER3 = a16.MEMBER3)
join A_MSTR_STORE a17
on (a11.MEMBER1 = a17.MEMBER1)
join A_MSTR_PLAN a18
on (a11.PlanNo = a18.PlanNo and
a12.YEAR_NBR = a18.YEAR_NBR and
a13.YEAR_ID = a18.YEAR_ID)
where (a11.PlanNo in (1547)
and a15.MEMBER_PARENT_LEVEL_ID4 in (91329))
group by a11.PlanFormat,
a15.MEMBER_PARENT_LEVEL_ID2,
a15.MEMBER_PARENT_LEVEL_ID3,
a15.MEMBER_PARENT_LEVEL_ID4,
a16.MSTR_SEASON_ID,
a17.MSTR_STORE_ID,
a13.MSTR_C_MNTH_ID,
a13.MSTR_CALENDAR_GRP_ID,
a12.YEAR_NBR,
a11.PlanNo
}
This is a follow up. to my previous post. As i mentioned below, this error goes away after i dropped the table, perform DBCC clean up and recreate the table. Now, it is coming back but this time, only when you select two or more criterias. The problem seems to be with the where condition where it filter out my product. My product table does not have any nvarchar or varbinary.
I am so frustrated... :-(
I would suggest modifying the query to remove the IN clause(s) and replace it with additional joins or a WHERE EXISTS. This may generate a more efficient query plan which avoids the problem you're seeing.
If your physical data size is less than 8060 then too you are getting this error, try following command.
DBCC CLEANTABLE
For example:
DBCC CLEANTABLE (Databasename,'tablename', 0)
WITH NO_INFOMSGS;
Related
I am trying to insert a row in the table which throws an Arithmetic overflow error. However, I am having difficulty understanding the reason !
Here's the table Columns-
[id] [int] IDENTITY(1,1) NOT NULL,
[batch_number] [bigint] NOT NULL,
[mixer_batch_history_id] [int] NULL,
[lab_test_type_id] [int] NOT NULL,
[lab_test_compound_id] [int] NOT NULL,
[machine_number] [int] NULL,
[sample_location_id] [int] NULL,
[sample_date] [datetime] NOT NULL,
[pass_customer_specs] [int] NOT NULL,
[pass_internal_specs] [int] NOT NULL,
[tested_by] [nvarchar](25) NULL,
[last_test] [bit] NULL,
[date_created] [datetime] NOT NULL
I am trying the following query -
DECLARE #batchNo as BIGINT = 2010000976009
INSERT INTO [LoMOS].dbo.lab_test
(batch_number, mixer_batch_history_id, lab_test_type_id, lab_test_compound_id, machine_number, sample_location_id, sample_date, pass_customer_specs, pass_internal_specs, date_created)
VALUES (#batchNo, 16579, 1, 128, 1, 1, '2020-07-06 08:43:25.227', 0, 1, '2020-07-08 07:44:27.993')
SELECT SCOPE_IDENTITY()
Which Throws an Error as -
Arithmetic overflow error converting expression to data type int.
Msg 3609, Level 16, State 1, Line 3
Any Idea ?
Based on the information provided, it seems highly likely the error is coming from a trigger, not the insert as posted.
I have created a partition table and trying to query the partition but not able to do so in sql server 2016. Could somebody tell me where I am going wrong
CREATE PARTITION FUNCTION [financialStatementPartition](datetime)
AS RANGE RIGHT FOR VALUES (N'2013-01-01T00:00:00.000', N'2014-01-01T00:00:00.000',
N'2015-01-01T00:00:00.000', N'2016-01-01T00:00:00.000',
N'2017-01-01T00:00:00.000')
GO
Table schema
CREATE TABLE [dbo].[FinancialStatementIds]
(
[financialCollectionId] [int] NOT NULL,
[companyId] [int] NOT NULL,
[dataItemId] [int] NOT NULL,
[dataItemName] [varchar](200) NULL,
[dataItemvalue] [decimal](18, 0) NULL,
[unittypevalue] [int] NULL,
[fiscalyear] [int] NULL,
[fiscalquarter] [int] NULL,
[periodenddate] [datetime] NULL,
[filingdate] [datetime] NULL,
[restatementtypename] [varchar](200) NULL,
[latestforfinancialperiodflag] [bit] NULL,
[latestfilingforinstanceflag] [bit] NULL,
[currencyconversionflag] [int] NULL,
[currencyname] [varchar](200) NULL,
[periodtypename] [varchar](200) NULL
)
Query
SELECT *
FROM dbo.FinancialStatementIds
WHERE $PARTITION.financialStatementPartition(periodenddate) = '2013-01-01T00:00:00.000'
Error
Conversion failed when converting the varchar value '2013-01-01T00:00:00.000' to data type int.
Use following query instead:
SELECT *
FROM dbo.FinancialStatementIds
WHERE periodenddate = '20130101'
Anyway, the table is still not partitioned properly. You need to apply Partition Scheme to connect the table and Partition function.
I am using SQL Server version 2012. I have a table which has more than 10 million rows. I have to count records using a SQL filter.
My query is this:
select count(*)
from reconcil
where tenantid = 101
which is taking more than 5 minutes for 5 millions records.
Is there any fastest way to count records?
Reconcil table structure is
CREATE TABLE [dbo].[RECONCIL]
(
[AckCode] [nvarchar](50) NULL,
[AckExpireTime] [int] NULL,
[AckFileName] [nvarchar](255) NULL,
[AckKey] [int] NULL,
[AckState] [int] NULL,
[AppMsgKey] [nvarchar](30) NULL,
[CurWrkActID] [nvarchar](50) NULL,
[Date_Time] [datetime] NULL,
[Direction] [nvarchar](1) NULL,
[ErrorCode] [nvarchar](50) NULL,
[FGLOGKEY] [int] NOT NULL,
[FolderID] [int] NULL,
[FuncGCtrlNo] [nvarchar](14) NULL,
[INLOGKEY] [int] NULL,
[InputFileName] [nvarchar](255) NULL,
[IntCtrlNo] [nvarchar](14) NULL,
[IsAssoDataPresent] [nvarchar](1) NULL,
[JobState] [int] NULL,
[LOGDATA] [nvarchar](max) NULL,
[MessageID] [nvarchar](25) NULL,
[MessageState] [int] NULL,
[MessageType] [int] NULL,
[NextWrkActID] [nvarchar](50) NULL,
[NextWrkHint] [nvarchar](20) NULL,
[NONFAERRORLOG] [nvarchar](max) NULL,
[NumberOfBytes] [int] NULL,
[NumberOfSegments] [int] NULL,
[OutputFileName] [nvarchar](255) NULL,
[Priority] [nvarchar](1) NULL,
[ReceiverID] [nvarchar](30) NULL,
[RecNo] [int] NULL,
[RecordID] [int] IDENTITY(1,1) NOT NULL,
[RelationKey] [int] NULL,
[SEGLOG] [nvarchar](max) NULL,
[SenderID] [nvarchar](30) NULL,
[ServerID] [nvarchar](255) NULL,
[Standard] [int] NULL,
[TenantID] [int] NULL,
[TPAgreementKey] [int] NULL,
[TSetCtrlNo] [nvarchar](35) NULL,
[UserKey1] [nvarchar](255) NULL,
[UserKey2] [nvarchar](255) NULL,
[UserKey3] [nvarchar](255) NULL,
CONSTRAINT [RECONCIL_PK]
PRIMARY KEY CLUSTERED ([RecordID] ASC)
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
Unless you materialized the count, this non-clustered index on TenentID will provide better performance because it is narrower than the clustered primary key index and will scan only the matching rows:
CREATE INDEX idx ON [dbo].[RECONCIL](TenantID);
If performance of the aggregate query with this index isn't acceptable, you could create an indexed view with the count. The indexed view will provide the fastest performance for this query but will incur additional costs for storage and index maintenance for inserts and deletes. Also, queries that modify the table must have required SET options for indexed views. Those costs may be justified if the count query is executed often.
SQL Server can use the indexed view automatically in Enterprise (or Developer) editions even if not directly referenced in the query as long as the optimizer can match the semantics of the query using the view. In lesser editions, you'll need to query the indexed view directly and specify the NOEXPAND hint.
CREATE VIEW dbo.VW_RECONCIL_COUNT
WITH SCHEMABINDING
AS
SELECT
TenantID
, COUNT_BIG(*) AS TenentRowCount
FROM [dbo].[RECONCIL]
GROUP BY TenantID;
GO
CREATE UNIQUE CLUSTERED INDEX cdx ON dbo.VW_RECONCIL_COUNT(TenantID);
GO
--Enterprise Edition can use the view index automatically
SELECT COUNT_BIG(*) AS TenentRowCount
FROM [dbo].[RECONCIL]
WHERE TenantID = 101
GROUP BY TenantID;
GO
--other editions require the view to be specified plus the NOEXPAND hint
SELECT TenentRowCount
FROM dbo.VW_RECONCIL_COUNT WITH (NOEXPAND)
WHERE TenantID = 101;
GO
As being suggested, create an index or even partition your table by tenantId if you have so many items. This way you would have one data file per partition which increases performance.
select count(tenantid)
from reconcil
where tenantid = 101 group by tenantid ;
not sure but try using this.
I have a database that can have data updated from two external parties.
Each of those parties sends a pipe delimited text file that is BULK INSERTED into the staging table.
I now want to change the scheme for one of the parties by adding a few columns, but this is unfortunately breaking the BULK INSERT for the other party even though the new columns are all added as NULLABLE.
Is there any obvious solution to this?
TABLE SCHEMA:
CREATE TABLE [dbo].[CUSTOMER_ENTRY_LOAD](
[CARD_NUMBER] [varchar](12) NULL,
[TITLE] [varchar](6) NULL,
[LAST_NAME] [varchar](34) NULL,
[FIRST_NAME] [varchar](40) NULL,
[MIDDLE_NAME] [varchar](40) NULL,
[NAME_ON_CARD] [varchar](26) NULL,
[H_ADDRESS_PREFIX] [varchar](50) NULL,
[H_FLAT_NUMBER] [varchar](5) NULL,
[H_STREET_NUMBER] [varchar](10) NULL,
[H_STREET_NUMBER_SUFFIX] [varchar](5) NULL,
[H_STREET] [varchar](50) NULL,
[H_SUBURB] [varchar](50) NULL,
[H_CITY] [varchar](50) NULL,
[H_POSTCODE] [varchar](4) NULL,
[P_ADDRESS_PREFIX] [varchar](50) NULL,
[P_FLAT_NUMBER] [varchar](5) NULL,
[P_STREET_NUMBER] [varchar](10) NULL,
[P_STREET_NUMBER_SUFFIX] [varchar](5) NULL,
[P_STREET] [varchar](50) NULL,
[P_SUBURB] [varchar](50) NULL,
[P_CITY] [varchar](50) NULL,
[P_POSTCODE] [varchar](4) NULL,
[H_STD] [varchar](3) NULL,
[H_PHONE] [varchar](7) NULL,
[C_STD] [varchar](3) NULL,
[C_PHONE] [varchar](10) NULL,
[W_STD] [varchar](3) NULL,
[W_PHONE] [varchar](7) NULL,
[W_EXTN] [varchar](5) NULL,
[DOB] [smalldatetime] NULL,
[EMAIL] [varchar](50) NULL,
[DNS_STATUS] [bit] NULL,
[DNS_EMAIL] [bit] NULL,
[CREDITCARD] [char](1) NULL,
[PRIMVISACUSTID] [int] NULL,
[PREFERREDNAME] [varchar](100) NULL,
[STAFF_NUMBER] [varchar](50) NULL,
[CUSTOMER_ID] [int] NULL,
[IS_ADDRESS_VALIDATED] [varchar](50) NULL
) ON [PRIMARY]
BULK INSERT STATEMENT:
SET #string_temp = 'BULK INSERT customer_entry_load FROM '+char(39)+#inpath
+#current_file+'.txt'+char(39)+' WITH (FIELDTERMINATOR = '+char(39)+'|'+char(39)
+', MAXERRORS=1000, ROWTERMINATOR = '+char(39)+'\n'+char(39)+')'
SET DATEFORMAT dmy
EXEC(#string_temp)
The documentation describes how to use a format file to handle the scenario where the target table has more columns than the source file. An alternative that can sometimes be easier is to create a view on the table and BULK INSERT into the view instead of the table; this possibility is described in the same documentation.
And please always mention your SQL Server version.
Using OPENROWSET with BULK allows you to use your file in a query. You can use that to format the data and select only the columns you need.
In the end I have handled the two different cases with two different BULK INSERT statements (depending on which file is being processed). It seems like there isn't a way to do what I was trying to do with one statement.
You could use the format file idea supplied by #Pondlife.
Adapt your insert dynamically based on the input file name (provided there are unique differneces between the external parties). Using a CASE statement, simply select the correct format file based on the unique identifier in the file name.
DECLARE #formatFile varchar (max);
Set #formatFile =
CASE
WHEN #current_file LIKE '%uniqueIdentifier%'
THEN 'file1'
ELSE 'file2'
END
SET #string_temp = 'BULK INSERT customer_entry_load FROM '+char(39)+#inpath
+#current_file+'.txt'+char(39)+' WITH (FORMATFILE = '+char(39)+#formatFile+char(39)
')'
SET DATEFORMAT dmy
EXEC(#string_temp)
Hope that helps!
I have been playing around with a quite complex SQL Statement for a few days, and have gotten most of it working correctly.
I am having trouble with one last part, and was wondering if anyone could shed some light on the issue, as I have no idea why it isnt working:
INSERT INTO ExistingClientsAccounts_IMPORT
SELECT DISTINCT
cca.AccountID, cca.SKBranch, cca.SKAccount, cca.SKName, cca.SKBase,
cca.SyncStatus, cca.SKCCY, cca.ClientType, cca.GFCID, cca.GFPID, cca.SyncInput,
cca.SyncUpdate, cca.LastUpdatedBy, cca.Deleted, cca.Branch_Account, cca.AccountTypeID
FROM ClientsAccounts AS cca
INNER JOIN
(SELECT DISTINCT ClientAccount, SKAccount, SKDesc,
SKBase, SKBranch, ClientType, SKStatus, GFCID,
GFPID, Account_Open_Date, Account_Update
FROM ClientsAccounts_IMPORT) AS ccai
ON cca.Branch_Account = ccai.ClientAccount
Table definitions follow:
CREATE TABLE [dbo].[ExistingClientsAccounts_IMPORT](
[AccountID] [int] NOT NULL,
[SKBranch] [varchar](2) NOT NULL,
[SKAccount] [varchar](12) NOT NULL,
[SKName] [varchar](255) NULL,
[SKBase] [varchar](16) NULL,
[SyncStatus] [varchar](50) NULL,
[SKCCY] [varchar](5) NULL,
[ClientType] [varchar](50) NULL,
[GFCID] [varchar](10) NULL,
[GFPID] [varchar](10) NULL,
[SyncInput] [smalldatetime] NULL,
[SyncUpdate] [smalldatetime] NULL,
[LastUpdatedBy] [varchar](50) NOT NULL,
[Deleted] [tinyint] NOT NULL,
[Branch_Account] [varchar](16) NOT NULL,
[AccountTypeID] [int] NOT NULL
) ON [PRIMARY]
CREATE TABLE [dbo].[ClientsAccounts_IMPORT](
[NEWClientIndex] [bigint] NOT NULL,
[ClientGroup] [varchar](255) NOT NULL,
[ClientAccount] [varchar](255) NOT NULL,
[SKAccount] [varchar](255) NOT NULL,
[SKDesc] [varchar](255) NOT NULL,
[SKBase] [varchar](10) NULL,
[SKBranch] [varchar](2) NOT NULL,
[ClientType] [varchar](255) NOT NULL,
[SKStatus] [varchar](255) NOT NULL,
[GFCID] [varchar](255) NULL,
[GFPID] [varchar](255) NULL,
[Account_Open_Date] [smalldatetime] NULL,
[Account_Update] [smalldatetime] NULL,
[SKType] [varchar](255) NOT NULL
) ON [PRIMARY]
The error message I get is:
Msg 8152, Level 16, State 14, Line 1
String or binary data would be truncated.
The statement has been terminated.
The error is because you are trying to insert data into a column in ExistingClientsAccounts_IMPORT where the column size is smaller than the length of data attempting to be inserted into it.
e.g.
SKAccount column is VARCHAR(12) in the ExistingClientsAccounts_IMPORT table but is VARCHAR(255) in ClientsAccounts_IMPORT.
So if ClientsAccounts_IMPORT contains any rows where that field is longer than 12 characters, you will get that error as obv. e.g. 100 characters will not fit into a 12 character field.
You need to make sure all the columns in the table you are inserting into, are big enough - make sure each column definition matches the source table.
The third column of your SELECT column list means that ExistingClientsAccounts_IMPORT.SKAccount is populated from ClientsAccounts.SKAccount - however, the source is up to 255 characters, while the destination has a capacity of 12. If there's any data that wouldn't fit, you'll get this message.
I haven't gone through all the other columns.
You are trying to insert values which are greater than tha max length specified for a column. Use a profiler to check the data being passed to this query and verify the length of data against the permissible length for all columns.
There is a clear mismatch in the column lenghts of common columns of these two tables.
ClientsAccounts_IMPORT.SKBase is 10 whereas it is 16 in the other table.
Check the field definitions. You can see some are smaller than the original ones. Now run a query on the old data - you will find some of the larger fields were used, so the insert is not possible without losing data.
Example: SKAccount - from 255 length to 12.