I found my SQL Server job gets stuck occasionally, about once every two months. Since I am not from DBA background, I need some to help to rectify the issue.
So far, I have tried to pinpoint the issue by checking the activity monitor. I found the issue is caused by one of my stored procedures which it will create a temp table to collect data, then the data will be inserted into one of my transaction tables. This table have 400 millions of records.
Whenever this issue occur, I stop the job and:
I rerun the job, the stored procedure can complete
I execute the stored procedure manually, the stored procedure completes
I implemented the SP_BlitzCache, and execute it. I can see it suggest DBCC FREEPROCCACHE (0x0...) on the stored procedure.
CREATE TABLE #dtResult
(
RunningNumber INTEGER,
, AlphaID BIGINT
, BetaID BIGINT
, Content varchar(100)
, X varchar(10)
, Y varchar(10)
)
INSERT INTO #dtResult ( RunningNumber, ...)
SELECT RowId AS RunningNumber,
...
FROM
...
/*** Based on activity monitor, the highest CPU caused by this statement ***/
INSERT INTO tblTransaction ( ... )
SELECT DISTINCT
RES.AlphaID
, b.UnitId
, RES.BetaID
, CASE WHEN RES.BinData IS NULL THEN [dbo].[fnGetCode](B.Data, RES.X, RES.Y) ELSE RES.Content END
, CONVERT(DATETIME, SUBSTRING(RES.Timestamp, 1, 4) + '-' + SUBSTRING(RES.Timestamp, 5, 2) + '-' + SUBSTRING(RES.Timestamp, 7, 2) + ' ' + SUBSTRING(RES.Timestamp, 9, 2) + ':' + SUBSTRING(RES.Timestamp, 11, 2) + ':' + SUBSTRING(RES.Timestamp, 13, 2) + '.' + SUBSTRING(RES.Timestamp, 15, 3), 121)
FROM
#dtResult RES
INNER JOIN
tblA a with(nolock) ON RES.AlphaID = a.AlphaID
INNER JOIN
tblB b with(nolock) ON a.UnitId = b.UnitId AND CAST(RES.X AS INTEGER) = b.X AND CAST(RES.Y AS INTEGER) = b.Y
INNER JOIN
tblC c with(nolock) ON RES.BetaID = c.BetaID
LEFT OUTER JOIN
tblTransaction t with(nolock) ON RES.AlphaID = t.AlphaID AND RES.BetaID = t.BetaID AND t.UnitId = b.UnitId
WHERE
t.BetaID IS NULL
/* FUNCTION */
CREATE FUNCTION [dbo].[fnGetCode]
(
#Data VARCHAR(MAX),
#SearchX INT,
#SearchY INT
)
RETURNS CHAR(4)
WITH ENCRYPTION
AS
BEGIN
DECLARE #SearchResult CHAR(4)
DECLARE #Pos INT
SET #Pos = (#SearchY * #SearchX) + 1
SET #SearchResult = CONVERT(char(1),SUBSTRING(#Data,#Pos,1), 1)
RETURN #SearchResult
END
Related
Using the Data Explorer (SEDE), I would like to find which users have more than 200000 reputation on Stack Overflow, and then find details for any accounts they have on other Stack Exchange sites.
Here is the query which provides the list with this threshold:
Select id, reputation, accountid
From users
Where reputation > 200000
AccountId is the key for all Stack Exchange sites.
I have found this query for aggregating across SEDE databases, but how is it possible to do that based on the dynamic results of the previous/baseline query?
Here is the kind of output I'm aiming for:
id_so, reputation_so, accounted, other_stackexchange_site_name, reputation_othersite, number_of_answers_other_site, number_of_questions_other_site
1, 250000, 23, serverfault, 500, 5, 1
1, 250000, 23, superuser, 120, 1, 0
2, 300000, 21, serverfault, 300, 3, 2
2, 300000, 21, webmasters, 230, 1, 1
3, 350000, 20, NA, NA, NA, NA
#the case with id 3 has an SO profile with reputation but it has no other profile in other Stack Exchange site
To run non-trivial queries across databases, based on an initial query:
Figure out the common key in all databases. In this case it's AccountId (which is a user's Stack-Exchange-wide Id).
Create your initial query to feed that key into a temp table. In this case:
CREATE TABLE #UsersOfInterest (AccountId INT)
INSERT INTO #UsersOfInterest
SELECT u.AccountId
FROM Users u
Where u.Reputation > 200000
Create Another temp table to hold the final results (see below).
Determine the query, to run on each site, that gets the info you want. EG:
SELECT u.AccountId, u.DisplayName, u.Reputation, u.Id
, numQst = (SELECT COUNT(q.Id) FROM Posts q WHERE q.OwnerUserId = u.Id AND q.PostTypeId = 1)
, numAns = (SELECT COUNT(q.Id) FROM Posts q WHERE q.OwnerUserId = u.Id AND q.PostTypeId = 2)
FROM Users u
WHERE u.AccountId = ##seAccntId##
Use a system query to get the appropriate databases. For the Data Explorer (SEDE), a query of this type:
SELECT name
FROM sys.databases
WHERE CASE WHEN state_desc = 'ONLINE'
THEN OBJECT_ID (QUOTENAME (name) + '.[dbo].[PostNotices]', 'U')
END
IS NOT NULL
Create a cursor on the above query and use it to step through the databases.
For each database:
Build a query string that takes the query of step 4 and puts it into the temp table of step 3.
Run the query string using sp_executesql.
When the cursor is done, perform the final query on the temp table from step 3.
Refer to this other answer, for a working template for querying all of the Stack Exchange sites.
Putting it all together, results in the following query, which you can run live on SEDE:
-- MinMasterSiteRep: User's must have this much rep on whichever site this query is run against
-- MinRep: User's must have this much rep on all other sites
CREATE TABLE #UsersOfInterest (
AccountId INT NOT NULL
, Reputation INT
, UserId INT
, PRIMARY KEY (AccountId)
)
INSERT INTO #UsersOfInterest
SELECT u.AccountId, u.Reputation, u.Id
FROM Users u
Where u.Reputation > ##MinMasterSiteRep:INT?200000##
CREATE TABLE #AllSiteResults (
[Master Rep] INT
, [Mstr UsrId] NVARCHAR(777)
, AccountId NVARCHAR(777)
, [Site name] NVARCHAR(777)
, [Username on site] NVARCHAR(777)
, [Rep] INT
, [# Ans] INT
, [# Qst] INT
)
DECLARE #seDbName AS NVARCHAR(777)
DECLARE #seSiteURL AS NVARCHAR(777)
DECLARE #sitePrettyName AS NVARCHAR(777)
DECLARE #seSiteQuery AS NVARCHAR(max)
DECLARE seSites_crsr CURSOR FOR
WITH dbsAndDomainNames AS (
SELECT dbL.dbName
, STRING_AGG (dbL.domainPieces, '.') AS siteDomain
FROM (
SELECT TOP 50000 -- Never be that many sites and TOP is needed for order by, below
name AS dbName
, value AS domainPieces
, row_number () OVER (ORDER BY (SELECT 0)) AS [rowN]
FROM sys.databases
CROSS APPLY STRING_SPLIT (name, '.')
WHERE CASE WHEN state_desc = 'ONLINE'
THEN OBJECT_ID (QUOTENAME (name) + '.[dbo].[PostNotices]', 'U') -- Pick a table unique to SE data
END
IS NOT NULL
ORDER BY dbName, [rowN] DESC
) AS dbL
GROUP BY dbL.dbName
)
SELECT REPLACE (REPLACE (dadn.dbName, 'StackExchange.', ''), '.', ' ' ) AS [Site Name]
, dadn.dbName
, CASE -- See https://meta.stackexchange.com/q/215071
WHEN dadn.dbName = 'StackExchange.Mathoverflow.Meta'
THEN 'https://meta.mathoverflow.net/'
-- Some AVP/Audio/Video/Sound kerfuffle?
WHEN dadn.dbName = 'StackExchange.Audio'
THEN 'https://video.stackexchange.com/'
-- Ditto
WHEN dadn.dbName = 'StackExchange.Audio.Meta'
THEN 'https://video.meta.stackexchange.com/'
-- Normal site
ELSE 'https://' + LOWER (siteDomain) + '.com/'
END AS siteURL
FROM dbsAndDomainNames dadn
WHERE (dadn.dbName = 'StackExchange.Meta' OR dadn.dbName NOT LIKE '%Meta%')
-- Step through cursor
OPEN seSites_crsr
FETCH NEXT FROM seSites_crsr INTO #sitePrettyName, #seDbName, #seSiteURL
WHILE ##FETCH_STATUS = 0
BEGIN
SET #seSiteQuery = '
USE [' + #seDbName + ']
INSERT INTO #AllSiteResults
SELECT
uoi.Reputation AS [Master Rep]
, ''site://u/'' + CAST(uoi.UserId AS NVARCHAR(88)) + ''|'' + CAST(uoi.UserId AS NVARCHAR(88)) AS [Mstr UsrId]
, [AccountId] = ''https://stackexchange.com/users/'' + CAST(u.AccountId AS NVARCHAR(88)) + ''?tab=accounts|'' + CAST(u.AccountId AS NVARCHAR(88))
, ''' + #sitePrettyName + ''' AS [Site name]
, ''' + #seSiteURL + ''' + ''u/'' + CAST(u.Id AS NVARCHAR(88)) + ''|'' + u.DisplayName AS [Username on site]
, u.Reputation AS [Rep]
, (SELECT COUNT(q.Id) FROM Posts q WHERE q.OwnerUserId = u.Id AND q.PostTypeId = 2) AS [# Ans]
, (SELECT COUNT(q.Id) FROM Posts q WHERE q.OwnerUserId = u.Id AND q.PostTypeId = 1) AS [# Qst]
FROM #UsersOfInterest uoi
INNER JOIN Users u ON uoi.AccountId = u.AccountId
WHERE u.Reputation > ##MinRep:INT?200##
'
EXEC sp_executesql #seSiteQuery
FETCH NEXT FROM seSites_crsr INTO #sitePrettyName, #seDbName, #seSiteURL
END
CLOSE seSites_crsr
DEALLOCATE seSites_crsr
SELECT *
FROM #AllSiteResults
ORDER BY [Master Rep] DESC, AccountId, [Rep] DESC
It gives results like:
-- where the blue values are hyperlinked.
Note that a user must have 200 rep on a site for it to be "significant". That's also the rep needed for the site to be included in the Stack Exchange flair.
I have the following query:
if object_id('tempdb..#tAJ88') is not null
drop table #tAJ88
create table #tAJ88 (
conv_raw_AJ88_ECO_key int,
case_id numeric(14,0),
account_key int,
account_period_key int,
aj_number varchar(25),
county_code varchar(25)
)
insert into #tAJ88(conv_raw_AJ88_ECO_key,account_key,account_period_key,aj_number,county_code)
select ac.conv_raw_AJ88_ECO_key,a.account_key, ap.account_period_key, ac.aj_number, ac.county_code
from [Conv].[dbo].[conv_raw_AJ88_ECO] ac
inner join [IT].[dbo].[entity_identifier] ei on ei.identifier_value = ac.account_number
and ei.identifier_type_key = #MITS
inner join [IT].[dbo].[account_x_entity_id] axe on axe.entity_identifier_key = ei.entity_identifier_key
inner join [IT].[dbo].[account] a on a.account_key = axe.account_key
and a.account_type_key = (select account_type_key from [IT].[dbo].[r_account_type] where code = ac.tax_type)
inner join [IT].[dbo].[account_period] ap on ap.account_key = a.account_key
and cnsd.NEXT_STEP_NAME not in ('A','B')
where (convert(datetime, substring(ac.periods,4,4) + '-' + substring(ac.periods,1,2) + '-01' ) >= ap.period_begin_dt and convert(datetime, substring(ac.periods,4,4) + '-' + substring(ac.periods,1,2) + '-01' ) <= ap.period_end_dt)
and len(rtrim(substring(ac.periods,4,4))) = 4
The query inserts the data from a select statement. The select statement itself only takes 1 second to run and only 1500 records appear in the select statement. However, when I try to insert into the temp table, I takes more than 10 minutes. I have never seen this issue before. Is this a tech issue where we don't have enough disk space or does it have to do with indexing which should not matter.
Is it possible you are having contention in tempdb? You can read about it here from Paul Randal: https://www.sqlskills.com/blogs/paul/the-accidental-dba-day-27-of-30-troubleshooting-tempdb-contention/
Have you tried doing this insert, but instead create a real table and do the insert? That would give you a clue if it was tempdb or not.
This question already has answers here:
TSQL Generate 5 character length string, all digits [0-9] that doesn't already exist in database
(6 answers)
Closed 7 years ago.
i want to create a coupon code generator by using SQL database but i don't know how to generate 1000 of random number without repeating them. so can someone help me it's important. thanks
Records in a relational database tables are unordered by nature.
therefor, you can simply create a table that has all the values between #First and #Last (0 and 9999 in your case), and then use a random order by when selecting from that table. you can also use a simple int in the database table and just format it when you select the data from the table.
Since my main database is Sql server, and I have no experience with sqlite, I will use Sql Server syntax in my code example, and leave it up to you to find the sqllite equivalent.
First, create the table:
CREATE TABLE Tbl
(
IntValue int PRIMARY KEY,
IsUsed bit NOT NULL DEFAULT 0
)
Then, populate it with numbers between 0 and 9999:
;With CTE AS (
SELECT 0 As IntValue
UNION ALL
SELECT IntValue + 1
FROM CTE
WHERE IntValue + 1 < 10000
)
INSERT INTO Tbl (IntValue)
SELECT IntValue
FROM CTE
OPTION(MAXRECURSION 0)
Then, you want to select multiple values each time, so I would write a stored procedure like this:
CREATE PROCEDURE stp_GetCouponCodes
(
#Number int = 5 -- or whatever number is default
)
AS
BEGIN
DECLARE #UsedValues AS TABLE
(
IntValue int
)
BEGIN TRY
BEGIN TRANSACTION
INSERT INTO #UsedValues
SELECT TOP(#Number) IntValue
FROM Tbl
WHERE IsUsed = 0
ORDER BY NEWID()
UPDATE Tbl
SET IsUsed = 1
FROM Tbl
INNER JOIN
#UsedValues uv ON(Tbl.IntValue = uv.IntValue)
SELECT RIGHT('00000' + CAST(IntValue as varchar), 5)
FROM #UsedValues
COMMIT TRANSACTION
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
ROLLBACK TRANSACTION
END CATCH
END
Then, when ever you want to generate coupons, simply execute the stored procedure with the number of coupons you want:
EXEC stp_GetCouponCodes 10;
See working fiddle example here.
The code below uses a quick method to generate 100 random 5-character strings based on the alphabet provided. You'll still need to perform duplicate checking, but this should get you started.
DECLARE #Quantity INT = 1000
DECLARE #Alphabet VARCHAR(100) = '0123456789'
DECLARE #Length INT = LEN(#Alphabet)
DECLARE #Top INT = SQRT(#Quantity) + 1
;WITH CTE AS (
SELECT TOP (#Top) *
FROM sys.objects
)
SELECT TOP (#Quantity)
SUBSTRING(#Alphabet, ABS(CHECKSUM(NEWID())) % #Length + 1, 1)
+ SUBSTRING(#Alphabet, ABS(CHECKSUM(NEWID())) % #Length + 1, 1)
+ SUBSTRING(#Alphabet, ABS(CHECKSUM(NEWID())) % #Length + 1, 1)
+ SUBSTRING(#Alphabet, ABS(CHECKSUM(NEWID())) % #Length + 1, 1)
+ SUBSTRING(#Alphabet, ABS(CHECKSUM(NEWID())) % #Length + 1, 1)
AS [Code]
FROM CTE X
CROSS JOIN CTE Y
I have an existing stored procedure. I have been asked to attempt to find a way to fit a specific set of logic into the procedure in order to avoid having to create a new one. However, I am not the best with SQL, but I would still like to do everything I can to accomplish my task.
My current goal: use the existing table generated from the select top 400 statement and somehow fit the update I wrote (second chunk of code) to work with that.
My existing procedure:
USE [cph]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER proc [dbo].[PatientSynch]
#EnvironmentKey varchar(1)
AS
BEGIN
DECLARE #patId VARCHAR(25)
select top 400
c.pat_id as cpPatId,
(LEFT(c.fname, 1)
+LEFT(c.lname, 1)
+ #EnvironmentKey
+ RIGHT('00000000'
+ convert(varchar,c.pat_id),8 )) AS PRN,
c.pref_meth_cont_cn as PreferredChannel
--,p.cppatid,p.prn
from
dbo.cppat c
left outer join dbo.patient p on c.pat_id=p.cppatid
where
p.cppatid is null or p.prn is null
order by
c.pat_id desc
END
The statement I have created to suit my needs:
UPDATE dbo.cppat
SET chart_id = CONVERT(VARCHAR(10), pat_id) + '+'
WHERE pat_id IN
(
SELECT pat_id
FROM cppat
)
I've added Chart_id in the query since you need it there to update it. This creates a common table expression that you can use to update records.
;WITH Update_Complex_Query AS
(
SELECT TOP 400
c.pat_id AS cpPatId
, (LEFT(c.fname, 1) + LEFT(c.lname, 1) + #EnvironmentKey + + RIGHT('00000000' + convert(VARCHAR, c.pat_id), 8)) AS PRN
, c.pref_meth_cont_cn AS PreferredChannel
--,p.cppatid,p.prn
, c.Chart_Id
FROM dbo.cppat c
LEFT JOIN dbo.patient p
ON c.pat_id = p.cppatid
WHERE p.cppatid IS NULL
OR p.prn IS NULL
ORDER BY c.pat_id DESC
)
UPDATE Update_Complex_Query
SET Chart_id = CONVERT(VARCHAR(10), cpPatId) + '+'
Can you give me some pointers (or point in the right direction on what search terms for google)? In a stored procedure I have a parameter #TAG (string). I receive '(14038314,14040071)' (for example) from another application that cannot be altered. In the stored procedure, I need to split apart '(14038314,14040071)' to put quotes around each string value, rebuild it, strip out the outer quotes,strip out the parens and pass it to #TAG in the query below so that it looks like the line commented out below?
SELECT
V.NAME AS VARIETY, TAGID
FROM
mfinv.dbo.onhand h
INNER JOIN
mfinv.dbo.onhand_tags t on h.onhand_id = t.onhand_id
INNER JOIN
mfinv.dbo.onhand_tag_details d on t.onhand_tag_id = d.onhand_tag_id
INNER JOIN
mfinv.dbo.FM_IC_PS_VARIETY V ON V.VARIETYIDX = d.VARIETYIDX
LEFT JOIN
mfinv.dbo.FM_IC_TAG TG ON TG.TAGIDX = t.TAGIDX
WHERE
h.onhand_id = (SELECT onhand_id FROM mfinv.dbo.onhand
WHERE onhand_id = IDENT_CURRENT('mfinv.dbo.onhand'))
AND TG.ID IN (#TAG)
--AND TG.ID IN ('14038314','14040071')
You can Use Dynamic SQL Like This
DECLARE #TAG Nvarchar(MAX)='14038314,14040071'
set #TAG=''''+REPLACE(#TAG,',',''',''')+''''
--select #TAG
DECLARE #SQL NVARCHAR(MAX)=N'
Select V.NAME AS VARIETY, TAGID
FROM mfinv.dbo.onhand h
INNER JOIN mfinv.dbo.onhand_tags t on h.onhand_id = t.onhand_id
INNER JOIN mfinv.dbo.onhand_tag_details d on t.onhand_tag_id = d.onhand_tag_id
INNER JOIN mfinv.dbo.FM_IC_PS_VARIETY V ON V.VARIETYIDX = d.VARIETYIDX
LEFT JOIN mfinv.dbo.FM_IC_TAG TG ON TG.TAGIDX = t.TAGIDX
WHERE h.onhand_id = (SELECT onhand_id FROM mfinv.dbo.onhand
WHERE onhand_id = IDENT_CURRENT (''mfinv.dbo.onhand''))
AND TG.ID IN ('+#TAG+')'
PRINT #SQL
EXEC (#SQL)
Here's what I did. Thank you all for responding. Thanks to dasblinkenlight for answering "How to replace first and last character of column in sql server?". Thanks to SQLMenace for answering "How Do I Split a Delimited String in SQL Server Without Creating a Function?".
Here's how I removed parenthesis:
#Tag nvarchar(256)
SET #Tag = SUBSTRING(#Tag, 2, LEN(#Tag)-2)
Here's how I split and rebuilt #Tag:
AND TG.ID in
(
SELECT SUBSTRING(',' + #Tag + ',', Number + 1,
CHARINDEX(',', ',' + #Tag + ',', Number + 1) - Number -1)AS VALUE
FROM master..spt_values
WHERE Type = 'P'
AND Number <= LEN(',' + #Tag + ',') - 1
AND SUBSTRING(',' + #Tag + ',', Number, 1) = ','
)