Repro steps:
Create a stream on table.
Create a view on Stream.
Desc view.
Observation
Length of column METADATA$ACTION is VARCHAR(1) but when I select the length of column[length(METADATA$ACTION) ] it is coming as 6(value can be INSERT, UPDATE and DELETE).
Can someone please help to explain this discrepancy?
Related
I have a stored procedure that performs a bulk insert of a large number of DNS log entries. I wish to summarise this raw data in a new table for analysis. The new table takes a given log entry for FQDN and Record Type and holds one record only with a hitcount.
Source table might include 100 rows of:
FQDN, Type
www.microsoft.com,A
Destination table would have:
FQDN, Type, HitCount
www.microsoft.com, A, 100
The SP establishes a unique ID made up of [FQDN] +'|'+ [Type], which is then used as the primary key in the destination table.
My plan was to have the SP fire a trigger that did an UPDATE...IF ##ROWCOUNT=0...INSERT. However, that of course failed because the trigger receives all the [inserted] rows as a single set so always throws a key violation error.
I'm having trouble getting my head around a solution and need some fresh eyes and better skills to take a look. The bulk insert SP works just fine and the raw data is exactly as desired. However trying to come up with a suitable method to create the summary data is beyond my present skills/mindset.
I have several 10s of Tb of data to process, so I don't see the summary as a something we could do dynamically with a SELECT COUNT - which is why I started down the trigger route.
The relevant code in the SP is driven by a cursor consisting of a list of compressed log files needing to be decompressed and bulk-inserted, and is as follows:
-- Bulk insert to a view because bulk insert cannot populate the UID field
SET #strDynamicSQL = 'BULK INSERT [DNS_Raw_Logs].[dbo].[vwtblRawQueryLogData] FROM ''' + #strTarFolder + '\' + #strLogFileName + ''' WITH (FIRSTROW = 1, FIELDTERMINATOR = '' '', ROWTERMINATOR = ''0x0a'', ERRORFILE = ''' + #strTarFolder + '\' + #strErrorFile + ''', TABLOCK)'
--PRINT #strDynamicSQL
EXEC (#strDynamicSQL)
-- Update [UID] field after the bulk insert
UPDATE [DNS_Raw_Logs].[dbo].[tblRawQueryLogData]
SET [UID] = [FQDN] + '|' + [Type]
FROM [tblRawQueryLogData]
WHERE [UID] IS NULL
I know that the UPDATE...IF ##ROWCOUNT=0...INSERT solution is wrong because it assumes that the input data is a single row. I'd appreciate help on a way to do this.
Thank you
First, at that scale make sure you understand columnstore tables. They are very highly compressed and fast to scan.
Then write a query that reads from the raw table and returns the summarized
create or alter view DnsSummary
as
select FQDN, Type, count(*) HitCount
from tblRawQueryLogData
group by FQDN, Type
Then if querying that view directly is too expensive, write a stored procedure that loads a table after each bulk insert. Or make the view into an indexed view.
Thanks for the answer David, obvious when someone else looks at it!
I ran the view-based solution with 14M records (about 4 hours worth) and it took 40secs to return, so I think i'll modify the SP to drop and re-create summary table each time it runs the bulk insert.
The source table also includes a timestamp for each entry. I would like to grab the earliest and latest times associated with each UID and add that to the summary.
My current summary query (courtesy of David) looks like this:
SELECT [UID], [FQDN], [Type], COUNT([UID]) AS [HitCount]
FROM [DNS_Raw_Logs].[dbo].tblRawQueryLogData
GROUP BY [UID], [FQDN], [Type]
ORDER BY COUNT([UID]) DESC
And returns:
UID, FQDN, Type, HitCount
www.microsoft.com|A, www.microsoft.com, A, 100
If I wanted to grab first earliest and latest times then I think I'm looking at nesting 3 queries to grab the earliest time (SELECT TOP N...ORDER BY... ASC), the latest time (SELECT TOP N...ORDER BY... DESC) and the hitcount. Is there a more efficient way of doing this, before I try and wrap my head around this route?
I need to concatenate the job message descriptions by ett_date and create a new table. After this I am going to operate a different step. My problem is that while i am creating table with WITH DATA CLAUSE, I am getting ORA-22993: specified input amount is greater than actual source amount error. Could you help me about how to solve this problem ?
create table mytable as
WITH DATA AS(
SELECT
ett_date,JOB_NAME,PLAN_NAME,JOB_PERSON1_INFO,JOB_PERSON2_INFO,JOB_PROCEDURE
,RTRIM(XMLAGG(XMLELEMENT(e,REGEXP_REPLACE(job_message_desc, '[^0-9A-Za-z]', ' ') , ' ',',').EXTRACT('//text()')).getCLOBVal() ,',') a
from mytable group by ett_date,JOB_NAME,PLAN_NAME,JOB_PERSON1_INFO,JOB_PERSON2_INFO,JOB_PROCEDURE)
select *from DATA
TablesIs there a way for me to have a String and check within multiple columns in another table in order starting from 1 until I get a match?
I have table with a few fields
Medicine
-----------
Advil
Tylenol
Midol
I need to check it against another table and check column in order for the medicine above.
MedsToTry1 | MedsToTry2 | MedsToTry3 | MedsToTry4 | MedsToTry5 | MedsToTry6 |
------------|------------|------------|------------|------------|------------|
NotAdvil Advil Null Null Null Null
Tylenol Ibuprofen NotTylenol Null Null Null
NotMidol NotAdvil Ibuprofen Midol Null Null
So I have to go through each one of the fields in the first table and search for them in the 'MedsToTry1' field if not there then on 'MedsToTry2' and so on until found.
I've tried concatentation on all the strings in the MedsToTry fields and searching for the string in there but it doesn't guarantee that it'll be in order and I need for 'MedsToTry1' to be checked first.
I tried to use COALESCE but it returns the fields on MedsToTry1 since they're all not null but won't go to MedsToTry2 to see if it's there.
Is there a way for me to do this? Have a String and check within multiple columns in another table in order starting from 1 until I get a match?
If I need to provide more information please let me know. I pretty new to SQL so I'm take any and all help I can get.
Thank you.
Your table is - quite probably - not the real source table. If you can query against this, it was much better. This is - again my guessing - the result of a pivot operation, where a row-wise table is transformed to a side-by-side or column-wise format.
You did not state the expected output. For the next time, or to improve this question, please look at my code and try to prepare a stand-alone example to reproduce your issue yourself. This time I've done it for you:
First we have to declare mockup-tables to simulate your issue:
DECLARE #tblA TABLE(Medicine VARCHAR(100));
INSERT INTO #tblA VALUES
('Advil')
,('Tylenol')
,('Midol');
DECLARE #tblB TABLE(RowId INT,MedsToTry1 VARCHAR(100),MedsToTry2 VARCHAR(100),MedsToTry3 VARCHAR(100),MedsToTry4 VARCHAR(100),MedsToTry5 VARCHAR(100),MedsToTry6 VARCHAR(100));
INSERT INTO #tblB VALUES
(1,'NotAdvil','Advil',Null,Null ,Null,Null)
,(2,'Tylenol','Ibuprofen','NotTylenol',Null ,Null,Null)
,(3,'NotMidol','NotAdvil','Ibuprofen','Midol',Null,Null);
--This is the query (use your own table names to test this in your environment)
SELECT B.RowId
,C.*
FROM #tblB b
CROSS APPLY (VALUES(1,b.MedsToTry1)
,(2,b.MedsToTry2)
,(3,b.MedsToTry3)
,(4,b.MedsToTry4)
,(5,b.MedsToTry5)
,(6,b.MedsToTry6)) C(MedRank,Medicin)
WHERE EXISTS(SELECT 1 FROM #tblA A WHERE A.Medicine=C.Medicin);
The idea in short:
The trick with CROSS APPLY (VALUES... will return each name-numbered column (MedsToTry1, MedsToTry2...) in one row, together with a rank. This way we do not lose the information of the sort order or the position within the table.
The WHERE will reduce the set to rows, where there exists a corresponding medicine in the other table.
The result
RowId MedRank Medicin
1 2 Advil
2 1 Tylenol
3 4 Midol
I have inserted data from a sql trace in a table and I have a problem interpreting the data in table.
In the attached image the reads related to my SP are very low but on the next line for a value of NULL in Text Data column the reads are very high.
How do i interpret this.? Why the NULL rows have so high read values?
Edit: I have updated the image file. Now it has all the column names for initial 10 rows of my trace I could not find any EventType column, but there is an EventClass column which has value : 15 for every NULL row.
Screenshot
Check out the SQL Server Event Class reference. You determine the EventType by EventClass value. Some EventClass types come with a NULL value for TextData.
Also, here's a query that might help you out mapping the EventClass ID to the actual event type:
SELECT te.name
FROM dbo.Trace t
JOIN sys.trace_events te ON t.EventClass = te.trace_event_id
where dbo.Trace is the table where you save the EventClass values.
I have a table in sql server 2005. Yesterday an update was performed on the table and then updated back to the origional record.
Now i want to figure out what the updated values was. At least i want to learn if it was really updated or not.
Is there someway to determine what i want from the transaction logs?
Thanks.
There is an undocumented command:
DBCC log ( dbname, 0|1|2|3|4 )
where
0: minimum information (Default)
1: Returns info available using 0 +
flags, tags and the log record length.
2: Returns info available using
1 + object, index, page ID and slot ID.
3: Maximum information about
each operation.
4: Maximum information about each operation +
hexadecimal dump of the current transaction log row
And read this: Looking for a SQL Transaction Log file viewer
Although is not a precise method (ex. you have data only from the last SQL Server restart), you can try to use sys.dm_exec_query_stats view:
CREATE TABLE dbo.LongTableName (ID INT IDENTITY(1,1) PRIMARY KEY, Column1 VARCHAR(10) NOT NULL);
INSERT LongTableName VALUES ('A');
INSERT LongTableName VALUES ('BB');
INSERT LongTableName VALUES ('CCC');
WAITFOR DELAY '00:00:05';
INSERT LongTableName VALUES ('DDDD');
GO
SELECT ca.[text], s.last_execution_time, s.last_logical_reads, s.last_logical_writes, s.execution_count
FROM sys.dm_exec_query_stats s
CROSS APPLY sys.dm_exec_sql_text(s.sql_handle) ca
WHERE ca.[text] LIKE '%INSERT%LongTableName%'
GO
DROP TABLE LongTableName;
GO
For example, one of the records will be:
text last_execution_time last_logical_reads last_logical_writes execution_count
(#1 varchar(8000))INSERT INTO [LongTableName] values(#1)
2011-10-04 10:51:17.070 2 0 4