I'm exporting data out to a flat file to import into a SIEM. Is there a way to display as "column name = data" for each item?
SELECT
[Description], [MessageDescription], [CardNumber],
[TimeStamp_Server], [SPMDescription] [CardHolderID],
[FirstName], [MiddleName], [LastName],
[CardStatusID], [CardStatusDescription], [Imprint],
[TransactionNumber]
FROM
[DB].[dbo].[Message]
WHERE
CONVERT(varchar(10), TimeStamp_Server, 120) = CONVERT(varchar(10), GETDATE(), 120)
Here is how it currently presents in the flat file.
Description,MessageDescription,CardNumber,TimeStamp_Server,CardHolderID,FirstName,MiddleName,LastName,CardStatusID,CardStatusDescription,Imprint,TransactionNumber
North Entry,AccessGrantedNoEntry,0,2023-02-08 09:52:19,Retail Center C004 Converted PXL250-2DOOR,,,,,,,527312
I'd like it to display as this for each row
Description = North Entry,
MessageDescription = AccessGrantedNoEntry,
CardNumber = 0,
TimeStamp_Server = 2023-02-08
... and so on.
This is a side issue (so community wiki), but you can significantly improve performance of this query by changing the WHERE clause like this (assuming TimeStamp_Server is a DateTime column):
WHERE TimeStamp_Server >= cast(cast(getdate() as date) as datetime)
AND TimeStamp_Server < cast(dateadd(day, 1, cast(getdate() as date)) as datetime)
This helps in three ways:
Thanks to cultural/internationalization issues, converting dates to and from string values is far slower and more error-prone than we'd like to believe. Sticking with Date functions and types will always perform better and be more accurate.
By shifting all the modifications to getdate(), so TimeStamp_Server is unaltered, we avoid needing to do the conversion on every row in the table.
By shifting all the modifications to getdate(), so TimeStamp_Server is unaltered, we preserve the use of any index that might exist for the column. This cuts to the core of database performance.
With a bit of JSON and string_agg()
Select B.NewVal
From (
--Your Query Here--
) A
Cross Apply (
Select NewVal = string_agg(concat([key],' = ',value),',')
From openjson( (Select A.* For JSON Path,Without_Array_Wrapper ) )
) B
A minimal reproducible example is not provided. So, I am shooting from the hip.
Please try the following conceptual example.
IMHO, the JSON or XML output format would be much more reliable.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (id INT PRIMARY KEY, case_date DATE, Product INT);
INSERT INTO #tbl (id, case_date, Product) VALUES
(55, '2022-08-01', 11),
(66, '2022-05-21', 51);
-- DDL and sample data population, end
SELECT t.*
, result = STUFF(XMLData.query('
for $x in /root/*
return concat(", ", local-name($x), "=", $x/text()[1])
').value('.', 'VARCHAR(4096)'), 1,2,'')
FROM #tbl AS t
CROSS APPLY (SELECT t.* FOR XML PATH(''), TYPE, ROOT('root')) AS t1(XMLData);
Output
id
case_date
Product
result
55
2022-08-01
11
id=55 , case_date=2022-08-01 , Product=11
66
2022-05-21
51
id=66 , case_date=2022-05-21 , Product=51
Thank you #Stu for providing what I was seeking in a comment:
So you want every column name reproduced on every row? That's going to significantly increase the size of the file! Just use
select concat('columnName = ', ColumnName)Columename, ...
Related
I am auditing values in 2 identical structure tables. The T-SQL EXCEPT statement is ignoring the trailing space on a value in one table, so the values don't match, but also do not show up in our audit.
I have tried searching for ways to change how SQL is comparing the columns. I did something similar to ensure it was case sensitive, but couldn't find something that would make it include the white space/padding in the field value.
Example data would have the value in MyTable as "Product Name ", while the RemoteTable has the value "Product Name".
To quickly reproduce, here is a slimmed down version of what I'm doing now:
DECLARE #SampleLocal TABLE(ProductName varchar(50))
DECLARE #RemoteTable TABLE(ProductName varchar(50))
INSERT INTO #SampleLocal (ProductName) VALUES ('Product Name')
INSERT INTO #RemoteTable (ProductName) VALUES ('Product Name ')
SELECT ProductName COLLATE SQL_Latin1_General_CP1_CS_AS ProductName
FROM #SampleLocal
EXCEPT
SELECT ProductName COLLATE SQL_Latin1_General_CP1_CS_AS ProductName
FROM #RemoteTable
This currently returns no results, showing that the values are the same. But the value in the second table has a space at the end.
I would expect to get a result back that has "Product Name"
When I needed to compare things with case sensitivity I was able to add
COLLATE SQL_Latin1_General_CP1_CS_AS
Is there something similar that would show the value being different because of the blank space?
According to this article (https://support.microsoft.com/en-us/help/316626/inf-how-sql-server-compares-strings-with-trailing-spaces) :
The ANSI standard requires padding for the character strings used in comparisons so that their lengths match before comparing them. The padding directly affects the semantics of WHERE and HAVING clause predicates and other Transact-SQL string comparisons. For example, Transact-SQL considers the strings 'abc' and 'abc ' to be equivalent for most comparison operations.
This behavior is intended.
You can use a slower method to achieve what you wanted:
SELECT innerItems.ProductName
FROM
(
SELECT DATALENGTH(ProductName) as realLength, ProductName COLLATE SQL_Latin1_General_CP1_CS_AS as ProductName
FROM #SampleLocal
EXCEPT
SELECT DATALENGTH(ProductName) as realLength, ProductName COLLATE SQL_Latin1_General_CP1_CS_AS as ProductName
FROM #RemoteTable
) innerItems
Comparing the values and real lengths together does the magic here. (The len method would give the 'wrong' result in this case)
This is old, and already answered, but I was struggling with this as a result of comparison in a JOIN condition too. While it is true that SQL Server ignores trailing spaces, it doesn't ignore leading spaces. Therefore for my JOIN condition, I compared forwards and backwards (using the reverse function) and that gave me a more accurate set of results.
In the example below, I want rows with a trailing space in one table to only match rows with a trailing space in the JOINed table. The REVERSE function helped with this.
DECLARE #DbData AS TABLE (
Id INT,
StartDate DATETIME,
OrgName VARCHAR(100)
)
DECLARE #IncomingData AS TABLE (
Id INT,
StartDate DATETIME,
OrgName VARCHAR(100)
)
INSERT INTO #DbData (Id, StartDate, OrgName)
SELECT 1, CAST('1 Jan 2022' AS DATE), 'Test ' UNION ALL
SELECT 2, CAST('1 Jan 2022' AS DATE), 'Test' UNION ALL
SELECT 3, CAST('1 Jan 2022' AS DATE), 'Other Test' UNION ALL
SELECT 4, CAST('1 Jan 2022' AS DATE), 'Other Test '
INSERT INTO #IncomingData (Id, StartDate, OrgName)
SELECT 1, CAST('1 Jan 2022' AS DATE), 'Test ' UNION ALL
SELECT 2, CAST('1 Jan 2022' AS DATE), 'Test' UNION ALL
SELECT 3, CAST('1 Jan 2022' AS DATE), 'Other Test'
SELECT '~' + dd.OrgName + '~', '~' + id.OrgName + '~', *
FROM #DbData dd
JOIN #IncomingData id ON id.StartDate = dd.StartDate
AND dd.OrgName = id.OrgName
AND REVERSE(dd.OrgName) = REVERSE(id.OrgName) -- Try the query with and without this line to see the difference it makes
Data example:
AtrxId AtrxDate AtrxTime AtrxDes
------------------------------------------
CAS-000001 05-03-2018 12:43 PM Cash
INV-000001 05-03-2018 11:04 AM Credit
I need the output sorted according to time in a particular day:
AtrxId AtrxDate AtrxTime AtrxDes
-----------------------------------------
INV-000001 05-03-2018 11:04 AM Credit
CAS-000001 05-03-2018 12:43 PM Cash
1) Why is it nvarchar? Just asking... this will likely cause many problems for you down the line, unless you need it to be nvarchar because it needs to sometimes accept invalid dates.
This should work. This assumes that both AtrxDate and AtrxTime are nvarchar. It also assumes that there are always valid dates and times in the fields. If either are not valid, then the TRY_CONVERT will return null, which basically means that those will be sorted first:
SELECT
AtrxId
, AtrxDate
, AtrxTime
, AtrxDes
FROM tablenamehere t
ORDER BY TRY_CONVERT(AtrxDate + ' ' + AtrxTime AS datetime) DESC
I'm not sure if you want it listed in forward or backward order by date/time (the example is listed in forward order, but the question's subject line says DESC), but if you want it in forward order, it's just a matter of replacing DESC with ASC. :)
You can try the following query. According to your sorting requirement, put "ASC" or "DESC"
DECLARE #TABLE TABLE (AtrxId VARCHAR(100), AtrxDate DATE, AtrxTime TIME, AtrxDes VARCHAR(100))
INSERT INTO #TABLE
SELECT 'CAS-000001', '05-03-2018', '12:43 PM', 'Cash' UNION ALL SELECT 'INV-000001','05-03-2018','11:04 AM','Credit'
SELECT * FROM #TABLE ORDER BY AtrxDate,AtrxTime
Or you can try the following select query
SELECT * FROM #TABLE ORDER BY CONVERT(DATETIME, CONVERT(CHAR(8), AtrxDate, 112)
+ ' ' + CONVERT(CHAR(8), AtrxTime, 108))
I have a table with a VARCHAR field called ArrivalDate in format yymmdd (such as 170202).
I am writing a query which converts it to yyyymmdd so it should become 20170202.
However my problem is that I need to cater for the case when inappropriate data is entered into the field, and my query needs to exclude that data. I am achieving this exclusion by using the ISDATE function of TSQL. I also need to select the least recent entry (I'm using order by asc for this).
I am using a variety of converts to write this query, below is my implementation with a sample table and data.
Declare #tmp TABLE (theDates VARCHAR(MAX))
INSERT INTO #tmp VALUES('170202')
SELECT TOP 1 t.theDates
WHEN (ISDATE(t.theDates) = 1) THEN CONVERT( VARCHAR(max),CONVERT(datetime t.theDates), 112)
FROM #tmp t
WHERE (ISDATE(t.theDates) = 1)
ORDER BY CAST(t.theDates as DATE)
However I do not like my approach and it occasionally fails conversion and throws an error with values such as 02/02/02 which breaks the query. Can someone please show me a better way of writing this functionality.
Much appreciated!
You can use TRY_CONVERT and CONVERT to get the correct format and convert the value. Then check that the string is exactly 6 character to prevent other formats from being returned.
SELECT
convert(char(10),convert(date, theDates, 12),112)
FROM
(values('02/02/02'),('170202')) x(theDates)
WHERE
try_convert(date, theDates, 12) is not null
and len(theDates) = 6
You can use cast(#date as datetime)
declare #date varchar(max);
set #date='170202';
select
CASE WHEN (ISDATE(cast(#date as datetime)) = 1)
THEN CONVERT(VARCHAR(max), CONVERT(datetime, cast(#date as datetime)), 112) end
from table
set #date='02/02/02';
select
CASE WHEN (ISDATE(cast(#date as datetime)) = 1)
THEN CONVERT(VARCHAR(max), CONVERT(datetime, cast(#date as datetime)), 112) end
from table
please use create function for check dateformat is Valid or not and use this fun in your query inside cash clouse.
ALTER FUNCTION dbo.f_CheckDate
(#InDate nvarchar(25))
RETURNS DATE
AS
BEGIN
declare #Return DATETIME
select #return = CASE WHEN ISDATE(#InDate) = 1
THEN #InDate
ELSE NULL
END
return #return
END
You could use TRY_CAST or TRY_CONVERT if value cannot be cast it will return NULL.
SELECT
TRY_CAST('20170228' AS DATETIME),
TRY_CAST('170228' AS DATETIME),
TRY_CONVERT(DATETIME, '20170228'),
TRY_CONVERT(DATETIME, '170228')
This works for SQL Server 2012 and newer.
I am creating a web app in which I am executing a select command on my stored procedure, but I want to insert the same fetched data into another table.
So I tried to do something like the following
CREATE PROCEDURE profinalinstexpensesonid
(#from varchar(5000),
#to varchar(5000),
#trainer varchar(5000),
#sonvinid varchar(5000)
)
AS
BEGIN
INSERT INTO invoice(sonvinid, tid, date, brandname, zone, location, area, venuename, venue, instructore, amount)
SELECT
instructoreexpense.sonvinid,
sonvininsert.trainer,
CONVERT(VARCHAR, sonvininsert.date, 105) AS date,
sonvininsert.brandname,
SUBSTRING(sonvininsert.zone, 1, 1) AS zone,
sonvininsert.location,
sonvininsert.area,
companysonvinunitvenue.venuename,
sonvininsert.venue,
sonvininsert.instructore,
instructoreexpense.amount
FROM
instructoreexpense
LEFT OUTER JOIN
sonvininsert ON sonvininsert.sonvinid = instructoreexpense.sonvinid
AND sonvininsert.status = '0'
LEFT OUTER JOIN
finalinstructoreexpense ON finalinstructoreexpense.sonvinid = instructoreexpense.sonvinid
LEFT OUTER JOIN
companysonvinunitvenue ON companysonvinunitvenue.id = sonvininsert.comsonvinid
WHERE
sonvininsert.date BETWEEN CONVERT(DATETIME, #from, 105)
AND CONVERT(DATETIME, #to, 105)
AND sonvininsert.trainer = (SELECT empname
FROM trainerdetails
WHERE trid = #trainer)
AND instructoreexpense.sonvinid NOT IN (SELECT CAST(Item AS INTEGER)
FROM SplitString(#sonvinid, ','))
ORDER BY
instructoreexpense.sonvinid
END
and when I execute the stored procedure like
exec profinalinstexpensesonid '01-01-2013','01-01-2017','andrews'
I am getting the following error
Msg 8152, Level 16, State 13, Procedure profinalinstexpensesonid, Line 10
String or binary data would be truncated.
On my line 10 I have the following code
insert into invoice(sonvinid, tid, date, brandname, zone, location, area, venuename, venue, instructore, amount)
I don't know what is wrong here?
The error message states the size of a column in invoice table is less compared to the size of the data being inserted into it.
For example if column brandname has data type varchar(50) and you are trying to insert more than 50 characters then it will cause error.
To resolve this compare the size of columns in invoice with the size of the columns being inserted.
You need to check column size of invoice table as well as columns in select list from which you are populating data.
Let's say you are inserting column "B" having data type as varchar(70) from table2 in column "A" having data type varchar(50) in table1; this won't work as you are trying to insert 70 characters in 50 varchar sized column.
Check source & destination column data type & it's length; and change it and try again.
I am trying to get data after year 2012.
Date is saved in nvarchar format in a table. For example: 12/31/2010
Column also has some other values like 'Confidential', I don't want this row.
I am trying a query (shown below) but it is not succeed :-
select *
from tbl_ProductionWells
where CONVERT(NVARCHAR(10), wellstatusdate, 103) > CONVERT(NVARCHAR(10), '01/01/2012', 103)
Edited :-
I tried this :-
SELECT *
FROM tbl_ProductionWells
WHERE DATEPART(YEAR, CAST(wellstatusdate AS date)) > 2012
But it is giving an error (shown below), This column also has some text values like 'not available','Confidential' .. :-
Msg 241, Level 16, State 1, Line 1
Conversion failed when converting date and/or time from character string.
Note:- I can't change column datatype as it also contains some other texts.
Thanks in advance
First of all: Store date values in DATE columns, datetimes in DATETIME2 columns. Always choose proper data type for your data
You have to convert your NVARCHAR to DATE, then compare it to 2012-01-01
OR you can extract the 'year' part of your string.
SELECT *
FROM tbl_ProductionWells
WHERE CONVERT(DATE, wellstatusdate) >= '2012-01-01'
The best choice is to change your column's data type to DATE. After that, you can do lots of magicial things with those values. Store the 'Confidental' flag in another column.
EDIT
Some additional info:
Please note, that the STRING -> DATE conversion depends on the current session's language.
Run this batch to see the difference:
DECLARE #DateAsChar VARCHAR(32) = '01/02/12';
SET LANGUAGE us_english
SELECT CONVERT(VARCHAR(32), CONVERT(DATE, #DateAsChar), 120)
SET LANGUAGE Hungarian
SELECT CONVERT(VARCHAR(32), CONVERT(DATE, #DateAsChar), 120)
SET LANGUAGE Deutsch
SELECT CONVERT(VARCHAR(32), CONVERT(DATE, #DateAsChar), 120)
How about:
WITH cte
AS ( SELECT *
FROM tbl_ProductionWells
WHERE ISDATE(wellstatusdate) = 1
)
SELECT *
FROM cte
WHERE DATEPART(YEAR, CAST(wellstatusdate AS DATE)) > 2012
Select all data from the table that is a date using IsDate, then work with that dataset only.
SELECT * FROM
(SELECT * FROM tbl_ProductionWells WHERE ISDATE(wellstatusdate) = 1)
WHERE CAST(wellstatusdate as Date) > #YOURDATE