Use findstr to replace null values with empty strings from sqlcmd command - sql-server

I have an sqlcmd command which generates a csv file from a view. Is it possible to replace the NULL values to empty string using the findstr command?
Here is what I tried.
sqlcmd -S . -d SAMPLEDB -U sa -P pass -s"|" -W -Q "SET NOCOUNT ON SET ANSI_WARNINGS OFF select * from view_Table" > Sample.csv -h -1 | findstr /v /c:"NULL"

You can easily build the extraction SQL for each view using the system management views. This simple query:
SELECT v.[name]
,c.[name]
,c.[column_id]
,c.[is_nullable]
FROM sys.views V
INNER JOIN sys.columns C
ON V.[object_id] = C.[object_id];
will return everything we need to perform the task:
the view name
the column name
the column order
if the column is nullable
So, we need only to build the extraction SQL statements:
SELECT v.[name]
,'SELECT ' + DS.[definition] + ' FROM ' + v.[name]
FROM sys.views V
CROSS APPLY
(
SELECT STUFF
(
(
SELECT ',' + CASE WHEN c.[is_nullable] = 1 THEN 'ISNULL(' + c.[name] + ','''')' ELSE c.[name] END
FROM sys.columns C
WHERE V.[object_id] = C.[object_id]
ORDER BY c.[column_id]
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1
,1
,''
)
) DS ([definition]);
Depending on your SQL version you can reduce the code above - for example using IIF or STRING_AGG.
Also, you can add WHERE clause to filter the query for specific views.

Change the sql in the command string to select fields individually and do an ISNULL on them
select * from view_Table
To
select ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') from view_Table
in you calling string
use the following code to build it and check to see the final statement runs
DECLARE #Temp VARCHAR(MAX)
SET #Temp = 'SELECT ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') FROM view_Table'
SELECT #Temp

Related

How to get all table counts from database in JSON output in TSQL script

I would like to have a TSQL script to output all table counts from a SQL Server database to a JSON file. Getting the table counts is not the problem, but outputting this to JSON is the issue. I can't get it to work.
This is what I have to get the table counts:
USE databasename
GO
SELECT
QUOTENAME(SCHEMA_NAME(sOBJ.schema_id)) + '.' + QUOTENAME(sOBJ.name) AS [TableName]
, SUM(sPTN.Rows) AS [RowCount]
FROM
sys.objects AS sOBJ
INNER JOIN sys.partitions AS sPTN
ON sOBJ.object_id = sPTN.object_id
WHERE
sOBJ.type = 'U'
AND sOBJ.is_ms_shipped = 0x0
AND index_id < 2 -- 0:Heap, 1:Clustered
GROUP BY
sOBJ.schema_id
, sOBJ.name
ORDER BY [TableName]
GO
How can I output the results to JSON?
To convert a simple resultset to Json, add for json auto to the end of the statement eg,
...GROUP BY
sOBJ.schema_id
, sOBJ.name
ORDER BY [TableName]
FOR JSON AUTO
See this example DB<>Fiddle

How to add a variable in the open query in a stored procedure

I have an stored procedure that receives a parameter of store id. I need that variable to be added in a openquery tsql script I am not able to do it.. Look in my code where it says #var thats where the variable will need to work
I tried to reference to this https://support.microsoft.com/en-us/help/314520/how-to-pass-a-variable-to-a-linked-server-query but I cant get it to work
SELECT x.*
FROM (
SELECT tl.title_log_sts_cd
,tl.ro_vin
,tl.ro_store_id
,ll.CurrentLoanStatus
,bc.vin
,bc.BorrowerId
,ll.DisplayLoanNumber
,CONVERT(DATE, CONVERT(VARCHAR(10), ll.CreateDateKey, 7)) LoanDate
,CONCAT (
bb.LastName
,','
,bb.FirstName
) CustomerName
,ll.CurrentPrincipalBalanceAmt + ll.CurrentFeeBalanceAmt TotalDue
FROM OPENQUERY(TLXPRD, 'SELECT * FROM TITLE_LOG
WHERE title_log_sts_cd in (''Ready to Send to DMV'', ''Sent to DMV'')
and ro_store_id =#VAR ') AS tl
JOIN EIS.Borrower.Collateral bc WITH (NOLOCK) ON tl.ro_vin = bc.Vin
LEFT JOIN eis.loan.loan ll WITH (NOLOCK) ON ll.BorrowerId = bc.BorrowerId
AND convert(DATE, tl.created_ts) = CONVERT(DATE, CONVERT(VARCHAR(10), ll.CreateDateKey, 7))
LEFT JOIN EIS.Borrower.Borrower BB ON bb.borrowerid = ll.BorrowerId
) AS x
ORDER BY vin
Expected get some data from oracle join it in with tsql
I didn't totally get what you are trying to do but I hope this is what you are looking for.
FROM OPENQUERY(TLXPRD, concat( 'SELECT * FROM TITLE_LOG
WHERE title_log_sts_cd in (''Ready to Send to DMV'', ''Sent to DMV'')
and ro_store_id =', cast(#VAR as nvarchar(50) ) ))
basically, you had your variable in your string but you want to have it as a value.
so we get the value and put it in your string.

passing multiple datatypes into dynamic sql-values not passing in parameterized query [duplicate]

This question already has answers here:
Dynamic SQL Not Converting VARCHAR To INT (shouldn't anyway)
(2 answers)
Closed 4 years ago.
I have a dynamic SQL query inside a stored procedure that works and gives me the correct results. But it is taking too long-because I have to compare as varchar instead of int. I believe #query variable in SQL server requires the statement to be a unicode.
Here is the dynamic sql part
ALTER PROCEDURE [dbo].[sp_GetRows]( #Id varchar(64))
AS
BEGIN
DECLARE #Query nvarchar(4000),
#Comp varchar(256)
SELECT #Comp
= STUFF((
SELECT DISTINCT ',' + char(39)+
tci.Component +char(39)
FROM TCI tci WITH(NOLOCK)
JOIN CDetail cd WITH(NOLOCK)
ON tci.ParentCId = cd.CIdentifier
WHERE tci.ParentCId = #Id
AND cd.ParentBranch IS NULL
FOR XML PATH('')),1,1,'')
SET #Query
= 'WITH CTE AS
(
SELECT '+#Id+' as ParentCId, CIdentifier as ChildCId,
a.Comp as Comp
from dbo.CD cd WITH(NOLOCK)
INNER JOIN
(SELECT DISTINCT ChildCId,Comp
FROM TCI tc WITH(NOLOCK)
WHERE ParentCId = '+ #Id + '
) a
ON cd.CIdentifier= a.ChildCId
);
EXEC (#Query)
END;
Here is the comparison-
SELECT CIdentifier FROM #tempTable temp WITH(NOLOCK)
WHERE temp.CIdentifier < '+#Id+'....
This compares as CIdentifier =1122233 instead of CIdentifier ='1122233' because dynamic SQL is not allowing me to pass it as an int. I keep getting the 'cannot convert varchar to int error'
So I used parameterized query - hoping that would enable me to pass int values.Here is the query part
SET #Query
= N';WITH CTE AS
(
......
(SELECT DISTINCT ChildCId,Comp
FROM TCI tc WITH(NOLOCK)
WHERE ParentCId = #Id
AND ChildCId + tc.Comp
NOT IN
(SELECT ChildId + Comp FROM dbo.TCI WITH(NOLOCK)
WHERE ParentId IN (SELECT CIdentifier FROM #tempTable WITH(NOLOCK)
WHERE temp.CIdentifier < #Idn
AND Comp IN ( #Comp))
)
)
)a
ON cd.CIdentifier= a.ChildId
)
SELECT * FROM CTE;'
EXEC sp_executeSQL #Query,'#Id VARCHAR(64),#Idn INT,#comp VARCHAR(256)',#Id=#Id,#Idn=#Idn,#comp =#comp
This gives me incorrect results and when I saw the execution using a trace - saw that values are not being passed onto the query. How can I get the query to pick up the variables?
Just change WHERE ParentCId = '+ #Id + ' to WHERE ParentCId = '+ cast(#Id as varchar(16)) + ' in the first query. The problem is SQL Server see's + as addition when the value is a numeric type, or date, and concatenation when it isn't. This is where you get the error from. However, when you do this, it will not make SQL Server compare it as a string literal so you don't have to worry about that. You can see this if you use PRINT (#Query) at the end instead of EXEC (#Query)
Note, this needs to be changed at the other locations you have any NUMERIC data type, like in the SELECT portion, SELECT '+ cast(#Id as varchar(16)) +'
Also, you code doesn't show where #Id value comes from, so be cautious of SQL injection here.

SQL Export Data from a Query to Use Tilde (~)

I have a basic Select query which brings me back a set of results (roughly around 100,000 records) which I currently have to export to .CSV Format , issue is I then have to remove the commas out of the results and replace it with a (~). Which I do using a file format application that I got off the web.
But I'm trying to go about making this automated (if possible) to save time. Like e.g run off a stored procedure that can do this for me export the file in a (~) format.
Does anyone have any tips how this stored procedure can be written or an pointers would be appreciated.
p.s I have tried to use the export wizard but it just just crashes due to too many records.
Expected Result
Test1~Test2~Test3
5~6~7
(sql Script which I am running)
select
'SPK' as [AGENCY_CODE], -- should be set to SPK
'OBCALL' as [MEDIA_CODE], -- should be set to OBCALL
isnull(c.salutation,'') as [TITLE],
isnull(c.otherName,'') as [FORENAME],
isnull(c.name,'') as [SURNAME],
isnull(c.attTXT64,'') as [STANDARDISED_NAME],
replace(isnull(c.addr1, ''), ',', '.') AS [BEST_ADDRESS_LINE_1],
replace(isnull(c.addr2, ''), ',', '.') AS [BEST_ADDRESS_LINE_2],
replace(isnull(c.addr3, ''), ',', '.') AS [BEST_ADDRESS_LINE_3],
replace(isnull(c.addr4, ''), ',', '.') AS [BEST_ADDRESS_LINE_4],
isnull(c.postCode,'') as [BEST_POSTCODE],
--'0' + isnull(c.phone1,'') as [TELEPHONE_N2O],-- should be populated with the spare field ORIG_TEL .
RIGHT('0' + CONVERT(VARCHAR(11), c.phone1), 11) as [TELEPHONE_NO],-- should be populated with the spare field ORIG_TEL .
convert(varchar(100),c.attDT03,120) as [DATE_TIMESTAMP],
isnull(c.attTXT10,'') as [SM_CONTACT_KEY],
isnull(c.attTXT89,'') as [ SM_ADDRESS_KEY],
isnull(c.attTXT11,'') as [ CAMPAIGN_IDENTIFIER],
isnull(c.attTXT12,'') as [ WAVE_ID],
isnull(c.attTXT13,'') as [OLDSTACK_NEWSTACK_FLAG],
isnull(c.attTXT14,'') as [MARKET_3_FLAG],
isnull(c.attTXT15,'') as [ADSL_2_FLAG],
isnull(c.attTXT16,'') as [FIBRE_FLAG],
isnull(c.attTXT17,'') as [LOAD_ID],
isnull(c.attTXT18,'') as [CONTACT_POINT_KEY],
isnull(c.attTXT19,'') as [DATA_POOL_URN],
isnull(c.attTXT20,'') as [EVENT_KEY],
isnull(c.attTXT21,'') as [BILLING_ACCOUNT_KEY] ,
isnull(c.attTXT22,'') as [CAMPAIGN_SOURCE] ,
isnull(c.attTXT23,'') as [CAMPAIGN_CODE] ,
isnull(c.attTXT24,'') as [ CMT_ROLE_KEY],
isnull(c.attTXT25,'') as [ CMT_LOCATION_KEY],
isnull(c.attTXT26,'') as [BILL_ACCNT_NUM],
isnull(c.attTXT27,'') as [BILLING_ACCOUNT_TYPE],
--All other fields are as per the import record values
--DATE/TIME_OF_CONTRACT to CONTRACT_END_DATE_SUPPLIER3
CASE WHEN dx.datetime IS NULL THEN convert(varchar,getdate(),120) ELSE CONVERT(varchar, dx.datetime, 120) END
as [DATE/TIMEOF CONTACT],
Case when uc.campaignid = 3 then 'CT001' when uc.campaignid = 22 then 'CT001' when uc.campaignid = 18 then 'CT011'
when uc.campaignid = 26 then 'CT013'
end as [CAMPAIGN_TYPE], -- map to BT/DATA/10.CAMPAIGN_Code (return CAMPAIGN_TYPE)
ISNULL(( CASE
WHEN dx.[Abandon] = 1 THEN 'OC039'
ELSE d.code END),'OC042')
AS [OUTCOME_CODE], -- populate with (OC001-OCxxx)
'TM' as [CHANNEL_MEDIA_CODE], --= (TBC)
isnull(c.email,'') as [EMAIL_ADDRESS], -- populate with EMAIL_ADDRESS
'' as [EMAIL_CONSENT], --= populate with EMAIL_CONSENT ***************
'' as [INBOUND_TELEPHONE_NUMBER], --*****************
'' as [COMPETITOR_SUPPLIER_1], -- CONT CONTRACT_END_DATE_SUPPLIER3 – poplulate ***********
'' as [PRODUCT_FROM_SUPPLIER1], -- populate ************
'' as [CONTRACT_START_DATE_SUPPLIER1], --*****
'' as [CONTRACT_END_DATE_SUPPLIER1], --*****
'' as [COMPETITOR_SUPPLIER2], --****
'' as [PRODUCT_FROM_SUPPLIER2], --****
'' as [CONTRACT_START_DATE_SUPPLIER2],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER2], --*****
'' as [COMPETITOR_SUPPLIER3],-- ****
'' as [PRODUCT_FROM_SUPPLIER3],--****
'' as [CONTRACT_START_DATE_SUPPLIER3],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER3],-- *****
--ORDER_NUM to NUMBER_OF_CALLS_MADE
isnull(c.attTXT02,'') as [ORDER_NUM], -- Captured by Operator
isnull(dx.duration,0) as [CALL_DURATION],
-------------dxi.talk as [CALL_DURATION], -- populate difference start/end time (seconds)******
isnull(c.attTXT09,'') as [WARMTH_RATING_NOW], -- captured by agent
isnull(c.attTXT80,'') as [WARMTH_RATING_FUTURE_CAMPAIGNS], -- captured by agent
isnull(dx.callid,'') [SOURCE_INTERACTION_ID],
-------------isnull(a.id,'') as [SOURCE_INTERACTION_ID], -- Tpoints unique call Identifier --- Activity ID *******
isnull(uc.callcount,'') as [NUMBER_OF_CALLS_MADE],
(select left(ethnicOrigin, 1)) as [CALL_CONSENT_VALIDATE_FLG], -- ******** gift aid status - first byte only
(select left(nationality, 5)) as [CALL_CONSENT_OUTCOME_CD], -- ******** nationality - first 5 bytes only
--QUESTION _1 to ANSWER_3
isnull(c.attTXT56,'') as [QUESTION_1], -- populate from SPARE_FIELD_28
isnull(c.attTXT51,'') as [ANSWER_1], -- captured by the agent if SPARE_FIELD_1 populated
isnull(c.attTXT29,'') as [QUESTION_2], -- poaddingpulate from SPARE_FIELD_28
isnull(c.attTXT67,'') as [ANSWER_2], -- captured by the agent if SPARE_FIELD_2 populated
isnull(c.attTXT37,'') as [QUESTION_3], -- populate from SPARE_FIELD_28
isnull(c.attTXT58,'') as [ANSWER_3] -- captured by the agent if SPARE_FIELD_3 populated
--isnull(c.attdt18,GETDATE()) as [Export_Date]
from
u_contact c with (nolock)
inner join u_campaigncontact uc with (nolock) on uc.contactid = c.id
inner join u_dispcode d with (nolock) on d.id = uc.resultcodeid
outer apply (select top 1 duration, callid,
case when outcome = 113 then 1 else 0 end [Abandon], [datetime] from dxi_cdrlog cdr where cdr.urn = c.id order by callid desc) dx
where
uc.campaignid in (3, 18, 22, 26) and d.dmc = 1 and c.created between DATEADD(week, -1, getdate()) AND getdate()
and c.importid > 0
Since you are using SQL-Server you could use the command line tool sqlcmd which comes with the package. You might have to re-run the installation of the SSMS to acticvate the option.
Open cmd.exe. When you use the sqlcmd command with the options
sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q "SET NOCOUNT ON;SELECT 1,'hello world'"
It should get you
1~hello world
-W removes blanks between the columns, -s ~ sets ~ as the column separator, -h -1 removes the header line at the top and -Q then expects the actual query statement as the next argument.
Obviously for the actual job you need to call your SELECt statement. It might be a good idea to define a view (myview) for the job and in the sqlcmd just do a SELECT * FROM myview since writing a multiline SQL command within a cmd environment is not so much fun .... Of course it is possible but it is not really worth the hassle.
The command should then redirect its output directly into a file like
set sql=SET NOCOUNT ON;SELECT * FROM myview
set scmd=sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q
%scmd% "%sql" > exportfile.csv

Generate Insert statement with "#" for SQLCommand?

If I use SQLCommand, I usually need something like:
INSERT INTO klant(klant_id,naam,voornaam) VALUES(#param1,#param2,#param3)";
Is there some easy way I can generate this string?
SSMS generate something like this
,[years]
,[source]
,[TimeStamp])
VALUES
(<count, int,>
,<sex_male, bit,>
,<Ethnicity, tinyint,>
Can I use it somehow?
If you need this frequently then consider SSMS tools pack. This generates CRUD according to user specified templates.
An adhoc not particularly robust alternative is pretty simple with find and replace though.
SSMS generates something like
INSERT INTO [HumanResources].[Employee]
([BusinessEntityID]
,[NationalIDNumber]
,[LoginID]
,[CurrentFlag]
,[rowguid]
,[ModifiedDate])
VALUES
(<BusinessEntityID, int,>
,<NationalIDNumber, nvarchar(15),>
,<LoginID, nvarchar(256),>
,<CurrentFlag, Flag,>
,<rowguid, uniqueidentifier,>
,<ModifiedDate, datetime,>)
Then copy the top section down to the bottom
INSERT INTO [HumanResources].[Employee]
([BusinessEntityID]
,[NationalIDNumber]
,[LoginID]
,[CurrentFlag]
,[rowguid]
,[ModifiedDate])
VALUES
([BusinessEntityID]
,[NationalIDNumber]
,[LoginID]
,[CurrentFlag]
,[rowguid]
,[ModifiedDate])
And select the bottom section and replace [ with # and ] with an empty string.
Or alternatively it would be fairly trivial to write something that queries sys.columns and generates the desired string.
(Again a not robust solution that assumes you are using column names conforming to the rules for standard identifiers- quotename would help with the column names but not the parameter names if you aren't.)
DECLARE #QualifiedName NVARCHAR(500) = '[HumanResources].[Employee]';
WITH C
AS (SELECT *
FROM sys.columns
WHERE object_id = object_id(#QualifiedName)
AND is_computed = 0
AND is_identity = 0)
SELECT '
INSERT INTO ' + #QualifiedName + '
(' + SUBSTRING((SELECT ',' + name
FROM C
ORDER BY column_id
FOR XML PATH('')), 2, 8000) + ')
VALUES
(' + SUBSTRING((SELECT ',#' + name
FROM C
ORDER BY column_id
FOR XML PATH('')), 3, 8000) + ')

Resources