SQL Export Data from a Query to Use Tilde (~) - sql-server

I have a basic Select query which brings me back a set of results (roughly around 100,000 records) which I currently have to export to .CSV Format , issue is I then have to remove the commas out of the results and replace it with a (~). Which I do using a file format application that I got off the web.
But I'm trying to go about making this automated (if possible) to save time. Like e.g run off a stored procedure that can do this for me export the file in a (~) format.
Does anyone have any tips how this stored procedure can be written or an pointers would be appreciated.
p.s I have tried to use the export wizard but it just just crashes due to too many records.
Expected Result
Test1~Test2~Test3
5~6~7
(sql Script which I am running)
select
'SPK' as [AGENCY_CODE], -- should be set to SPK
'OBCALL' as [MEDIA_CODE], -- should be set to OBCALL
isnull(c.salutation,'') as [TITLE],
isnull(c.otherName,'') as [FORENAME],
isnull(c.name,'') as [SURNAME],
isnull(c.attTXT64,'') as [STANDARDISED_NAME],
replace(isnull(c.addr1, ''), ',', '.') AS [BEST_ADDRESS_LINE_1],
replace(isnull(c.addr2, ''), ',', '.') AS [BEST_ADDRESS_LINE_2],
replace(isnull(c.addr3, ''), ',', '.') AS [BEST_ADDRESS_LINE_3],
replace(isnull(c.addr4, ''), ',', '.') AS [BEST_ADDRESS_LINE_4],
isnull(c.postCode,'') as [BEST_POSTCODE],
--'0' + isnull(c.phone1,'') as [TELEPHONE_N2O],-- should be populated with the spare field ORIG_TEL .
RIGHT('0' + CONVERT(VARCHAR(11), c.phone1), 11) as [TELEPHONE_NO],-- should be populated with the spare field ORIG_TEL .
convert(varchar(100),c.attDT03,120) as [DATE_TIMESTAMP],
isnull(c.attTXT10,'') as [SM_CONTACT_KEY],
isnull(c.attTXT89,'') as [ SM_ADDRESS_KEY],
isnull(c.attTXT11,'') as [ CAMPAIGN_IDENTIFIER],
isnull(c.attTXT12,'') as [ WAVE_ID],
isnull(c.attTXT13,'') as [OLDSTACK_NEWSTACK_FLAG],
isnull(c.attTXT14,'') as [MARKET_3_FLAG],
isnull(c.attTXT15,'') as [ADSL_2_FLAG],
isnull(c.attTXT16,'') as [FIBRE_FLAG],
isnull(c.attTXT17,'') as [LOAD_ID],
isnull(c.attTXT18,'') as [CONTACT_POINT_KEY],
isnull(c.attTXT19,'') as [DATA_POOL_URN],
isnull(c.attTXT20,'') as [EVENT_KEY],
isnull(c.attTXT21,'') as [BILLING_ACCOUNT_KEY] ,
isnull(c.attTXT22,'') as [CAMPAIGN_SOURCE] ,
isnull(c.attTXT23,'') as [CAMPAIGN_CODE] ,
isnull(c.attTXT24,'') as [ CMT_ROLE_KEY],
isnull(c.attTXT25,'') as [ CMT_LOCATION_KEY],
isnull(c.attTXT26,'') as [BILL_ACCNT_NUM],
isnull(c.attTXT27,'') as [BILLING_ACCOUNT_TYPE],
--All other fields are as per the import record values
--DATE/TIME_OF_CONTRACT to CONTRACT_END_DATE_SUPPLIER3
CASE WHEN dx.datetime IS NULL THEN convert(varchar,getdate(),120) ELSE CONVERT(varchar, dx.datetime, 120) END
as [DATE/TIMEOF CONTACT],
Case when uc.campaignid = 3 then 'CT001' when uc.campaignid = 22 then 'CT001' when uc.campaignid = 18 then 'CT011'
when uc.campaignid = 26 then 'CT013'
end as [CAMPAIGN_TYPE], -- map to BT/DATA/10.CAMPAIGN_Code (return CAMPAIGN_TYPE)
ISNULL(( CASE
WHEN dx.[Abandon] = 1 THEN 'OC039'
ELSE d.code END),'OC042')
AS [OUTCOME_CODE], -- populate with (OC001-OCxxx)
'TM' as [CHANNEL_MEDIA_CODE], --= (TBC)
isnull(c.email,'') as [EMAIL_ADDRESS], -- populate with EMAIL_ADDRESS
'' as [EMAIL_CONSENT], --= populate with EMAIL_CONSENT ***************
'' as [INBOUND_TELEPHONE_NUMBER], --*****************
'' as [COMPETITOR_SUPPLIER_1], -- CONT CONTRACT_END_DATE_SUPPLIER3 – poplulate ***********
'' as [PRODUCT_FROM_SUPPLIER1], -- populate ************
'' as [CONTRACT_START_DATE_SUPPLIER1], --*****
'' as [CONTRACT_END_DATE_SUPPLIER1], --*****
'' as [COMPETITOR_SUPPLIER2], --****
'' as [PRODUCT_FROM_SUPPLIER2], --****
'' as [CONTRACT_START_DATE_SUPPLIER2],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER2], --*****
'' as [COMPETITOR_SUPPLIER3],-- ****
'' as [PRODUCT_FROM_SUPPLIER3],--****
'' as [CONTRACT_START_DATE_SUPPLIER3],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER3],-- *****
--ORDER_NUM to NUMBER_OF_CALLS_MADE
isnull(c.attTXT02,'') as [ORDER_NUM], -- Captured by Operator
isnull(dx.duration,0) as [CALL_DURATION],
-------------dxi.talk as [CALL_DURATION], -- populate difference start/end time (seconds)******
isnull(c.attTXT09,'') as [WARMTH_RATING_NOW], -- captured by agent
isnull(c.attTXT80,'') as [WARMTH_RATING_FUTURE_CAMPAIGNS], -- captured by agent
isnull(dx.callid,'') [SOURCE_INTERACTION_ID],
-------------isnull(a.id,'') as [SOURCE_INTERACTION_ID], -- Tpoints unique call Identifier --- Activity ID *******
isnull(uc.callcount,'') as [NUMBER_OF_CALLS_MADE],
(select left(ethnicOrigin, 1)) as [CALL_CONSENT_VALIDATE_FLG], -- ******** gift aid status - first byte only
(select left(nationality, 5)) as [CALL_CONSENT_OUTCOME_CD], -- ******** nationality - first 5 bytes only
--QUESTION _1 to ANSWER_3
isnull(c.attTXT56,'') as [QUESTION_1], -- populate from SPARE_FIELD_28
isnull(c.attTXT51,'') as [ANSWER_1], -- captured by the agent if SPARE_FIELD_1 populated
isnull(c.attTXT29,'') as [QUESTION_2], -- poaddingpulate from SPARE_FIELD_28
isnull(c.attTXT67,'') as [ANSWER_2], -- captured by the agent if SPARE_FIELD_2 populated
isnull(c.attTXT37,'') as [QUESTION_3], -- populate from SPARE_FIELD_28
isnull(c.attTXT58,'') as [ANSWER_3] -- captured by the agent if SPARE_FIELD_3 populated
--isnull(c.attdt18,GETDATE()) as [Export_Date]
from
u_contact c with (nolock)
inner join u_campaigncontact uc with (nolock) on uc.contactid = c.id
inner join u_dispcode d with (nolock) on d.id = uc.resultcodeid
outer apply (select top 1 duration, callid,
case when outcome = 113 then 1 else 0 end [Abandon], [datetime] from dxi_cdrlog cdr where cdr.urn = c.id order by callid desc) dx
where
uc.campaignid in (3, 18, 22, 26) and d.dmc = 1 and c.created between DATEADD(week, -1, getdate()) AND getdate()
and c.importid > 0


Since you are using SQL-Server you could use the command line tool sqlcmd which comes with the package. You might have to re-run the installation of the SSMS to acticvate the option.
Open cmd.exe. When you use the sqlcmd command with the options
sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q "SET NOCOUNT ON;SELECT 1,'hello world'"
It should get you
1~hello world
-W removes blanks between the columns, -s ~ sets ~ as the column separator, -h -1 removes the header line at the top and -Q then expects the actual query statement as the next argument.
Obviously for the actual job you need to call your SELECt statement. It might be a good idea to define a view (myview) for the job and in the sqlcmd just do a SELECT * FROM myview since writing a multiline SQL command within a cmd environment is not so much fun .... Of course it is possible but it is not really worth the hassle.
The command should then redirect its output directly into a file like
set sql=SET NOCOUNT ON;SELECT * FROM myview
set scmd=sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q
%scmd% "%sql" > exportfile.csv

Related

How to export Sybase table contents to a text file?

I want to export the tables with contents from a Sybase database to a text file. I am using Sybase ASE 15.5 and that can't be changed/upgraded.
I have tried using sp_tables and different Select commands, but they don't give the result I'm looking for.
The output format I'm looking for is something like this:
FIRST_TABLE
Column_1 Column_2 Column_3
Roger Male 51
Anne Female 46
SECOND_TABLE
Column_1 Column_2 Column_3
BMW German Car
Schwinn American Bicycles
etc.etc.

Create a view that generates the output you want and use bcp to copy the data from the view.
Consider the following table, view and data:
create table t1 (
k int not null,
v varchar(255) null)
go
create view v1 as
select
'k' as k,
'v' as v
union all
select
convert(varchar, k),
v
from
t1
go
insert into t1 (k, v) values (1, 'Line_1')
insert into t1 (k, v) values (2, 'Line_2')
insert into t1 (k, v) values (3, 'Line_3')
go
Check the data returned from the view, notice the column names are in the result set. They need to here. Ideally you would query against syscolumns, but there is no pivot statement in ASE, so you need to know the names in advance :-(
select * from v1
go
k v
1 Line_1
2 Line_2
3 Line_3
(4 rows affected)
Now copy the data from the view into the text file:
$ bcp <db_name>..v1 out v1.txt -c -U login_name -S server_name
Password:
Starting copy...
4 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1 Average : (4000.0 rows per sec.)
$ cat v1.txt
k v
1 Line_1
2 Line_2
3 Line_3

Without using BCP
declare cur cursor for
select sc.name from sysobjects so, syscolumns sc
where so.id = sc.id and so.name = 'FIRST_TABLE'
order by sc.colid
go
declare #l_name varchar(30), #l_sql varchar(2000)
begin
open cur
fetch cur into #l_name
while ##sqlstatus = 0
begin
set #l_sql = #l_sql + '''' + #l_name + ''','
fetch cur into #l_name
end
close cur
deallocate cur
set #l_sql = 'select ' + substring(#l_sql,1,len(#l_sql)-1)
set #l_sql = #l_sql + ' union all select * from FIRST_TABLE'
exec (#l_sql)
end
go

Use findstr to replace null values with empty strings from sqlcmd command

I have an sqlcmd command which generates a csv file from a view. Is it possible to replace the NULL values to empty string using the findstr command?
Here is what I tried.
sqlcmd -S . -d SAMPLEDB -U sa -P pass -s"|" -W -Q "SET NOCOUNT ON SET ANSI_WARNINGS OFF select * from view_Table" > Sample.csv -h -1 | findstr /v /c:"NULL"

You can easily build the extraction SQL for each view using the system management views. This simple query:
SELECT v.[name]
,c.[name]
,c.[column_id]
,c.[is_nullable]
FROM sys.views V
INNER JOIN sys.columns C
ON V.[object_id] = C.[object_id];
will return everything we need to perform the task:
the view name
the column name
the column order
if the column is nullable
So, we need only to build the extraction SQL statements:
SELECT v.[name]
,'SELECT ' + DS.[definition] + ' FROM ' + v.[name]
FROM sys.views V
CROSS APPLY
(
SELECT STUFF
(
(
SELECT ',' + CASE WHEN c.[is_nullable] = 1 THEN 'ISNULL(' + c.[name] + ','''')' ELSE c.[name] END
FROM sys.columns C
WHERE V.[object_id] = C.[object_id]
ORDER BY c.[column_id]
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1
,1
,''
)
) DS ([definition]);
Depending on your SQL version you can reduce the code above - for example using IIF or STRING_AGG.
Also, you can add WHERE clause to filter the query for specific views.

Change the sql in the command string to select fields individually and do an ISNULL on them
select * from view_Table
To
select ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') from view_Table
in you calling string
use the following code to build it and check to see the final statement runs
DECLARE #Temp VARCHAR(MAX)
SET #Temp = 'SELECT ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') FROM view_Table'
SELECT #Temp

SQL string before and after certain characters

SELECT NAME
FROM SERVERS
returns:
SDACR.hello.com
SDACR
SDACR\AIR
SDACR.hello.com\WATER
I need the SELECT query for below result:
SDACR
SDACR
SDACR\AIR
SDACR\WATER
Kindly help ! I tried using LEFT and RIGHT functions as below, but not able to get combined output correctly:
SELECT
LEFT(Name, CHARINDEX('.', Name) - 1)
FROM
SERVERS
SELECT
RIGHT(Name, LEN(Name) - CHARINDEX('\', Name))
FROM
SERVERS

It looks like you're just trying to REPLACE a substring of characters in your column. You should try this:
SELECT REPLACE(Name,'.hello.com','') AS ReplacementName
FROM SERVERS

In tsql, you can concatenate values with CONCAT(), or you can simply add strings together with +.
SELECT LEFT(Name, CHARINDEX('.',Name)-1) + RIGHT(Name,LEN(Name)-CHARINDEX('\',Name)) from SERVERS
Also, be careful with doing arithmetic with CHARINDEX(). A value without a '.' or a '\' will return a NULL and you will get an error.

You can use LEFT for this to select everything up to the first period (dot) and add on everything after the last \
declare #servers table ([NAME] varchar(64))
insert into #servers
values
('SDACR.hello.com '),
('SDACR'),
('SDACR\AIR'),
('SDACR.hello.com\WATER')
select
left([NAME],case when charindex('.',[NAME]) = 0 then len([NAME]) else charindex('.',[NAME]) -1 end) +
case when charindex('\',left([NAME],case when charindex('.',[NAME]) = 0 then len([NAME]) else charindex('.',[NAME]) -1 end)) = 0 then right([NAME],charindex('\',reverse([NAME]))) else '' end
from #servers

Throwing my hat in.... Showing how to use Values and APPLY for cleaner code.
-- sample data in an easily consumable format
declare #yourdata table (txt varchar(100));
insert #yourdata values
('SDACR.hello.com'),
('SDACR'),
('SDACR\AIR'),
('SDACR.hello.com\WATER');
-- solution
select
txt,
newTxt =
case
when loc.dot = 0 then txt
when loc.dot > 0 and loc.slash = 0 then substring(txt, 1, loc.dot-1)
else substring(txt, 1, loc.dot-1) + substring(txt, loc.slash, 100)
end
from #yourdata
cross apply (values (charindex('.',txt), (charindex('\',txt)))) loc(dot,slash);
Results
txt newTxt
------------------------------ --------------------
SDACR.hello.com SDACR
SDACR SDACR
SDACR\AIR SDACR\AIR
SDACR.hello.com\WATER SDACR\WATER

bcp append a line at the end of file

I am trying to export a .txt file with bcp:
EXEC xp_CMDSHELL 'BCP "SELECT v.ItemId
, v.InventLocationId
, CAST(v.AvailPhysical AS int)
FROM D_R1.dbo.vwStockOnHand AS v
JOIN D_R1.DBO.vwProducts AS m
ON v.ItemId = m.ITEMID
WHERE NOT EXISTS
(SELECT 1
FROM D_R1.DBO.ExportExcludedFamilies AS magExport
WHERE magExport.REFRECID = m.FamilyRecId)
AND AVAILPHYSICAL > 0 AND v.PICKFROMZ = 1
ORDER BY v.InventLocationId, v.ITEMID"
queryout "C:\temp\1.txt" -c -t"|" -T -S D_R1'
All good, but i would need a last line in the file like "END OF FILE" and i am not able to figure it out ..
Can someone give me a hint on this?

1) First solution
SELECT CONVERT(VARCHAR(11), v.ItemId) AS ItemId
, v.InventLocationId
, CAST(v.AvailPhysical AS int)
, 1 AS Priority
FROM ...
UNION ALL
SELECT 'End of export file', NULL, NULL, 2 AS Priority
ORDER BY Priority, InventLocationId, ItemID
or
SELECT... ;
SELECT 'End of report' ;
or
SELECT... ;
PRINT 'End of report' ;
2) Second solution
bcp "the same SQL query", ...
echo End of export
echo command

Extract Substring from text in SQL sever

I capture MDX statements fired on the SSAS Cube using a SQL profiler into a table. What I want to do is to extract the Cube name from the MDX statement.
The problem I have is the fact that the MDX statements are pretty huge and random (Users connect to the Cube and create Adhoc reports) and have multiple Sub Cubes constructed making it difficult to fetch the Cube Name.
I was able to figure out a pattern for search.
First string: 'FROM ['
Second string: ']'
I need to now pickup a substring from the variable.
Example below:
DECLARE #TEXT varchar(max) = 'SELECT NON EMPTY (((( [[ XXXXX ]] }) ON ROWS FROM (SELECT ({XXXXXXXX }) ON COLUMNS FROM [Sales Reporting]))
WHERE XXXXX ))'
DECLARE #FirstPosition int = (SELECT CHARINDEX('FROM [',#TEXT)+5)
DECLARE #SecondPosition int = (SELECT CHARINDEX(']',#TEXT,#FirstPosition))
SELECT #FirstPosition, #SecondPosition
SELECT SUBSTRING(#TEXT,CHARINDEX('FROM [',#TEXT)+5,(CHARINDEX(']',#TEXT,#FirstPosition)-CHARINDEX('[',#TEXT))-1)
Desired Result = Sales Reporting

Got the solution turned out to be simple than I expected.
DECLARE #TEXT varchar(max) = 'SELECT NON EMPTY (((( [[ XXXXX ]] }) ON ROWS FROM (SELECT ({XXXXXXXX }) ON COLUMNS FROM [Sales Reporting]))
WHERE XXXXX ))'
DECLARE #FirstPosition int = (SELECT CHARINDEX('FROM [',#TEXT)+5)
DECLARE #ExtractString nvarchar(max) = (SELECT SUBSTRING(#TEXT,#FirstPosition, LEN(#Text)))
DECLARE #SecondPosition int = (SELECT CHARINDEX(']',#ExtractString))
SELECT SUBSTRING(#ExtractString,CHARINDEX('[',#ExtractString)+1,(CHARINDEX(']',#ExtractString)-CHARINDEX('[',#ExtractString))-1) AS CubeName

Resources