Exporting CSV's from SQL Server - Iterating different Key - sql-server

Question

I am currently going through the process of exporting a CSV file from an SQL Server DB for each unique key in a database that contains meter readings. I am doing this one at a time manually via Export data in SSMS.... It is making me want to die since I have 200 unique key values.
Here is my query:
SELECT DataTime, DataValue
FROM [i96X].[dbo].[PointValue]
WHERE PointID = 68352
The bulk of the query stays the same, the only value i change is the PointID. I need tab separated CSV's as output where the filename = the PointID.
Can someone help?

Answer 1

You can use xp_cmdshell and bcp to achieve what you want but it requires more privileges, in case you're thinking about running this in production.
You need to configure enable advanced options in order to enable xp_cmdshell and then enable xp_cmdshell
EXEC sp_configure 'show advanced options', 1 RECONFIGURE
EXEC sp_configure 'xp_cmdshell', 1 RECONFIGURE
Then we gather all PointIDs into a cursor so we can process individually
Declare curPointIDS Cursor For
Select Distinct PointID From [i96X].[dbo].[PointValue]
Open curPointIDS
Declare #pointId Int, #cmd Varchar(max)
Fetch Next From curPointIDS Into #pointId
While ##FETCH_STATUS = 0
Begin
Set #cmd = 'bcp "SELECT DataTime, DataValue FROM [i96X].[dbo].[PointValue] '
+ 'WHERE PointID = ' + LTRIM(#pointId) + '" queryout '
+ '"C:\Temp\Results_' + LTRIM(#pointId) + '.csv" -T -c -t,'
EXEC xp_cmdshell #cmd
End
Close curPointIDS
Deallocate curPointIDS
xp_cdshell: Runs commands in command prompt (https://msdn.microsoft.com/en-us/library/ms175046.aspx)
bcp: Sql Server's data extract utility https://msdn.microsoft.com/en-us/library/ms162802.aspx
Parameters:
-T: Use trusted connection
-c: Data type will be characters
-t: Specify delimiter, in this case it's comma (,)

Answer 2

You can use a loop and a temp table to accomplish this.
declare #i int = (select count(distinct PointID) from [i96X].[dbo].[PointValue])
select row_number() over(order by PointID) as rn, *
into #tmp --temp table
from [i96X].[dbo].[PointValue]
declare #id int
declare #sql varchar(4000)
while #i > 0
begin
set #id = (select PointID from #tmp where rn = #i)
SELECT DataTime, DataValue
FROM [i96X].[dbo].[PointValue]
WHERE PointID = #id;
select #sql = 'bcp "SELECT DataTime, DataValue FROM [i96X].[dbo].[PointValue] WHERE PointID = '+str(#id)+'" queryout "C:\Temp\'+str(#id)+'+'.csv'+'" -S servername -d databasename -U username -P password'
exec master..xp_cmdshell #sql
set #i = #i-1;
end

Answer 3

If you have a fix list of meters you can create a batch file and extract this information, on a regular basis, using the Task Scheduler.
Basically you should put on this batch file one row for each meter you want to extract and use sqlcmd to extract the historical records.
For example, let's say you want to extract the historical data for the pointIDs (1, 2, 3). You can create a batch file called "extract.bat" with the following lines:
sqlcmd -Q "select * from i96x.dbo.PointValue where Pointid=1" -s "\t" -o "C:\Results\1.csv"
sqlcmd -Q "select * from i96x.dbo.PointValue where Pointid=2" -s "\t" -o "C:\Results\2.csv"
sqlcmd -Q "select * from i96x.dbo.PointValue where Pointid=3" -s "\t" -o "C:\Results\3.csv"
You just need to run this batch file and get your files. This files will use tab as a column separator. If you want to use comma as column separator just change -s , on each row.
You can use excel to make easier to generate this batch file.
Another head up, with that query you will get ALL the values for each pointid. If you are working with meter probably you will be interested in extract only the last month. To do that you will need to adjust the query.