Importing Grouped Report Data to Database - sql-server

Question

My company receives data from a client that is unable to provide data in any direct format so we have to import several reports that are in a grouped layout like the one below. We have to develop in house methods to ungroup the report and then import the data to get all of the data we need. Currently a member on my team is using MS Access / VBA to generate the needed detail records but I want to move this to a server based and automated process. We are using SQL Server 2008R2 for storage and I would like to use SSIS to accomplish the task. Does anyone know of a way I can generate the detail records and import the data directly into SQL Server?

Answer 1

Hmm - well you will definitely have to do some programmatic adjusting of the data set to add that group date to the detail line. I'm unsure of how you will be importing the xlsx but I would recommend first off just using a SSIS package and then doing the adjustments in a script task as the "best" way to do this. See here on how to handle Excel in SSIS Script tasks.
If you don't know SSIS or especially programming though, you're next best bet (in my opinion) is to just import the data into a staging table, do the manipulations with T-SQL and then insert that table into your main table. I did a SQL Fiddle of this here.
CREATE TABLE ActivitySummary
(
id int identity(1,1),
activity_date date,
activity varchar(100),
paid_time decimal(5,2),
unpaid_time decimal(5,2),
total_time decimal(5,2)
)
CREATE TABLE ActivitySummary_STG
(
id int identity(1,1),
activity_date date,
activity varchar(100),
paid_time decimal(5,2),
unpaid_time decimal(5,2),
total_time decimal(5,2)
)
GO
-- Simulate import of Excel sheet into staging table
truncate table ActivitySummary_STG;
GO
INSERT INTO ActivitySummary_STG (activity_date, activity, paid_time, unpaid_time, total_time)
select '8/14/17',null,null,null,null
UNION ALL
select null,'001 Lunch',0,4.4,4.4
UNION ALL
select null,'002 Break',4.2,0,4.2
UNION ALL
select null,'007 System Down',7.45,0,7.45
UNION ALL
select null,'019 End of Work Day',0.02,0,0.02
UNION ALL
select '8/15/17',null,null,null,null
UNION ALL
select null,'001 Lunch',0,4.45,4.45
UNION ALL
select null,'002 Break',6.53,0,6.53
UNION ALL
select null,'007 System Down',0.51,0,0.51
UNION ALL
select null,'019 End of Work Day',0.02,0,0.02
GO
-- Code to massage data
declare #table_count int = (select COALESCE(count(id),0) from ActivitySummary_STG);
declare #counter int = 1;
declare #activity_date date,
#current_date date;
WHILE (#table_count > 0 AND #counter <= #table_count)
BEGIN
select #activity_date = activity_date
from ActivitySummary_STG
where id = #counter;
if (#activity_date is not null)
BEGIN
set #current_date = #activity_date;
delete from ActivitySummary_STG
where id = #counter;
END
else
BEGIN
update ActivitySummary_STG SET
activity_date = #current_date
where id = #counter;
END
set #counter += 1;
END
INSERT INTO ActivitySummary (activity_date, activity, paid_time, unpaid_time, total_time)
select activity_date, activity, paid_time, unpaid_time, total_time
from ActivitySummary_STG;
truncate table ActivitySummary_STG;
GO
select * from ActivitySummary;

Answer 2

I'd do it with a script component.
Total Data Flow:
ExcelSource --> Script Component (Tranformation) --> Conditional Split --> SQL Destination
In script component:
Check accountSummary on InputColumns
Add ActivityDate as output column.
Open Script:
outside of your row processing.
Add:
public datetime dte;
inside row processing:
if (DateTime.TryParse(Row.ActivitySummary.ToString()))
{dte=DateTime.Parse(Row.ActivitySummary.ToString());}
else
{Row.ActivityDate = dte;}
Then add a Conditional Split to remove null Activity Dates