Dynamic columns depend on previous dynamic columns - TSQL - sql-server

Question

I try to create dynamic forecast for 18(!) months depend on previous columns (months) and i am stuck:
I have three columns:
Stock
SafetyStock
Need for production - another select with clause WHERE date = getdate()
what i need to achieve:
Index, Stock- Current month, SafetyStock-Current month, Need for production (select * from Nfp where date = getdate()), Stock - Current month + 1, Safetystock - Current Month + 1, Need for Production - Current Month + 1 ... etc till 18 months
calculations:
Stock - Current month + 1 = Stock previous month + SafetyStock previous month - Needs for production of current month
there is any possibility to create something like this ? it has to be dynamic and get calculation for current date and next 18 months. So now i have to calculate from 2020-10 till let's say 2022-04
What i have tried:
I prepared 18 cte and joins everything. Then i do calculations - it works but it slow and i think it is not profesional.
I have tried to do dynamic sql, below you can see my code but i have stucked when i wanted to do computed column depended on previous computed column:
------------------- CODE -------------------------
if object_id('tempdb..#tmp') is not null
drop table #tmp
if object_id('tempdb..#tmp2') is not null
drop table #tmp2
declare #cols as int
declare #iteration as int
declare #Mth as nvarchar(30)
declare #data as date
declare #sql as nvarchar(max)
declare #sql2 as nvarchar(max)
set #cols = 18
set #iteration = 0
set #Mth = month(getdate())
set #data = cast(getdate() as date)
select
10 as SS,
12 as Stock
into #tmp
WHILE #iteration < #cols
begin
set #iteration = #iteration + 1
set #sql =
'
alter table #tmp
add [StockUwzgledniajacSS - ' + cast(concat(year(DATEADD(Month, #Iteration, #data)),'-', month(DATEADD(Month, #Iteration, #data))) as nvarchar(max)) +'] as (Stock - SS)
'
exec (#sql)
set #Mth= #Mth+ 1
set #sql2 =
'
alter table #tmp
add [StockUwzgledniajacSS - ' + #Mth +'] as ([StockUwzgledniajacSS - ' + #Mth +'])
'
end
select * from #tmp
thanks in advance!

Answer 1

Update 1 note: I wrote this before you posted your data. This still holds I believe but, of course, stock levels are way different. Given that your NFP data is by day, and your report is by month, I suggest adding something to preprocess that data into months e.g., sum of NPS values, grouped by month.
Update 2 (next day) note: From the OPs comments below, I've tried to integrate this with what was written and more directly answering the question e.g., creating a reporting table #tmp.
Given that the OP also mentions millions of rows, I imagine each row represents a specific part/item - I've included this as a field called StockNum.
I have done something that probably doesn't do your calculations properly, but demonstrates the approach and should get you over your current hurdle. Indeed, if you haven't used these before, then updating this code with your own calculations will help you to understand how it works so you can maintain it.
I'm assuming the key issue here for calculation is that this month's stock is based on last month's stock and then new stock minus old stock for this month.
It is possible to calculate this in 18 separate statements (update table set col2 = some function of col1, then update table set col3 = some function of col2, etc). However, updating the same table multiple times is often an anti-pattern causing poor performance - especially if you need to read the base data again and again.
Instead, something like this is often best calculated using a Recusive CTE (here's an example description), where it 'builds' a set of data based on previous results.
The key difference in this approach is that it
Creates the reporting table (without any data/calculations going in)
Calculates the data as a separate step - but with columns/fields that can be used to link to the reporting table
Inserts the data from calculations into the reporting table as a single insert statement.
I have used temporary tables/etc liberally, to help demonstrate the process.
You haven't explained what safety stock is, nor how you measure what's coming in, so for the example below, I have assumed safety stock is the amount produced and is 5 per month. I've then assumed that NFP is amount going out each month (e.g., forward estimates of sales). The key result will be stock at the end of month (e.g., which you could then review whether it's too high or too low).
As you want to store it in a table that has each month as columns, the first step is to create a list with the relevant buckets (months). These include fields used for matching in later calculations/etc. Note I have included some date fields (startdate and enddate) which may be useful when you customise the code. This part of the SQL is designed to be as straightforward as possible.
We then create the scratch table that has our reference data for stock movements, replacing your SELECT * FROM NFP WHERE date = getdate()
/* SET UP BUCKET LIST TO HELP CALCULATION */
CREATE TABLE #RepBuckets (BucketNum int, BucketName nvarchar(30), BucketStartDate datetime, BucketEndDate datetime)
INSERT INTO #RepBuckets (BucketNum) VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
(11),(12),(13),(14),(15),(16),(17),(18)
DECLARE #CurrentBucketStart date
SET #CurrentBucketStart = DATEFROMPARTS(YEAR(getdate()), MONTH(getdate()), 1)
UPDATE #RepBuckets
SET BucketName = 'StockAtEnd_' + FORMAT(DATEADD(month, BucketNum, #CurrentBucketStart), 'MMM_yy'),
BucketStartDate = DATEADD(month, BucketNum, #CurrentBucketStart),
BucketEndDate = DATEADD(month, BucketNum + 1, #CurrentBucketStart)
/* CREATE BASE DATA */
-- Current stock
CREATE TABLE #Stock (StockNum int, MonthNum int, StockAtStart int, SafetyStock int, NFP int, StockAtEnd int, PRIMARY KEY(StockNum, MonthNum))
INSERT INTO #Stock (StockNum, MonthNum, StockAtStart, SafetyStock, NFP, StockAtEnd) VALUES
(12422, 0, NULL, NULL, NULL, 10)
-- Simulates SELECT * FROM NFP WHERE date = getdate()
CREATE TABLE #NFP_by_month (StockNum int, MonthNum int, StockNFP int, PRIMARY KEY(StockNum, MonthNum))
INSERT INTO #NFP_by_month (StockNum, MonthNum, StockNFP) VALUES
(12422, 1, 4), (12422, 7, 4), (12422, 13, 4),
(12422, 2, 5), (12422, 8, 5), (12422, 14, 5),
(12422, 3, 2), (12422, 9, 2), (12422, 15, 2),
(12422, 4, 7), (12422, 10, 7), (12422, 16, 7),
(12422, 5, 9), (12422, 11, 9), (12422, 17, 9),
(12422, 6, 3), (12422, 12, 3), (12422, 18, 3)
We then use the recursive CTE to get calculate our data. It stores these in table #StockProjections.
What this does is
Start with your current stock (last row in the #Stock table). Note that the only value that matters in that is the stock at end of month.
Uses that stock level at the end of last month, as the stock level at the start of the new month
Adds the safety stock, minuses the NFP, and calculates your stock at end.
Note that within the recursive part of the CTE, 'SBM' (StockByMonth) refers to last month's data). This is then used with whatever external data (e.g., #NFP) to calculate new data.
These calculations create a table with
StockNum (the ID number of the relevant stock item - for this example, I've used one stock item 12422)
MonthNum (I've used integers this rather than dates, for clarity/simplicity)
BucketName (an nvarchar representing the month, used for column names)
Stock at start of month
Safety stock (which I assume is incoming stock, 5 per month)
NFP (which I assume is outgoing stock, varies by month and comes from a scratch table here - you'll need to adjust this to your select)
Stock at end of month
/* CALCULATE PROJECTIONS */
CREATE TABLE #StockProjections (StockNum int, BucketName nvarchar(30), MonthNum int, StockAtStart int, SafetyStock int, NFP int, StockAtEnd int, PRIMARY KEY (StockNum, BucketName))
; WITH StockByMonth AS
(-- Anchor
SELECT TOP 1 StockNum, MonthNum, StockAtStart, SafetyStock, NFP, StockAtEnd
FROM #Stock S
ORDER BY MonthNum DESC
-- Recursion
UNION ALL
SELECT NFP.StockNum,
SBM.MonthNum + 1 AS MonthNum,
SBM.StockAtEnd AS NewStockAtStart,
5 AS Safety_Stock,
NFP.StockNFP,
SBM.StockAtEnd + 5 - NFP.StockNFP AS NewStockAtEnd
FROM StockByMonth SBM
INNER JOIN #NFP_by_month NFP ON NFP.MonthNum = SBM.MonthNum + 1
WHERE NFP.MonthNum <= 18
)
INSERT INTO #StockProjections (StockNum, BucketName, MonthNum, StockAtStart, SafetyStock, NFP, StockAtEnd)
SELECT StockNum, BucketName, MonthNum, StockAtStart, SafetyStock, NFP, StockAtEnd
FROM StockByMonth
INNER JOIN #RepBuckets ON StockByMonth.MonthNum = #RepBuckets.BucketNum
Now we have the data, we set up a table for reporting purposes. Note that this table has the month names embedded into the column names (e.g., StockAtEnd_Jun_21). It would be easier to use a generic name (e.g., StockAtEnd_Month4) but I've gone for the slightly more complex case here for demonstration.
/* SET UP TABLE FOR REPORTING */
DECLARE #cols int = 18
DECLARE #iteration int = 0
DECLARE #colname nvarchar(30)
DECLARE #sql2 as nvarchar(max)
CREATE TABLE #tmp (StockNum int PRIMARY KEY)
WHILE #iteration <= #cols
BEGIN
SET #colname = (SELECT TOP 1 BucketName FROM #RepBuckets WHERE BucketNum = #iteration)
SET #sql2 = 'ALTER TABLE #tmp ADD ' + QUOTENAME(#colname) + ' int'
EXEC (#sql2)
SET #iteration = #iteration + 1
END
The last step is to add the data to your reporting table. I've used a pivot here but feel free to use whatever you like.
/* POPULATE TABLE */
DECLARE #columnList nvarchar(max) = N'';
SELECT #columnList += QUOTENAME(BucketName) + N' ' FROM #RepBuckets
SET #columnList = REPLACE(RTRIM(#columnList), ' ', ', ')
DECLARE #sql3 nvarchar(max)
SET #sql3 = N'
;WITH StockPivotCTE AS
(SELECT *
FROM (SELECT StockNum, BucketName, StockAtEnd
FROM #StockProjections
) StockSummary
PIVOT
(SUM(StockAtEnd)
FOR [BucketName]
IN (' + #columnList + N')
) AS StockPivot
)
INSERT INTO #tmp (StockNum, ' + #columnList + N')
SELECT StockNum, ' + #columnList + N'
FROM StockPivotCTE'
EXEC (#sql3)
Here's a DB<>fiddle showing it running with results of each sub-step.