SQL Server Denominator Pushed to Zero - sql-server

Question

I am using the following formula to calculate the Pearson correlation in my data. Note: I am using a CASE WHEN to account for a divide by zero error. The code below represents solely the formula.
( COUNT(*) * SUM(X * Y) - SUM(X) * SUM(Y) )
/ ( SQRT(COUNT(*) * SUM(X * X) - SUM(X) * SUM(x)) * SQRT(COUNT(*) * SUM(Y* Y) - SUM(Y) * SUM(Y) ) )
Edit added query:
DROP TABLE IF EXISTS #test;
SELECT year
,product_id
,score_range
,reporting_year
/* used to manually calculate correlation in excel */
,COUNT(*) AS n_count
,COUNT(*) * SUM(1_x * 2_score) - SUM(1_x) * SUM(2_score) AS numerator
,SUM(1_x * 1_x) AS 1_sumprod
,SUM(1_x) AS 1_sum
,SUM(2_score * 2_score) AS 2_sumprod
,SUM(2_score) AS 2_sum
INTO #test
FROM #acct_details
GROUP BY year
,product_id
,score_range
,reporting_year
;
SELECT year
,product_id
,score_range
,reporting_year
,CASE
WHEN ( ( SQRT(n_count * 1_sumprod - 1_sum * 1_sum) * SQRT(n_count * 2_sumprod - 2_sum * 2_sum) ) ) = 0
THEN NULL
ELSE numerator / ( ( SQRT(n_count * 1_sumprod - 1_sum * 1_sum) * SQRT(n_count * 2_sumprod - 2_sum * 2_sum) ) )
END AS sql_corr
,(n_count * 1_sumprod - 1_sum * 1_sum) 1_denom
,( SQRT(n_count * 2_sumprod - 2_sum * 2_sum) ) AS 2_denom
FROM #test
ORDER BY year
,reporting_year
,score_range
;
The output of my data looks like the table below. Note that excel_corr is the correlation manually calculated in Excel, which is my expected output.
The column sql_corr is the result from my sql code above. The columns from count to the end represent the X and Y values that get plugged into the formula above. My problem is that the sql_corr does not match the output from manually calculating the correlation by groupings in Excel.
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| year | product_id | score_range | reporting_year | sql_corr | count | numerator | 1_sumprod | 1_sum | 2_sumprod | 2_sum | excel_corr |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 1-2 | 2016 | NULL | 1 | 0 | 0.000124 | -0.011155 | 195364 | 442 | #DIV/0! |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 3-4 | 2016 | NULL | 1272 | -0.0683 | 4.9E-11 | -0.000007 | 304648060 | 622434 | -0.02911 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 5-6 | 2016 | -0.06416 | 3913 | -11.845 | 2.89E-09 | -0.000459 | 1.089E+09 | 2063948 | -0.06391 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 7-8 | 2016 | 0.00573 | 2593 | 1.63663 | 2.27E-08 | -0.000975 | 848560006 | 1482872 | 0.00573 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 9-10 | 2016 | -0.02106 | 1420 | -3.2855 | 4.13E-08 | -0.00131 | 555096971 | 887587 | -0.02106 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 11-12 | 2016 | 0.05231 | 917 | 6.64768 | 1.06E-07 | -0.000987 | 413059274 | 615312 | 0.052438 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 13-14 | 2016 | 0.006704 | 359 | 0.5064 | 6.18E-07 | 0.000271 | 185781413 | 258205 | 0.006705 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 15-16 | 2016 | 0.017846 | 55 | 0.14095 | 3.79E-06 | 0.000349 | 31849498 | 41850 | 0.017839 |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
| 2016 | 1 | 17-18 | 2016 | NULL | 1 | 0 | 0 | 0 | 641601 | 801 | #DIV/0! |
+------+------------+-------------+----------------+----------+-------+-----------+-----------+-----------+-----------+---------+------------+
For example, in score_range 3-4 the sql_corr value is NULL but in excel the value is -0.02911. If we plug in the values manually into the formula -0.02911 is the correct result.
numerator
/ ( ( SQRT(n_count * 1_sumprod - 1_sum * 1_sum) * SQRT(n_count * 2_sumprod - 2_sum * 2_sum) ) )
In SQL Server the denominator is getting pushed to 0. When I calculate this manually in Excel the denominator is 2.344354. Why is my denominator being pushed to 0 in SQL Server when the same data results in a different calculation when done manually?
Edit
The first part of the denominator is being pushed to 0. ( ( SQRT(n_count * 1_sumprod - 1_sum * 1_sum). When the multiplication occurs the whole denominator gets pushed to 0 in SQL activating the CASE statement to return NULL. This is incorrect confirmed by manual calculation. The following represents the output from both parts of the denominator 0.000000 and 9394.0387480572. The actual value for the first part of the denominator via manual calculation is ~0.00025.
Edit
The value of (n_count * 1_sumprod - 1_sum * 1_sum) = 6.2279E-08 -- before taking the square root. However, SQL is pushing this part of the equation to 0.
I am using SQL Server 2016 v14.0.2037.2. I thought maybe my value was too small but it appears that values greater than 5E-18 should remain. This was confirmed in the documentation here.

Answer 1

Credit to TomPhillips here.
They problem you have is you have a mix of integers and floats. This causes confusions and conversions. Convert all your values to float to get the value you are expecting. This is what Excel does.
Even though I CAST the initial values going into #test from #acct_details as DECIMAL, I needed to explicitly CAST the values in #test to the formula all as DECIMAL also. The result kept getting rounded to 0 here: SUM(1_x * 1_x). CASTing the values explicitly resolved this by forcing the necessary precision.
DROP TABLE IF EXISTS #test;
SELECT year
,product_id
,score_range
,reporting_year
/* used to manually calculate correlation in excel */
,CAST(COUNT(*) AS DECIMAL(16,9)) AS n_count
,CAST( ( COUNT(*) * SUM(1_x * 2_score) - SUM(1_x) * SUM(2_score) ) AS DECIMAL(16,9)) AS numerator
,CAST(SUM(1_x * 1_x) AS DECIMAL(16,9)) AS 1_sumprod
,CAST(SUM(1_x) AS DECIMAL(16,9)) AS 1_sum
,CAST(SUM(2_score * 2_score) AS DECIMAL(16,9)) AS 2_sumprod
,CAST(SUM(2_score) AS DECIMAL(16,9)) AS 2_sum
INTO #test
FROM #acct_details
GROUP BY year
,product_id
,score_range
,reporting_year
;
Changing them to FLOAT also worked. Documentation here.
DROP TABLE IF EXISTS #test;
SELECT year
,product_id
,score_range
,reporting_year
/* used to manually calculate correlation in excel */
,CAST(COUNT(*) AS FLOAT) AS n_count
,CAST( ( COUNT(*) * SUM(1_x * 2_score) - SUM(1_x) * SUM(2_score) ) AS FLOAT) AS numerator
,CAST(SUM(1_x * 1_x) AS FLOAT) AS 1_sumprod
,CAST(SUM(1_x) AS FLOAT) AS 1_sum
,CAST(SUM(2_score * 2_score) AS FLOAT) AS 2_sumprod
,CAST(SUM(2_score) AS FLOAT) AS 2_sum
INTO #test
FROM #acct_details
GROUP BY year
,product_id
,score_range
,reporting_year
;