Dynamic SQL query with cell reference in Excel - sql-server

I have a query connected to a pivot table and I want it to show results based on a date from cell (so the user can change it without editing the query). I've read one way to do it is creating a connection to the cell
And drilling down that query so its data that can be referenced by another query
I've read I have to reference that connected table (just a date) through
SQL1 = Text.Combine({SQL,Date}),
SQL2 = Text.Combine({SQL1, "‘" }),
Source = Sql.Database("servername", "db", [Query= SQL2])
So I think it should look like this:
let
SQL =
"select trim(codart) AS REF,
descart AS 'DESCRIPTION',
codalm AS WAREHOUSE,
descalm AS WAREHOUSEDESCRIP,
unidades AS UNITS,
entran AS 'IN',
salen AS 'OUT',
m.entran*1 + m.salen*-1 as MOVEMENT,
(select sum(m1.entran*1 + m1.salen*-1)
from MOVSTOCKS m1
where m1.codart = m.codart and m1.codalm =
m.codalm and m.fecdoc >= m1.fecdoc) as 'CUMULATIVE',
PRCMEDIO as 'VALUE',
FECDOC as 'DATE',
REFERENCIA as 'REF',
tipdoc as 'DOCUMENT'
from (select m.*,
sum(m.entran*1 - m.salen*-1) over (partition by m.codart, m.codalm order by fecdoc) as cumulative,
max(fecdoc) over (partition by m.codart, m.codalm) as max_fecdoc
from MOVSTOCKS m
where fecdoc <= ‘ ), m
where fecdoc = max_fecdoc
and (m.entran <> 0 or m.salen <> 0)
order by fecdoc",
SQL1 = Text.Combine({SQL,Date}),
SQL2 = Text.Combine({SQL1, "‘" }),
Source = Sql.Database("servername", "db", [Query= SQL2])
in
Source
The problem is... if that dynamic cell is formated as date, the power query shows this error:
Expression.Error: We cannot convert the value #date (2020, 9, 30) to type Text.
Details:
Value= 30/09/2020
Type=Type
And if I format date as text, then it shows this error (after changing privacy options):
DataSource.Error: Microsoft SQL: Incorrect syntax near '‘'.
Details:
DataSourceKind=SQL
DataSourcePath=servername;db
Message=Incorrect syntax near '‘'.
Number=102
Class=15
I want SQL to read this
where fecdoc <= ‘ ), m
as
where fecdoc <= 30/09/2020 ), m
Having in mind that date comes from an excel cell that can be changed by the user.
I'm new with SQL and M, I don't even know if what I'm trying makes any sense.
Any feedback will be helpful. Thank you very much

Instead of whatever you're trying to do with SQL1 and SQL1, in your query, replace
where fecdoc <= ‘ ), m
with
where fecdoc <= '" & Date & "' ), m
where Date is a reference to your date in string format.

Related

SQL Server : Sum function is returning wrong value

So this is my query
select a.ERSDataValues_ERSCommodity_ID,c.ersGeographyDimension_country,
b.ERSTimeDimension_Year,
sum(a.ERSDataValues_AttributeValue) as Total
from cosd.ERSDataValues a ,cosd.ERSTimeDimension_LU
b,cosd.ERSGeographyDimension_LU c
where a.ERSDataValues_ERSCommodity_ID in (SELECT
ERSBusinessLogic_InputDataSeries
FROM [AnimalProductsCoSD].[CoSD].[ERSBusinessLogic]
where ERSBusinessLogic_InputGeographyDimensionID = 7493
and ERSBusinessLogic_InputTimeDimensionValue = 'all months'
and ERSBusinessLogic_Type = 'time aggregate')
and a.ERSDataValues_ERSTimeDimension_ID = b.ERSTimeDimension_ID
and c.ersGeographyDimension_country != 'WORLD'
and a.ERSDataValues_ERSGeography_ID=c.ERSGeographyDimension_ID
group by b.ERSTimeDimension_Year,a.ERSDataValues_ERSCommodity_ID,
c.ersGeographyDimension_country
order by b.ERSTimeDimension_Year,a.ERSDataValues_ERSCommodity_ID
This is the updated query, I know it is long but I am facing the same issue the sum does not match with values when I sum up manually
It's most likely a problem with your JOINs (which you aren't technically using). Try removing the SUM and GROUP functions and see if you're getting duplicate rows (maybe limit your time frame to get fewer rows if this is a large data set):
SELECT b.TimeDimension_Year ,
c.GeographyDimension_country ,
a.DataValues_AttributeValue
FROM cosd.DataValues a ,
cosd.TimeDimension_LU b ,
cosd.GeographyDimension_LU c
WHERE a.DataValues_Commodity_ID = 2257
AND a.DataValues_TimeDimension_ID = b.TimeDimension_ID
AND c.GeographyDimension_ID = 7493
AND b.TimeDimension_Year = 2015;

SSIS Package with User Variable in Where Clause

Simply put, I'm trying to take user input, store it in a variable, and then use that variable in a query.
First, I have a Script Task that opens a window with a simple input text box and a button. On click, the text gets put into a user variable and a message box pops up to display what is in the user variable. Clicking Ok on the message box closes both the message box and original window. This seems to work fine.
Next, I have an Execute SQL Task. Settings are as follows:
General
ResultSet = None
ConectionType = OLE DB
Connection = localhost.DB
SQLSourceType = Direct Input
SQLStatement = [SQL Code displayed below]
Parameter Mapping
Variable Name = User::VarName
Direction = Input
DataType = VARCHAR
Parameter Name = 0
Parameter Size = 65535
Result Set - N/A
Expressions - N/A
[SQL CODE]
if object_id('TEST.dbo.TEST','U') is not null
drop table TEST.dbo.TEST;
SELECT
coalesce(FSC.a, format(cast(SVY.b as int),'000')) as ab,
SVY.c,
PP.d,
SV1.e,
PP.f,
PP.g,
cast(PP.g as float) AS g,
SV1.h,
SV1.i,
SVY.j,
SVY.k,
format(cast(SVY.l as int), '00000') as l,
CAST(SVY.m AS float) AS m,
SVY.n,
SV1.o,
SVY.p,
cast(PID.q as float) as q,
cast(PID.r as float) as r,
cast(PID.s as float) as s,
cast(PID.t as float) as t,
PID.u as u,
PID.v as v,
PP.w,
CAL_B.x as x,
CAL_B.y as y,
CAL_B.z as z,
CAL_R.aa as aa,
CAL_R.bb as bb,
CAL_R.cc as cc
into TEST.dbo.TEST
FROM AAA.dbo.AAA PP
INNER JOIN BBB.dbo.BBB PPC
ON PP.x = PPC.x
AND cast(PP.x as date) = cast(PPC.x as date)
RIGHT OUTER JOIN CCC.dbo.CCC SVY
ON PPC.x = SVY.x
AND cast(PPC.x as date) = cast(SVY.x as date)
LEFT OUTER JOIN DDD.dbo.DDD FSC
ON SVY.x = FSC.x
AND cast(SVY.x as date) = cast(FSC.x as date)
LEFT OUTER JOIN EEE.dbo.EEE SV1
ON SVY.x = SV1.x
AND SVY.x = SV1.x
AND SVY.x = SV1.x
AND SVY.x = SV1.x
AND cast(SVY.x as date) = cast(SV1.x as date)
AND PPC.x = SV1.x
AND cast(PPC.x as date) = cast(SV1.x as date)
INNER JOIN FFF.dbo.FFF RLS
ON SV1.x = RLS.x
AND SV1.x = RLS.x
AND cast(SV1.x as date) = cast(RLS.x as date)
AND SVY.x = RLS.x
AND SVY.x = RLS.x
AND SVY.x = RLS.x
AND cast(SVY.x as date) = cast(RLS.x as date)
LEFT OUTER JOIN GGG.dbo.GGG PID
ON PP.x = PID.x
AND coalesce(FSC.x, format(cast(SVY.x as int),'000')) = PID.x
LEFT OUTER JOIN HHH.dbo.HHH CAL_B
ON cast('20' + SUBSTRING(RLS.x,4,2) + '-' + SUBSTRING(RLS.x,6,2) + '-' + SUBSTRING(RLS.x,8,2) as date) = CAL_B.x
AND CAL_B.x = 1
LEFT OUTER JOIN III.dbo.III CAL_R
ON cast('20' + SUBSTRING(RLS.x,4,2) + '-' + SUBSTRING(RLS.x,6,2) + '-' + SUBSTRING(RLS.x,8,2) as date) = CAL_R.x
AND CAL_R.x = 1
where
cast(SVY.x as date) = (select cast(max(x) as date) from JJJ.dbo.JJJ)
and
PP.x is not null
and
SVY.x in ( ? )
THE ISSUE: I run the package. Text input box opens. I put in my text, " 'CN05','CN06' " (without double quotes), I click OK. Box pops up showing me my input. I click OK. Workflow moves on to Execute SQL Task. In 1 second the task completes with a green check, no errors. I verify that TEST.dbo.TEST has been created but it is empty.
Now, if I run the above code in SSMS hardcoding the last bit [...SVY.x in ('CN05','CN06')], I pull back over 5 million records in about 4 minutes. I am stumped as to why this isn't working in SSIS. Any ideas out there?
In general, parameterized WHERE IN clauses don't play nice. They never work the way we think they should. Instead, you can use SSIS' version of dynamic SQL to achieve the same thing. Create a string variable to store the concatenation of your larger SQL statement with your user input. Then, wire your Execute SQL Task to use this variable.
For example, please create a new SSIS string variable called SqlStatement. In the properties window for the variable, open the expression builder window by clicking the ellipses. Use this expression builder to concatenate your SQL with the user input. Sample below ...
Be sure to set the EvaluateAsExpression property of the variable to True.
Lastly, change your Execute SQL Task over from Direct Input to Variable and select this variable as the source.
All set. Your Execute SQL Task should now be wired to use the dynamic SQL you built inside of the expression builder. Good luck!

Why do I get "Conversion failed when converting date and/or time from character string" when subtracting a number from GETDATE()?

I cannot figure out what I am doing incorrect here. I have the following values for these variables.
my $sql = qq~
SELECT COUNT(*)
FROM tableName u
WHERE 1=1 AND u.ManufacturerPartNumber IN ('X','Y','Z')
AND CAST(u.InspectionStartDate AS DATETIME) > (GETDATE() - ?)
~;
my $argsRef->{lookBack} = 30;
And when I try to run a selectrow_array on the sql like such:
my $qnCount = $dbh->selectrow_array($sql, undef, $argsRef->{lookBack});
I get the following error:
DBD::ODBC::db selectrow_array failed: [Microsoft][ODBC SQL Server Driver][SQL Server]Conversion failed when converting date and/or time from character string. (SQL-22007) [for Statement "
SELECT COUNT(*)
FROM tableName u
WHERE 1=1 AND u.ManufacturerPartNumber IN ('X','Y','Z')
AND CAST(u.InspectionStartDate AS DATETIME) > (GETDATE() - ?)
"]
So, it is my understanding that the third parameter in the selectrow_array call from a database handle should be the bind variables. Yet, they seem not to be binding....
the database handle is created as such:
my $dbh = DBI->connect_cached("dbi:ODBC:$dsn", undef, undef, {
PrintError => 0,
RaiseError => 1,
ShowErrorStatement => 1,
LongReadLen => 500000,
})
$dsn is the correct DSN but not shown for various reasons, but we know it works due the error message coming back from SQL Server.
Any idea what I am doing wrong?
Thanks in advance for your help.
First, a shout out to ThisSuitIsBlackNot for pointing out that the problem. I made the assumption that the DBI/driver would be able to determine the value type. That was wrong. So, I explicitly stated the type:
$sth = $dbh->prepare($sql);
$sth->bind_param( 1, ($argsRef->{lookBack} * -1), { TYPE => SQL_INTEGER });
$sth->execute();
my $qnCount = $sth->fetchrow_array();
This allowed it to go through as an integer and the SQL ran without problem.
In case you were wondering why I am multiplying by -1, is because I changed the query to use DATEADD:
SELECT COUNT(*)
FROM tableName u
WHERE 1=1 AND u.ManufacturerPartNumber IN ('x')
AND CAST(u.InspectionStartDate AS DATETIME) > DATEADD(dd, ?, GETDATE())
Although there seems to be some continuing discussion in regards to its use, it is working here.
Thank you all for your help.

T-SQL Temp Tables and Stored Procedures from R using RevoScaleR

In the example below, I was able to get the query to work with one exception. When I use q in place of source.query during the RxSqlServerData step, I get the error rxCompleteClusterJob Execution halted.
The first goal is to use a stored procedure in place of a longer query. Is this possible?
The second goal would be to create and call upon a #TEMPORARY table within the stored procedure. I'm wondering if that is possible, as well?
library (RODBC)
library (RevoScaleR)
sqlConnString <- "Driver=SQL Server;Server=SAMPLE_SERVER; Database=SAMPLE_DATABASE;Trusted_Connection=True"
sqlWait <- TRUE
sqlConsoleOutput <- FALSE
sql_share_directory <- paste("D:\\RWork\\AllShare\\", Sys.getenv("USERNAME"), sep = "")
sqlCompute <- RxInSqlServer(connectionString = sqlConnString, wait = sqlWait, consoleOutput = sqlConsoleOutput)
rxSetComputeContext(sqlCompute)
#This Sample Query Works
source.query <- paste("SELECT CASE WHEN [Order Date Key] = [Picked Date Key]",
"THEN 1 ELSE 0 END AS SameDayFulfillment,",
"[City Key] AS city, [STOCK ITEM KEY] AS item,",
"[PICKER KEY] AS picker, [QUANTITY] AS quantity",
"FROM [WideWorldImportersDW].[FACT].[ORDER]",
"WHERE [WWI ORDER ID] >= 63968")
#This Query Does Not
q <- paste("EXEC [dbo].[SAMPLE_STORED_PROCEDURE]")
inDataSource <- RxSqlServerData(sqlQuery=q, connectionString=sqlConnString, rowsPerRead=500)
order.logit.rx <- rxLogit(SameDayFulfillment ~ city + item + picker + quantity, data = inDataSource)
order.logit.rx
Currently, only T-SQL SELECT statements are allowed as input data-set, not stored procedures.

LINQ to SQL anomaly

I'm trying to learn LINQ to SQL. I've run into something I just don't get. Here's the LINQ program (vb.net):
Imports System.IOModule Module1
Sub Main()
Dim crs = New DataClasses1DataContext()
Dim sw As New StringWriter()
crs.Log = sw
Dim reports = From report In crs.CRS_Report_Masters
Group report By report_id = report.Report_ID Into grouped = Group
Select New With {
.reportId = report_id,
.two = grouped.Sum(
Function(row) row.active_report * row.Report_ID)
}
For Each report In reports
Console.WriteLine("{0} {1}", report.reportId, report.two)
Next
MsgBox(sw.GetStringBuilder().ToString())
End Sub
End Module
Here's the SQL it produces:
SELECT SUM([t1].[value]) AS [two], [t1].[Report_ID] AS [reportId] FROM (
SELECT (-(CONVERT(Float,[t0].[active_report]))) *
(CONVERT(Float,CONVERT(Float,[t0].[Report_ID]))) AS [value], [t0].[Report_ID]
FROM [dbo].[CRS_Report_Master] AS [t0]
) AS [t1] GROUP BY [t1].[Report_ID]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.30319.1
What I don't get is why there is a minus sign in the SQL before the math in the parentheses. I didn't specify that in the LINQ query.
If I recall correctly, VB.NET treats -1 as Boolean True, while SQL treats +1 as Boolean True. Therefore the minus sign is necessary for VB.NET to properly interpret the active_report field.
So, now I see what's going on. The LINQ to SQL definition of the column corresponding to a SQL bit is a .net Boolean. Fair enough. When doing math using it, however, the compiler negates the value as described by ekolis. I did not expect that -- in fact I think it's an error, since I do not get the results I expect. e.g. if active_report=1 and report_id=123, I expect to get "123" but I'm getting "-123". I modified the query like this:
Dim reports = (From report In crs.CRS_Report_Masters
Group report By report_id = report.Report_ID Into grouped = Group
Select New With {
.reportId = report_id,
.two = grouped.Sum(
Function(row) If(row.active_report, 1, 0) * CInt(row.Report_ID))
})
which changed the generated SQL to:
SELECT SUM([t1].[value]) AS [two], [t1].[Report_ID] AS [reportId] FROM (
SELECT (
(CASE
WHEN (COALESCE([t0].[active_report],#p0)) = 1 THEN #p1
ELSE #p2
END)) * (CONVERT(Int,[t0].[Report_ID])) AS [value], [t0].[Report_ID]
FROM [dbo].[CRS_Report_Master] AS [t0]
) AS [t1] GROUP BY [t1].[Report_ID]
-- #p0: Input Int (Size = -1; Prec = 0; Scale = 0) [0]
-- #p1: Input Int (Size = -1; Prec = 0; Scale = 0) [1]
-- #p2: Input Int (Size = -1; Prec = 0; Scale = 0) [0]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.30319.1
Makes sense I suppose, though it's a roundabout way get the desired result. In T-SQL, a simple SELECT [BOOLEAN]*[INTEGER] will achieve the same result, I believe. Lesson learned: Don't trust LINQ to always do the right thing. Check its work!

Resources