I have created a report in Cognos Report Studio using pass through SQL syntax. My query includes a couple of common table expressions. How can i pass prompts to my query? I would like to use one optional date filter, which then is used in two cte:s. Then another required and multi choice text-filter that will be used in the final select statement.
Below is a simplified version of my query:
WITH in_date AS
(SELECT * FROM in_dates WHERE in_date > optional_date_prompt),
out_date AS
(SELECT * FROM out_dates WHERE out_date > optional_date_prompt),
organisation AS
(SELECT * FROM organisation)
-- some joins and unions later i end up with this table
SELECT * FROM final_table
WHERE organisation_name = 'required_text_prompt' OR
organisation_name = 'optional_text_prompt_value'
To use the prompts as regular cognos filters applied on the report page is not an option as the report would take hours to run.
To pass in parameters directly to SQL use a macro. A prompt macro for a string prompt called org would look like this:
#prompt('org','string')#
The first parameter is the prompt name and the second the type. This is the minimum amount of parameters that are needed to be specified. There are other optional parameters, such as default value that can be specified as well. You can consult the Cognos documention for more options.
You put the prompt macro in your WHERE clause:
WHERE organisation_name = #prompt('org','string')#
Cognos will see the macro and resolve it before sending the SQL on to the data source.
Related
I would like to create a function which runs a specific query that will extract any DimPatientID that is listed within the DimTestPatient table. Currently, I use the WHERE clause to exclude the test IDs from my query. I would like a function because I will use the WHERE clause on almost any query that I will run in the future.
SELECT
COUNT(*)
FROM
dbo.DimPatient dp
LEFT JOIN
dbo.FactTreatmentHistory th ON dp.DimPatientID = th.DimPatientID
WHERE
dp.DimPatientID NOT IN (SELECT DimPatientID
FROM dbo.DimTestPatient)
try using STORED PROCEDURE , it also supports parameters which are handy when you want to return a result set in the form of table.
it can utilise whole query , including WHERE clause as well.
for details follow the below link :
https://learn.microsoft.com/en-us/sql/relational-databases/stored-procedures/create-a-stored-procedure?view=sql-server-ver16
I want to set local variables or pass parameters from Excel to SQL. I've found similar questions, but all referred to old versions of Excel and/or the answers showed how to filter or manipulate output from a generic SQL query in the Power Query Editor, rather than pass a parameter or modify the SQL, so that the SQL Server supplies data in the needed form.
I'm building a large Microsoft Excel spreadsheet that depends on ten different SQL queries, all against a common SQL Server database. Excel and SQL Server are installed on my laptop and are current versions (as of 16 Mar 2022). All ten queries share a common date restriction, imposed in the WHERE clauses of the queries. The tables accessed and the form of output are very different, so there is no easy way to combine the ten queries into a single query. The queries contain multiple levels of aggregation (e.g. SUM(...)) so I need to restrict the records accessed prior to aggregation and passing results from the query back to Excel.
As currently written, each query begins by setting two date values in local variables. For example,
DECLARE #BEGIN_DATE AS smalldatetime;
DECLARE #END_DATE AS smalldatetime;
#BEGIN_DATE = CAST('2021-03-01 00:00' AS smalldatetime);
#END_DATE = CAST('2021-03-02 23:59' AS smalldatetime);
Every one of the ten queries includes a line in the WHERE clause similar to
WHERE
PickUpDate BETWEEN #BEGIN_DATE AND #END_DATE
Every query will use the same pair of dates. However, the column filtered (PickUpDate above) changes from one query to the next.
As it is, I have to manually edit each of the ten queries to change the two dates--twenty edits in all. This is time-consuming and error-prone. Ideally, I'd like to set the date range in the spreadsheet, in a pop-up dialog box, or any other convenient way and pass the dates to the SQL queries. Then by selecting Data > Refresh All in Excel, update all my tables at once.
Is this possible, and if so, how does one do it?
The answer from David Browne is generally on-target. But I found some difficulties reading data from an Excel table directly into the SQL, given security restrictions in the latest version of Excel/Power Query. Also, since this was the first time I worked directly in M-code and the advanced editor, it was challenging to fill-in the gaps.
I finally got a nice solution running; here is what worked for me.
First, I stored the parameter values in a two-column table. My table is named "ParameterTable" with column headers named "Parameter_Name" and "Value". The value(s) to pass to SQL Server are stored in the Value column. My table has two rows with row entries labeled "Begin_DateTime" and "End_DateTime".
Secondly I created a callable function named “ftnGetParameter.” Select Data > Get Data > From Other Sources > Blank Query. Then select “Advanced Editor.” Delete any boilerplate added by Excel, and enter and save this function
let theParameter=(TableName,ParameterLabel) =>
let
Source=Excel.CurrentWorkbook(){[Name=TableName]}[Content],
value = Source{[Parameter_Name=ParameterLabel]}[Value]
in
value
in
theParameter
Thirdly, code-up your SQL statement as usual. I was trying to pass dates to SQL, so I initially coded with string literals. Enter the query in the usual way. I used Data > Get Data > From Database > From SQL Server Database. Then pasted in the SQL. The two relevant lines in my query looked like this:
DECLARE #BEGIN_DATE AS SMALLDATETIME='2021-01-01 00:00';
DECLARE #END_DATE AS SMALLDATETIME='2021-12-31 23:59';
You could skip this step, but it allowed me to get complex SQL code entered, formatted, and running before I invoked the function to pass the parameters.
Finally, simply replace the string literals in the SQL with code to call the function. My first few lines of M-code looks like this:
let
Source = Sql.Database("DESKTOP-04P8E8C", "nfbdata",
[Query=
"
DECLARE #BEGIN_DATE AS SMALLDATETIME= '" & ftnGetParameter("ParameterTable","Begin_DateTime") & "';
DECLARE #END_DATE AS SMALLDATETIME='" & ftnGetParameter("ParameterTable","End_DateTime") & "' (… the query continues )
Excel will issue some warnings about running the query and prompt you to edit permissions. Once permission has been granted, the function reads the text from the parameter table and passes it into the SQL.
I found that the function call was not optional. Apparently, importing the code directly into a native call to SQL Server is considered an unacceptable security risk.
Many thanks to Mr. David Browne. His post definitely points in the right direction.
You can reference a table on a sheet from Power Query and integrate values from that table into your other queries. Eg if ParameterTable is a single-row table on some worksheet with a column called "StartDate", something like
let
theDate = Date.From( Record.Field(Table.First(ParameterTable),"StartDate") ),
Source = Sql.Databases("localhost"),
AdventureWorksDW2017 = Source{[Name="AdventureWorksDW2017"]}[Data],
dbo_DimDate = AdventureWorksDW2017{[Schema="dbo",Item="DimDate"]}[Data],
#"Filtered Rows" = Table.SelectRows(dbo_DimDate, each [FullDateAlternateKey] = theDate )
in
#"Filtered Rows"
for M query folding, or
let
theDate = Date.From( Record.Field(Table.First(ParameterTable),"StartDate") ),
sql = "
select *
from dimDate
where FullDateAlternateKey = '" & Text.From(theDate) & "'
",
Source = Sql.Database("localhost", "adventureworksdw2017", [Query=sql])
in
Source
for dynamic SQL.
I have looked at the articles on stackoverflow about this issue.
I have also reviewed the article for calling stored procedures with parameters at https://www.c-sharpcorner.com/article/execute-sql-server-stored-procedure-with-user-parameter-in-power-bi/.
In my case, I have a stored procedure with no parameters.
I am unclear on how I would apply a fix-up to the M script in Power Query Editor to call a stored procedure with no parameters so that the stored procedure can be recognized and used by Power BI.
Could someone provide guidance for my scenario and steps below?
Scenario
I am using a Power BI with DirectQuery.
I need an ordered list or rows from my database. So I created a stored procedure in my SQL database that simply wraps a SQL SELECT statement with an ORDER BY clause.
The stored procedure has no parameters.
Steps
In SQL Server Management Studio, I create and test my stored procedure.
CREATE PROCEDURE [dbo].[pbiGetFileInfo]
AS
BEGIN
SET NOCOUNT ON;
SELECT dbo.CurrentReport.JobId AS CurrentJobId,
dbo.jobs.id AS JobId,
dbo.JobInstruments.Id AS JobInstrumentId,
dbo.JobInstruments.InstrumentDescription,
dbo.JobInstruments.Notes,
dbo.JobInstruments.Latitude,
dbo.JobInstruments.Longitude,
dbo.JobInstruments.Depth,
dbo.jobinstrumentimport.filename,
dbo.jobinstrumentimport.mindate AS FromDate,
dbo.jobinstrumentimport.maxdate AS ToDate,
DATEDIFF(hour, dbo.jobinstrumentimport.mindate, dbo.jobinstrumentimport.maxdate) AS duration_hours
FROM dbo.CurrentReport INNER JOIN
dbo.jobs ON dbo.CurrentReport.JobId = dbo.jobs.id INNER JOIN
dbo.JobInstruments ON dbo.jobs.id = dbo.JobInstruments.JobId INNER JOIN
dbo.jobinstrumentimport ON dbo.JobInstruments.Id = dbo.jobinstrumentimport.jobinstrumentid
ORDER BY JobInstruments.Id, FromDate
END
GO
In Power BI, I click the Transform Data button to launch the Power Query Editor.
Under queries, I right-click the first empty entry in the Queries pane and highlight the New Query item and click SQL Server from the context menu.
In the SQL Server Database dialog, I enter the Server and Database.
In the SQL Server Database dialog, I click the Advanced Options link to expand the dialog and show the SQL statement (optional, requires database) field.
In the SQL statement (optional, requires database), I enter EXEC [dbo].[pbiGetFileInfo] and click the OK button.
A truncated preview of the data returned by the stored procedure is displayed.
I click OK at the bottom of the preview.
A new entry Query1 appears in the Queries pane.
I right-click the new Query1 entry and rename it to pbiGetFileInfo. The M syntax that appears for the query at this point is:
= Sql.Database("Server Name", "NWBDatabase", [Query="EXEC [dbo].[pbiGetFileInfo]"])
At this point, if I click "Apply" from the Power Query Editor Ribbon, I will get the error message:
Incorrect syntax near the keyword 'EXEC'. Incorrect syntax near ')'
I click the Advanced Editor button on the toolbar. The M script for the pbiGetFileInfo query is:
let
Source = Sql.Database("Server Name", "NWBDatabase", [Query="EXEC [dbo].[pbiGetFileInfo]"])
in
Source
At this point, I am stuck.
My questions are:
The stored procedure has no parameters. Do I need to add a SQLSource prefix to the M script? If I do need a SQLSource, what would that look like?
let
SQLSource ...
let
Source = Sql.Database("Server Name", "NWBDatabase", [Query="EXEC [dbo].[pbiGetFileInfo]"])
in
Source
in
SQLSource
One thought is to create a view in SQL that calls the stored procedure. I have tried this and found that the view returns the same warning in Power BI as you would see if you tried to create a View with an ORDER BY in SQL. Calling views from Power BI is problematic at best.
Is there any way to write a stored procedure in SQL that minimizes the workarounds required to use them from Power BI?
Updates
I cannot call a stored procedure from Power BI under DirectQuery. It returns the same error `Incorrect syntax near 'EXEC' message. I need to see the DAX that is created to find the source of this error.
If I try the raw SQL Select from the stored procedure that I am trying to call, I get the following error:
Microsoft SQL: The ORDER BY clause is invalid in views.
Note: this is using straight SQL SELECT. The word VIEW is does not exist in the SQL SYNTAX at all.
A SQL Select that calls a VIEW only works if the calling outer SELECT contains a TOP (100) PERCENT clause. For example:
My view named [pbiGetFileInfo] contains the following SELECT statement:
SELECT dbo.CurrentReport.JobId AS CurrentJobId,
dbo.jobs.id AS JobId,
dbo.JobInstruments.Id AS JobInstrumentId,
dbo.JobInstruments.InstrumentDescription,
dbo.JobInstruments.Notes,
dbo.JobInstruments.Latitude,
dbo.JobInstruments.Longitude,
dbo.jobinstrumentimport.filename,
dbo.jobinstrumentimport.mindate AS FromDate,
dbo.jobinstrumentimport.maxdate AS ToDate,
DATEDIFF(hour, dbo.jobinstrumentimport.mindate, dbo.jobinstrumentimport.maxdate) AS duration_hours
FROM dbo.CurrentReport INNER JOIN
dbo.jobs ON dbo.CurrentReport.JobId = dbo.jobs.id INNER JOIN
dbo.JobInstruments ON dbo.jobs.id = dbo.JobInstruments.JobId INNER JOIN
dbo.jobinstrumentimport ON dbo.JobInstruments.Id = dbo.jobinstrumentimport.jobinstrumentid
The view itself does not contain an ORDER BY clause.
When I try to call this from a SQL SELECT statement:
SELECT * FROM [dbo].[pbiGetFileInfo] ORDER BY Id,FromDate
I get the error:
Microsoft SQL: The ORDER BY clause is invalid in views...
It works if I revise the SELECT to:
SELECT TOP (100) PERCENT * FROM [dbo].[pbiGetFileInfo] ORDER BY Id,FromDate
But, I am not sure it this will work correctly in Power BI DirectQuery.
My first thought is that Power BI seems to treat everything as a SQL VIEW. So all data sources are subject to the limitations of views. None of the advantages of sorting on a SQL Server are actually available in Power BI under DirectQuery. If you have to set the sort order in Power BI, there may be significant performance penalties.
I am experimenting with Table-Valued Functions (but have no faith that this will work in Power BI).
Is it possible to get exactly the same parameters as shown in the All execution Overview report (see the printscreen below)?
I was trying to use the table [internal].[execution_parameter_values] from SSISDB and filter it via execution_id, nevertheless it returns much more parameters than in the report. I have also tried to filter it with table attribute "value_set", "object_type", etc. but still it did not return the same list as in the report.
Reference:
https://learn.microsoft.com/en-us/sql/integration-services/system-views/views-integration-services-catalog?view=sql-server-2017
execution_parameter_value:
Displays the actual parameter values that
are used by Integration Services packages during an instance of
execution.
Whenever the package is executed, records are inserted into that table. You need to determine the execution_id that you want to filter on.
You can get that from [catalog].[executions] in the SSIS DB. Filter based on your project or package and when it was executed.
Or you will also see that in the execution overview report as "Operation ID":
You can then filter based on that value:
SELECT * FROM [internal].[execution_parameter_values]
WHERE [execution_id] = 16529
Overview report in the SSIS catalog shows only TOP 25 used parameters sorted by parameter_name ASC.
Also, it is needed to filter out the records with parameter_name without "." character.
So the result T-SQL script would be:
SELECT TOP 25
[parameter_name]
,[parameter_value]
,[parameter_data_type]
FROM [SSISDB].[internal].[execution_parameter_values]
WHERE execution_id = #execution_id AND parameter_name not like '%.%'
ORDER BY parameter_name
I want to send the date range and employee ids from SSRS to Oracle package. Does anyone knows how to do it? Even if i try to declare an oracle package with three in parameters from_date, to_date and employee_id it works fine with one employee id. But fails if i select multiple employees in SSRS web interface saying wrong number of parameters. Does anyone know how to handle this?
Normally with SQL and SSRS you would do something like
Select * From Table where Table.Field in #Param
Oracle is different of course and depends on the Connection Type you are using.
ODBC connection you would use something like:
Select * From Table where Table.Field = ?
Order of the parameters is important here so be careful.
OLEDB connection you would use something like:
Select * From Table where Table.Field = :Param
However, none of the above work with multi selecting of the Parameters. You could do this:
=”Select * From Table where Table.Field in (‘” + Join(Parameters!parameter.Value,”‘, ‘”) + “‘)”
or
=”Select * From Table where Table.Field in(” + Join(Parameters!parameter.Value,”, “) + “)” if the values or your field is only numeric.
Here are a couple of good articles that you can look at. It has a better explanation than I can give personally without more information; Details of your Oracle Environment, Connection Type, etc.
Quick Reference:
Be sure your Oracle version is 9 or greater, none of this works on
versions 8 or older.
When using parameters with Oracle, use a : instead of an # sign –
:param instead of #param.
Add an ‘ALL’ option to your datasets that supply values for
multivalued drop down parameters.
Check for the ALL in your where clause by using “where ( field1 in (
:prmField1 ) or ‘ALL’ in ( :prmField1 ) )” syntax.
You can execute your query from the dataset window, but can only
supply 1 value. However that value can be ‘ALL’.
Educate your users on ‘ALL’ versus ‘(select all)’ .
Another good article about Multiple Parameters in SSRS with Oracle: Davos Collective Which is referenced in the top portion.
Hope this helps!