I have seen a similar post here but my situation is slightly different from anything I've found so far. I am trying to call a postgres function with parameters that I can leverage in the function logic as they pertain to the jsonb query. Here is an example of the query I'm trying to recreate with parameters.
SELECT *
from edit_data
where ( "json_field"#>'{Attributes}' )::jsonb #>
'{"issue_description":"**my description**",
"reporter_email":"**user#generic.com**"}'::jsonb
I can run this query just fine in PGAdmin but all my attempts thus far to run this inside a function with parameters for "my description" and "user#generic.com" values have failed. Here is a simple example of the function I'm trying to create:
CREATE OR REPLACE FUNCTION get_Features(
p1 character varying,
p2 character varying)
RETURNS SETOF edit_metadata AS
$BODY$
SELECT * from edit_metadata where ("geo_json"#>'{Attributes}' )::jsonb #> '{"issue_description":**$p1**, "reporter_email":**$p2**}'::jsonb;
$BODY$
LANGUAGE sql VOLATILE
COST 100
ROWS 1000;
I know that the syntax is incorrect and I've been struggling with this for a day or two. Can anyone help me understand how to best deal with these double quotes around the value and leverage a parameter here?
TIA
You could use function json_build_object:
select json_build_object(
'issue_description', '**my description**',
'reporter_email', '**user#generic.com**');
And you get:
json_build_object
-----------------------------------------------------------------------------------------
{"issue_description" : "**my description**", "reporter_email" : "**user#generic.com**"}
(1 row)
That way there's no way you will input invalid syntax (no hassle with quoting strings) and you can swap the values with parameters.
Related
If it's relevant I'm using Django with Django Rest Framework, django-mssql-backend and pyodbc
I am building some read only models of a legacy database using fairly complex queries and Django's MyModel.objects.raw() functionality. Initially I was executing the query as a Select query which was working well, however I received a request to try and do the same thing but with a table-valued function from within the database.
Executing this:
MyModel.objects.raw(select * from dbo.f_mytablefunction)
Gives the error: Invalid object name 'myapp_mymodel'.
Looking deeper into the local variables at time of error it looks like this SQL is generated:
'SELECT [myapp_mymodel].[Field1], '
'[myapp_mymodel].[Field2] FROM '
'[myapp_mymodel] WHERE '
'[myapp_mymodel].[Field1] = %s'
The model itself is mapped properly to the query as executing the equivalent:
MyModel.objects.raw(select * from dbo.mytable)
Returns data as expected, and dbo.f_mytablefunction is defined as:
CREATE FUNCTION dbo.f_mytablefunction
(
#param1 = NULL etc etc
)
RETURNS TABLE
AS
RETURN
(
SELECT
field1, field2 etc etc
FROM
dbo.mytable
)
If anyone has any explanation as to why these two modes of operation are treated substantially differently then I would be very pleased to find out.
Guess you've figured this out by now (see docs):
MyModel.objects.raw('select * from dbo.f_mytablefunction(%s)', [1])
If you'd like to map your table valued function to a model, this gist has a quite thorough approach, though no license is mentioned.
Once you've pointed your model 'objects' to the new TableFunctionManager and added the 'function_args' OrderedDict (see tests in gist), you can query it as follows:
MyModel.objects.all().table_function(param1=1)
For anyone wondering about use cases for table valued functions, try searching for 'your_db_vendor tvf'.
Given the following procedure
CREATE OR REPLACE PROCEDURE z()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
AS
$$
const what = snowflake.execute( {sqlText: "select $1 as value from values (1), (2), (3);"} )
return 'Hi There';
$$
;
If I run the 2 statements below
CALL z();
select * from table(result_scan(-2));
I get an error "SQL Error [709] [02000]: Statement 01912a0c-01c1-0574-0000-4de50036137e not found"
If I run the 2 statements below
CALL z();
SELECT LAST_QUERY_ID(), LAST_QUERY_ID(-1), LAST_QUERY_ID(-2);
It shows me that LAST_QUERY_ID() and LAST_QUERY_ID(-1) are identical but also that LAST_QUERY_ID(-2) returns NULL...
Any idea why it returns NULL rather than something that would allow me to retrieve the result of my query "select $1 as value from values (1), (2), (3);"?
Thanks
It shows me that LAST_QUERY_ID() and LAST_QUERY_ID(-1) are identical but also that LAST_QUERY_ID(-2) returns NULL
It's expected to see the same query ID for LAST_QUERY_ID() and LAST_QUERY_ID(-1), because they are same (the default value for LAST_QUERY_ID is "-1").
On the other hand, the LAST_QUERY_ID(-2) should not returns NULL, and the "select * from table(result_scan(-2))" query should not fail.
If you define the function with CALLER rights, you can see that the LAST_QUERY_ID(-2) returns the query ID and the "select * from table(result_scan(-2))" query works:
CREATE OR REPLACE PROCEDURE z()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
const what = snowflake.execute( {sqlText: "select $1 as value from values (1), (2), (3);"} )
return 'Hi There';
$$
;
As a workaround, you can use the history window to check the latest queries. You should be able to see your query (01912a0c-01c1-0574-0000-4de50036137e) in the history tab. You will notice that these queries are executed by same user, and in the same session. Therefore, there shouldn't be a restriction on listing these queries.
I am able to reproduce the issue on my test environment, and I will report the issue to the development team. If you have access to Snowflake support, it could be better to submit a support case regarding to this issue, so you can easily follow the process.
For the record, here is the answer I got from Snowflake Support
This is an expected behavior because of security limitation placed on purpose (queries from inside a SP do not have access to queries outside and vice versa).
We put a blanket limitation that the world of inside and outside such SPs should be separated. From one the "result" of a query from another cannot be accessed.
Yes, If we declare the procedure with caller rights, everything (LAST_QUERY_ID and RESULT_SCAN) works fine in this case.
There is a discussion going on to implement this use-case when the owner and caller of an "owner's right" SP are the same, all this limitations should be lifted. But we cannot be sure about the timeline, I can link this case with that discussion so that we have this record.
Sounds good to me :-)
My last question Passing an array to stored to postgres was a bit unclear. Now, to clarify my objective:
I want to create an Postgres stored procedure which will accept two input parameters. One will be a list of some amounts like for instance (100, 40.5, 76) and the other one will be list of some invoices ('01-2222-05','01-3333-04','01-4444-08'). After that I want to use these two lists of numbers and characters and do something with them. For example I want to take each amount from this array of numbers and assign it to corresponding invoice.
Something like that in Oracle would look like this:
SOME_PACKAGE.SOME_PROCEDURE (
789,
SYSDATE,
SIMPLEARRAYTYPE ('01-2222-05','01-3333-04','01-4444-08'),
NUMBER_TABLE (100,40.5,76),
'EUR',
1,
P_CODE,
P_MESSAGE);
Of course, the two types SIMPLEARRAYTYPE and NUMBER_TABLE are defined earlier in DB.
You will love this new feature of Postgres 9.4:
unnest(anyarray, anyarray [, ...])
unnest() with the much anticipated (at least by me) capability to unnest multiple arrays in parallel cleanly. The manual:
expand multiple arrays (possibly of different types) to a set of rows. This is only allowed in the FROM clause;
It's a special implementation of the new ROWS FROM feature.
Your function can now just be:
CREATE OR REPLACE FUNCTION multi_unnest(_some_id int
, _amounts numeric[]
, _invoices text[])
RETURNS TABLE (some_id int, amount numeric, invoice text) AS
$func$
SELECT _some_id, u.* FROM unnest(_amounts, _invoices) u;
$func$ LANGUAGE sql;
Call:
SELECT * FROM multi_unnest(123, '{100, 40.5, 76}'::numeric[]
, '{01-2222-05,01-3333-04,01-4444-08}'::text[]);
Of course, the simple form can be replaced with plain SQL (no additional function):
SELECT 123 AS some_id, *
FROM unnest('{100, 40.5, 76}'::numeric[]
, '{01-2222-05,01-3333-04,01-4444-08}'::text[]) AS u(amount, invoice);
In earlier versions (Postgres 9.3-), you can use the less elegant and less safe form:
SELECT 123 AS some_id
, unnest('{100, 40.5, 76}'::numeric[]) AS amount
, unnest('{01-2222-05,01-3333-04,01-4444-08}'::text[]) AS invoice;
Caveats of the old shorthand form: besides being non-standard to have set-returning function in the SELECT list, the number of rows returned would be the lowest common multiple of each arrays number of elements (with surprising results for unequal numbers). Details in these related answers:
Parallel unnest() and sort order in PostgreSQL
Is there something like a zip() function in PostgreSQL that combines two arrays?
This behavior has finally been sanitized with Postgres 10. Multiple set-returning functions in the SELECT list produce rows in "lock-step" now. See:
What is the expected behaviour for multiple set-returning functions in SELECT clause?
Arrays are declared by adding [] to the base datatype. You declare them as a parameter the same way you declare regular parameters:
The following function accepts an array of integers and and array of strings and will return some dummy text:
create function array_demo(p_data integer[], p_invoices text[])
returns text
as
$$
select p_data[1] || ' => ' || p_invoices[1];
$$
language sql;
select array_demo(array[1,2,3], array['one', 'two', 'three']);
SQLFiddle demo: http://sqlfiddle.com/#!15/fdb8d/1
I have a database table in SQL Server 2014 with only an ID column (int) and a column xmldata of type XML.
This xmldata column contains for example:
<book>
<title>a nice Novel</title>
<author>Maria</author>
<author>Peter</author>
</book>
As expected, I have multiple books, therefore multiple rows with xmldata.
I now want to execute a query for all books, where Peter is an Author. I tried this in some xPath2.0 testers and got to the conclusion that:
/book/author/concat(text(), if(position() != last())then ',' else '')
works.
If you try to port this success into SQL Server 2014 Express it looks like this, which is correctly escaped syntax etc.:
SELECT id
FROM books
WHERE 'Peter' IN (xmldata.query('/book/author/concat(text(), if(position() != last())then '','' else '''')'))
SQL Server however does not seem to support a construction like /concat(...) because of:
The XQuery syntax '/function()' is not supported.
I am at a loss then however, why /text() would work in:
SELECT id, xmldata.query('/book/author/text()')
FROM books
which it does.
My constraints:
I am bound to use SQL Server
I am bound to xpath or something else that can be "injected" as the statement above (if the structure of the xml or the database changes, the xpath above could be changed isolated and the application logic above that constructs the Where clause will not be touched) SEE EDIT
Is there a way to make this work?
regards,
BillDoor
EDIT:
My second constraint boils down to this:
An Application constructs the Where clause by
expression <operator> value(s)
expression is stored in a database and is mapped by the xmlTag eg.:
| tokenname| querystring
| "author" | "xmldata.query(/book/author/text())"
the values are presented by the Requesting user. so if the user asks for the author "Peter" with operator "EQUALS" the application constructs:
xmaldata.query(/book/author/text()) = "Peter"
as where clause.
If the customer now decides that author needs to be nested in an <authors> element, i can simply change the expression in the construction-database and the whole machine keeps running without any changes to code, simply manageable.
So i need a way to achieve that
<xPath> <operator> "Peter"
or any other combination of this three isolated components (see above: "Peter" IN <xPath>...) gets me all of Peters' books, even if there are multiple unsorted authors.
This would not suffice either (its not sqlserver syntax, but you get the idea):
WHERE xmldata.exist('/dossier/client[text() = "$1"]', "Peter") = 1;
because the operator is still nested in the expression, i could not request <> "Peter".
I know this is strange, please don't question the concept as a whole - it has a history :/
EDIT: further clarification:
The filter-rules come into the app in an XML structure basically:
Operator: "EQ"
field: "name"
value "Peter"
evaluates to:
expression = lookupExpressionForField("name") --> "table2.xmldata.value('book/author/name[1]', 'varchar')"
operator = lookUpOperatorMapping("EQ") --> "="
value = FormatValues("Peter") --> "Peter" (if multiple values are passed FormatValues cosntructs a comma seperated list)
the application then builds:
- constructClause(String expression,String operator,String value)
"table2.xmldata.value('book/author/name[1]', 'varchar')" + "=" + "Peter"
then constructs a Select statement with the result as WHERE clause.
it does not build it like this, unescaped, unfiltered for injection etc, but this is the basic idea.
i can influence how the input is Transalted, meaning I can implement the methods:
lookupExpressionForField(String field)
lookUpOperatorMapping(String operator)
Formatvalues(List<String> values) | Formatvalues(String value)
constructClause(String expression,String operator,String value)
however i choose to do, i can change the parameter types, I can freely implement them. The less the better of course. So simply constructing a comma-seperated list with xPath would be optimal (like if i could somewhere just tick "enable /function()-syntax in xPath" in sqlserver and the /concat(if...) would work)
How about something like this:
SET NOCOUNT ON;
DECLARE #Books TABLE (ID INT NOT NULL IDENTITY(1, 1) PRIMARY KEY, BookInfo XML);
INSERT INTO #Books (BookInfo)
VALUES (N'<book>
<title>a nice Novel</title>
<author>Maria</author>
<author>Peter</author>
</book>');
INSERT INTO #Books (BookInfo)
VALUES (N'<book>
<title>another one</title>
<author>Bob</author>
</book>');
SELECT *
FROM #Books bk
WHERE bk.BookInfo.exist('/book/author[text() = "Peter"]') = 1;
This returns only the first "book" entry. From there you can extract any portion of the XML field using the "value" function.
The "exist" function returns a boolean / BIT. This will scan through all "author" nodes within "book", so there is no need to concat into a comma-separated list only for use in an IN list, which wouldn't work anyway ;-).
For more info on the "value" and "exist" functions, as well as the other functions for use with XML data, please see:
xml Data Type Methods
I understand that in Postgres pure, you can pass an integer array into a function but that this isn't supported in the .NET data provider Npgsql.
I currently have a DbCommand into which I load a call to a stored proc, add in a parameter and execute scalar to get back an Id to populate an object with.
This now needs to take n integers as arguments. These are used to create child records linking the newly created record by it's id to the integer arguments.
Ideally I'd rather not have to make multiple ExecuteNonQuery calls on my DbCommand for each of the integers, so I'm about to build a csv string as a parameter that will be split on the database side.
I normally live in LINQ 2 SQL savouring the Db abstraction, working on this project with manual data access it's all just getting a bit dirty, how do people usually go about passing these kinds of parameters into postgres?
See: http://www.postgresql.org/docs/9.1/static/arrays.html
If your non-native driver still does not allow you to pass arrays, then you can:
pass a string representation of an array (which your stored procedure can then parse into an array -- see string_to_array)
CREATE FUNCTION my_method(TEXT) RETURNS VOID AS $$
DECLARE
ids INT[];
BEGIN
ids = string_to_array($1,',');
...
END $$ LANGUAGE plpgsql;
then
SELECT my_method(:1)
with :1 = '1,2,3,4'
rely on Postgres itself to cast from a string to an array
CREATE FUNCTION my_method(INT[]) RETURNS VOID AS $$
...
END $$ LANGUAGE plpgsql;
then
SELECT my_method('{1,2,3,4}')
choose not to use bind variables and issue an explicit command string with all parameters spelled out instead (make sure to validate or escape all parameters coming from outside to avoid SQL injection attacks.)
CREATE FUNCTION my_method(INT[]) RETURNS VOID AS $$
...
END $$ LANGUAGE plpgsql;
then
SELECT my_method(ARRAY [1,2,3,4])
I realize this is an old question, but it took me several hours to find a good solution and thought I'd pass on what I learned here and save someone else the trouble. Try, for example,
SELECT * FROM some_table WHERE id_column = ANY(#id_list)
where #id_list is bound to an int[] parameter by way of
command.Parameters.Add("#id_list", NpgsqlDbType.Array|NpgsqlDbType.Integer).Value = my_id_list;
where command is a NpgsqlCommand (using C# and Npgsql in Visual Studio).
You can always use a properly formatted string. The trick is the formatting.
command.Parameters.Add("#array_parameter", string.Format("{{{0}}}", string.Join(",", array));
Note that if your array is an array of strings, then you'll need to use array.Select(value => string.Format("\"{0}\", value)) or the equivalent. I use this style for an array of an enumerated type in PostgreSQL, because there's no automatic conversion from the array.
In my case, my enumerated type has some values like 'value1', 'value2', 'value3', and my C# enumeration has matching values. In my case, the final SQL query ends up looking something like (E'{"value1","value2"}'), and this works.
Full Coding Structure
postgresql function
CREATE OR REPLACE FUNCTION admin.usp_itemdisplayid_byitemhead_select(
item_head_list int[])
RETURNS TABLE(item_display_id integer)
LANGUAGE 'sql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
SELECT vii.item_display_id from admin.view_item_information as vii
where vii.item_head_id = ANY(item_head_list);
$BODY$;
Model
public class CampaignCreator
{
public int item_display_id { get; set; }
public List<int> pitem_head_id { get; set; }
}
.NET CORE function
DynamicParameters _parameter = new DynamicParameters();
_parameter.Add("#item_head_list",obj.pitem_head_id);
string sql = "select * from admin.usp_itemdisplayid_byitemhead_select(#item_head_list)";
response.data = await _connection.QueryAsync<CampaignCreator>(sql, _parameter);