I am trying to flatten an array using UNNEST function in the Table API.
Am I doing something wrong or it is not a supported function ? This page suggests it though: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html
Thanks !
Code
Python udf
#udf(input_types=DataTypes.STRING(), result_type=DataTypes.ARRAY(DataTypes.STRING()))
def explode(s):
return 3*[s]
t_env.register_function("explode", explode)
Processing
tab = t_env.from_path('mySource').select("id, explode(dummy) as dummy_list")
t_env.register_table("temp_table", tab)
t_env.sql_query("SELECT t.item as dummy_item FROM UNNEST(select dummy_list from temp_table) AS t(item)").insert_into("mySink")
Execution
t_env.execute("dummy_unnest")
Error
TableException: Cannot generate a valid execution plan for the given query:
LogicalProject(dummy_item=[$0])
Uncollect
LogicalProject(EXPR$0=[org$apache$flink$table$functions$python$PythonScalarFunction$908596b4671476ee325743dba92ed6c7($1)])
LogicalTableScan(table=[[default_catalog, default_database, mySource]])
This exception indicates that the query uses an unsupported SQL feature.
Please check the documentation for the set of currently supported SQL features.
I think you can change the query to
select id, dummy_item from temp_table CROSS JOIN UNNEST(dummy_list) AS t (dummy_item)
Related
If it's relevant I'm using Django with Django Rest Framework, django-mssql-backend and pyodbc
I am building some read only models of a legacy database using fairly complex queries and Django's MyModel.objects.raw() functionality. Initially I was executing the query as a Select query which was working well, however I received a request to try and do the same thing but with a table-valued function from within the database.
Executing this:
MyModel.objects.raw(select * from dbo.f_mytablefunction)
Gives the error: Invalid object name 'myapp_mymodel'.
Looking deeper into the local variables at time of error it looks like this SQL is generated:
'SELECT [myapp_mymodel].[Field1], '
'[myapp_mymodel].[Field2] FROM '
'[myapp_mymodel] WHERE '
'[myapp_mymodel].[Field1] = %s'
The model itself is mapped properly to the query as executing the equivalent:
MyModel.objects.raw(select * from dbo.mytable)
Returns data as expected, and dbo.f_mytablefunction is defined as:
CREATE FUNCTION dbo.f_mytablefunction
(
#param1 = NULL etc etc
)
RETURNS TABLE
AS
RETURN
(
SELECT
field1, field2 etc etc
FROM
dbo.mytable
)
If anyone has any explanation as to why these two modes of operation are treated substantially differently then I would be very pleased to find out.
Guess you've figured this out by now (see docs):
MyModel.objects.raw('select * from dbo.f_mytablefunction(%s)', [1])
If you'd like to map your table valued function to a model, this gist has a quite thorough approach, though no license is mentioned.
Once you've pointed your model 'objects' to the new TableFunctionManager and added the 'function_args' OrderedDict (see tests in gist), you can query it as follows:
MyModel.objects.all().table_function(param1=1)
For anyone wondering about use cases for table valued functions, try searching for 'your_db_vendor tvf'.
So, I have Hibernate 5.3.1 in a project which connects to different enginees (MySql, Oracle, PostgreSQL and MS SQL), so I can't use native queries.
Let's say I have 3 records in a table, which all of them have the same datetime, but I need to group them only by date (not time). For example, 2019-12-04;
I execute this query:
SELECT
CONCAT(year(tx.date_), month(tx.date_), day(tx.date_)),
iss.code,
COUNT(tx.id)
FROM
tx_ tx
JOIN
issuer_ iss
ON
tx.id_issuer = iss.id
GROUP BY
CONCAT(year(tx.date_), month(tx.date_), day(tx.date_)), iss.code
But, when I test it connected to SQL SERVER 2017, instead of return 20191204, it's returning 2035. In Oracle and MySQL is working fine.
Anyone has any idea why is this happen? I've tried different ways, like use + instead of CONCAT but the result is the same.
I've also tried to extract them for separate (without concat), and they have been returning correct. The problem is, I need to group them by the complete date.
And just for the record, the field is declared as datetime2 in DDBB
How about simply adding them, instead of using CONCAT.
(year(tx.date_)*10000 + month(tx.date_)*100 + day(tx.date_)*1) AS datenum
Thus, try this:
SELECT
CAST((year(tx.date_)*10000 + month(tx.date_)*100 + day(tx.date_)*1) AS string) AS datenum,
iss.code
FROM tx_ tx
JOIN issuer_ iss
ON tx.id_issuer = iss.id
GROUP BY year(tx.date_), month(tx.date_), day(tx.date_), iss.code
Thanks for the hint Gert Arnold gave me. I just didn't realize that the query was adding like if they were numbers in MSSQL.
Finally, I manage to make it work in the 4 RDBMS casting to string first
SELECT
CONCAT(CAST(year(tx.date_) AS string), CAST(month(tx.date_) AS string), CAST(day(tx.date_) AS string)),
iss.code
FROM
tx_ tx
JOIN
issuer_ iss
ON
tx.id_issuer = iss.id
GROUP BY
CONCAT(year(tx.date_), month(tx.date_), day(tx.date_)), iss.code
I tried also casting to TEXT, but it throws exception in MySQL
Why use concat() to begin with?
Assuming Hibernate takes care of converting the non-standard year(), month() and day() functions, then the following should work on any DBMS
SELECT year(tx.date_), month(tx.date_), day(tx.date_), iss.code
FROM tx_ tx
JOIN issuer_ iss ON tx.id_issuer = iss.id
GROUP BY year(tx.date_), month(tx.date_), day(tx.date_), iss.code
I need to make hierarchical queries, and I need to get the results of CONNECT_BY_ISCYCLE and CONNECT_BY_ISLEAF, but these features are supported in Oracle not in Snowflake.
What are the alternative ways to implement the functionalities of CONNECT_BY_ISCYCLE and CONNECT_BY_ISLEAF in snowflake without using them as these keywords are not supported there?
Wonder if you have taken a look at the following Snowflake features?
https://docs.snowflake.net/manuals/user-guide/queries-hierarchical.html#using-connect-by-or-recursive-ctes-to-query-hierarchical-data
Yes I took a look there. I also took a look at https://docs.snowflake.net/manuals/sql-reference/constructs/connect-by.html where it clearly says that these features are not supported in Snowflake.
I was trying below code block to find an alternative but facing varieties of error in snowflake.
person_vertex as (
select
emp_number,
user_id
from person
),
person_edges as (
select
supervisor_emp_number,
emp_number
from person
where supervisor_emp_number is not null
),
select
pv.emp_number emp_id_pk,
level,
CONNECT_BY_ROOT pv.emp_number AS root,
concat(SYS_CONNECT_BY_PATH(pv.emp_number,':'),':') as path,
-- CONNECT_BY_ISCYCLE AS iscyclic, ------------------- no idea how to implement this
-- CONNECT_BY_ISLEAF as isleaf ------------------- i tried below block, but it is not working
case
when (pe.supervisor_emp_number in (select emp_number from pv)) then 0
else 1
end AS isleaf
from person_vertex pv
left join person_edges pe on pv.emp_number = pe.emp_number
connect by prior A.emp_number = A.supervisor_emp_number
start with A.supervisor_emp_number is null
Any help with this block is really appreciated.
Thanks.
enter code here
I am trying to query the metadata from a tabular cube using DMVs. I was able to get it to work without a where clause, but I can't seem to get a where clause to work. Any Advice?
Here is the code that works:
SELECT
[MEASURE_CAPTION] AS [Measure]
,[MEASURE_IS_VISIBLE] AS [Visable]
,[DESCRIPTION] AS [Description]
,[MEASURE_DISPLAY_FOLDER] AS [Display Folder]
,[EXPRESSION] AS [Calculation]
FROM $SYSTEM.MDSCHEMA_MEASURES
The WHERE clauses that I have tried are:
WHERE ([MEASURE_IS_VISIBLE].[members].[true])
I get the following error:
The dot expression is not allowed in the context at line 9, column 1.
Also:
WHERE [MEASURE_IS_VISIBLE] = TRUE
I get the following error:
Error: A Boolean expression is not allowed in the context at line 9, column 7.
I have tried many veriations on these themes, but always get the same result. I am not at all familiar with how MDX works, so any assistance would be appreciated.
MEASURE_IS_VISIBLE is a boolean column and must be filtered as such. The following query filters the results to only those with a MEASURE_IS_VISIBLE value of true. Change this to WHERE NOT MEASURE_IS_VISIBLE for rows where this is false. You can see the datatypes for this DMV from the documentation here.
SELECT
[MEASURE_CAPTION] AS [Measure]
,[MEASURE_IS_VISIBLE] AS [Visable]
,[DESCRIPTION] AS [Description]
,[MEASURE_DISPLAY_FOLDER] AS [Display Folder]
,[EXPRESSION] AS [Calculation]
FROM $SYSTEM.MDSCHEMA_MEASURES
WHERE MEASURE_IS_VISIBLE
I want to execute query in my yii2 application. I'm using PostgreSQl. There is a table called list inside the user schema. If I try to build any query it returns 1. My code is here:
$numUsers = Yii::$app->db->createCommand('
SELECT COUNT(*) FROM "user"."list"
')->execute();
Please show me my mistake in the query above.
This is not related to the DB type in Yii2 if you want the result of a single value you should use queryScalar() instead of execute()
$numUsers = Yii::$app->db->createCommand('
SELECT COUNT(*) FROM "user"."list" ')->queryScalar();