I have the following custom type on Postgres:
CREATE TYPE my_custom_type AS (
field_a VARCHAR,
field_b NUMERIC(10,3)
);
and the following table:
CREATE TABLE my_table
(
COL1 VARCHAR(120) NOT NULL,
CUSTOM_COLUMN my_custom_type,
CUSTOM_COLUMN_ARRAY my_custom_type[]
);
Everything works fine when I use my custom type with JOOQ:
#Test
public void testWithoutArray(){
MyTableRecord record = dsl.newRecord(MyTable.MY_TABLE);
record.setCol1("My Col1");
MyCustomType customType = new MyCustomType();
customType.setFieldA("Field A Val");
customType.setFieldB(BigDecimal.ONE);
record.setCustomColumn(customType);
record.store();
}
However, when I try to set some value in the field mapped to a custom type array, I have the following error:
#Test
public void testWithArray(){
MyTableRecord record = dsl.newRecord(MyTable.MY_TABLE);
record.setCol1("My Col1");
MyCustomTypeRecord customType = new MyCustomTypeRecord();
customType.setFieldA("Field A Val 1");
customType.setFieldB(BigDecimal.ONE);
MyCustomTypeRecord customType2 = new MyCustomTypeRecord();
customType2.setFieldA("Field A Val 2");
customType2.setFieldB(BigDecimal.TEN);
record.setCustomColumnArray(new MyCustomTypeRecord[]{customType, customType2});
record.store();
}
org.jooq.exception.DataAccessException: SQL [insert into "my_table" ("col1", "custom_column_array") values (?, ?::my_custom_type[]) returning "my_table"."col1"]; ERROR: malformed record literal: "my_custom_type"(Field A Val 1, 1)"
Detail: Missing left parenthesis.
at org.jooq.impl.Utils.translate(Utils.java:1553)
at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:571)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:347)
at org.jooq.impl.TableRecordImpl.storeInsert0(TableRecordImpl.java:176)
at org.jooq.impl.TableRecordImpl$1.operate(TableRecordImpl.java:142)
at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123)
at org.jooq.impl.TableRecordImpl.storeInsert(TableRecordImpl.java:137)
at org.jooq.impl.UpdatableRecordImpl.store0(UpdatableRecordImpl.java:185)
at org.jooq.impl.UpdatableRecordImpl.access$000(UpdatableRecordImpl.java:85)
at org.jooq.impl.UpdatableRecordImpl$1.operate(UpdatableRecordImpl.java:135)
at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123)
at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:130)
at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:123)
The query generated by JOOQ debugg is the following:
DEBUG [main] org.jooq.tools.LoggerListener#debug:255 - Executing query : insert into "my_table" ("col1", "custom_column_array") values (?, ?::my_custom_type[]) returning "my_table"."col1"
DEBUG [main] org.jooq.tools.LoggerListener#debug:255 - -> with bind values : insert into "my_table" ("col1", "custom_column_array") values ('My Col1', array[[UDT], [UDT]]) returning "my_table"."col1"
Am I missing some configuration or is it a bug?
Cheers
As stated in the relevant issue (https://github.com/jOOQ/jOOQ/issues/4162), this is a missing piece of support for this kind of PostgreSQL functionality. The answer given in the issue so far is:
Unfortunately, this is an area where we have to work around a couple of limitations of the PostgreSQL JDBC driver, which doesn't implement SQLData and other API (see also pgjdbc/pgjdbc#63).
Currently, jOOQ binds arrays and UDTs as strings. It seems that this particular combination is not yet supported. You will probably be able to work around this limitation by implementing your own custom data type Binding:
http://www.jooq.org/doc/latest/manual/code-generation/custom-data-type-bindings/
Related
I am using SQL Server 2016.
I have a stored procedure GET_RECORDS that takes input parameters for filter and outputs a CURSOR parameter
I want to get this cursor in my SSIS package
I had created data flow task, OleDb source and variables for parameter values. Then mapped parameters
Params mapping screen
but when I wanted to save the component - I got an error
error screen
I tried to add clause WITH RESULT SETS with some dummy columns, but my procedure doesn't return any result set
What am I doing wrong?
Any advices will be helpful.
Thank you.
With regards, Yuriy.
The source component is trying to determine what columns and types will be returned. Because you are using dynamic SQL the metadata can change each time you run it.
With result sets allows you to define the data being returned but should only be used if you are guaranteed to have those results every time you execute.
EDIT:
I create a connection and run the command so that it populates a data table. Then I put the column headers into a string array. There are plenty of examples out there.
Then I use the following function to create a destination table. Finally I create a datareader and pass that to the .Net SqlBulkCopy. Hope this helps.
private void CreateTable(string TableName, string[] Fields)
{
if (TableExists(TableName) && Overwrite)
{
SqlCommand = new SqlCommand($"Drop Table [{TableName}]", SqlConnection);
SqlCommand.ExecuteNonQuery();
}
string Sql = $"Create Table [{TableName}] (";
int ColumnNumber = 1;
foreach (string Field in Fields)
{
string FieldValue = Field;
if (! HasHeaders)
{
FieldValue = "Column" + ColumnNumber;
ColumnNumber++;
}
Sql += $"[{FieldValue}] Varchar(8000),";
}
Sql = Sql + "ImportFileID Int, ID Int Identity(1,1) Not Null, Constraint [PK_" + TableName + "] Primary Key Clustered ([ID] Asc))";
SqlCommand = new SqlCommand(Sql, SqlConnection);
SqlCommand.ExecuteNonQuery();
}
Use ado.net source instead of oledb source, define a simple select and get the columns you wish to return. Now you can define expresión in the dataflow properties.
Search ado.net source dynamic sql
:)
try to return the records and use foreach in ETL instead of cursor
https://www.simple-talk.com/sql/ssis/implementing-foreach-looping-logic-in-ssis/
I think you can do it from a simple way, but I don't know what you are you doing, exactly...
There is a table in postgres DB test1 having schema :
We are using spring frameworks jdbcTemplate to insert data as below:
Object[] params = {"978","tour"};
jdbcTemplate.update("insert into test1 values (?,?)", params);
But this gives the exception :
org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [insert into test1 values (?,?)]; nested exception is org.postgresql.util.PSQLException: ERROR: column "id" is of type integer but expression is of type character varying
ERROR: column "id" is of type integer but expression is of type character varying
This works for Oracle database through implicit type conversion, but postgres does nOt seem to work that way.
Could this be an issue with postgres driver?
A workaround would be to cast explicitly:
insert into test1 values (?::numeric ,?)
But is there better way to do the conversion as this does not seem like a good solution since there are lot of queries to be modified and also there can be other such casting issues too.
Is there some parameter that can be set at DB level to perform an auto cast?
We found the answer here
Storing json, jsonb, hstore, xml, enum, ipaddr, etc fails with "column "x" is of type json but expression is of type character varying"
A new connection propertyshould be added :
String url = "jdbc:postgresql://localhost/test";
Properties props = new Properties();
props.setProperty("user","fred");
props.setProperty("password","secret");
props.setProperty("stringtype", "unspecified");
Connection conn = DriverManager.getConnection(url, props);
https://jdbc.postgresql.org/documentation/94/connect.html
"If stringtype is set to unspecified, parameters will be sent to the server as untyped values, and the server will attempt to infer an appropriate type. This is useful if you have an existing application that uses setString() to set parameters that are actually some other type, such as integers, and you are unable to change the application to use an appropriate method such as setInt()"
Yeah, drop the double quotes here:
Object[] params = {"978","tour"};
Becomes
Object[] params = {978,"tour"};
Alternatively do the casting as you mentioned.
This is a followup post to this post. I am writing an accounting system backed by an h2 database. The tree of accounts is stored in the ACCOUNTS table, with the PARENT_ID column storing the links in the tree.
To get the path to a given node in the tree, I have the following stored procedure:
public static Long[] getAncestorPKs(Long id)
whose job is to produce an array of integers, being the PARENT_ID values between the given node and the root of the tree. Let's imagine it is defined like this (because I have tried this and I get the same error):
public static Long[] getAncestorPKs(Long id)
{
return new Long[]{new Long(1), new Long(2), new Long(3)};
}
It is properly registered in the database and I can call it from within a SQL query. My problem is that h2 seems to be unable to deal with the return value: if I use it like this:
SELECT ID FROM ACCOUNTS WHERE ID IN (ANCESTOR_PKS(5))
then I get the following error:
Data conversion error converting "(1, 2, 3)"; SQL statement:
SELECT ID FROM ACCOUNTS WHERE ID IN (ANCESTOR_PKS(5)) [22018-167]
If, instead, I send the following to the database:
SELECT ID FROM ACCOUNTS WHERE ID IN (1, 2, 3)
I get back a result set with 3 rows, containing the three integers (exactly what I expect).
I really can't see what is the problem here! I am returning an array of Longs, which are to be used in comparing against a column which contains BIGINTS. Why is h2 refusing to convert this array? I have tried making the return value be Object[], because the h2 documentation is not entirely clear whether this is required on the return side as well as on the call side, but that makes no difference at all. I'm just banging my head against a brick wall here. This ain't rocket science! Surely someone has written similar code before?
Many thanks in advance, before I go mad!
If the method returns an array of objects, then for the database this is one value of data type ARRAY. And not a table with 3 rows. But of course you don't use the data type ARRAY in your table, you use INT or BIGINT. So your query is incorrect.
Either the method needs to return a ResultSet, or you need to convert the array value to a table. To do that, you could use the function TABLE(..) as follows:
select x from table(x bigint = getAncestorPKs(1));
So what you could do is:
drop table accounts;
create table accounts(id int);
insert into accounts values(1), (2), (10), (20);
drop alias getAncestorPKs;
create alias getAncestorPKs as 'Long[] getAncestorPKs(Long id) {
return new Long[]{new Long(1), new Long(2), new Long(3)};
}';
select * from accounts where id in
(select x from table(x bigint = getAncestorPKs(1)));
I have one query that does a count/group by where I don't need a parameter (there is no where clause).
What is the syntax to run a parameterless query with dapper?
var _results = _conn.Query<strongType>("Select Count(columnA) as aCount, ColumnB, ColumnC from mytable group by ColumnB, ColumnC");
does not work.
I've tried it a few different ways but I still keep getting "ArgumentNullException was unhandled by user code".
Tried to figure it out myself, searched all over and I'm giving up. Thanks in advance.
Edit: Below is the line of code from SqlMapper.cs that throws the error. It's line 1334
il.Emit(OpCodes.Newobj, typeof(T).GetConstructor(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic, null, Type.EmptyTypes, null));
The error details: Value cannot be null. Parameter name: con
Mapping a single result back works just fine:
var a = cnn.Query<int>("select 1").Single()
// a is 1
You may face trouble if somehow your query returns no results, for example:
select count(Id) from
(
select top 0 1 as Id, 2 as Title
) as X
group by Title
return 0 results, so doing a Single on an empty result set is not going to work.
try
var _results = _conn.Query("Select columnB, Count(columnA) C from mytable group by columnB");
int ColumnB = ((int)_results[0].ColumnB);
int C = ((int)_results[0].C);
Value cannot be null. Parameter name: con
This is error is thrown by several dynamic ORMs including Dappper, PetaPoco and Massive but it's usually the same problem: Make sure you're using the [Ignore] attribute on the properties you don't want to include. This includes properties inherited from base classes. The error is useless but that's what it means.
This error can occur because a property you're trying to set in your return object is get-only. Dapper, of course, requires being able to set all properties. You might consider having a separate database DTO object that then gets converted to your properly immutable domain object after reading from the database.
Change this:
public string MyProperty { get; }
to this:
public string MyProperty { get; set; }
I have a CLR UDT that would benefit greatly from table-valued methods, ala xml.nodes():
-- nodes() example, for reference:
declare #xml xml = '<id>1</id><id>2</id><id>5</id><id>10</id>'
select c.value('.','int') as id from #xml.nodes('/id') t (c)
I want something similar for my UDT:
-- would return tuples (1, 4), (1, 5), (1, 6)....(1, 20)
declare #udt dbo.FancyType = '1.4:20'
select * from #udt.AsTable() t (c)
Does anyone have any experience w/ this? Any help would be greatly appreciated. I've tried a few things and they've all failed. I've looked for documentation and examples and found none.
Yes, I know I could create table-valued UDFs that take my UDT as a parameter, but I was rather hoping to bundle everything inside a single type, OO-style.
EDIT
Russell Hart found the documentation states that table-valued methods are not supported, and fixed my syntax to produce the expected runtime error (see below).
In VS2010, after creating a new UDT, I added this at the end of the struct definition:
[SqlMethod(FillRowMethodName = "GetTable_FillRow", TableDefinition = "Id INT")]
public IEnumerable GetTable()
{
ArrayList resultCollection = new ArrayList();
resultCollection.Add(1);
resultCollection.Add(2);
resultCollection.Add(3);
return resultCollection;
}
public static void GetTable_FillRow(object tableResultObj, out SqlInt32 Id)
{
Id = (int)tableResultObj;
}
This builds and deploys successfully. But then in SSMS, we get a runtime error as expected (if not word-for-word):
-- needed to alias the column in the SELECT clause, rather than after the table alias.
declare #this dbo.tvm_example = ''
select t.[Id] as [ID] from #this.GetTable() as [t]
Msg 2715, Level 16, State 3, Line 2
Column, parameter, or variable #1: Cannot find data type dbo.tvm_example.
Parameter or variable '#this' has an invalid data type.
So, it seems it is not possible after all. And even if were possible, it probably wouldn't be wise, given the restrictions on altering CLR objects in SQL Server.
That said, if anyone knows a hack to get around this particular limitation, I'll raise a new bounty accordingly.
You have aliased the table but not the columns. Try,
declare #this dbo.tvm_example = ''
select t.[Id] as [ID] from #this.GetTable() as [t]
According to the documentation, http://msdn.microsoft.com/en-us/library/ms131069(v=SQL.100).aspx#Y4739, this should fail on another runtime error regarding incorrect type.
The SqlMethodAttribute class inherits from the SqlFunctionAttribute class, so SqlMethodAttribute inherits the FillRowMethodName and TableDefinition fields from SqlFunctionAttribute. This implies that it is possible to write a table-valued method, which is not the case. The method compiles and the assembly deploys, but an error about the IEnumerable return type is raised at runtime with the following message: "Method, property, or field '' in class '' in assembly '' has invalid return type."
They may be avoiding supporting such a method. If you alter the assembly with method updates this can cause problems to the data in UDT columns.
An appropriate solution is to have a minimal UDT, then a seperate class of methods to accompany it. This will ensure flexibility and fully featured methods.
xml nodes method will not change so it is not subject to the same implemetation limitations.
Hope this helps Peter and good luck.