There is a table in postgres DB test1 having schema :
We are using spring frameworks jdbcTemplate to insert data as below:
Object[] params = {"978","tour"};
jdbcTemplate.update("insert into test1 values (?,?)", params);
But this gives the exception :
org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [insert into test1 values (?,?)]; nested exception is org.postgresql.util.PSQLException: ERROR: column "id" is of type integer but expression is of type character varying
ERROR: column "id" is of type integer but expression is of type character varying
This works for Oracle database through implicit type conversion, but postgres does nOt seem to work that way.
Could this be an issue with postgres driver?
A workaround would be to cast explicitly:
insert into test1 values (?::numeric ,?)
But is there better way to do the conversion as this does not seem like a good solution since there are lot of queries to be modified and also there can be other such casting issues too.
Is there some parameter that can be set at DB level to perform an auto cast?
We found the answer here
Storing json, jsonb, hstore, xml, enum, ipaddr, etc fails with "column "x" is of type json but expression is of type character varying"
A new connection propertyshould be added :
String url = "jdbc:postgresql://localhost/test";
Properties props = new Properties();
props.setProperty("user","fred");
props.setProperty("password","secret");
props.setProperty("stringtype", "unspecified");
Connection conn = DriverManager.getConnection(url, props);
https://jdbc.postgresql.org/documentation/94/connect.html
"If stringtype is set to unspecified, parameters will be sent to the server as untyped values, and the server will attempt to infer an appropriate type. This is useful if you have an existing application that uses setString() to set parameters that are actually some other type, such as integers, and you are unable to change the application to use an appropriate method such as setInt()"
Yeah, drop the double quotes here:
Object[] params = {"978","tour"};
Becomes
Object[] params = {978,"tour"};
Alternatively do the casting as you mentioned.
Related
I'm implementing a data analysis pipeline in Flink and I have a problem converting a DataStream to a Table. I have this table defined from a join between two Kafka sources:
Table legalFileEventsTable = legalFilesTable.join(eventsTable)
.where($("id").isEqual($("id_fascicolo")))
.select(
$("id").as("id_fascicolo"),
$("id_evento"),
$("giudice"),
$("nrg"),
$("codice_oggetto"),
$("ufficio"),
$("sezione"),
$("data_evento"),
$("evento"),
$("data_registrazione_evento")
);
Then I convert the joined table to a DataStream to apply some computation on the data. Here's the code I'm using:
DataStream<Row> phasesDurationsDataStream = tEnv.toChangelogStream(legalFileEventsTable)
.keyBy(r -> r.<Long>getFieldAs("id_fascicolo"))
.process(new PhaseDurationCounterProcessFunction());
phasesDurationsDataStream.print();
The PhaseDurationCounterProcessFunction emits a Row like this:
Row outputRow = Row.withNames(RowKind.INSERT);
outputRow.setField("id_fascicolo", currentState.getId_fascicolo());
outputRow.setField("nrg", currentState.getNrg());
outputRow.setField("giudice", currentState.getGiudice());
outputRow.setField("codice_oggetto", currentState.getCodice_oggetto());
outputRow.setField("ufficio", currentState.getUfficio());
outputRow.setField("sezione", currentState.getSezione());
outputRow.setField("fase", currentState.getPhase());
outputRow.setField("fase_completata", false);
outputRow.setField("durata", currentState.getDurationCounter());
out.collect(outputRow);
After collecting the results from the process function I reconvert the DataStream to a Table and execute the pipeline:
Table phasesDurationsTable = tEnv.fromChangelogStream(
phasesDurationsDataStream,
Schema.newBuilder()
.column("id_fascicolo", DataTypes.BIGINT())
.column("nrg", DataTypes.STRING())
.column("giudice", DataTypes.STRING())
.column("codice_oggetto", DataTypes.STRING())
.column("ufficio", DataTypes.STRING())
.column("sezione", DataTypes.STRING())
.column("fase", DataTypes.STRING())
.column("fase_completata", DataTypes.BOOLEAN())
.column("durata", DataTypes.BIGINT())
.primaryKey("id_fascicolo", "fase")
.build(),
ChangelogMode.upsert()
);
env.execute();
But during the startup I receive this exception:
Unable to find a field named 'id_fascicolo' in the physical data type derived from the given type information for schema declaration. Make sure that the type information is not a generic raw type. Currently available fields are: [f0]
It seems that the row information (name and type) aren't available yet and so the exception is generated. I tried to invoke the env.execute() before the DataStream->Table conversion and in this case the job starts but I have no output if I print the phasesDurationsTable. Any suggestions on how to make this work?
You need to specify the correct type information for Flink because it cannot figure out the schema from the generic Row type.
Here is an example, given our data stream is producing records like this:
Row exampleRow = Row.ofKind(RowKind.INSERT, "sensor-1", 32);
We need to define the correct type information the following way:
TypeInformation<?>[] types = {
BasicTypeInfo.STRING_TYPE_INFO,
BasicTypeInfo.LONG_TYPE_INFO
};
Using this we can define the row type information:
RowTypeInfo rowTypeInfo = new RowTypeInfo(
types,
new String[]{"sensor_name", "temperature"}
);
The last step is to specify the return type of this DataStream:
DataStream<Row> stream = env.fromSource(...).returns(rowTypeInfo)
Note that when using fromChangelogStream you only need to provide a schema if you want to use a different one from the type info returned by the DataStream, so tEnv.fromChangelogStream(stream) works just fine and will use sensor_name and temperature as the schema by default.
I am trying to decrease the value of a field in DB2 Table using Spring JDBC template.Below is my code
String sql= "UPDATE XYZ SET CURR_COUNT = CURR_COUNT-1 WHERE COMPANY_CODE =?";
int rowsUpdated = jdbcTemplate.update(sql, new Object[] {compCode});
This gives me error "**PreparedStatementCallback; bad SQL grammar**"
If I am not wrong the error is because of this **"CURR_COUNT = CURR_COUNT-1"** as program dont have value of CURR_COUNT. Can anybody suggest me a solution for this.
You could take value from database via SELECT to variable (int or long), decrement it in java, and then save new value in UPDATE query.
I have the following custom type on Postgres:
CREATE TYPE my_custom_type AS (
field_a VARCHAR,
field_b NUMERIC(10,3)
);
and the following table:
CREATE TABLE my_table
(
COL1 VARCHAR(120) NOT NULL,
CUSTOM_COLUMN my_custom_type,
CUSTOM_COLUMN_ARRAY my_custom_type[]
);
Everything works fine when I use my custom type with JOOQ:
#Test
public void testWithoutArray(){
MyTableRecord record = dsl.newRecord(MyTable.MY_TABLE);
record.setCol1("My Col1");
MyCustomType customType = new MyCustomType();
customType.setFieldA("Field A Val");
customType.setFieldB(BigDecimal.ONE);
record.setCustomColumn(customType);
record.store();
}
However, when I try to set some value in the field mapped to a custom type array, I have the following error:
#Test
public void testWithArray(){
MyTableRecord record = dsl.newRecord(MyTable.MY_TABLE);
record.setCol1("My Col1");
MyCustomTypeRecord customType = new MyCustomTypeRecord();
customType.setFieldA("Field A Val 1");
customType.setFieldB(BigDecimal.ONE);
MyCustomTypeRecord customType2 = new MyCustomTypeRecord();
customType2.setFieldA("Field A Val 2");
customType2.setFieldB(BigDecimal.TEN);
record.setCustomColumnArray(new MyCustomTypeRecord[]{customType, customType2});
record.store();
}
org.jooq.exception.DataAccessException: SQL [insert into "my_table" ("col1", "custom_column_array") values (?, ?::my_custom_type[]) returning "my_table"."col1"]; ERROR: malformed record literal: "my_custom_type"(Field A Val 1, 1)"
Detail: Missing left parenthesis.
at org.jooq.impl.Utils.translate(Utils.java:1553)
at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:571)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:347)
at org.jooq.impl.TableRecordImpl.storeInsert0(TableRecordImpl.java:176)
at org.jooq.impl.TableRecordImpl$1.operate(TableRecordImpl.java:142)
at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123)
at org.jooq.impl.TableRecordImpl.storeInsert(TableRecordImpl.java:137)
at org.jooq.impl.UpdatableRecordImpl.store0(UpdatableRecordImpl.java:185)
at org.jooq.impl.UpdatableRecordImpl.access$000(UpdatableRecordImpl.java:85)
at org.jooq.impl.UpdatableRecordImpl$1.operate(UpdatableRecordImpl.java:135)
at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:123)
at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:130)
at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:123)
The query generated by JOOQ debugg is the following:
DEBUG [main] org.jooq.tools.LoggerListener#debug:255 - Executing query : insert into "my_table" ("col1", "custom_column_array") values (?, ?::my_custom_type[]) returning "my_table"."col1"
DEBUG [main] org.jooq.tools.LoggerListener#debug:255 - -> with bind values : insert into "my_table" ("col1", "custom_column_array") values ('My Col1', array[[UDT], [UDT]]) returning "my_table"."col1"
Am I missing some configuration or is it a bug?
Cheers
As stated in the relevant issue (https://github.com/jOOQ/jOOQ/issues/4162), this is a missing piece of support for this kind of PostgreSQL functionality. The answer given in the issue so far is:
Unfortunately, this is an area where we have to work around a couple of limitations of the PostgreSQL JDBC driver, which doesn't implement SQLData and other API (see also pgjdbc/pgjdbc#63).
Currently, jOOQ binds arrays and UDTs as strings. It seems that this particular combination is not yet supported. You will probably be able to work around this limitation by implementing your own custom data type Binding:
http://www.jooq.org/doc/latest/manual/code-generation/custom-data-type-bindings/
We are in the process to migrate our SQL 2000 box to SQL 2008. But we ran into an issue; when a result set (rows or not) is returned by using a query that has a UNION. Later in the code we try to add a new row and assign field to it but because a UNION was used, when we try to assign a value to the field it gives us a Multiple-step operation generated errors. Check each status value. error. We tried the following code on a Windows XP & Windows 7 and got the same result. But when we change our connection string to point back to our SQL 2000 box we don't get that error any more.
The following example show the problem we are having.
var c = new ADODB.Connection();
var cmd = new ADODB.Command();
var rs = new ADODB.Recordset();
object recordsAffected;
c.Open("Provider=SQLOLEDB;Server=*****;Database=*****;User Id=*****;Password=*****;");
cmd.ActiveConnection = c;
cmd.CommandType = ADODB.CommandTypeEnum.adCmdText;
cmd.CommandText = "create table testing2008 (id int)";
cmd.Execute(out recordsAffected);
try {
cmd.CommandText = "select * from testing2008 union select * from testing2008";
rs.CursorLocation = ADODB.CursorLocationEnum.adUseClient;
rs.Open(cmd, Type.Missing, ADODB.CursorTypeEnum.adOpenDynamic, ADODB.LockTypeEnum.adLockBatchOptimistic, -1);
rs.AddNew();
rs.Fields["id"].Value = 0; //throws exception
rs.Save();
}
catch (Exception ex) {
MessageBox.Show(ex.ToString());
}
finally {
cmd.CommandText = "drop table testing2008";
cmd.Execute(out recordsAffected);
c.Close();
}
The link below is an article that gives a great breakdown of the 6 scenarios this error message can occur:
Scenario 1 - Error occurs when trying to insert data into a database
Scenario 2 - Error occurs when trying to open an ADO connection
Scenario 3 - Error occurs inserting data into Access, where a fieldname has a space
Scenario 4 - Error occurs inserting data into Access, when using adLockBatchOptimistic
Scenario 5 - Error occurs inserting data into Access, when using Jet.OLEDB.3.51 or ODBC driver (not Jet.OLEDB.4.0)
Scenario 6 - Error occurs when using a Command object and Parameters
http://www.adopenstatic.com/faq/80040e21.asp
Hope it may help others that may be facing the same issue.
It is type mismatch, try
rs.Fields["id"].Value = "0";
or make sure you assign a Variant to the value.
Since I posted this problem, we figured out that the problem was when you do a union the attributes on the fields are not bound (i.e. the attributes: basecatalog, basetable & basecolumn are empty) to remedy our problem we had to force the values of those attributes, by saving the recordset to xml (adPersistXML), change the xml and reopen the recordset from the xml. This rebound the fields and we were able to continue. We know this may not be the most efficient solution, but it was for an older app and we didn't want to rewrite the sql statements. It looks like the main error Multiple-step operation generated errors. Check each status value. is related to when an error occurs when a value is assigned to a field.
Two things I can think of... Make sure your "ID" column will accept a zero (0). Also - I've stopped this issue on one occasion by not using the adUseClient cursor (try server).
Many times this is a type mismatch, trying to stuff a NULL into a non-null column, or attempting to write more characters into a column than it's designed to take.
Hope this helps. - Freddo
Same issue occurred to me the problem was that i violated an object property , in my case it was size the error came out as
"IntegrationException: Problem (Multiple-step operation generated errors. Check each status value.)"
Imports ADODB
Dim _RecordSet As Recordset
_rs.Fields.Append("Field_Name", DataTypeEnum.adVarChar, 50)
_Recordset("Field_Name").Value = _RecordDetails.Field_NameValue
_RecordDetails.Field_NameValue length was more than 50 chars , so this property was violated , hence the error occurred .
I found another scenario:
When I was trying to set the value of a adLongVarChar field in a new record for a memory-only adodb.recordset. In my case, the error was triggered because the string I was passing had a buried unicode character.
I found this error when our legacy application was trying to parse 1/1/0001 12AM date and time. Looks like VB6 recordsets doesn't like that value.
To get rid of the errors, I had to set all the offending dates to null.
I was getting this error when trying to insert/update the field with a value that did not match the table>field type.
For example, the database table > field was
char(1)
however, I was trying to insert/update
"apple"
into the record.
Once I change the inputted value to "a" and it worked.
I have a CLR UDT that would benefit greatly from table-valued methods, ala xml.nodes():
-- nodes() example, for reference:
declare #xml xml = '<id>1</id><id>2</id><id>5</id><id>10</id>'
select c.value('.','int') as id from #xml.nodes('/id') t (c)
I want something similar for my UDT:
-- would return tuples (1, 4), (1, 5), (1, 6)....(1, 20)
declare #udt dbo.FancyType = '1.4:20'
select * from #udt.AsTable() t (c)
Does anyone have any experience w/ this? Any help would be greatly appreciated. I've tried a few things and they've all failed. I've looked for documentation and examples and found none.
Yes, I know I could create table-valued UDFs that take my UDT as a parameter, but I was rather hoping to bundle everything inside a single type, OO-style.
EDIT
Russell Hart found the documentation states that table-valued methods are not supported, and fixed my syntax to produce the expected runtime error (see below).
In VS2010, after creating a new UDT, I added this at the end of the struct definition:
[SqlMethod(FillRowMethodName = "GetTable_FillRow", TableDefinition = "Id INT")]
public IEnumerable GetTable()
{
ArrayList resultCollection = new ArrayList();
resultCollection.Add(1);
resultCollection.Add(2);
resultCollection.Add(3);
return resultCollection;
}
public static void GetTable_FillRow(object tableResultObj, out SqlInt32 Id)
{
Id = (int)tableResultObj;
}
This builds and deploys successfully. But then in SSMS, we get a runtime error as expected (if not word-for-word):
-- needed to alias the column in the SELECT clause, rather than after the table alias.
declare #this dbo.tvm_example = ''
select t.[Id] as [ID] from #this.GetTable() as [t]
Msg 2715, Level 16, State 3, Line 2
Column, parameter, or variable #1: Cannot find data type dbo.tvm_example.
Parameter or variable '#this' has an invalid data type.
So, it seems it is not possible after all. And even if were possible, it probably wouldn't be wise, given the restrictions on altering CLR objects in SQL Server.
That said, if anyone knows a hack to get around this particular limitation, I'll raise a new bounty accordingly.
You have aliased the table but not the columns. Try,
declare #this dbo.tvm_example = ''
select t.[Id] as [ID] from #this.GetTable() as [t]
According to the documentation, http://msdn.microsoft.com/en-us/library/ms131069(v=SQL.100).aspx#Y4739, this should fail on another runtime error regarding incorrect type.
The SqlMethodAttribute class inherits from the SqlFunctionAttribute class, so SqlMethodAttribute inherits the FillRowMethodName and TableDefinition fields from SqlFunctionAttribute. This implies that it is possible to write a table-valued method, which is not the case. The method compiles and the assembly deploys, but an error about the IEnumerable return type is raised at runtime with the following message: "Method, property, or field '' in class '' in assembly '' has invalid return type."
They may be avoiding supporting such a method. If you alter the assembly with method updates this can cause problems to the data in UDT columns.
An appropriate solution is to have a minimal UDT, then a seperate class of methods to accompany it. This will ensure flexibility and fully featured methods.
xml nodes method will not change so it is not subject to the same implemetation limitations.
Hope this helps Peter and good luck.