How to find which field is giving error in query:
com.abc.fast.common.db.exception.UncategorizedSQLException: CallableStatementCallback; SQL [{call p_proc_u(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; [ErrorCode: 257; SQLState: 42000; Message: Implicit conversion from datatype 'VARCHAR' to 'NUMERIC' is not allowed. Use the CONVERT function to run this query. (DataSource: 1, Type: SYBASE)]; nested exception is com.sybase.jdbc3.jdbc.SybSQLException: Implicit conversion from datatype 'VARCHAR' to 'NUMERIC' is not allowed. Use the CONVERT function to run this query.
This indicates that a numeric value is getting enclosed in quotes, or a character value is missing quotes.
Double check your code to make sure values are being properly passed into the database.
If you're experiencing this using Spring's Stored Procedure template, make sure the order of your 'declareParameter' is in the same order as the parameter list as specified in the destination stored procedure.
Given a stored proc that looks like:
CREATE PROCEDURE dbo.sp_ac_createStoredProc
(
#pFirst_name VARCHAR(100),
#pLast_name VARCHAR(100) = NULL,
#pAge NUMERIC(2,0) = NULL
declare your parameters like this
declareParameter(new SqlParameter(FIRST_NAME, Types.VARCHAR));
declareParameter(new SqlParameter(LAST_NAME, Types.VARCHAR));
declareParameter(new SqlParameter(AGE, Types.NUMERIC));
and it will be fine. Declare your parameters like this:
declareParameter(new SqlParameter(AGE, Types.NUMERIC));
declareParameter(new SqlParameter(FIRST_NAME, Types.VARCHAR));
declareParameter(new SqlParameter(LAST_NAME, Types.VARCHAR));
Well, basically, just don't.
You can see the variable values it finally used to call stored proc. This can be acheived by seting jdbctemplate logger to DEBUG mode. For example if you are using Spring jdbctempate then you can set property log4j.logger.org.springframework.jdbc.core to DEBUG. This can be done in log4j.properties or logback.xml.
Related
I have a table like the one below in SQL Server:
I have an API that receives 15,000 records every 10 seconds to save in the table above. As if userid exist update that row if not exist userid insert record. I use the following code to write the record of each user with pyodbc (in python) That means I run the following code 15,000 times :
update Mytable
set buy = ?, model = ?, price = ?, color = ?, number = ?,
balance = ?, time = ?, type = ?,
where userid = ?
if ##ROWCOUNT = 0
insert into Mytable (userid, buy, model, price, color,
number, balance, time, type)
values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
The above code works fine but takes 700 seconds for 15,000 records. I have just this API and I have no other information about the number of users and ...
How can I save 15,000 records in less than seven seconds?
How can I save 15,000 records in less than seven seconds?
The most important things (in order of importance) are:
to not to send 15,000 separate batches or RPCs to SQL Server
to not run 15,000 separate transactions
and
to not run 15,000 separate DML statements.
The easiest way to do this from Python is to send the data as a JSON document and insert/update the data using batch update/insert, or merge.
You can send all 15,000 or break it into a few batches. See eg, Trying to insert pandas dataframe to temporary table
Once you have the data on the server you can use TSQL's MERGE, or a single UPDATE and single INSERT statement for the whole batch.
Thanks from #David Browne and This tutorial for use OPENJSON in SQL server.
I use OPENJSON and Write 15000 rows just in 1 second with below code (in pyodbc) and then delete duplicate old records with 1 query.
Write 1500 rows:
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=server_name;'
'Database=database_name;'
'Trusted_Connection=yes;')
cursor = conn.cursor()
SqlSave= DECLARE #json NVARCHAR(max) = ' [
{ "userid" : 14F2G34, "buyV":"pen" ,"modelM" : "Bic", "color" : "red","numberB" : 4000,"balanceVal" : 750,"timeBuy" : 1631303488,"type":"simple" },
{ "userid" : 14F2G35, "buyV":"pen" ,"modelM" : "blueBic", "color" : "blue","numberB" : 1000,"balanceVal" : 150,"timeBuy" : 1631303488,"type":"coly" },
{ "userid" : 14F2G36, "buyV":"pen" ,"modelM" : "oli", "color" : "yellow","numberB" : 6000,"balanceVal" : 200,"timeBuy" : 1631303488,"type":"ni" },
...
]';
INSERT INTO Mytable
SELECT *
FROM OPENJSON(#json, '$')
WITH (
useruserid int '$.userid',
buy varchar(60) '$.buyV',
model varchar(60) '$.modelM',
color varchar(60) '$.color',
number varchar(60) '$.numberB',
balance varchar(60) '$.balanceVal',
time varchar(60) '$.timeBuy',
type varchar(60) '$.type',
);
cursor.execute(SqlSave)
cursor.commit()
Note: if length your string above is more than 4000 use NVARCHAR(max).
For update exist rows I delete old rows, I have ID column that auto incremental and I delete all duplicate rows by userid column except duplicate rows which have max ID by below code:
delete FROM mytable
WHERE ID NOT IN
( SELECT max(ID)
FROM mytable
GROUP BY userid
)
I am utilising Spring Data JPA and am getting some strange behaviour that I cannot work out how to fix.
#Query(value="EXEC dbo.spChartPracticeInternalDoctor ?, ?, ?, ?", nativeQuery = true)
List<ReportDataSet> spChartPracticeInternalDoctor(Long id, String date,Long months, String tests);
When I pass in a CSV string in tests that contains no spaces the stored procedure executes correctly ( i.e. SC,ECG )
But when I pass in a tests string with a space it errors.
e.g. SC,ECG,IP Standard Consult
2018-04-12 12:45:13.309 DEBUG 684 --- [ XNIO-2 task-2] com.pci.dprm.aop.logging.LoggingAspect : Enter: com.pci.dprm.web.rest.ReportDataSetResource.spChartPracticeInternalDoctor() with argument[s] = [36, 2018-04-12T12:05:01 08:00, 24, SC,ECG,IP Standard Consult]
Hibernate: EXEC dbo.spChartPracticeInternalDoctor ?, ?, ?, ?
2018-04-12 12:45:13.316 WARN 684 --- [ XNIO-2 task-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 8114, SQLState: S0001
2018-04-12 12:45:13.317 ERROR 684 --- [ XNIO-2 task-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Error converting data type nvarchar to date.
It seems that hibernate is not escaping the string correctly when passing it to SQL, but I cannot work out how to overwrite the escaping.
I would appreciate any ideas to deal with this issue
I think error is not because of 4th parameter it is because of 2nd parameter. Logs shows that --> Error converting data type nvarchar to date.
I think you need to convert second argument to Date.
Please see below code snippet for your reference.
#Query(value="EXEC dbo.spChartPracticeInternalDoctor ?, ?, ?, ?", nativeQuery = true)
List<ReportDataSet> spChartPracticeInternalDoctor(Long id, Date date,Long months, String tests);
I'm having a problem while i'm trying to collect explain plan with the db2exfmt tool.
Can some body explain me the process of how to use that tool?
My requirement is to collect cost of a stored procedure. I have the EXPLAIN tables created in instance 'XYZ' schema and I have a procedure named "UNNAMED", which has the package name "P123456" and the schema "ABCD".
I used the following commands:
! db2exfmt -d SAMPLE -e DB2INST1 -s ABCD -n P123456 -g TIC -w -1 -#***5*** -t
Wherein 5 is the Section Number of the part of the procedure I'm trying to collect cost for.
Furthermore, I have also tried to do the following:
1) Identify the package corresponding to the stored proc :
select r.routineschema, r.routinename, rd.bname as packagename
from syscat.routines r, syscat.routinedep rd
where
r.specificname=rd.specificname and
r.routineschema=rd.routineschema and
rd.btype='K' and
r.routineschema = 'XYZ' and
r.routinename = 'ABCD'
2) Identify the section number for the SQL statement :
select sectno, text
from syscat.statements
where pkgschema='XYZ' and pkgname='P123456'
3) Populate Explain tables :
call EXPLAIN_FROM_CATALOG( 'XYZ', 'P123456', ' ', 5, 'SYSTOOLS', ?, ?, ?, ?, ? )
The latter throws an error:
Message: The parameter mode OUT or INOUT is not valid for a parameter in the routine named "EXPLAIN_FROM_CATALOG" with specific name "EXPLAIN_FROM_CATALOG" (parameter number "5", name "EXPLAIN_SCHEMA").. SQLCODE=-469, SQLSTATE=42886, DRIVER=3.50.152
I am logged in as USER : "MNO" and want the explain tables under SYSTOOLS schema to be populated.
Can someone please help me resolve the problem?
As the error message indicates, and the manual says, explain_schema is an INOUT parameter, so you cannot specify a literal value.
You may want to try wrapping the procedure call in a compound statement, providing declared variables for each OUT and INOUT parameter, something like:
begin
declare v_schema varchar(50) default 'SYSTOOLS';
declare v_req, v_srcname, v_srcschema, v_srcver varchar(128);
declare v_ts timestamp;
call EXPLAIN_FROM_CATALOG( 'XYZ', 'P123456', ' ', 5, v_schema,
v_req, v_ts, v_srcname, v_srcschema, v_srcver );
end
PS. Code is not tested
You have to specify every IN/OUT parameter with a "?" - running it will then prompt you for the input value ('SYSTOOLS').
So try
call EXPLAIN_FROM_CATALOG( 'XYZ', 'P123456', ' ', 5, **?**, ?, ?, ?, ?, ? )
Here is my problem in detail.
I have created a data access layer class that allows me to create most of objects I needed to communicate with databases (Odbc, OleDb and SqlClient). I also have created a business object handling layer class with intensive use of Reflection to handle lots of tasks with my business object(s). Among other things this class generates every single property/object I needed for my DAL handling (SQL stream, list of values, properties, set retrieve values etc.). Take a look the code below for further explanation:
Public Shared Function InvokeParam(Of T)(_classObject As T, _commandType As AdapterCommandType, _arguments As Object()) As Boolean
Dim s As String = DAL.SCRFL.GetParamStatement(_classObject, _commandType, _arguments)
'Debug.Print(s)
Dim hT As Hashtable = DAL.SCRFL.GetProperties(_classObject)
Using cnn As IDbConnection = DataFactory.CreateConnection()
Dim cmd As IDbCommand = DataFactory.CreateCommand(s, cnn)
'cmd.CommandType = CommandType.Text
cmd.CommandText = s
For Each k In hT
Dim param As IDbDataParameter = cmd.CreateParameter()
'param.DbType = DataFactory.ConvertToDbType(k.value.GetType)
param.Value = k.value
param.ParameterName = k.key
'param.Direction = ParameterDirection.Input
'Debug.Print("value:={0}, name:={1}", TypeName(k.value), TypeName(k.key))
Debug.Print("typeMatch:={0}, value:={1}, name:={2}", TypeName(param.Value) = TypeName(k.value), param.Value, param.ParameterName)
cmd.Parameters.Add(param)
Next
If (cmd.ExecuteNonQuery > 0) Then
Return True
End If
End Using
Return False
End Function
So, DAL.SCRFL.GetParamStatement returns string formatted as INSERT INTO t1 (f1, f2, f3...) values (?, ?, ?...) for insert and appropriate strings for update, delete, select statements. All are done with reflection. There is no syntax error here. I can manually execute returned values through direct provider type commands.
The DAL.SCRFL.GetProperties method returns a hashtable formatted as key=property (field), value=field value.
Now, I need to create parameters for each property and add that to my command parameters then execute it. This attempt you will see in my code (I’m creating parameters for each propert/value pair by looping the hash table). However at the end I'm getting an exception with Data type mismatch in criteria expression. description. I've tried adding type property to parameter object, size etc., all was unsuccessful (I commented them). I tried changing param.Value = k.value to param.Value = If(IsDBNull(k.value), DBNull.Value, k.value) thinking that this might be the problem, though k.value is from my business class and I intentionally prevent from null values. Nothing worked! here is the test; business class returned value from DAL.SCRFL.GetParamStatement call: The test is done for OleDb/Access database and, as you can see, I enclosed the Memo field in single quotes. My reflection methods read class properties' attributes (which I set to be table field names) and DAL.SCRFL.GetParamStatement builds basic sql statements for insert, update, delete and select use. AdapterCommandType is a built in enum type for it).
INSERT INTO Clinics
(ClinicId, ClinicName, Phone, Fax, FederalId, DateContracted, Address, City, State, Zip, Inactive, [Memo], DateEntered, EnteredBy, DateModified, ModifiedBy)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Note that I have another method similar to this that executes an sql statement (InvokeSql) where I thoroughly check value types for each property to construct property=value pairs in my sql statement. Using a fully qualified sql statement in this, InvokeSql, method works w/out a single warning (Rouphly: cnn As IDbConnection = CreateConnection(), cmd = CreateCommand(_cmdText, cnn), cmd.ExecuteNonQuery() where _cmdText is the sql statement. No parameters as you can see!). I'm mentioning this to point out that the problem arises whenever I use parameters with generic IDbCommands. Even though inside my DataFactory the IDbCommand set to be provider specific command type (my DataFactory.CreateCommand(s, cnn) returns generic IDbCommand).
Prior to my DAL development, I was doing all above steps manually though all objects (commands, connections etc.) were explicitly declared to be provider specific types. Technically speaking I’m exercising exact same scenarios as I used to with generic type of objects (not provider specific). But I can’t make it work, there is somewhere, probably, I’m missing something.
I had posted this question on codeproject and the answer is there.
http://www.codeproject.com/Questions/446516/How-to-add-parameters-and-execute-a-generic-IDbCom
I want to map a SQL Server stored procedure with MyBatis, using annotations.
#Select(value = "{call sp_cen_obliczcene(" +
"#{wytworId, mode=IN, jdbcType=NUMERIC}, " +
"#{rodzajCenyId, mode=IN, jdbcType=NUMERIC}, " +
"#{walutaId, mode=IN, jdbcType=NUMERIC}, " +
"#{jmId, mode=IN, jdbcType=NUMERIC}, " +
"#{ilosc, mode=IN, jdbcType=DECIMAL}, " +
"#{data, mode=IN, jdbcType=DATE})}")
#Result(property = "kwota", column = "kwota", javaType = BigDecimal.class, jdbcType = JdbcType.DECIMAL)
#Options(statementType = StatementType.CALLABLE)
public DtoCena dajCene(CriteriaCena parametry);
The procedure selects one row - I am interested in one column. Now, I've mapped a procedure before, only I had multiple rows and selected more then one column from them. Everything worked perfectly fine. When I mapped new procedure, in a similar way I got an error:
### The error occurred while setting parameters
### SQL: {call sp_cen_obliczcene(?, ?, ?, ?, ?, ?)}
### Cause: java.lang.NullPointerException
I started the SQL Profiler and saw that the procedure is called properly with the given parameters. I've noticed that the procedure I'm mapping is executing other procedures. They're performing some updates. When I changed my annotation to #Update I got an other error: that Integer cannot be cast to DtoCena type. I changed the return value of the method to Integer and I got no errors but as you can guess it did not return what I was looking for.
The question is, can I map a stored procedure which updates tables AND returns a ResultSet? I can do this using JDBC, but is this possible with MyBatis? Am I doing something wrong when using the #Select annotation?
Looks like the #Update returns the affected row count ...
Anyway, I don't think the issue is related to calling stored procedure, this is merely a mapping issue that would occur with simple select.
You must use #Result annotation inside #Results annotation, otherwise it is ignored.
Here is a simplified, yet functional, code:
#Select("select 'hello' as h, 1 as n from dual")
#Results({
#Result(column="n")
})
Integer test();
Just add a property attribute and change return type to retrieve result into an object.