Weird SQL Error (Bug) - sql-server

So this is really weird.
I run a sql command from .net on sqlserver with a 'Select Count(*)' and get a response like "Needs attention CA" (which is in a varchar of one field of one record of the inner joined tables).
Huh? How can Count(*) return a string? 999 out of 1000 times this code executes correctly. Just sometimes on some clients servers it will throw a string of errors for an hour or so only to miraculously stop again.
This is my sqlcommand:
SELECT Count(*)
FROM patientsappointments
INNER JOIN appointmenttypes
ON patientsappointments.appointmenttypeid =
appointmenttypes.appointmenttypeid
WHERE ( ( patientsappointments.date > #WeekStartDate
AND patientsappointments.date < #WeekFinishDate )
AND ( patientsappointments.status = 'Pending' )
AND ( patientsappointments.doctorid = #DoctorID )
AND ( appointmenttypes.appointmentname <> 'Note' ) )
And these are the parameters:
#WeekStartDate = 24/06/2013 12:00:00 AM (DateTime)
#WeekFinishDate = 1/07/2013 12:00:00 AM (DateTime)
#DoctorID = 53630c67-3a5a-406f-901c-dbf6b6d1b20f (UniqueIdentifier)
I do a sqlcmd.executescalar to get the result. Any ideas?
The actual executed code is:
SyncLock lockRefresh
Dim WeekFulfilled, WeekPending As Integer
Using conSLDB As New SqlConnection(modLocalSettings.conSLDBConnectionString)
Dim mySQL As SqlCommand
mySQL = New SqlCommand("SELECT COUNT(*) FROM PatientsAppointments INNER JOIN AppointmentTypes ON PatientsAppointments.AppointmentTypeID = AppointmentTypes.AppointmentTypeID " & _
"WHERE ((PatientsAppointments.Date > #WeekStartDate AND PatientsAppointments.Date < #WeekFinishDate) AND (PatientsAppointments.Status = 'Pending') " & _
"AND (PatientsAppointments.DoctorID = #DoctorID) AND (AppointmentTypes.AppointmentName <> 'Note'))", conSLDB)
Try
mySQL.Parameters.Add("#WeekStartDate", SqlDbType.DateTime).Value = MonthCalendar1.SelectionStart.Date.AddDays(-MonthCalendar1.SelectionStart.Date.DayOfWeek).AddDays(1)
mySQL.Parameters.Add("#WeekFinishDate", SqlDbType.DateTime).Value = MonthCalendar1.SelectionStart.Date.AddDays(-MonthCalendar1.SelectionStart.Date.DayOfWeek).AddDays(8)
mySQL.Parameters.Add("#DoctorID", SqlDbType.UniqueIdentifier).Value = cboDoctors.SelectedValue
conSLDB.Open()
'got errors here like "Conversion from string "R2/3" to type 'Integer' is not valid." Weird.
'failing on deadlock - maybe due to simultaneous updating from udp event. Try adding random delay to refresh
WeekPending = mySQL.ExecuteScalar
Catch ex As Exception
ErrorSender.SendError("frmAppointmentBook - RefreshHeader 1", ex, New String() {String.Format("mySQL.commandtext: {0}", mySQL.CommandText), _
String.Format("mySQL.Parameters: {0}", clsErrorSender.ParamsListToString(mySQL.Parameters))})
End Try
Me.lblPendingWeek.Text = WeekPending
Try
mySQL.CommandText = "SELECT COUNT(*) FROM PatientsAppointments INNER JOIN AppointmentTypes ON PatientsAppointments.AppointmentTypeID = AppointmentTypes.AppointmentTypeID WHERE " & _
"(PatientsAppointments.Date > #WeekStartDate AND PatientsAppointments.Date < #WeekFinishDate) AND (PatientsAppointments.Status = 'Fulfilled') AND " & _
"(PatientsAppointments.DoctorID = #DoctorID) AND (AppointmentTypes.AppointmentName <> 'Note')"
'didn't get the error here... but just in case...
WeekFulfilled = mySQL.ExecuteScalar
Catch ex As Exception
ErrorSender.SendError("frmAppointmentBook - RefreshHeader 2", ex, New String() {String.Format("mySQL.commandtext: {0}", mySQL.CommandText)})
End Try
conSLDB.Close()
End Using
End SyncLock
The exact error message is:
System.InvalidCastException
Conversion from string "Needs Attention DC" to type 'Integer' is not valid.

Your problem has nothing to do with the COUNT(*) portion of your code. The problem is somewhere else in your query. What that particular error is telling you is that at some point you are comparing a character field (it probably usually contains numbers) to an integer field. One of the values of the character field happens to be "Needs Attention DC". If I had to guess it is probably either patientsappointments.appointmenttypeid or appointmenttypes.appointmenttypeid. Double check the datatype of each of those columns to make sure they are in fact INT. If they are both INT then start checking the other explicitly named columns in your query to see if you have any surprises.

You must have an error somewhere in your implementation...
Per the documentation, count always returns an int data type value.

Since this doesn't always happen, it must be a result of one of the paramenter values that is sent in. This is one of the lbuiggest problems with using dynamic SQL. What I would do is create the dymanic SQl and then store it in a database logging table with the date and time and user who executed it. Then when you get the exception, you can find the exact SQL code that was sent. Most likely you need more controls on the input variables to ensure the data placed in them is of the correct data type.

I am going to make another guess. I am guessing that this is a multi threading issue. You probably are sharing the connection between multiple threads. Once in a while the thread will get that man from somewhere else and execute it. Make sure that the connection variable is local, and only one thread can access it at a time.
As Martin points out, the following answer is wrong. I'm keeping this here to show that this is wrong.
From what everyone has already said, there is a type mismatch on your columns. Since your where clause appears to be fine, and your join is fine, it must be elsewhere. I would check to see if patientsappointments or appointmenttypes are views. Maybe the view has a join that's throwing the exception. Check the schema definition of all your joins/where's. Somewhere in there you're storing integers in a character field. It's fine for most rows, but one of them has your string.
If it's not in your views, it may be a trigger somewhere. The point is that somewhere there is a schema mismatch. Once you find your schema mismatch, you can find the row by querying for that string.

Related

SQL Server query using case statement IN Clause doesn't work [duplicate]

What are the best workarounds for using a SQL IN clause with instances of java.sql.PreparedStatement, which is not supported for multiple values due to SQL injection attack security issues: One ? placeholder represents one value, rather than a list of values.
Consider the following SQL statement:
SELECT my_column FROM my_table where search_column IN (?)
Using preparedStatement.setString( 1, "'A', 'B', 'C'" ); is essentially a non-working attempt at a workaround of the reasons for using ? in the first place.
What workarounds are available?
An analysis of the various options available, and the pros and cons of each is available in Jeanne Boyarsky's Batching Select Statements in JDBC entry on JavaRanch Journal.
The suggested options are:
Prepare SELECT my_column FROM my_table WHERE search_column = ?, execute it for each value and UNION the results client-side. Requires only one prepared statement. Slow and painful.
Prepare SELECT my_column FROM my_table WHERE search_column IN (?,?,?) and execute it. Requires one prepared statement per size-of-IN-list. Fast and obvious.
Prepare SELECT my_column FROM my_table WHERE search_column = ? ; SELECT my_column FROM my_table WHERE search_column = ? ; ... and execute it. [Or use UNION ALL in place of those semicolons. --ed] Requires one prepared statement per size-of-IN-list. Stupidly slow, strictly worse than WHERE search_column IN (?,?,?), so I don't know why the blogger even suggested it.
Use a stored procedure to construct the result set.
Prepare N different size-of-IN-list queries; say, with 2, 10, and 50 values. To search for an IN-list with 6 different values, populate the size-10 query so that it looks like SELECT my_column FROM my_table WHERE search_column IN (1,2,3,4,5,6,6,6,6,6). Any decent server will optimize out the duplicate values before running the query.
None of these options are ideal.
The best option if you are using JDBC4 and a server that supports x = ANY(y), is to use PreparedStatement.setArray as described in Boris's anwser.
There doesn't seem to be any way to make setArray work with IN-lists, though.
Sometimes SQL statements are loaded at runtime (e.g., from a properties file) but require a variable number of parameters. In such cases, first define the query:
query=SELECT * FROM table t WHERE t.column IN (?)
Next, load the query. Then determine the number of parameters prior to running it. Once the parameter count is known, run:
sql = any( sql, count );
For example:
/**
* Converts a SQL statement containing exactly one IN clause to an IN clause
* using multiple comma-delimited parameters.
*
* #param sql The SQL statement string with one IN clause.
* #param params The number of parameters the SQL statement requires.
* #return The SQL statement with (?) replaced with multiple parameter
* placeholders.
*/
public static String any(String sql, final int params) {
// Create a comma-delimited list based on the number of parameters.
final StringBuilder sb = new StringBuilder(
String.join(", ", Collections.nCopies(possibleValue.size(), "?")));
// For more than 1 parameter, replace the single parameter with
// multiple parameter placeholders.
if (sb.length() > 1) {
sql = sql.replace("(?)", "(" + sb + ")");
}
// Return the modified comma-delimited list of parameters.
return sql;
}
For certain databases where passing an array via the JDBC 4 specification is unsupported, this method can facilitate transforming the slow = ? into the faster IN (?) clause condition, which can then be expanded by calling the any method.
Solution for PostgreSQL:
final PreparedStatement statement = connection.prepareStatement(
"SELECT my_column FROM my_table where search_column = ANY (?)"
);
final String[] values = getValues();
statement.setArray(1, connection.createArrayOf("text", values));
try (ResultSet rs = statement.executeQuery()) {
while(rs.next()) {
// do some...
}
}
or
final PreparedStatement statement = connection.prepareStatement(
"SELECT my_column FROM my_table " +
"where search_column IN (SELECT * FROM unnest(?))"
);
final String[] values = getValues();
statement.setArray(1, connection.createArrayOf("text", values));
try (ResultSet rs = statement.executeQuery()) {
while(rs.next()) {
// do some...
}
}
No simple way AFAIK.
If the target is to keep statement cache ratio high (i.e to not create a statement per every parameter count), you may do the following:
create a statement with a few (e.g. 10) parameters:
... WHERE A IN (?,?,?,?,?,?,?,?,?,?) ...
Bind all actuall parameters
setString(1,"foo");
setString(2,"bar");
Bind the rest as NULL
setNull(3,Types.VARCHAR)
...
setNull(10,Types.VARCHAR)
NULL never matches anything, so it gets optimized out by the SQL plan builder.
The logic is easy to automate when you pass a List into a DAO function:
while( i < param.size() ) {
ps.setString(i+1,param.get(i));
i++;
}
while( i < MAX_PARAMS ) {
ps.setNull(i+1,Types.VARCHAR);
i++;
}
You can use Collections.nCopies to generate a collection of placeholders and join them using String.join:
List<String> params = getParams();
String placeHolders = String.join(",", Collections.nCopies(params.size(), "?"));
String sql = "select * from your_table where some_column in (" + placeHolders + ")";
try ( Connection connection = getConnection();
PreparedStatement ps = connection.prepareStatement(sql)) {
int i = 1;
for (String param : params) {
ps.setString(i++, param);
}
/*
* Execute query/do stuff
*/
}
An unpleasant work-around, but certainly feasible is to use a nested query. Create a temporary table MYVALUES with a column in it. Insert your list of values into the MYVALUES table. Then execute
select my_column from my_table where search_column in ( SELECT value FROM MYVALUES )
Ugly, but a viable alternative if your list of values is very large.
This technique has the added advantage of potentially better query plans from the optimizer (check a page for multiple values, tablescan only once instead once per value, etc) may save on overhead if your database doesn't cache prepared statements. Your "INSERTS" would need to be done in batch and the MYVALUES table may need to be tweaked to have minimal locking or other high-overhead protections.
Limitations of the in() operator is the root of all evil.
It works for trivial cases, and you can extend it with "automatic generation of the prepared statement" however it is always having its limits.
if you're creating a statement with variable number of parameters, that will make an sql parse overhead at each call
on many platforms, the number of parameters of in() operator are limited
on all platforms, total SQL text size is limited, making impossible for sending down 2000 placeholders for the in params
sending down bind variables of 1000-10k is not possible, as the JDBC driver is having its limitations
The in() approach can be good enough for some cases, but not rocket proof :)
The rocket-proof solution is to pass the arbitrary number of parameters in a separate call (by passing a clob of params, for example), and then have a view (or any other way) to represent them in SQL and use in your where criteria.
A brute-force variant is here http://tkyte.blogspot.hu/2006/06/varying-in-lists.html
However if you can use PL/SQL, this mess can become pretty neat.
function getCustomers(in_customerIdList clob) return sys_refcursor is
begin
aux_in_list.parse(in_customerIdList);
open res for
select *
from customer c,
in_list v
where c.customer_id=v.token;
return res;
end;
Then you can pass arbitrary number of comma separated customer ids in the parameter, and:
will get no parse delay, as the SQL for select is stable
no pipelined functions complexity - it is just one query
the SQL is using a simple join, instead of an IN operator, which is quite fast
after all, it is a good rule of thumb of not hitting the database with any plain select or DML, since it is Oracle, which offers lightyears of more than MySQL or similar simple database engines. PL/SQL allows you to hide the storage model from your application domain model in an effective way.
The trick here is:
we need a call which accepts the long string, and store somewhere where the db session can access to it (e.g. simple package variable, or dbms_session.set_context)
then we need a view which can parse this to rows
and then you have a view which contains the ids you're querying, so all you need is a simple join to the table queried.
The view looks like:
create or replace view in_list
as
select
trim( substr (txt,
instr (txt, ',', 1, level ) + 1,
instr (txt, ',', 1, level+1)
- instr (txt, ',', 1, level) -1 ) ) as token
from (select ','||aux_in_list.getpayload||',' txt from dual)
connect by level <= length(aux_in_list.getpayload)-length(replace(aux_in_list.getpayload,',',''))+1
where aux_in_list.getpayload refers to the original input string.
A possible approach would be to pass pl/sql arrays (supported by Oracle only), however you can't use those in pure SQL, therefore a conversion step is always needed. The conversion can not be done in SQL, so after all, passing a clob with all parameters in string and converting it witin a view is the most efficient solution.
Here's how I solved it in my own application. Ideally, you should use a StringBuilder instead of using + for Strings.
String inParenthesis = "(?";
for(int i = 1;i < myList.size();i++) {
inParenthesis += ", ?";
}
inParenthesis += ")";
try(PreparedStatement statement = SQLite.connection.prepareStatement(
String.format("UPDATE table SET value='WINNER' WHERE startTime=? AND name=? AND traderIdx=? AND someValue IN %s", inParenthesis))) {
int x = 1;
statement.setLong(x++, race.startTime);
statement.setString(x++, race.name);
statement.setInt(x++, traderIdx);
for(String str : race.betFair.winners) {
statement.setString(x++, str);
}
int effected = statement.executeUpdate();
}
Using a variable like x above instead of concrete numbers helps a lot if you decide to change the query at a later time.
I've never tried it, but would .setArray() do what you're looking for?
Update: Evidently not. setArray only seems to work with a java.sql.Array that comes from an ARRAY column that you've retrieved from a previous query, or a subquery with an ARRAY column.
My workaround is:
create or replace type split_tbl as table of varchar(32767);
/
create or replace function split
(
p_list varchar2,
p_del varchar2 := ','
) return split_tbl pipelined
is
l_idx pls_integer;
l_list varchar2(32767) := p_list;
l_value varchar2(32767);
begin
loop
l_idx := instr(l_list,p_del);
if l_idx > 0 then
pipe row(substr(l_list,1,l_idx-1));
l_list := substr(l_list,l_idx+length(p_del));
else
pipe row(l_list);
exit;
end if;
end loop;
return;
end split;
/
Now you can use one variable to obtain some values in a table:
select * from table(split('one,two,three'))
one
two
three
select * from TABLE1 where COL1 in (select * from table(split('value1,value2')))
value1 AAA
value2 BBB
So, the prepared statement could be:
"select * from TABLE where COL in (select * from table(split(?)))"
Regards,
Javier Ibanez
I suppose you could (using basic string manipulation) generate the query string in the PreparedStatement to have a number of ?'s matching the number of items in your list.
Of course if you're doing that you're just a step away from generating a giant chained OR in your query, but without having the right number of ? in the query string, I don't see how else you can work around this.
You could use setArray method as mentioned in this javadoc:
PreparedStatement statement = connection.prepareStatement("Select * from emp where field in (?)");
Array array = statement.getConnection().createArrayOf("VARCHAR", new Object[]{"E1", "E2","E3"});
statement.setArray(1, array);
ResultSet rs = statement.executeQuery();
Here's a complete solution in Java to create the prepared statement for you:
/*usage:
Util u = new Util(500); //500 items per bracket.
String sqlBefore = "select * from myTable where (";
List<Integer> values = new ArrayList<Integer>(Arrays.asList(1,2,4,5));
string sqlAfter = ") and foo = 'bar'";
PreparedStatement ps = u.prepareStatements(sqlBefore, values, sqlAfter, connection, "someId");
*/
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;
public class Util {
private int numValuesInClause;
public Util(int numValuesInClause) {
super();
this.numValuesInClause = numValuesInClause;
}
public int getNumValuesInClause() {
return numValuesInClause;
}
public void setNumValuesInClause(int numValuesInClause) {
this.numValuesInClause = numValuesInClause;
}
/** Split a given list into a list of lists for the given size of numValuesInClause*/
public List<List<Integer>> splitList(
List<Integer> values) {
List<List<Integer>> newList = new ArrayList<List<Integer>>();
while (values.size() > numValuesInClause) {
List<Integer> sublist = values.subList(0,numValuesInClause);
List<Integer> values2 = values.subList(numValuesInClause, values.size());
values = values2;
newList.add( sublist);
}
newList.add(values);
return newList;
}
/**
* Generates a series of split out in clause statements.
* #param sqlBefore ""select * from dual where ("
* #param values [1,2,3,4,5,6,7,8,9,10]
* #param "sqlAfter ) and id = 5"
* #return "select * from dual where (id in (1,2,3) or id in (4,5,6) or id in (7,8,9) or id in (10)"
*/
public String genInClauseSql(String sqlBefore, List<Integer> values,
String sqlAfter, String identifier)
{
List<List<Integer>> newLists = splitList(values);
String stmt = sqlBefore;
/* now generate the in clause for each list */
int j = 0; /* keep track of list:newLists index */
for (List<Integer> list : newLists) {
stmt = stmt + identifier +" in (";
StringBuilder innerBuilder = new StringBuilder();
for (int i = 0; i < list.size(); i++) {
innerBuilder.append("?,");
}
String inClause = innerBuilder.deleteCharAt(
innerBuilder.length() - 1).toString();
stmt = stmt + inClause;
stmt = stmt + ")";
if (++j < newLists.size()) {
stmt = stmt + " OR ";
}
}
stmt = stmt + sqlAfter;
return stmt;
}
/**
* Method to convert your SQL and a list of ID into a safe prepared
* statements
*
* #throws SQLException
*/
public PreparedStatement prepareStatements(String sqlBefore,
ArrayList<Integer> values, String sqlAfter, Connection c, String identifier)
throws SQLException {
/* First split our potentially big list into lots of lists */
String stmt = genInClauseSql(sqlBefore, values, sqlAfter, identifier);
PreparedStatement ps = c.prepareStatement(stmt);
int i = 1;
for (int val : values)
{
ps.setInt(i++, val);
}
return ps;
}
}
Spring allows passing java.util.Lists to NamedParameterJdbcTemplate , which automates the generation of (?, ?, ?, ..., ?), as appropriate for the number of arguments.
For Oracle, this blog posting discusses the use of oracle.sql.ARRAY (Connection.createArrayOf doesn't work with Oracle). For this you have to modify your SQL statement:
SELECT my_column FROM my_table where search_column IN (select COLUMN_VALUE from table(?))
The oracle table function transforms the passed array into a table like value usable in the IN statement.
try using the instr function?
select my_column from my_table where instr(?, ','||search_column||',') > 0
then
ps.setString(1, ",A,B,C,");
Admittedly this is a bit of a dirty hack, but it does reduce the opportunities for sql injection. Works in oracle anyway.
Sormula supports SQL IN operator by allowing you to supply a java.util.Collection object as a parameter. It creates a prepared statement with a ? for each of the elements the collection. See Example 4 (SQL in example is a comment to clarify what is created but is not used by Sormula).
Generate the query string in the PreparedStatement to have a number of ?'s matching the number of items in your list. Here's an example:
public void myQuery(List<String> items, int other) {
...
String q4in = generateQsForIn(items.size());
String sql = "select * from stuff where foo in ( " + q4in + " ) and bar = ?";
PreparedStatement ps = connection.prepareStatement(sql);
int i = 1;
for (String item : items) {
ps.setString(i++, item);
}
ps.setInt(i++, other);
ResultSet rs = ps.executeQuery();
...
}
private String generateQsForIn(int numQs) {
String items = "";
for (int i = 0; i < numQs; i++) {
if (i != 0) items += ", ";
items += "?";
}
return items;
}
instead of using
SELECT my_column FROM my_table where search_column IN (?)
use the Sql Statement as
select id, name from users where id in (?, ?, ?)
and
preparedStatement.setString( 1, 'A');
preparedStatement.setString( 2,'B');
preparedStatement.setString( 3, 'C');
or use a stored procedure this would be the best solution, since the sql statements will be compiled and stored in DataBase server
I came across a number of limitations related to prepared statement:
The prepared statements are cached only inside the same session (Postgres), so it will really work only with connection pooling
A lot of different prepared statements as proposed by #BalusC may cause the cache to overfill and previously cached statements will be dropped
The query has to be optimized and use indices. Sounds obvious, however e.g. the ANY(ARRAY...) statement proposed by #Boris in one of the top answers cannot use indices and query will be slow despite caching
The prepared statement caches the query plan as well and the actual values of any parameters specified in the statement are unavailable.
Among the proposed solutions I would choose the one that doesn't decrease the query performance and makes the less number of queries. This will be the #4 (batching few queries) from the #Don link or specifying NULL values for unneeded '?' marks as proposed by #Vladimir Dyuzhev
SetArray is the best solution but its not available for many older drivers. The following workaround can be used in java8
String baseQuery ="SELECT my_column FROM my_table where search_column IN (%s)"
String markersString = inputArray.stream().map(e -> "?").collect(joining(","));
String sqlQuery = String.format(baseSQL, markersString);
//Now create Prepared Statement and use loop to Set entries
int index=1;
for (String input : inputArray) {
preparedStatement.setString(index++, input);
}
This solution is better than other ugly while loop solutions where the query string is built by manual iterations
I just worked out a PostgreSQL-specific option for this. It's a bit of a hack, and comes with its own pros and cons and limitations, but it seems to work and isn't limited to a specific development language, platform, or PG driver.
The trick of course is to find a way to pass an arbitrary length collection of values as a single parameter, and have the db recognize it as multiple values. The solution I have working is to construct a delimited string from the values in the collection, pass that string as a single parameter, and use string_to_array() with the requisite casting for PostgreSQL to properly make use of it.
So if you want to search for "foo", "blah", and "abc", you might concatenate them together into a single string as: 'foo,blah,abc'. Here's the straight SQL:
select column from table
where search_column = any (string_to_array('foo,blah,abc', ',')::text[]);
You would obviously change the explicit cast to whatever you wanted your resulting value array to be -- int, text, uuid, etc. And because the function is taking a single string value (or two I suppose, if you want to customize the delimiter as well), you can pass it as a parameter in a prepared statement:
select column from table
where search_column = any (string_to_array($1, ',')::text[]);
This is even flexible enough to support things like LIKE comparisons:
select column from table
where search_column like any (string_to_array('foo%,blah%,abc%', ',')::text[]);
Again, no question it's a hack, but it works and allows you to still use pre-compiled prepared statements that take *ahem* discrete parameters, with the accompanying security and (maybe) performance benefits. Is it advisable and actually performant? Naturally, it depends, as you've got string parsing and possibly casting going on before your query even runs. If you're expecting to send three, five, a few dozen values, sure, it's probably fine. A few thousand? Yeah, maybe not so much. YMMV, limitations and exclusions apply, no warranty express or implied.
But it works.
No one else seems to have suggested using an off-the-shelf query builder yet, like jOOQ or QueryDSL or even Criteria Query that manage dynamic IN lists out of the box, possibly including the management of all edge cases that may arise, such as:
Running into Oracle's maximum of 1000 elements per IN list (irrespective of the number of bind values)
Running into any driver's maximum number of bind values, which I've documented in this answer
Running into cursor cache contention problems because too many distinct SQL strings are "hard parsed" and execution plans cannot be cached anymore (jOOQ and since recently also Hibernate work around this by offering IN list padding)
(Disclaimer: I work for the company behind jOOQ)
Just for completeness: So long as the set of values is not too large, you could also simply string-construct a statement like
... WHERE tab.col = ? OR tab.col = ? OR tab.col = ?
which you could then pass to prepare(), and then use setXXX() in a loop to set all the values. This looks yucky, but many "big" commercial systems routinely do this kind of thing until they hit DB-specific limits, such as 32 KB (I think it is) for statements in Oracle.
Of course you need to ensure that the set will never be unreasonably large, or do error trapping in the event that it is.
Following Adam's idea. Make your prepared statement sort of select my_column from my_table where search_column in (#)
Create a String x and fill it with a number of "?,?,?" depending on your list of values
Then just change the # in the query for your new String x an populate
There are different alternative approaches that we can use for IN clause in PreparedStatement.
Using Single Queries - slowest performance and resource intensive
Using StoredProcedure - Fastest but database specific
Creating dynamic query for PreparedStatement - Good Performance but doesn't get benefit of caching and PreparedStatement is recompiled every time.
Use NULL in PreparedStatement queries - Optimal performance, works great when you know the limit of IN clause arguments. If there is no limit, then you can execute queries in batch.
Sample code snippet is;
int i = 1;
for(; i <=ids.length; i++){
ps.setInt(i, ids[i-1]);
}
//set null for remaining ones
for(; i<=PARAM_SIZE;i++){
ps.setNull(i, java.sql.Types.INTEGER);
}
You can check more details about these alternative approaches here.
For some situations regexp might help.
Here is an example I've checked on Oracle, and it works.
select * from my_table where REGEXP_LIKE (search_column, 'value1|value2')
But there is a number of drawbacks with it:
Any column it applied should be converted to varchar/char, at least implicitly.
Need to be careful with special characters.
It can slow down performance - in my case IN version uses index and range scan, and REGEXP version do full scan.
After examining various solutions in different forums and not finding a good solution, I feel the below hack I came up with, is the easiest to follow and code:
Example: Suppose you have multiple parameters to pass in the 'IN' clause. Just put a dummy String inside the 'IN' clause, say, "PARAM" do denote the list of parameters that will be coming in the place of this dummy String.
select * from TABLE_A where ATTR IN (PARAM);
You can collect all the parameters into a single String variable in your Java code. This can be done as follows:
String param1 = "X";
String param2 = "Y";
String param1 = param1.append(",").append(param2);
You can append all your parameters separated by commas into a single String variable, 'param1', in our case.
After collecting all the parameters into a single String you can just replace the dummy text in your query, i.e., "PARAM" in this case, with the parameter String, i.e., param1. Here is what you need to do:
String query = query.replaceFirst("PARAM",param1); where we have the value of query as
query = "select * from TABLE_A where ATTR IN (PARAM)";
You can now execute your query using the executeQuery() method. Just make sure that you don't have the word "PARAM" in your query anywhere. You can use a combination of special characters and alphabets instead of the word "PARAM" in order to make sure that there is no possibility of such a word coming in the query. Hope you got the solution.
Note: Though this is not a prepared query, it does the work that I wanted my code to do.
Just for completeness and because I did not see anyone else suggest it:
Before implementing any of the complicated suggestions above consider if SQL injection is indeed a problem in your scenario.
In many cases the value provided to IN (...) is a list of ids that have been generated in a way that you can be sure that no injection is possible... (e.g. the results of a previous select some_id from some_table where some_condition.)
If that is the case you might just concatenate this value and not use the services or the prepared statement for it or use them for other parameters of this query.
query="select f1,f2 from t1 where f3=? and f2 in (" + sListOfIds + ");";
PreparedStatement doesn't provide any good way to deal with SQL IN clause. Per http://www.javaranch.com/journal/200510/Journal200510.jsp#a2 "You can't substitute things that are meant to become part of the SQL statement. This is necessary because if the SQL itself can change, the driver can't precompile the statement. It also has the nice side effect of preventing SQL injection attacks." I ended up using following approach:
String query = "SELECT my_column FROM my_table where search_column IN ($searchColumns)";
query = query.replace("$searchColumns", "'A', 'B', 'C'");
Statement stmt = connection.createStatement();
boolean hasResults = stmt.execute(query);
do {
if (hasResults)
return stmt.getResultSet();
hasResults = stmt.getMoreResults();
} while (hasResults || stmt.getUpdateCount() != -1);
OK, so I couldn't remember exactly how (or where) I did this before so I came to stack overflow to quickly find the answer. I was surprised I couldn't.
So, how I got around the IN problem a long time ago was with a statement like this:
where myColumn in ( select regexp_substr(:myList,'[^,]+', 1, level) from dual connect by regexp_substr(:myList, '[^,]+', 1, level) is not null)
set the myList parameter as a comma delimited string: A,B,C,D...
Note: You have to set the parameter twice!
This is not the ideal practice, yet it's simple and works well for me most of the time.
where ? like concat( "%|", TABLE_ID , "|%" )
Then you pass through ? the IDs in this way: |1|,|2|,|3|,...|

SSMS Error "An expression of non-boolean type specified in a context where a condition is expected, near '('"

I am getting the titled error from a number of SQL Server views I am trying to create. They are modified from a MS Access database I'm upgrading to use SQL as the back end. See SQL below:
SELECT dbo.Site_Info_All.SiteID,
IIf(dbo.Site_Info_All.Fixed_Charge, N'Yes', N'No') AS [Fixed Charge],
dbo.Site_Info_All.Fixed_Charge_Date, dbo.MG_Definition.MG_Definition
FROM dbo.MG_Definition INNER JOIN
dbo.Site_Info_All ON dbo.MG_Definition.MG_DefinitionID =
dbo.Site_Info_All.MG_DefinitionID
It looks as though I'm being told that the Fixed_Charge field is not boolean, except that it is. I'm encountering this issue with multiple views. What am I doing wrong?
It's not boolean. It's a bit. :)
A bit column doesn't directly evaluate to true or false, the way a bool does in other languages. You actually have to compare it to another bit value to return the boolean value.
declare #bit bit = 1;
if (#bit) print '#bit was true'; -- this does not work
if (#bit = 1) print '#bit = 1 was true'; -- this works
What you want is:
... IIf(dbo.Site_Info_All.Fixed_Charge = 1, N'Yes', N'No') ...
SELECT dbo.Site_Info_All.SiteID,
IIf(dbo.Site_Info_All.Fixed_Charge = 1, N'Yes', N'No') AS [Fixed Charge],
dbo.Site_Info_All.Fixed_Charge_Date,
dbo.MG_Definition.MG_Definition
FROM dbo.MG_Definition
INNER JOIN dbo.Site_Info_All ON dbo.MG_Definition.MG_DefinitionID = dbo.Site_Info_All.MG_DefinitionID
In the end, I decided that IIF simply wasn't working for whatever reason. An answer to that would be very welcome, btw.
I used
SELECT dbo.Site_Info_All.SiteID,
CASE WHEN dbo.Site_Info_All.Fixed_Charge = 1
THEN N'Yes'
ELSE N'No'
END AS [Fixed Charge], dbo.Site_Info_All.Fixed_Charge_Date,
dbo.MG_Definition.MG_Definition
FROM dbo.MG_Definition INNER JOIN
dbo.Site_Info_All ON dbo.MG_Definition.MG_DefinitionID =
dbo.Site_Info_All.MG_DefinitionID
That gave me the results I was after, but only after pulling out what little hair I have left.

Is conversion from string to decimal needed and if so how should it be done?

I am trying to write a code for correction of entries to my SQL Server database. I am a mechanical engineering student who has a programming class and I have never programmed before so I am not sure should I convert string to decimal and how. Last 2 rows contain 2 options I came up with. Second one is what I use for pure string, first one is a modification of formatting datetime.
This is my stored procedure:
ALTER PROCEDURE [dbo].[SP_RN_O_Ispravak]
#Br_RN_O bigint,
#Datum_O DateTime OUTPUT,
#Sifra_p int OUTPUT,
#Ime_P nvarchar (30) output,
#Prezime_P NVarChar(30) OUTPUT,
#Naziv_P nvarchar (50) output,
#Adresa_P nvarchar (50) OUTPUT,
#Telefon_P NVarChar(15) OUTPUT,
#Sifra_z int OUTPUT,
#Ime_Z nvarchar (30) output,
#Prezime_Z nvarchar (30) output,
#Sifra_kul nvarchar (3) OUTPUT,
#Naziv_Kul NVarChar(20) OUTPUT,
#Masa_O decimal (5,0) OUTPUT,
#Vlaga_O decimal (4,1) OUTPUT,
#Hek_Masa_O decimal (3,1) OUTPUT,
#Protein_O decimal (3,1) output,
#Cijena_O decimal (3,2) output
AS
SELECT #Br_RN_O=T_Otkup.Br_RN_O,
#Datum_O=T_Otkup.Datum_O,
#Sifra_p=T_Otkup.Sifra_p,
#Sifra_z=T_Otkup.Sifra_z,
#Sifra_kul=T_Otkup.Sifra_kul,
#Masa_O=T_Otkup.Masa_O,
#Vlaga_O=T_Otkup.Vlaga_O,
#Hek_Masa_O=T_Otkup.Hek_Masa_O,
#Protein_O=T_Otkup.Protein_O
FROM T_Otkup
WHERE (T_Otkup.Br_RN_O = #Br_RN_O)
SELECT #Prezime_P=Prezime_P
FROM T_Poljoprivrednik
WHERE Sifra_P=#Sifra_p
SELECT #Prezime_z=Prezime_Z
FROM T_Zaposlenik
WHERE Sifra_Z=#Sifra_z
SELECT #Naziv_kul=Naziv_Kul
FROM T_Kultura
WHERE Sifra_Kul=#Sifra_kul
RETURN
This procedure is supposed to pull the data from the database and place it in textboxes shown in the image.visual of whati'm trying to make
I managed to use the following code to convert decimal to string:
Dim cijenao As SqlParameter = New SqlParameter("#Cijena_O", Data.SqlDbType.Decimal, 3, 2)
cijenao.Direction = Data.ParameterDirection.Output
cijenao.Value = Cijena_O.Text
cmd.Parameters.Add(cijenao)
Masa_O.Text = Format(masao.Value, "#####").ToString
Vlaga_O.Text = Format(vlagao.Value, "###.#").ToString
Hek_Masa_O.Text = Format(hmasao.Value, "##.#").ToString
Protein_O.Text = Format(proto.Value, "##.#").ToString
However, it doesn't work for 2 decimal places like this:
Cijena_O.Text = Format(cijenao.Value, "#.##").ToString
I tried using the code posted by Mary, but it get the following message:
System.Data.SqlClient.SqlException: 'Procedure or function SP_RN_O_Ispravak has too many arguments specified.'
I've cleaned up this Sub, to properly scope the connection, and make sure the Connection and Command objects get disposed (via Using). It is always best to explicitly handle the data type conversions, such as using .ToString() on the .Value property of the parameters. Note I also parse the Long before assigning it to the input parameter (although you should Google the .TryParse() method and use that).
Protected Sub ISPRAVAK_NALOGA()
Using conn As New SqlConnection(<your connection string here>)
Using cmd As New SqlCommand("SP_RN_O_Ispravak", conn) With {.CommandType = CommandType.StoredProcedure}
With cmd
.Parameters.Add("#Br_RN_O", SqlDbType.BigInt).Value = Long.Parse(Br_RN_O.Text)
.Parameters.Add("#Masa_O", Data.SqlDbType.Decimal, 5, 0)
.Parameters("#Masa_O").Direction = ParameterDirection.Output
.Parameters.Add("#Vlaga_O", Data.SqlDbType.Decimal, 4, 1)
.Parameters("#Vlaga_O").Direction = ParameterDirection.Output
conn.Open()
.ExecuteNonQuery()
Masa_O.Text = .Parameters("#Masa_O").Value.ToString
Vlaga_O.Text = .Parameters("#Vlaga_O").Value.ToString
End With
End Using
End Using
End Sub
As to your code...
Check the available overloads for the Constructors for the Parameter class. https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlparameter?view=netframework-4.8#constructors There are none that match your code. Dim masao As SqlParameter = New SqlParameter("#Masa_O", Data.SqlDbType.Decimal, 5, 0) The final parameter of the constructor with 4 parameters is a string holding the name of the source column.
A Bigint in Sql Server maps to and Int64 in .net. (A Long in vb.net) This is a good reference for mapping datatypes from Sql Server to .net. https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-data-type-mappings.
Now my code...
Declare the variable outside the using blocks.
The TryParse is a very clever method that not only test a string but fills the variable with the converted string when it succeeds. Return is functionly equivalent, in this case, to the vb.net specific Exit Sub . Return is often used in other languages (think C#).
Keep your databse objects local so you can control their closing and disposing. A Using block will do this for you even if there is an error. You don't need to create new variable for the parameters. They can be referred to by name in the Parameters collection. Set the values of maso an vlaga inside the Using block before the command is disposed.
After the database objects are duly discharges, we can set the values in the User Interface. Reguarding the .ToString method; N0 (the 0 is a zero) will give you a string containing the number with no decimal protion. The N stands for Number and the 0 is the number of decimal places. It adds commas to make the number easier to read and it will round as appropriate. See https://learn.microsoft.com/en-us/dotnet/standard/base-types/standard-numeric-format-strings for more details.
Ideally the database code and the UI code would be separated but that is for another day.
Protected Sub ISPRAVAK_NALOGA()
Dim maso As Decimal
Dim vlaga As Decimal
Dim InputNumber As Long
If Not Long.TryParse(Br_RN_O.Text, InputNumber) Then
MessageBox.Show("Please enter a valid number.")
Return
End If
Using conn As New SqlConnection("Your connection string")
Using cmd As New SqlCommand("SP_RN_O_Ispravak", conn)
cmd.CommandType = CommandType.StoredProcedure
With cmd.Parameters
.Add("#Br_RN_O", SqlDbType.BigInt).Value = InputNumber
.Add("#Masa_O", Data.SqlDbType.Decimal)
.Add("#Vlaga_O", Data.SqlDbType.Decimal)
End With
cmd.Parameters("#Vlaga_O").Direction = ParameterDirection.Output
cmd.Parameters("#Masa_O").Direction = Data.ParameterDirection.Output
conn.Open()
cmd.ExecuteNonQuery()
maso = CDec(cmd.Parameters("#Masa_O").Value)
vlaga = CDec(cmd.Parameters("#Vlaga_O").Value)
End Using
End Using
Masa_O.Text = maso.ToString("N0")
Vlaga_O.Text = vlaga.ToString
End Sub
I am a bit unsure of what you intent.
But if you want to know if you need to convert a number into a string before assigning the value to a textbox, then the answer is yes. You do need to convert it.
But there are a few things that you can do to display decimal values in a more readable way. For example, you can set the textbox customFormat to #########0.00 or ######,##0.00

Stored procedure output parameters return empty

In my SQL Server 2014 I have a Stored procedure that returns 2 values in 2 variables as output:
#TotalNoRatio
#TotalRatio
Here are the results after execution:
#TotalNoRatio #TotalRatio
34510793 31857292
Return Value 0
Now I want those 2 values to be display in a Label on my form.
Here is the code:
cmd2.CommandType = CommandType.StoredProcedure
cmd2.Parameters.Add("#TotalNoRatio", SqlDbType.Decimal)
cmd2.Parameters.Add("#TotalRatio", SqlDbType.Decimal)
cmd2.ExecuteNonQuery()
Me.LTotal1.Text = cmd2.Parameters("#TotalNoRatio").Value
Me.LTotal2.Text = cmd2.Parameters("#TotalRatio").Value
Everything runs fine without errors except that the results are empty.
You need to define direction as return something like this:
SqlParameter retval = sqlcomm.Parameters.Add("#TotalNoRatio", SqlDbType.Decimal);
retval.Direction = ParameterDirection.ReturnValue;
You will need to specify the direction of you parameters as ParameterDirection.Output. You will also need to declare your parameters on your procedure as OUTPUT.
I have put together a small example below.
This is my procedure:
CREATE PROCEDURE [dbo].[procedureName]
#TotalNoRatio DECIMAL(18,2) OUTPUT,
#TotalRatio DECIMAL(18,2) OUTPUT
AS
SET #TotalNoRatio = 2
SET #TotalRatio = 3
This is my VB.NET code:
Using con As New SqlConnection(conString),
cmd As New SqlCommand("procedureName", con) With {.CommandType = CommandType.StoredProcedure}
con.Open()
cmd.Parameters.Add(New SqlParameter("#TotalNoRatio", SqlDbType.Decimal) With {.Direction = ParameterDirection.Output})
cmd.Parameters.Add(New SqlParameter("#TotalRatio", SqlDbType.Decimal) With {.Direction = ParameterDirection.Output})
cmd.ExecuteNonQuery()
lTotal1.Text = "TotalNoRatio: " & cmd.Parameters("#TotalNoRatio").Value.ToString()
lTotal2.Text = "TotalRatio: " & cmd.Parameters("#TotalRatio").Value.ToString()
End Using
This is a screenshot of the output:
On a seperate note consider turning Option Strict On:
Restricts implicit data type conversions to only widening conversions, disallows late binding, and disallows implicit typing that results in an Object type.
cmd.Parameters("#TotalNoRatio").Value returns type Object. You should be appending .ToString() to it if you're assigning to Label.Text.
Also note that I have implemented Using. You may already have, it's difficult to tell but if you haven't it's worth doing:
Sometimes your code requires an unmanaged resource, such as a file handle, a COM wrapper, or a SQL connection. A Using block guarantees the disposal of one or more such resources when your code is finished with them. This makes them available for other code to use.

SqlHelper.ExecuteReader() throws an exception when no results found

I have an SqlDataReader that is declared like this:
Dim myReader As SqlDataReader
myReader = SqlHelper.ExecuteReader(ConnectionString, "storedProcedure1", CInt(myTextBox.Text))
Later I use the results like this:
If myReader.HasRows Then
While myReader.Read()
Row = Table1.NewRow()
Row.Item("REF") = myReader.GetString(0)
Row.Item("CD") = myReader.GetString(1)
Row.Item("NAME") = myReader.GetString(2)
Row.Item("KEY") = myReader.GetDecimal(3)
Row.Item("STRING") = myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")
Table1.Rows.Add(Row)
'Fill Drop Down
drpMenu.Items.Add(New ListItem(myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")))
End While
End If
myTextBox is a textbox that the user enters a possible location number that gets searched for using the stored procedure. If the user enters a valid location number, this works great and I have no problems. If they enter a non-existent location number, I get an exception:
System.IndexOutOfRangeException: Index was outside the bounds of the array.
I would think that the If myReader.HasRows line would keep me from trying to read and manipulate results that don't exist but there must be something I'm missing. myTextBox is already being validated elsewhere in the code to make sure the user typed in an integer without any wacky characters so bad input doesn't seem to be the problem either.
How do I find out whether the location number exists before calling SqlHelper.ExecuteReader()? Or maybe the better question is how do I gracefully handle this exception and tell the user the location wasn't found?
EDIT: Here's the stored procedure.
ALTER PROCEDURE [dbo].[storedProcedure]
-- Add the parameters for the stored procedure here
#MBR as integer
AS
BEGIN
EXEC ('{CALL RM#IMLIB.spGETLOC( ?)}', #MBR) at AS400
END
EDIT #2: When I run the stored procedure in SMS and pass a valid location, it returns what I expect. If I pass an invalid location number, it returns nothing.
First. create a local variable...and cast the textbox value to the local variable...to make sure that isn't the error.
dim myValue as Int32
myValue = '' convert textbox value to an int.
Second...typically, my datareader code looks like this.
If (Not ( reader Is Nothing) ) then
If reader.HasRows Then
Do While reader.Read()
Console.WriteLine(reader.GetInt32(0) _
& vbTab & reader.GetString(1))
Loop
End If
End If
(You'll have to adjust the GetInt32 or GetString and the ordinal number to your specific case of course).
My guess is that your child-procedure (RM#IMLIB.spGETLOC) has logic in it that does NOT return a row if there isn't a match.
You can still return a result....that has no rows in it. ** (Read that again).
For example
Select ColA, ColB from dbo.MyTable where 0=1
This will return a result, with no rows. That is different from not returning a(ny) select statement..(Read that again)
My guess is that this little nuance is where you get an issue.
APPEND:
If you cannot change the child-stored procedure....
Create a #TempTable...
Populate the #TempTable with the child-stored procedure.
Do a select from the #TempTable.
Here is a generic example:
IF OBJECT_ID('tempdb..#TempOrders') IS NOT NULL
begin
drop table #TempOrders
end
CREATE TABLE #TempOrders
(
ColumnA int
, [ColumnB] nchar(5)
)
/* Note, your #temp table must have the exact same columns as returned by the child procedure */
INSERT INTO #TempOrders ( ColumnA, ColumnB )
exec dbo.uspChildProcedure ParameterOne
Select ColumnA, ColumnB from #TempOrders
IF OBJECT_ID('tempdb..#TempOrderDetails') IS NOT NULL
begin
drop table #TempOrderDetails
end
I finally figured out my issue.
Later in my code, I had a DataRow array that was empty if the location couldn't be found. I was getting the exception because it was trying to grab an Item from array(0) which makes complete sense.
I missed it initially because I thought it was a DataRow, not a DataRow array. Man, I hate fixing up code I didn't write...

Resources