I'm first reading lower bound and upper bound using a:
select max(timestamp) ,min(timestamp) from table name
extracting
Row row=query.collectasList().get(0).getString(0) as lowerbound and upperbound
respectively
then passing lowerbound and upper bound
spark.read("jdbc")
.option("url", url)
.option("dbtable", "sample")
.option("user", user)
.option("driver","com.sqlserver")
.option("password", password)
.option("numPartitions", 100)
.option("partitionColumn", "timestamp")
.option("lowerBound", lowerbound)
.option("upperBound", upperbound )
lowerbound and upperbound format "2022-02-09 17:13:22.353"
I understand lowerbound and upper bound has to be string but when passed I'm facing the below issue can you please help?
Facing this issue Lost task 6.0 in stage 1.0 (TID 7, , executor 1):
com.microsoft.sqlserver.jdbc.SQLServerException: Conversion failed when converting date and/or time from character string.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java
Related
I converted a datastream using table api and have a time, key and value field
t = t_env.from_data_stream(
ds,
Schema.new_builder()
.column_by_expression("ts","TO_TIMESTAMP(FROM_UNIXTIME(f0))")
.column("f1", DataTypes.STRING())
.column("f2", DataTypes.INT())
.watermark("ts", "ts - INTERVAL '1' SECONDS")
.build()
).alias("time","key","value")
The row time for "ts" works fine for t, when I do
t.order_by(col("time").asc)
I then tried to agg the value field for each key and also keep the last time field:
t1 = t.group_by(col("key")).select(col("key"), col("time").max.alias("time"), col("value").sum.alias("value")
and do
t1.order_by(col("time").asc)
It always print TableException: org.apache.flink.table.api.TableException: Sort on a non-time-attribute field is not supported.
Thanks a lot!
I checked the type of the "time" field of t1 after group by and it has the same type as the "time" field in t, which is TIMESTAMP(3) ROWTIME, looks like the SQL_TIMESTAMP type.
I am aware that there are differences between Oracle SQL and SQL Server. The query runs fine and displays the result well but the issue arises when I want to display it on a pie chart. I am thinking that it might be some Visual Studio restriction.
Here is my SQL statement:
SELECT
CAST(ROUND(CAST(COUNT(LoanAcNo) AS FLOAT) / 73 * 100, 1) AS VARCHAR) + '%' AS LoanPercentage,
LoanType
FROM
Loan
GROUP BY
LoanType;
This is how I implemented it:
public DataSet ReadLoanByLoanType()
{
SqlConnection myConn = new SqlConnection(DBConnect);
StringBuilder sqlStr = new StringBuilder();
sqlStr.AppendLine("SELECT cast( round( cast ( count(LoanAcNo) as float) / 73 * 100 , 1 ) as varchar ) + '%' as LoanPercentage , LoanType");
sqlStr.AppendLine("FROM Loan");
sqlStr.AppendLine("GROUP BY LoanType");
SqlDataAdapter da = new SqlDataAdapter(sqlStr.ToString(), myConn);
DataSet ds = new DataSet();
da.Fill(ds);
return ds;
}
If you want to download values from a database for inclusion in some reporting tool's pie chart, don't turn the value into a string by adding a '%' onto them
Your reports tool will be expecting some numeric value to chart, not a string
SQL:
SELECT
round(count(LoanAcNo)/ 73.0 * 100.0, 1) as LoanPercentage,
LoanType
FROM Loan
GROUP BY LoanType;
Tip: dividing an integer by a constant number that has a decimal place (e.g. I divided by 73.0 instead of 73) rather than an integer should make SQLS do the calc using floats - saves you having to cast the int to float/makes the SQL shorter and neater
With Visual Basic in Visual Studio I am trying to select a row of type rowversion from a SQL Server database.
'version' is a column of type rowversion
SELECT [Version] FROM Employees WHERE Employee_id = 1
Then in VB - To get value from version column
Dim myBuffer As Byte()
Dim reader As SqlDataReader
numRead = reader.GetBytes(0, 0, myBuffer, 0, 16)
I know there is data in mybuffer from this:
For i = 0 To myBuffer.Length - 1
MsgBox(myBuffer(i).ToString())
Next
Also myBuffer.Length = 9
But when I want to query my database with myBuffer...
cmd.Parameters.AddWithValue("version", myBuffer)
I get the error:
Procedure or function 'updatePerson' expects parameter '#version', which was not supplied.
As if myBuffer in NULL.
Does anybody know to get the rowversion out of a database and then use it in a query?
Hibernate 3.6 & postgresql 9.1.
Using SQLQuery how to get result array data (array of Long - assistants, array of rows of "Text, Long, Timestamp" - accounts)?
limit = 10000;
final SQLQuery sqlQuery = getSession().createSQLQuery("SELECT id, name, ts, " +
" array(SELECT assistant_id FROM user_assistant WHERE p_id=pr.id ORDER BY assistant_id) AS accounts," +
" array(SELECT row(type,uid,ts) FROM user_account WHERE p_id=pr.id ORDER BY type) AS accs," +
" FROM profile pr WHERE ts > ? ORDER BY ts LIMIT " + limit);
The most of DAO functions written with hibernate Entities & annotations.
But for a few statistics tasks easier to work with HQL or even SQL.
As opposed to pure JDBC in hibernateSQL working with arrays is not so intuitive.
JDBC could be a solution, but I haven't found any way to get JDBC Statement from Hibernate Session or Connection.
ResultTransformer doesn't help also, fails with:
org.hibernate.MappingException: No Dialect mapping for JDBC type: 2003
By referring this
http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/querysql.html
What you can do something like;
session.createSQLQuery("Your custom query")
.addScalar("field1", Hibernate.STRING)
.addScalar("field2", Hibernate.STRING)
.addScalar("field3", Hibernate.STRING)
and then
for(Object rows : query.list()){
Object[] row = (Object[]) rows;
String field1 = row[0] // contains field1
String field2 = row[1]
..
..
}
I have a table in my SQL server 2008 R2 database which includes two nullable decimal(16,6) columns. Let's call them column1 and column2.
When I try to run a Linq query against the entity generated from this table:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2) / 2m
}
);
I get a System.OverflowException if column1 + column2 >= 15846. The message of the exception is only:
Conversion overflows.
With a bit of trial and error I've managed to make the query work with the following:
Table.Select(r => new Foo
{
Bar = (r.Column1 + r.Column2).HasValue ?
(r.Column1 + r.Column2).Value / 2m : 0
}
);
However, I was wondering if anyone could explain what was going wrong with the initial query.
Edit
The first query generates this SQL:
SELECT
1 AS [C1],
([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18)) AS [C2]
FROM [dbo].[Table] AS [Extent1]
With a value of 10000 for both columns, running the query manually in SSMS the result is 10000.0000000000000000000000000 (25 decimal zeros).
The second query has this SQL:
SELECT
1 AS [C1],
CASE WHEN ([Extent1].[Column1] + [Extent1].[Column2] IS NOT NULL)
THEN ([Extent1].[Column1] + [Extent1].[Column2]) / cast(2 as decimal(18))
ELSE cast(0 as decimal(18))
END AS [C2]
FROM [dbo].[Table] AS [Extent1]
Running the query in SSMS returns 10000.00000000000000000000 (20 decimal zeros). Apparently there is a problem when EF tries to convert the first value (with 25 decimal zeros) into a decimal but with the second (with 20 decimal zeros) it works.
In the meantime it turned out that the problem also occurs with non-nullable columns and even a single decimal(16, 6) column. The following ...
Table.Select(r => new Foo
{
Bar = r.Column1 / 2m
}
);
... throws the same conversion exception (with a value of 20000 in the Column1).
Why do those two SQL queries result in two different numbers of digits?
And why can't the first number be converted into a decimal by EF?