Have a simple task of pulling data from oracle table and push to sql server.
Created a Variable to store the query and giving this in the expression builder:
select col1,col2,col3,col4
from <schema>.table
WHERE closedate between to_date('" + (DT_WSTR,10) #[User::ReportStartDate]+"','YYYY-MM-DD') and to_date('" + (DT_WSTR,10) #[User::ReportEndDate] + "','YYYY-MM-DD')
However, when evaluating the expression, it fails with the message: "The expression might contain an invalid token, an incomplete token or an invalid element. It might not be well-formed or might be missing part of the required element such as paranthesis"
Been looking at it for quite some time now, but cannot find anything obvious.
What am I doing wrong here?
TIA,
Bee
Added starting and ending quotes so expression turned into
"select col1,col2,col3,col4
from <schema>.table
WHERE closedate between to_date('" + (DT_WSTR,10) #[User::ReportStartDate]+"','YYYY-MM-DD') and to_date('" + (DT_WSTR,10) #[User::ReportEndDate] + "','YYYY-MM-DD')"
and it evaluated without errors yielding valid string.
Related
In an attempt to discover the possibilities and limitations of the Flink Table API for use in a current project, I was trying to translate a Flink SQL statement into its equivalent Flink Table API version.
For most parts, I am able to translate the statement using the documentation except for the window function row_number().
Flink SQL (working)
final Table someTable = tableEnvironment.sqlQuery("SELECT" +
" T.COLUMN_A," +
" T.COLUMN_B," +
" T.COLUMN_C," +
" row_number() OVER (" +
" PARTITION BY" +
" T.COLUMN_A" +
" ORDER BY" +
" T.EVENT_TIME DESC" +
" ) AS ROW_NUM" +
" FROM SOME_TABLE T"
)
.where($("ROW_NUM").isEqual(1))
.select(
$("COLUMN_A"),
$("COLUMN_B"),
$("COLUMN_C")
);
The closest I get, is the code below, but I don't seem to find what should be placed at the location of the question marks (/* ??? */).
Flink Table API (not working)
final Table someTable = tableEnvironment.from("SOME_TABLE")
.window(Over.partitionBy($("COLUMN_A"))
.orderBy($("EVENT_TIME").desc())
.as($("window"))
)
.select(
$("COLUMN_A"),
$("COLUMN_B"),
$("COLUMN_C"),
/* ??? */.over($("window")).as("ROW_NUM")
)
.where($("ROW_NUM").isEqual(1));
On https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/#over-window-aggregation I find how it works for other window functions like avg(), min(), max()...; but the one(s) I require (row_number(), rank(), dense_rank()) are not (yet) described on this page.
My question is twofold:
Does an equivalent exist in the Flink Table API?
If so, what does it look like?
Additional information:
The Flink SQL variant works without issues (for this specific part).
I am experimenting with Flink 1.15.1.
Thank you in advance for you help!
The page where you can look this up is at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/functions/systemfunctions/. You will see that ROW_NUMBER, RANK, and DENSE_RANK have examples for SQL, but not for Table API.
In the end, it shouldn't matter though. As you've done, you can just use SQL directly in your Table API program.
SQL Server 2016
SSRS 2016
IDE Visual Studio 2017
Problem: Report Field contains value of Doe, John
Solution/Output: Using SSRS expression require field to output John Doe
Current sample of my expression that gives me an #error when I run preview:
=Split(Fields!Name.Value,",")(1).ToString() &","& Split(Fields!Name.Value,",")(0).ToString()
I found example above online, however throws an error. Relatively new to SSRS advanced expressions.
I don't have the option to edit the T-SQL query
This should work...
=SWITCH(
split(Fields!Name.Value, ",").length = 1, Fields!Name.Value,
True, TRIM(split(Fields!Name.Value + ",", ",")(1))
+ " "
+ TRIM(split(Fields!Name.Value + ",", ",")(0))
)
What I've done here is...
for the first SWITCH expression pair, check if the name has only a single element, if it does return the name (e.g. for "not Applicable". This method is safer as it will handle anything with no commas present.
The second part True just acts like an ELSE
In this case I've added a comma to the end of the Name field value so that the evaluated string always has at least 1 comma (thus preventing the error). I've also trimmed the results so you don't get unwanted spaces.
We now get these results
Npgsql docs suggest performing a full text search based on an expression index using ToTsVector
.Where(p => EF.Functions.ToTsVector("english", p.Title + " " + p.Description).Matches("Npgsql"))
As I understand expression indexes, they require that the query uses the same expression that was used to create the index, ie "Name" || ' ' || "Description".
However it seems to me that p.Title + " " + p.Description is evaluated before being translated to SQL as ToTsVector takes a plain string
public static NpgsqlTsVector ToTsVector(this DbFunctions _, string config, string document);
Am I wrong or will the index not be utilized? If I'm correct, is there a way to query correctly without using raw SQL?
First, you may want to look at the other method, i.e. setting up a TsVector column with HasGeneratedTsVectorColumn.
Regardless, p.Title + " " + p.Description definitely isn't evaluated before being translated to SQL - that can't happen assuming p refers to a database column. If you turn on SQL logging, you should see the exact SQL being generated by EF Core against your database. To be extra sure that the query uses your expression index, you can use EXPLAIN on that SQL and examine the query plan.
I have a query that takes as a parameter a list of dates. In VB.NET the dates are in a string ArrayList and I am using the String.Join() method to get them in a comma delimited list. The problem is when I do that double quotes are put at the start and end of the string and SQL complains about that (I think; see below). How can I get a list of date from a string ArrayList without the quote.
My arraylist contains these values:
'2020-08-30'
'2020-08-27'
'2020-09-28'
'2020-09-09'
'2020-08-31'
'2020-08-29'
when I join them using String.Join(",", sDates) I get the following:
"'2020-08-30','2020-08-27','2020-09-28','2020-09-09','2020-08-31','2020-08-29'"
and when I use that in a parameter query it gets rejected.
comm.Parameters.AddWithValue("#dates", String.Join(",", sDates))
sql contains the following"
...where pj.ProjectName =#projectname And tcd.Date in (#dates)
Exact error I get is
System.Data.SqlClient.SqlException
HResult=0x80131904
Message=Incorrect syntax near ','.
Source=.Net SqlClient Data Provider
Any advice?
This error message:
Incorrect syntax near ','
Is not caused by the way you've done your IN. It is caused by something else, such as a misplaced comma in a select block:
SELECT , a FROM x
^
The way you've done your IN won't work either, because it's conceptually the same as writing this:
SELECT * FROM table WHERE dateColumn IN ('''2000-01-01'',''2000-02-01''')
There is no such date with a string value of '2000-01-01','2000-02-01'.
If you want to use IN in the style you're attempting here, you have to add a parameter per date value and set it up accordingly:
sqlCommand.CommandText = "SELECT * FROM table WHERE dateCol IN("
Dim p = 0
For Each d as DateTime in MyDateList
sqlCommand.CommandText &= "#p" & i & "," 'concat param placeholder on
sqlCommand.Parameters.AddWithValue("#p" & i, d)
i += 1
Next d
sqlCommand.CommandText = sqlCommand.CommandText.TrimEnd(","c) & ")" 'get rid of trailing comma and close the IN brackets
This will generate an sql like
SELECT * FROM table WHERE dateCol IN (#p0,#p1,#p2)
with 3 parameters, and a populated parameters collection. You've already been pointed to Joel's blog about AddWithValue, so I won't repeat it.. But i did want to say that the way you've presented your question implies you have a list of strings, not datetimes. You should definitely make sure your list has DateTimes in, and your db column should be a date based type, not a string based type
I have several variables in an SSIS package that I would like inserting into a table.
example:-
#financialMonth, #Status, #Comments
The Variables have been populated along the way with values based on lookups, filename, dates, etc, and I want to store them in a results table.
Is using the execute SQL task the way to do this ?
Do I need to call a sproc and pass those variales as parameters ?
I've tried putting the following T-SQL into the SQLStatement property
INSERT INTO FilesProcessed
(ProcessedOn, ProviderCode, FinancialMonth,
FileName, Status, Comments)
SELECT GETDATE(), 'ABC' , 201006,
'ABC_201005_Testology.csv',
'Imported','Success'
I tried hardcoding the values above to get it to work
These are the columns on the table I'm inserting into
Column_name Type Computed Length
fileID int no 4
ProcessedOn datetime no 8
ProviderCode nchar no 6
FinancialMonth int no 4
FileName nvarchar no 510
Status nvarchar no 40
Comments nvarchar no 510
This is the Expression code that feeds the SQLStatementSource property
"INSERT INTO FilesProcessed (ProcessedOn, ProviderCode, FinancialMonth,
FileName, Status, Comments) SELECT GETDATE() AS ProcessedOn, '"
+ #[User::providerCode] + "' , "
+ (DT_STR,6,1252)#[User::financialMonth] + ", '"
+ #[User::fileName] + "', 'Imported' AS Status,'Successfully' AS Comments "
Unfortunately I'm missing something, and can't quite get it to work.
The Error message I'm getting is ...
Error: 0xC002F210 at Log entry in
FilesProcessed, Execute SQL Task:
Executing the query "INSERT INTO
FilesProcessed (ProcessedOn,
ProviderCode, FinancialMonth,
FileName, Status, Comments) SELECT
GETDATE(), 'ABC' , 201006,
'DAG_201005_Testology.csv',
'Imported','Successfully'" failed with
the following error: "An error
occurred while extracting the result
into a variable of type (DBTYPE_I2)".
Possible failure reasons: Problems
with the query, "ResultSet" property
not set correctly, parameters not set
correctly, or connection not
established correctly.
Please
a). Advise whether the Execute SQL Task is the way to do what I want to do.
b). Give me any pointers or pitfalls to look out for and check.
Thanks in advance.
OK, here is what I did.
I created an Execute SQL task and configured, thus :-
General Tab
ConnectionType = OLE DB
SQLSourceType = Direct Input
SQLStatement = (left blank)
BypassPrepare = True
ResultSet = None
Parameter Mapping
(none - leave blank)
Result Set
(none - leave blank)
Expressions
SQLStatementSource = "INSERT INTO FilesProcessed (ProcessedOn, ProviderCode, FinancialMonth, FileName, Status, Comments) SELECT GETDATE(), '" + #[User::providerCode] + "' , " + (DT_STR,6,1252)#[User::financialMonth] + ", '" + #[User::fileName] + "', 'Import - Success', '" + #[User::fileComments] + "'"
Then as long as I set up the variables and populate them in the variables window (the Expression editor will not let you save an expression that references a variable that does not exist. Keep notepad handy to store the contents while you go back and edit the variables window, and add new variables in ;)
Build the expression slowly, using the Parse expression button regularly to check.
make sure that the data types of the VALUES match the destination column data types.
see: http://social.msdn.microsoft.com/forums/en-US/sqlintegrationservices/thread/e8f82288-b980-40a7-83a6-914e217f247d/
A couple of speculative suggestions
The Error message says An error occurred while extracting the result into a variable of type (DBTYPE_I2). But this is a straight insert statement. There shouldn't be a result except for rows affected. Do you have any parameter mappings erroneously set to Output?
What if you try and run the SQL Query from the error message directly in management studio? Does that give you an error?
In the above table definition FinancialMonth as int datatype as
FinancialMonth int no 4
while inseting casting as :
(DT_STR,6,1252)#[User::financialMonth]
I think it's purely a datatype mismatch with the target table definition.