Npgsql docs suggest performing a full text search based on an expression index using ToTsVector
.Where(p => EF.Functions.ToTsVector("english", p.Title + " " + p.Description).Matches("Npgsql"))
As I understand expression indexes, they require that the query uses the same expression that was used to create the index, ie "Name" || ' ' || "Description".
However it seems to me that p.Title + " " + p.Description is evaluated before being translated to SQL as ToTsVector takes a plain string
public static NpgsqlTsVector ToTsVector(this DbFunctions _, string config, string document);
Am I wrong or will the index not be utilized? If I'm correct, is there a way to query correctly without using raw SQL?
First, you may want to look at the other method, i.e. setting up a TsVector column with HasGeneratedTsVectorColumn.
Regardless, p.Title + " " + p.Description definitely isn't evaluated before being translated to SQL - that can't happen assuming p refers to a database column. If you turn on SQL logging, you should see the exact SQL being generated by EF Core against your database. To be extra sure that the query uses your expression index, you can use EXPLAIN on that SQL and examine the query plan.
Related
In an attempt to discover the possibilities and limitations of the Flink Table API for use in a current project, I was trying to translate a Flink SQL statement into its equivalent Flink Table API version.
For most parts, I am able to translate the statement using the documentation except for the window function row_number().
Flink SQL (working)
final Table someTable = tableEnvironment.sqlQuery("SELECT" +
" T.COLUMN_A," +
" T.COLUMN_B," +
" T.COLUMN_C," +
" row_number() OVER (" +
" PARTITION BY" +
" T.COLUMN_A" +
" ORDER BY" +
" T.EVENT_TIME DESC" +
" ) AS ROW_NUM" +
" FROM SOME_TABLE T"
)
.where($("ROW_NUM").isEqual(1))
.select(
$("COLUMN_A"),
$("COLUMN_B"),
$("COLUMN_C")
);
The closest I get, is the code below, but I don't seem to find what should be placed at the location of the question marks (/* ??? */).
Flink Table API (not working)
final Table someTable = tableEnvironment.from("SOME_TABLE")
.window(Over.partitionBy($("COLUMN_A"))
.orderBy($("EVENT_TIME").desc())
.as($("window"))
)
.select(
$("COLUMN_A"),
$("COLUMN_B"),
$("COLUMN_C"),
/* ??? */.over($("window")).as("ROW_NUM")
)
.where($("ROW_NUM").isEqual(1));
On https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/#over-window-aggregation I find how it works for other window functions like avg(), min(), max()...; but the one(s) I require (row_number(), rank(), dense_rank()) are not (yet) described on this page.
My question is twofold:
Does an equivalent exist in the Flink Table API?
If so, what does it look like?
Additional information:
The Flink SQL variant works without issues (for this specific part).
I am experimenting with Flink 1.15.1.
Thank you in advance for you help!
The page where you can look this up is at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/functions/systemfunctions/. You will see that ROW_NUMBER, RANK, and DENSE_RANK have examples for SQL, but not for Table API.
In the end, it shouldn't matter though. As you've done, you can just use SQL directly in your Table API program.
SQL Server 2016
SSRS 2016
IDE Visual Studio 2017
Problem: Report Field contains value of Doe, John
Solution/Output: Using SSRS expression require field to output John Doe
Current sample of my expression that gives me an #error when I run preview:
=Split(Fields!Name.Value,",")(1).ToString() &","& Split(Fields!Name.Value,",")(0).ToString()
I found example above online, however throws an error. Relatively new to SSRS advanced expressions.
I don't have the option to edit the T-SQL query
This should work...
=SWITCH(
split(Fields!Name.Value, ",").length = 1, Fields!Name.Value,
True, TRIM(split(Fields!Name.Value + ",", ",")(1))
+ " "
+ TRIM(split(Fields!Name.Value + ",", ",")(0))
)
What I've done here is...
for the first SWITCH expression pair, check if the name has only a single element, if it does return the name (e.g. for "not Applicable". This method is safer as it will handle anything with no commas present.
The second part True just acts like an ELSE
In this case I've added a comma to the end of the Name field value so that the evaluated string always has at least 1 comma (thus preventing the error). I've also trimmed the results so you don't get unwanted spaces.
We now get these results
Have a simple task of pulling data from oracle table and push to sql server.
Created a Variable to store the query and giving this in the expression builder:
select col1,col2,col3,col4
from <schema>.table
WHERE closedate between to_date('" + (DT_WSTR,10) #[User::ReportStartDate]+"','YYYY-MM-DD') and to_date('" + (DT_WSTR,10) #[User::ReportEndDate] + "','YYYY-MM-DD')
However, when evaluating the expression, it fails with the message: "The expression might contain an invalid token, an incomplete token or an invalid element. It might not be well-formed or might be missing part of the required element such as paranthesis"
Been looking at it for quite some time now, but cannot find anything obvious.
What am I doing wrong here?
TIA,
Bee
Added starting and ending quotes so expression turned into
"select col1,col2,col3,col4
from <schema>.table
WHERE closedate between to_date('" + (DT_WSTR,10) #[User::ReportStartDate]+"','YYYY-MM-DD') and to_date('" + (DT_WSTR,10) #[User::ReportEndDate] + "','YYYY-MM-DD')"
and it evaluated without errors yielding valid string.
I have a database table in SQL Server 2014 with only an ID column (int) and a column xmldata of type XML.
This xmldata column contains for example:
<book>
<title>a nice Novel</title>
<author>Maria</author>
<author>Peter</author>
</book>
As expected, I have multiple books, therefore multiple rows with xmldata.
I now want to execute a query for all books, where Peter is an Author. I tried this in some xPath2.0 testers and got to the conclusion that:
/book/author/concat(text(), if(position() != last())then ',' else '')
works.
If you try to port this success into SQL Server 2014 Express it looks like this, which is correctly escaped syntax etc.:
SELECT id
FROM books
WHERE 'Peter' IN (xmldata.query('/book/author/concat(text(), if(position() != last())then '','' else '''')'))
SQL Server however does not seem to support a construction like /concat(...) because of:
The XQuery syntax '/function()' is not supported.
I am at a loss then however, why /text() would work in:
SELECT id, xmldata.query('/book/author/text()')
FROM books
which it does.
My constraints:
I am bound to use SQL Server
I am bound to xpath or something else that can be "injected" as the statement above (if the structure of the xml or the database changes, the xpath above could be changed isolated and the application logic above that constructs the Where clause will not be touched) SEE EDIT
Is there a way to make this work?
regards,
BillDoor
EDIT:
My second constraint boils down to this:
An Application constructs the Where clause by
expression <operator> value(s)
expression is stored in a database and is mapped by the xmlTag eg.:
| tokenname| querystring
| "author" | "xmldata.query(/book/author/text())"
the values are presented by the Requesting user. so if the user asks for the author "Peter" with operator "EQUALS" the application constructs:
xmaldata.query(/book/author/text()) = "Peter"
as where clause.
If the customer now decides that author needs to be nested in an <authors> element, i can simply change the expression in the construction-database and the whole machine keeps running without any changes to code, simply manageable.
So i need a way to achieve that
<xPath> <operator> "Peter"
or any other combination of this three isolated components (see above: "Peter" IN <xPath>...) gets me all of Peters' books, even if there are multiple unsorted authors.
This would not suffice either (its not sqlserver syntax, but you get the idea):
WHERE xmldata.exist('/dossier/client[text() = "$1"]', "Peter") = 1;
because the operator is still nested in the expression, i could not request <> "Peter".
I know this is strange, please don't question the concept as a whole - it has a history :/
EDIT: further clarification:
The filter-rules come into the app in an XML structure basically:
Operator: "EQ"
field: "name"
value "Peter"
evaluates to:
expression = lookupExpressionForField("name") --> "table2.xmldata.value('book/author/name[1]', 'varchar')"
operator = lookUpOperatorMapping("EQ") --> "="
value = FormatValues("Peter") --> "Peter" (if multiple values are passed FormatValues cosntructs a comma seperated list)
the application then builds:
- constructClause(String expression,String operator,String value)
"table2.xmldata.value('book/author/name[1]', 'varchar')" + "=" + "Peter"
then constructs a Select statement with the result as WHERE clause.
it does not build it like this, unescaped, unfiltered for injection etc, but this is the basic idea.
i can influence how the input is Transalted, meaning I can implement the methods:
lookupExpressionForField(String field)
lookUpOperatorMapping(String operator)
Formatvalues(List<String> values) | Formatvalues(String value)
constructClause(String expression,String operator,String value)
however i choose to do, i can change the parameter types, I can freely implement them. The less the better of course. So simply constructing a comma-seperated list with xPath would be optimal (like if i could somewhere just tick "enable /function()-syntax in xPath" in sqlserver and the /concat(if...) would work)
How about something like this:
SET NOCOUNT ON;
DECLARE #Books TABLE (ID INT NOT NULL IDENTITY(1, 1) PRIMARY KEY, BookInfo XML);
INSERT INTO #Books (BookInfo)
VALUES (N'<book>
<title>a nice Novel</title>
<author>Maria</author>
<author>Peter</author>
</book>');
INSERT INTO #Books (BookInfo)
VALUES (N'<book>
<title>another one</title>
<author>Bob</author>
</book>');
SELECT *
FROM #Books bk
WHERE bk.BookInfo.exist('/book/author[text() = "Peter"]') = 1;
This returns only the first "book" entry. From there you can extract any portion of the XML field using the "value" function.
The "exist" function returns a boolean / BIT. This will scan through all "author" nodes within "book", so there is no need to concat into a comma-separated list only for use in an IN list, which wouldn't work anyway ;-).
For more info on the "value" and "exist" functions, as well as the other functions for use with XML data, please see:
xml Data Type Methods
print("select CustomerNo, CustomerName, Address, City, State, Zip,
Phone, Fax, ContactName, Email
from Customers where CustomerName like '%field%'");
Hi all. This is a simple question but I wasn't able to figure since I'm pretty new to tsql and sql in general.
I use the above stored procedure to do search. My question is for '%field%'. What variable do you use or how does it work in tsql? for example, "where Customers = #CustomerNo". how about for wildcard? how do you pass in a variable along with wildcard? I guess i can do "%" + "field" + "%" in the code but is there a way not to do that?
Wildcards are simply part of a string literal, e.g. '%field%' is just a string.
You can concatenate the wildcards onto your string and then use the string:
#Pattern = '%' + #CustomerName + '%';
...WHERE CustomerName LIKE #Pattern
Or else you can write an expression in the SQL involving concatenation:
WHERE CustomerName LIKE '%' + #CustomerName + '%'
There's no other magic solution for this.
It is very simple. "=" and "Like" are both operators. What you can do after one you can do after the other.
So, if in C# and using SQLClient calls, you can say:
string value;
...
value = "Some name";
...
myCommand.CommandText = "Select...from Customers Where CustomerName Like #Var";
myCommand.Parameters.AddWithValue("#Var", "%" + value + "%");
myCommand.ExecuteNonQuery();
If you use =, you say "equal", which won't use wildcards.
If you use LIKE, which only work on text fields, it can use wildcards.
There is no way you can get wildcard matches with =.
Note that depending on the data, a wildcard search might do a table-scan, so I would make sure that's what you want before you allow it.
For instance, this will do a table-scan:
WHERE CustomerID LIKE '%1'
ie. all customers who have a customer identifier (which is text) that ends with a 1. That can't be solved with an index.
Final thoughts. I'm not 100% sure I understand exactly what you're asking. Could you please clarify. What specifically do you mean by "pass in a variable along with wildcard"?