SQL Server full text search on column with alias - sql-server

I am using full-text search, its working fine on a direct column of table but not on a derived/aliased column.
SELECT ExpectationId
,ExpectationName
,(
CASE
WHEN ExpectationOrganization_OrganizationId IS NOT NULL
THEN (
SELECT OrganizationName
FROM Organizations
WHERE OrganizationId = ExpectationOrganization_OrganizationId
)
WHEN ExpectationBeneficiary_BeneficiaryId IS NOT NULL
THEN (
SELECT BeneficiaryName
FROM Beneficiaries
WHERE BeneficiaryId = ExpectationBeneficiary_BeneficiaryId
)
ELSE (
SELECT TeamName
FROM Teams
WHERE TeamId = ExpectationTeam_TeamId
)
END
) AS ParentName
FROM Expectations
WHERE
FREETEXT(ExpectationName, #Keyword) ---Working
OR FREETEXT(ParentName, #Keyword) ---Not working
All these columns ExpectationName, OrganizationName, BeneficiaryName, TeamName are full-text indexed.
How can I make it work for ParentName column?

You need to create a VIEW based on your query first then add a Full-Text index to it which will include the ParentName column. Without the Full-Text index over a column being searched neither FREETEXT nor CONTAINS will work.
Something like that should help you:
CREATE VIEW ExpectationsView AS
SELECT ExpectationId
,ExpectationName
,(
CASE
WHEN ExpectationOrganization_OrganizationId IS NOT NULL
THEN (
SELECT OrganizationName
FROM Organizations
WHERE OrganizationId = ExpectationOrganization_OrganizationId
)
WHEN ExpectationBeneficiary_BeneficiaryId IS NOT NULL
THEN (
SELECT BeneficiaryName
FROM Beneficiaries
WHERE BeneficiaryId = ExpectationBeneficiary_BeneficiaryId
)
ELSE (
SELECT TeamName
FROM Teams
WHERE TeamId = ExpectationTeam_TeamId
)
END
) AS ParentName
FROM Expectations
GO
-- This index is needed for FTS index.
-- Note, I trust ExpectationId column is unique in your SELECT above,
-- if it's not, the below CREATE INDEX will fail and you will need to provide
-- a new column to your VIEW which will uniquely identify each row, then use
-- that PK-like column in the below index
CREATE UNIQUE CLUSTERED INDEX PK_ExpectationsView
ON ExpectationsView (ExpectationId);
GO
CREATE FULLTEXT CATALOG fts_catalog;
GO
CREATE FULLTEXT INDEX ON ExpectationsView
(
ExpectationName Language 1033,
ParentName Language 1033
)
KEY INDEX PK_ExpectationsView
ON fts_catalog;
WITH (CHANGE_TRACKING = AUTO)
GO
Once the Full-Text index which includes the relevant columns is there you can use FREETEXT or CONTAINS in queries:
SELECT ExpectationId, ExpectationName, ParentName FROM ExpectationsView
WHERE FREETEXT(ExpectationName, #Keyword) OR FREETEXT(ParentName, #Keyword)
Note, the above code I've provided off the top of my head because I don't have data schema for your case so couldn't try running it. However, it should give you the general idea on how to proceed. HTH.

Related

How to create high performance SQL select query, if I need a condition for referrer table's records?

For example, I have 2 tables, which I need for my query, Property and Move for history of moving properties.
I must create a query which will return all properties + 1 additional boolean column, IsInService, which will have value true, in cases, when Move table has a record for property with DateTo = null and MoveTypeID = 1 ("In service").
I have created this query:
SELECT
[ID], [Name],
(SELECT COUNT(*)
FROM [Move]
WHERE PropertyID = p.ID
AND DateTo IS NULL
AND MoveTypeID = 1) AS IsInService
FROM
[Property] as p
ORDER BY
[Name] ASC
OFFSET 100500 ROWS FETCH NEXT 50 ROWS ONLY;
I'm not so strong in SQL, but as I know, subqueries are the evil :)
How to create high performance SQL query in my case, if it is expected that these tables will include millions of records?
I've updated the code based on your comment. If you need something else, please provide input and output data expected. This is about all I can do based on inference from the existing comments. Further, this isn't intended to give you an exact working solution. My intention was to give you a prototype from which you can build your solution.
That said:
The code below is the basic join that you need. However, keep in mind that indexing is probably going to play as big a part in performance as the structure of the table and the query. It doesn't matter how you query the tables if the indexes aren't there to support the queries once you reach a certain size. There are a lot of resources online for indexing but viewing querying plans should be at the top of your list.
As a note, your column [dbo].[Property] ([Name]) should probably be NVARCHAR to allow SQL to minimize data storage. Indexes on that column will then be smaller and searches/updates faster.
DECLARE #Property AS TABLE
(
[ID] INT
, [Name] NVARCHAR(100)
);
INSERT INTO #Property
([ID]
, [Name])
VALUES (1,N'A'),
(2,N'B'),
(3,N'C');
DECLARE #Move AS TABLE
(
[ID] INT
, [DateTo] DATE
, [MoveTypeID] INT
, [PropertyID] INT
);
INSERT INTO #Move
([ID]
, [DateTo]
, [MoveTypeID]
, [PropertyID])
VALUES (1,NULL,1,1),
(2,NULL,1,2),
(3,N'2017-12-07',1,2);
SELECT [Property].[ID] AS [property_id]
, [Property].[Name] AS [property_name]
, CASE
WHEN [Move].[DateTo] IS NULL
AND [Move].[MoveTypeID] = 1 THEN
N'true'
ELSE
N'false'
END AS [in_service]
FROM #Property AS [Property]
LEFT JOIN #Move AS [Move]
ON [Move].[PropertyID] = [Property].[ID]
WHERE [Move].[DateTo] IS NULL
AND [Move].[MoveTypeID] = 1;

Missing index (Impact 97) : Create Non Clustered Index

I am trying to optimize my stored procedure. When I look at the query plan, I can see tablescan on tempcompany is showing 97 percent. I am also seeing the following message Missing index (Impact 97) : Create Non Clustered Index on #tempCompany
I have already set non clustered indexes. Could somebody point out what the problem is
if object_id('tempdb..#tempCompany') is not null drop table #tempCompany else
select
fp.companyId,fp.fiscalYear,fp.fiscalQuarter,fi.financialperiodid, fi.periodEndDate,
fc.currencyId,fp.periodtypeid,ROW_NUMBER() OVER (PARTITION BY fp.companyId,
fp.fiscalYear, fp.fiscalQuarter ORDER BY fi.periodEndDate DESC) rowno
into #tempCompany
from
ciqFinPeriod fp
inner join #companyId c on c.val = fp.companyId
join ciqFinInstance fi on fi.financialperiodid = fp.financialperiodid
join ciqFinInstanceToCollection ic on ic.financialInstanceId = fi.financialInstanceId
left join ciqFinCollection fc on fc.financialCollectionId = ic.financialCollectionId
left join ciqFinCollectionData fd on fd.financialCollectionId = fc.financialCollectionId
where
fp.periodTypeId = #periodtypeId
and fi.periodenddate >= #date
--and fp.companyId in (select val from #companyId)
CREATE NONCLUSTERED INDEX id_companyId2 on #tempCompany(companyId,fiscalYear,fiscalQuarter,financialperiodid,periodEndDate,currencyId,periodtypeid,rowno)
if object_id('tempdb..#EstPeriodTbl') is not null drop table #EstPeriodTbl else
select
companyId,fiscalYear,fiscalQuarter,financialPeriodId,periodenddate,currencyId,
periodtypeid,rowno
into #EstPeriodTbl
from #tempCompany a
where a.rowno = 1
order by companyid, periodenddate
CREATE NONCLUSTERED INDEX id_companyId3 on #EstPeriodTbl(companyId,periodenddate,fiscalYear,fiscalQuarter,currencyId,financialPeriodId,rowno)
Execution Plan
You do not need to include everything in the #tempCompany index; just rowno:
CREATE NONCLUSTERED INDEX id_companyId2 on #tempCompany(rowno)
Short answer: The index you provided, does not help SQL Server in the query you are doing. If you create another non-clustered index, and have rowno as the first column in the index, Sql Server will probably be able to use that index.
Long explination:
The reason you have a problem, is because the index you have created isn't useful to SQL Server with this specific query. The order that records are sorted in on an index is determined by the order you specify them when creating an index.
(e.g. Your index orders your records by companyId first, and then orders the records with the same companyId by their fiscalYear, and then by their fiscalQuarter.)
Trying to use the provided index to find an item by just it's rowno value, would be like you trying to find the entries in the phone book based off of someone's phone number. The only way to locate all of the matching records is to search through every record in the book (i.e. a table scan).
In general, you can utalize nonclustered indexs only when the information you use in your where clause matches the first column in your index, (i.e. if you can provide a SARGable predicate for companyId in your where clause, you could probably use this index)
Using the phone book again: If I gave you a last name and a phone number, now you no longer need to do a full table scan on the phone book, you can do an index scan for the last name. Which would be more efficient. And if you were able to give a last name, and first name, and then middle initial and phone number, you could do an even more efficient table scan. But if I only provide you with a last name, middle initial and phone number; now I am back to scaning the index on just the value of last name.
So if you can narrow down your record set to use at least companyId (i.e. use companyID in your where clause) you can use the index you have provided.
Or, and I imagine this is what you will want to do, create an index that sorts by rowno, then companyId and periodendDate.
e.g.
CREATE NONCLUSTERED INDEX idx_temp_rowno ON #tempCompany(rowno, companyId, periodenddate)

Database Index when SQL statement includes "IN" clause

I have SQL statement which takes really a lot of time to execute and I really had to improve it somehow.
select * from table where ID=1 and GROUP in
(select group from groupteam where
department= 'marketing' )
My question is if I should create index on columns ID and GROUP would it help?
Or if not should I create index on second table on column DEPARTMENT?
Or I should create two indexes for both tables?
First table has 249003.
Second table has in total 900 rows while query in that table returns only 2 rows.
That is why I am surprised that response is so slow.
Thank you
You can also use EXISTS, depending on your database like so:
select * from table t
where id = 1
and exists (
select 1 from groupteam
where department = 'marketing'
and group = t.group
)
Create a composite index on individual indexes on groupteam's department and group
Create a composite index or individual indexes on table's id and group
Do an explain/analyze depending on your database to review how indexes are being used by your database engine.
Try a join instead:
select * from table t
JOIN groupteam gt
ON d.group = gt.group
where ID=1 AND gt.department= 'marketing'
Index on table group and id column and table groupteam group column would help too.

SQL Server : find index of string

User table
UserId UserName
55 FirstUser
22 SecondUser
5 ThirdUser
33 FourthUser
I have a username "SecondUser".
I want to match "SecondUser" name with the UserName column, after that I want to get it's index.
How can I find the index of SecondUser in SQL Server?
Any help will be appreciated with points.
Thanks.
Your question makes no sense .... what do you mean by index ??? You can fetch the UserId associated with your user name. But what do you mean by index ? What would the index of SecondUser be in your case??
Update: OK - so you want to have some ordering. But you want your data ordered by what criteria ?? You need some kind of a column - an ID, a DateTime column - SOMETHING that you can use in an ORDER BY clause.
SQL Server doesn't have any implicit ordering per se; if you want to have any ordering (and thus an "index" in that order), you must provide some kind of column(s) to order by.
If you do have an ID column - you could do something like this:
;WITH BaseData AS
(
SELECT
[UserId],
[Index] = ROW_NUMBER() OVER(ORDER BY ID),
UserName
FROM dbo.[User]
)
SELECT *
FROM BaseData
WHERE UserName = 'SecondUser'
Here is the SQLFiddel Demo
Below is the Sample Query :
select *,
row_number() over (order by (select 0)) as 'Index'
from [user]
Note : This answer is applicable only if there is no Clustered Index or Primary Key in [user] table. If there is Primary Key or Clustered Index in your table on any column then while querying Select * from [user] will give the output with Sorted rows on the primary key column or a column on which clustered index is formed.

Searchable text is in 2 tables, how to design full text index?

In a forum application, the actual name of the thread is stored in a table, and then replies is stored in another table.
Table_Thread
Subject varchar(255) e.g. "How to setup fulltext search"
Table_Replies (users replies here)
ReplyText text(not null)
Now I want to create a full-text search on both the subject and reply columns, but they seem very related so they should be in the same index.
Is it possible to do this?
I'm using sql server 2005.
Assuming there is an association between the subject and the replies you could create a view WITH SCHEMABINDING, create a UNIQUE CLUSTERED index on the view and then add that view to your fulltext catalog selecting the two columns you want included.
When huge concurrent query requests come, RDBMS cannot afford it by SQL. what's more, select SQL supports full-text search badly. So you need IR (Information Retrieval) library such as Lucene for java.
You could create a indexed view containing an union of both indexed columns + PK of the tables
e.g.
CREATE VIEW SearchText
WITH SCHEMABINDING
AS SELECT * FROM (
(Subject as Text, Table_Thread_ID as ID, 1 as Type FROM Table_Thread)
UNION ALL
(ReplyText as Text, Table_Replies_ID as ID, 2 as Type FROM Table_Replies));
I put type 1 and 2 as arbitrary, since you need a unique key to build a fulltext index.
And then create a unique index on (ID, Type), and finally your fulltext index.
CREATE UNIQUE INDEX SearchText_UK ON SearchText (ID, Type);
CREATE FULLTEXT CATALOG ft AS DEFAULT;
CREATE FULLTEXT INDEX ON SearchText(Text)
KEY INDEX SearchText_UK
WITH STOPLIST = SYSTEM;
I have seen what NopCommerce (C# MVC Open Source E-Commerce) has done using fulltext search on 'products' and 'variants' and only return 'products'. This is very similar to your case because you want to search on 'Thread' and 'Replies' but you obviously want to only return 'threads'. I have change it to use threads and replies for you:
First, create a function that generates an index name by table (optional):
CREATE FUNCTION [dbo].[nop_getprimarykey_indexname]
(
#table_name nvarchar(1000) = null
)
RETURNS nvarchar(1000)
AS
BEGIN
DECLARE #index_name nvarchar(1000)
SELECT #index_name = i.name
FROM sys.tables AS tbl
INNER JOIN sys.indexes AS i ON (i.index_id > 0 and i.is_hypothetical = 0) AND (i.object_id=tbl.object_id)
WHERE (i.is_unique=1 and i.is_disabled=0) and (tbl.name=#table_name)
RETURN #index_name
END
GO
Then, enable fulltext by creating the catalog and the indexes:
EXEC('
IF NOT EXISTS (SELECT 1 FROM sys.fulltext_catalogs WHERE [name] = ''myFullTextCatalog'')
CREATE FULLTEXT CATALOG [myFullTextCatalog] AS DEFAULT')
DECLARE #create_index_text nvarchar(4000)
SET #create_index_text = '
IF NOT EXISTS (SELECT 1 FROM sys.fulltext_indexes WHERE object_id = object_id(''[Table_Thread]''))
CREATE FULLTEXT INDEX ON [Table_Thread]([Subject])
KEY INDEX [' + dbo.[nop_getprimarykey_indexname] ('Table_Thread') + '] ON [myFullTextCatalog] WITH CHANGE_TRACKING AUTO'
EXEC(#create_index_text)
SET #create_index_text = '
IF NOT EXISTS (SELECT 1 FROM sys.fulltext_indexes WHERE object_id = object_id(''[Table_Replies]''))
CREATE FULLTEXT INDEX ON [Table_Replies]([ReplyText])
KEY INDEX [' + dbo.[nop_getprimarykey_indexname] ('Table_Replies') + '] ON [myFullTextCatalog] WITH CHANGE_TRACKING AUTO'
EXEC(#create_index_text)
Then, in the stored procedure to obtain products by keywords, build a temporary table with a list of product Ids that match the keywords.
INSERT INTO #KeywordThreads ([ThreadId])
SELECT t.Id
FROM Table_Thread t with (NOLOCK)
WHERE CONTAINS(t.[Subject], #Keywords)
UNION
SELECT r.ThreadId
FROM Table_Replies r with (NOLOCK)
WHERE CONTAINS(pv.[ReplyText], #Keywords)
Now you can use the temporary table #KeywordThreads to join with the list of threads and return them.
I hope this helps.

Resources