I have five related variables that I need to examine from three different external tables in the same database which will require two inner joins (I think). I can't seem to put the external file ref in the right spot to make the syntax work.
Almost every permutation you can think of, and all have syntax errors. The sample below is the easiest to conceptually read to describe what I'm trying to do, but it doesn't work.
SELECT
txntable1.AccountRef_FullName AS [GL Name],
txntable1.Amount AS [Bill Amount],
billtable1.Amount AS [Paid Amount],
billtable1.PayeeEntityRef_FullName AS Vendor,
billtable1.TxnDate AS [Check Date]
FROM
billtable1, linkedtable1, txntable1
IN
'c:\Users\database.mdb'
INNER JOIN
billtable1
ON
billtable1.TxnID = linkedtable1.TxnID
AND
INNER JOIN
linkedtable1
ON
linkedtable1.IDKEY = txntable1.IDKEY
Syntax error in FROM clause, even if "in" is moved to after each external table.
Related
I am trying to find a solution in order to improve the String searching process and I selected FULL-TEXT INDEX Strategy.
However, after implementing it, I still can see there is a performance hit when it comes to search by using multiple strings using multiple Full-Text Index tables with OR clauses.
(E.x. WHERE CONTAINS(F.*,'%Gayan%') OR CONTAINS(P.FirstName,'%John%'))
As a solution, I am trying to use CONTAINSTABLE expecting a performance improvement.
Now, I am facing an issue with CONTAINSTABLE when it comes to joining tables with a LEFT JOIN
Please go through the example below.
Query 1
SELECT F.Name,p.*
FROM P.Role PR
INNER JOIN P.Building F ON PR.PID = F.PID
LEFT JOIN CONTAINSTABLE(P.Building,*,'%John%') AS FFTIndex ON F.ID = FFTIndex.[Key]
LEFT JOIN P.Relationship PRSHIP ON PR.id = prship.ToRoleID
LEFT JOIN P.Role PR2 ON PRSHIP.ToRoleID = PR2.ID
LEFT JOIN P.Person p ON pr2.ID = p.PID
LEFT JOIN CONTAINSTABLE(P.Person,FirstName,'%John%') AS PFTIndex ON P.ID = PFTIndex.[Key]
WHERE F.Name IS NOT NULL
This produces the below result.
Query 2
SELECT F.Name,p.*
FROM P.Role PR
INNER JOIN P.Building F ON PR.PID = F.PID
INNER JOIN P.Relationship PRSHIP ON PR.id = prship.ToRoleID
INNER JOIN P.Role PR2 ON PRSHIP.ToRoleID = PR2.ID
INNER JOIN P.Person p ON pr2.ID = p.PID
WHERE CONTAINS(F.*,'%Gayan%') OR CONTAINS(P.FirstName,'%John%')
AND F.Name IS NOT NULL
Result
Expectation
To use query 1 in a way that works as the behavior of an SQL SERVER OR clause. As I can understand Query 1's CONTAINSTABLE, joins the data with the building table, and the rest of the results are going to ignore so that the CONTAINSTABLE of the Person table gets data that already contains the keyword filtered from the building table.
If the keyword = Building, I want to match the keyword in both the tables regardless of searching a saved record in both the tables. Having a record in each table is enough.
Summary
Query 2 performs well but is creates a slowness when the words in the indexes are growing. Query 1 seems optimized(When it comes to multiple online resources and MS Documentation),
however, it does not give me the expected output.
Is there any way to solve this problem?
I am not strictly attached to CONTAINSTABLE. Suggesting another optimization method will also be considerable.
Thank you.
Hard to say definitively without your full data set but a couple of options to explore
Remove Invalid % Wildcards
Why are you using '%SearchTerm%'? Does performance improve if you use the search term without the wildcards (%)? If you want a word that matches a prefix, try something like
WHERE CONTAINS (String,'"SearchTerm*"')
Try Temp Tables
My guess is CONTAINS is slightly faster than CONTAINSTABLE as it doesn't calculate a rank, but I don't know if anyone has ever attempted to benchmark it. Either way, I'd try saving off the matches to a temp table before joining up to the rest of the tables. This will allow the optimizer to create a better execution plan
SELECT ID INTO #Temp
FROM YourTable
WHERE CONTAINS (String,'"SearchTerm"')
SELECT *
FROM #Temp
INNER JOIN...
Optimize Full Text Index by Removing Noisy Words
You might find you have some noisy words aka words that reoccur many times in your data that are meaningless like "the" or perhaps some business jargon. Adding these to your stop list will mean your full text index will ignore them, making your index smaller thus faster
The query below will list indexed words with the most frequent at the top
Select *
From sys.dm_fts_index_keywords(Db_Id(),Object_Id('dbo.YourTable') /*Replace with your table name*/)
Order By document_count Desc
This OR That Criteria
For your WHERE CONTAINS(F.*,'%Gayan%') OR CONTAINS(P.FirstName,'%John%') criteria where you want this or that, is tricky. OR clauses generally perform even when using simple equality operators.
I'd try either doing two queries and union the results like:
SELECT * FROM Table1 F
/*Other joins and stuff*/
WHERE CONTAINS(F.*,'%Gayan%')
UNION
SELECT * FROM Table2 P
/*Other joins and stuff*/
WHERE CONTAINS(P.FirstName,'%John%')
OR this is much more work, but you could load all your data into giant denormalized table with all your columns. Then apply a full text index to that table and adjust your search criteria that way. It'd probably be the fastest method searching, but then you'd have to ensure the data is sync between the denormalized table and the underlying normalized tables
SELECT B.*,P.* INTO DenormalizedTable
FROM Building AS B
INNER JOIN People AS P
CREATE FULL TEXT INDEX ft ON DenormalizedTable
etc...
I'm getting the following error when running a query against my local database :
The query processor could not produce a query plan because a worktable is required, and its minimum row size exceeds the maximum allowable of 8060 bytes. A typical reason why a worktable is required is a GROUP BY or ORDER BY clause in the query. If the query has a GROUP BY or ORDER BY clause, consider reducing the number and/or size of the fields in the clause. Consider using prefix (LEFT()) or hash (CHECKSUM()) of fields for grouping or prefix for ordering. Note however that this will change the behavior of the query.
This occurs when running a query that looks similar to this (apolgies for the lack of detail) :
SELECT <about 183 columns>
FROM tableA
INNER JOIN tvf1(<params>) tvf1
ON tvf1.id = tableA.X1
INNER JOIN tvf2(<params>) tvf2
ON tvf2.id = tableA.X2
INNER JOIN tvf3(<params>) tvf3
ON tvf3.id = tableA.X3
INNER JOIN tvf4(<params>) tvf4
ON tvf4.id = tableA.X4
INNER JOIN tvf5(<params>) tvf5
ON tvf5.id = tableA.X5
The table-valued-functions above all use a combination of GROUP BY, ROW_NUMBER() and other aggregation functions.
While binary debugging, commenting out any 2 of the above joins results in the error not occurring, doesnt matter which though.
My database is running on Compatibility Level 2019.
If i try setting Legacy Cardinality Estimation to On then the error no longer happens but I dont understand what this setting does.
edit :
If the database compatibility level is 2016 then everything works as expected as well
A concern i have is that the production database might be upgraded in future and this error could occur.
Edit :
I've managed to get the column count down to a handful now however my results are inconsistent.
SELECT
Other = TvfGroupData.Other
,GroupA = TvfGroupData.GroupA
,GroupB = TvfGroupData.GroupB
,GroupC = TvfGroupData.GroupC
, [Max Created Date] =
(SELECT MAX(Value)
FROM (VALUES
(Tvf1.CreatedDate)
,(Tvf2.CreatedDate)
,(Tvf3.CreatedDate)
--,(TvfGroupData.CreatedDate)
,(Tvf3.CreatedDate)
,(Tvf4.CreatedDate)
) AS AllValues(Value)
)
FROM TableA
LEFT JOIN Tvf1() ...
LEFT JOIN Tvf2() ...
LEFT JOIN TvfGroupData() ...
LEFT JOIN Tvf3() ...
LEFT JOIN Tvf4() ...
In the above query the following scenarios work :
excluding only GroupA column.
excluding only GroupB, GroupC column
Other combinations all fail with the error :
The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information.
The cardinality estimator is how SQL Server generates the execution plan, meaning how the engine will execute and assemble the disparate sets of data.
The recent changes in the engine usually (but not always) result in better execution plans, leading to faster queries response while consuming less resources.
If you look into how SQL Server processes a query statement, the full data set is gathered before the unwanted columns are excluded. When multiple data sets are combined, the engine may exclude columns before joining to another set, or it may join the sets before excluding columns. It is based on what the engine "sees" in your data patterns (statistics).
UDFs are a frequent stumbling block for the query plan optimizer as a poorly formed function masks the data statistics and prevents the engine from efficiently piecing the data together.
All that to say, the updated engine is looking at your data and determining it is more efficient to combine multiple sets before eliminating unwanted columns.
I believe you may be able to fix this by subselecting the columns you need from your functions before joining to the outer set.
SELECT *
FROM (select SpecificColumns from tableA) as tableA
INNER JOIN (select SpecificColumns from tvf1(<params>)) as tvf1
ON tvf1.id = tableA.X1
INNER JOIN (select SpecificColumns from tvf2(<params>)) as tvf2
ON tvf2.id = tableA.X2
Alternatively, you may want to reconsider the Do-Everything query approach to reporting.
At ~8kb per row, you're likely passing a tremendous amount of data to your reporting system.
You may also give Cross Apply a try.
SELECT <about 183 columns>
FROM tableA
CROSS APPLY tvf1(<params>) tvf1
CROSS APPLY tvf2(<params>) tvf2
WHERE tvf1.id = tableA.X1
AND tvf2.id = tableA.X2
CROSS APPLY can instruct the optimizer to process sets in a different order.
Ive stil been unable to track down the exact problem.
My workaround for now has been to evaluate the table-valued-functions into table variables/temp tables and then join onto them instead
So it has been changed to something like this
DECLARE #tvf1 AS TABLE ....
INSERT INTO #tvf1 SELECT * FROM tvf1()...
DECLARE #tvf2 AS TABLE ....
INSERT INTO #tvf2 SELECT * FROM tvf2()...
DECLARE #tvf3 AS TABLE ....
INSERT INTO #tvf3 SELECT * FROM tvf3()...
SELECT <about 183 columns>
FROM tableA
INNER JOIN #tvf1 tvf1
ON tvf1.id = tableA.X1
INNER JOIN #tvf2 tvf2
ON tvf2.id = tableA.X2
INNER JOIN #tvf3 tvf3
ON tvf3.id = tableA.X3
I have a table name with spaces between say like 'Orders Details'.
I want to use that table in stored procedures with joins.I have tried using alias names,[],`` in queries but nothing seems to be working.Can anybody help me on this error!
Do not use order as a column alias. It is a SQL keyword. I would just use o:
select o.*, od.*
from orders o join
order_details od
on o.orderid = od.orderid
where year(o.orderdate) = #orderyear;
Notes:
Your JOIN condition is on ProductId. However, that is highly suspicious. Usually such joins are on the order id. In fact, ProductId doesn't belong in Orders (usually) if there is a detail table.
Do not define your tables with spaces in the name. That just makes it hard to reference the names.
orderdate does not seem to be defined, because it has a red underline.
I don't recommend select *. For one thing, you will have duplicate column names. More importantly, you want to be explicit about what this code returns, particularly in a stored function or procedure.
In advance, thank you for your assistance and explanation, I am a novice at best and trying to teach myself.
I have two different types of claims tables, [claim_a] and [claim_b]. Each of these tables capture similar data and have one claim record per row. I have a similar form to display data by record for each table. One of my forms has a grid that captures documents sent and the date sent.
In a third table, [documents], each instance of a document sent is associated with an ssn, claim_num, first, last and clm_type, doc_type and date_sent.
I want to create one query that would output all correspondence sent for both claim tables. I realize I could just do two individual queries but I think this can be done and is not too difficult, I am just missing something and would like to know what. I have tried various join type (inner, left, right) and get various results but nothing that is actually correct. With INNER JOIN, I only got 78 records but am expecting 2,261 and when I did LEFT OUTER, I got 3,070 which totals more than what I had in my [documents] table-I do understand that an outer join with one row in the LEFT table that matches two rows in the RIGHT table will return as two ROWS.
I have also been sure to use parenthesis in my first join statement which based on Google searches seems to be related to Access. I also tried using where clauses too.
I think the problem may be that some of the records in [documents] do not correspond to a record in either claims table. I also just tried joining one claim table to [documents] but even that did not return the expected number of results.
Here are few of the joins I have tried:
Inner Join for one table: My output was missing 4 records for an SSN with 6 total records and I could not figure out why it skipped over the remaining 4. It was only for this SSN. I had other SSNs with more than 6 records.
SELECT documents.date, documents.doc_type,
FROM documents INNER JOIN claim_a ON documents.ssn =
claim_a.ssn WHERE (((documents.clm_type)="Life Only")) OR
(((documents.clm_type)<>("Health")) AND (("Life/ ADB")<>False) AND
(("Life")<>False));
I got 78 records with this join
SELECT documents.date, documents.doc_type,
FROM (documents INNER JOIN claim_a ON documents.ssn =
claim_a.ssn) INNER JOIN claim_b ON documents.ssn =
claim_b.ssn;
I got 3070 records with this join
SELECT documents.date, documents.doc_type,
FROM (documents LEFT OUTER JOIN claim_a ON documents.ssn =
claim_a.ssn) LEFT OUTER JOIN claim_b ON documents.ssn =
claim_b.ssn;
I got the correct number of results with this query but I am concerned it will not work with my Master Form to display header specific information for my form associated with table, claim_b.
SELECT documents.date, documents.doc_type,
FROM documents LEFT JOIN claim_a ON documents.ssn =
claim_a.ssn WHERE (((documents.clm_type)<>""));
I am obviously doing something wrong. Can someone please advise?
Sounds like you need a Union query between the two claims tables to get a list of all claims. Then use the results of theat query to get the document list.
Union query
Select ssn from claim_a
Union all select ssn from claim_b
Save this with a name like SSN_List, then join it to Documents in another query
Select * from Documents
left join SSN_List on Documents.ssn=SSN_List.ssn
And of course change the 2nd query as needed to get the information you need from Documents.
This can probably be done in one query, but I find it easier to understand and use the 2 step approach.
How can I write a stored procedure in SQL Server 2005 so that i can display the repeated column names by having a prefix added to it?
Example: If I have 'Others' as the column name belonging to a multiple categories mapped to another table having columns as 'MyColumn','YourColumn'. I need to join these two tables so that my output should be 'M_Others' and 'Y_Others'. I can use a case but I am not sure of any other repeated columns in the table. How to write that dynamically to know the repetitions ?
Thanks In Advance
You should use aliases in the projection of the query: (bogus example, showing the usage)
SELECT c.CustomerID AS Customers_CustomerID, o.CustomerID AS Orders_CustomerID
FROM Customers c INNER JOIN Orders o ON c.CustomerID = o.CustomerID
You can't dynamically change the column names without using dynamic SQL.
You have to explicitly alias them. There is no way to change "A_Others" or "B_Others" in this query:
SELECT
A.Others AS A_Others,
B.Others AS B_Others
FROM
TableA A
JOIN
TableB B ON A.KeyCol = B.KeyCol
If the repeated columns contain the same data (i.e. they are the join fields), you should not be sending both in the query anyway as this is a poor practice and is wasteful of both server and network resources. You should not use select * in queries on production especially if there are joins. If you are properly writing SQL code, you would alias as you go along when there are two columns with the same name that mean different things (for instance if you joined twice to the person table, once to get the doctor name and once to get the patient name). Doing this dynamically from system tables would not only be inefficient but could end up giving you a big security hole depending on how badly you wrote the code. You want to save five minutes or less in development by permanently affecting performance for every user and possibly negatively impacing data security. This is what database people refer to as a bad thing.
select n.id_pk,
(case when groupcount.n_count > 1 then substring(m.name, 1, 1) + '_' + n.name
else n.name end)
from test_table1 m
left join test_table2 n on m.id_pk = n.id_fk
left join (select name, count(name) as n_count
from test_table2 group by name)
groupcount on n.name = groupcount.name