My query is related to how to build a module to take the string in a column in an SQL Server table, search on Google and scan search results for keywords and if the keywords are found put them into another column in the same table.
My initial idea was to use an R - SQL Server hybrid solution. But your guidance on any other possible tools would be great. I know it is an open ended question but I am looking more for guidance than a solution right now.
So basic steps I could gather...
Get strings in the column from SQL Server into R
Use RCurl to perform search of terms in R imported data.
Search for keywords I am looking for in the Google search results.
If results are what I am looking for, set a flag in SQL Server, else skip.
Related
I am trying to get some Oracle databases to work in MSSQL AZURE.
We seem to have most things working except the ability to search on file attachments - eg word,PDF etc.
Oracle lets us index a column in a table that uses a filepath link.
In MSSQL a column in a table can be added using:
[filepointer] VARBINARY(MAX) FILESTREAM
and then an index can be setup so files can then be searched.
I'm trying to use the same oracle table with this extra column to do the search in AZURE
EG - select * from [TESTATTACHSRCH] where contains ([filepointer],'Text in File')
I managed to get this working in MSSQL with the Oracle table we had used with this extra special column.
I know at this point FILESTREAM is not supported on AZURE & dropping FILESTREAM is not an option due to the size of the files we are searching on which would add too much size to the database.
I am hoping if there was a way i could still achieve this on AZURE, even if existing AZURE cannot do this on its own & there was 3rd party software to do something similar.
Hopefully somebody has hit the same roadblock & could provide some advice
Thanks
The alternative would be moving the attachments to blobs and store them on Azure Storage. Then, you can setup an Azure Cognitive Search which can extract and index content from pdf,word, ppt, etc.
Just using SQL Server / Azure SQL Database I don't think it will work. You can find more information about what I've described in the following link:
https://learn.microsoft.com/en-us/azure/search/search-indexer-overview
https://learn.microsoft.com/en-us/azure/search/search-howto-create-indexers?tabs=indexer-portal
I was given a task to write very specific queries from a microsoft sql server database that has over 250 tables and over 250 views. I am using sql server management studios and I am trying to figure out where a field location might exist in a table.
Are there ways to do a quick search of every table or the entire database for certain keywords to help narrow down my searches?
Or must I open every single table/view and try to see every single column in order to solve this issue?
Any help or advice with this would be greatly appreciated.
Try SQL Search from Redgate. Very powerful tool, you can download it for free:
https://www.red-gate.com/dynamic/products/sql-development/sql-search/download
The following query is written using SQL Server 2014 CONTAINS function.
SELECT
Org1.OrganizationPK
,*
FROM Organization Org1
WHERE Org1.Deleted = 0
AND CONTAINS (Org1.NAME,'"chris6*"')
AND org1.OrgDomainInd = 1
But, the above query is NOT working.
I have followed this article to understand more about CONTAINS function. I have verified those examples as-they-are. Even here also, I am not getting any results.
I have verified this post from stackoverflow, but I could not apply for my requirement.
Hence, I am getting doubt whether I need to do anything more to configure SQL Server 2014 to work with CONTAINS fuction? Please let me know if there is anything I need to do to make SQL Server 2014 ready to use CONTAINS function.
If there is nothing like that, please suggest me the solution.
Please don't suggest me to use LIKE operator. I am telling this why because, most of my colleagues suggested me same; hence, as a precautionary matter I am writing this statement here.
I am running behind my schedule to complete this task.
Before CONTAINS will work against a column you need setup full text index. This is actually a different engine which runs as a separate service in SQL Server, so make sure it is installed an running.
Once you're sure the component is installed (it may already be installed) You need to do the following:
Create a Full-Text Catalogue
Create a Full-Text Index (you can have multiple of these in the same catalogue) against the tables/columns you want to be able to use full-text keywords
Run a Population which will crawl the index created & populate the catalogue (these are seperate files which SQL Server needs to maintain in addition to mdf/ldf )
There's an ok tutorial on how to do this by using SSMS in SQL Server 2008 by Pinal Dave on CodeProject. SQL Server 2014 should be very similar.
You can also perform these steps with TSQL:
CREATE FULLTEXT CATALOG
CREATE FULLTEXT INDEX
ALTER FULLTEXT INDEX
I believe that contains functionality can only be used in tables configured to use/support Full Text Search -- an elderly feature of SQL Server that I have little experience with. If you are not using Full Text Search, I'm pretty sure contains will not work.
I have a table named as tblJobs on which i want to implement Full Text Search.
There is a column named as jobdescription on which search is to be implemented.
I m using SQL Server 2008 (RTM)..,but still my option of full-text Index is disabled. I did lots of thing, follow every steps from various websites.., but nothing seems to be helpfull. Can anyone pls help me to get out of this prolem.
Thanks in advance..!
So let's say I have two databases, one for production purposes and another one for development purposes.
When we copied the development database, the full-text catalog did not get copied properly, so we decided to create the catalog ourselves. We matched all the tables and indexes and created the database and the search feature seems to be working okay too (but been entirely tested yet).
However, the former catalog had a lot more files in its folder than the one we manually created. Is that fine? I thought they would have exact same number of files (but the size may vary)
First...when using full text search I would suggest that you don't manually try to create what the wizard does for you. I have to wonder about missing more than just some data. Why not just recreate the indexes?
Second...I suggest that you don't use freetext feature of sql server unless you have no other choice. I used to be a big believer in freetext but was shown an example of creating a Lucene(.net) index and searching it in comparison to creating an index in SQL Server and searching it. Creating a SQL Server index in comparison to creating a Lucene index is considerably slower and hard to maintain. Searching a SQL Server index is considerably less accurate (poor results) in comparison to Lucene. Lucene is like having your own personal Google for searching data.
How? Index your data (only the data you need to search) in Lucene and include the Primary Key of the data that you are indexing for use later. Then search the index using your language and the Lucene(.net) API (many articles written on this topic). In your search results make sure you return the PK. Once you have identified the records you are interested in you can then go get the rest of the data and/or any related data based on the PK that was returned.
Gotchas? Updating the index is also much quicker and easier. However, you have to roll your own for creating the index, updating the index, and searching the index. SUPER EASY to do...but still...there are no wizards or one handed coding here! Also, the index is on the file system. If the file is open and being searched and you try to open it again for another search you will obviously have some issues...so writing some form of infrastructure around opening and reading these indexes needs to be built.
How does this help in SQL Server? You can easily wrap your Lucene search in a CLR function or proc which can be installed in the database that you can then use as though it were native to your t-SQL queries.