We have one database full of strings of type:
Cannot write block. Device at EOM. dev=$device ($device_path)
And then, we have our program which generates log entries, like, for example
2013-10-10 15:37:07program-sd JobId 50345: Fatal error: block.c:434
Cannot write block. Device at EOM. dev="st0" (/dev/st0)
So, we can use SELECT * FROM X WHERE Y LIKE="%LOG%" but, the strings are not the same, in our database, we have a clean string, while in our log we have the timestamp, more info and also the data for $device and $device_path, so the query will return 0 results, because both doesn't match...
We're trying to return a error code based on what we have on the database, for this example it will be RC: 1019 if the result for the query is not 0...
Is there any way to use regex or something to accomplish this?
Suppose your table of error message templates looks like this:
create table error_templates(id serial primary key, template text);
then you can use a query like this:
select id
from error_templates
where $1 like '%' || regexp_replace(template, '\$\w+', '%', 'g') || '%';
$1 is a placeholder for the error log you are trying to find.
The regex_replace replaces the variables in the message with %. It assumes the variables consist of a dollar sign followed by one or more word characters (a-zA-Z0-9_) - you might need to adjust that depending on your actual variable names.
Also note that this may be slow. It will have to scan through the whole table; indexes cannot be used.
Related
I want to customize SQL Server FTS to handle language specific features better.
In many language like Persian and Arabic there are similar characters that in a proper search behavior they should consider as identical char like these groups:
['آ' , 'ا' , 'ء' , 'ا']
['ي' , 'ی' , 'ئ']
Currently my best solution is to store duplicate data in new column and replace these characters with a representative member and also normalize search term and perform search in the duplicated column.
Is there any way to tell SQL Server to treat any members of these groups as an identical character?
as far as i understand ,this would be used for suggestioning purposes so the being so accurate is not important. so
in farsi actually none of the character in list above doesn't share same meaning but we can say they do have a shared short form in some writing cases ('آ' != 'اِ' but they both can write as 'ا' )
SCENARIO 1 : THE INPUT TEXT IS IN COMPLETE FORM
imagine "محمّد" is a record in a table formatted (id int,text nvarchar(12))named as 'table'.
after removing special character we can use following command :
select * from [db].[dbo].[table] where text REPLACE(text,' ّ ','') = REPLACE(N'محمد',' ّ ','');
the result would be
SCENARIO 2: THE INPUT IS IN SHORT FORMAT
imagine "محمد" is a record in a table formatted (id int,text nvarchar(12))named as 'table'.
in this scenario we need to do some logical operation on text before we query in data base
for e.g. if "محمد" is input as we know and have a list of this special character ,it should be easily searched in query as :
select * from [db].[dbo].[table] where REPLACE(text,' ّ ','') = 'محمد';
note:
this solution is not exactly a best one because the input should not be affected in client side it, would be better if the sql server configure to handle this.
for people who doesn't understand farsi simply he wanna tell sql that َA =["B","C"] and a have same value these character in the list so :
when a "dad" word searched, if any word "dbd" or "dcd" exist return them too.
add:
some set of characters can have same meaning some of some times not ( ['ي','أ'] are same but ['آ','اِ'] not) so in we got first scenario :
select * from [db].[dbo].[table] where text like N'%هی[أي]ت' and text like N'هی[أي]ت%';
I have a table of swear words in SQL Server and I use LIKE query to search texts for words in the table. I need a way to include whitespaces around the swear word in LIKE query, like this:
... LIKE '%{whitespace}SWEAR-WORD{whitespace}%';
Putting space around the swear word is not enough, because it can be a part of another normal word in my language (like 'inter' that is part of 'international' or 'pointer').
Another solution I've tried was using this:
... LIKE '%[^a-zA-Z]SWEAR-WORD[^a-zA-Z]%';
But that did not work for me.
Is there any way to do this? Or alternatively is there any solution other than LIKE query?
Edit: For better understanding, it's our current way to find swear-words:
We have a table named Reviles which has 2 columns (Id and Text) and contains restricted words and phrases. We use this query to find out whether a content has any of those restricted words and phrases:
IF EXISTS (SELECT * dbo.Reviles WHERE #Text LIKE '%' + dbo.Reviles.Text + '%')
#IsHidden = 0
Note that this check is done before the content being inserted into its table. The code above is part of a stored procedure which gets information of a post and checks various things including swear words before inserting it.
Before we've stored restricted words like ' swear-word ' in the table, however this way we could not find and hide contents with swear words at the beginning or at the end of the line or contents which consists of only a swear word. For example:
This is my content with a swear-word
or
Swear-word in my content
or
Swear-word
So we decided to remove those spaces and store restricted words like 'swear-word'. But this causes some normal content to hide because some swear words can be part of another word which is normal (If we assume inter is a bad word, then pointer and international, etc. will be restricted).
Sorry for my bad English, I hope with this description, I've made it clear.
try to close your check statement in some chars and then compare:
some data:
declare #T table(stmt nvarchar(20))
insert into #T values ('inter'),('Inter.'),('My inter'),
('intermediate!'),('pointer '),('Good inter'),('inter inter inter')
try this:
select
stmt as stmt,
case
when '.'+stmt+'.' like '%[^a-z]inter[^a-Z]%' then 1 else 0 end as [has inter]
from
#T
results:
stmt has inter
-------------------- -----------
inter 1
Inter. 1
My inter 1
intermediate! 0
pointer 0
Good inter 1
inter inter inter 1
I'm a bit confuse what are u want to do, if u want to do like '{whitespace}swearword{whitespace}', then use like '% inter %' already work
but if u really have special requirement about filter, another way is enable SQL CLR, and Create Sql function from visualStudio and deploy to SQL Server. inside SQL function u can use Regular expression to return match or not.
Create SQL Databaase Project
Add SQL CLR(I use C#)
Add Code
public partial class UserDefinedFunctions
{
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlBoolean RegularMatch(string str, string pattern)
{
var regex = new Regex(pattern);
return new SqlBoolean (regex.IsMatch(str));
}
}
Public to SQL Server
Sorry I'm not good at format this.
I have a SQL Server engine in which I have a filed with filter clause. I need that clause to be compose to show all rows except those that contain :, -, ~.
My query is:
SELECT 1
WHERE '' LIKE '%[^:-~]%'
It is not working - it shows zero rows. Also I try this:
SELECT 1
WHERE 'aa:a' LIKE '%[^:-~]%'
And it shows as result 1, which is not desired result.
Is there a way to manage this?
REMARK: expression after like must be string which will be saved inside the table field (for exmaple: '%[^:-~]%' will be used as LIKE x.fldFilter)
EDIT: I need to make validation inside my engine inside SQL Server. I have a table with Parameters. Inside it I have column Format. For a specific parameter I need to check if value provided fits Format column.
For example:
DECLARE #value AS VARCHAR(1000) = 'aaa:aa';
SELECT 1 FROM dbo.ParameterDefinitions X WHERE #value LIKE X.[Format];
Where X.[Format] column contains '%[^:-~]%'.
When I test a value check must return 1 if it fits conditions and nothing if not fits.
So if I test value 'aaa:aa' or even ' ' it works. But when I have empty string ('') the condition does not working.
Unfortunately I can not change my engine and can not replace '' with space for example. I just wonder why '' does not fit the condition?
This is due to SQL Server not having a solid regex implementation.
Instead of negating your search with ^
Negate it with Not
SELECT 1
WHERE '' NOT LIKE '%[:-~]%'
Returns 1 row
SELECT 1
WHERE 'aa:a' NOT LIKE '%[:-~]%'
Returns 0 rows
EDIT:
Breaking down your search cases
'' LIKE '%[^:-~]%'
[^:-~] requires a single character so an empty string must fail
'aa:a' LIKE '%[^:-~]%'
% is a 0 or more wildcard which lets [^:-~] take its choice of 'a' while either % collects your forbidden character.
With a full regex engine we could repeat your negated range with the following [^:-~]* but SQL Server doesn't support that. Docs
The only option left to us is to perform a search for the forbidden characters '%[:-~]%' and to negate the like.
In my SQL Server database there is scenario like database have one primary key and primary key is in format like '0000100001' and 'C100001'
I want to delete the all records from database which starts with '0' but not the records starts with 'C'.
I tried the inbuilt function SUBSTRING('primary_key',1,1)='0' but it did not helped me..
Thank You..
SUBSTRING('primary_key',1,1)='0'
tests whether the string literal "primary_key" starts with the character 0 (which it doesn't so will return zero rows), Get rid of the single quotes to reference the column. (NB: If your column is not actually called primary_key you will need to reference its actual name of course!)
Or alternatively you can use WHERE primary_key LIKE '0%' which can use the index to locate the rows so is more efficient.
I don't know MS SQL, but in MySQL it would be something like this:
"DELETE * FROM your_table WHERE primary_key LIKE '0%' AND primary_key NOT LIKE 'C%'"
You can use the LIKE operator to essentially search for a occurances of either a string or a regular expression. It can take wildcards such as the % sign both in front, behind, or both in front and behind of the pattern you are looking for.
For example:
LIKE 'C%' would match anything starting with C
LIKE '%C' would match anything ending in C
LIKE '[A-Z]%' would match anything starting with a capital letter
LIKE '%LOL%' would match anything that has the word LOL(in caps) in it.
Further reading at
http://msdn.microsoft.com/en-us/library/ms179859.aspx
It's impossible to sqlite3_bind_text a table name because sqlite3_prepare_v2 fails to prepare a statement such as:
SELECT * FROM ? ;
I presume the table name is needed to parse the statement, so the quoting needs to have happened before sqlite3_prepare_v2.
Is there something like a sqlite3_quote_tablename? Maybe it already exists under a name I can't recognize, but I can't find anything in the functions list.
SQLite will escape identifiers for you with the %w format in the https://www.sqlite.org/printf.html family of functions.
your proposed sqlite3_quote_tablename function could sanitize the input to prevent sql injection attacks. To do this it could parse the input to make sure it is a string literal. http://sqlite.org/lang_expr.html#litvalue
If a table name has invalid characters in it you can enclose the table name in double quotes, like this.
sqlite> create table "test table" (id);
sqlite> insert into "test table" values (1);
sqlite> select * from "test table";
id
----------
1
Of course you should avoid using invalid characters whenever possible. It complicates development and is almost always unnecessary (IMO the only time it is necessary is when you inherit a project that is already done this way and it's too big to change).
When using SQLite prepared statements with parameters the parameter: "specifies a placeholder in the expression for a literal value that is filled in at runtime"
Before executing any SQL statement, SQLite "compiles" the SQL string into a series of opcodes that are executed by an internal Virtual Machine. The table names and column names upon which the SQL statement operates are a necessary part of the compilation process.
You can use parameters to bind "values" to prepared statements like this:
SELECT * FROM FOO WHERE name=?;
And then call sqlite3_bind_text() to bind the string gavinbeatty to the already compiled statement. However, this architecture means that you cannot use parameters like this:
SELECT * FROM ? WHERE name=?; // Can't bind table name as a parameter
SELECT * FROM FOO WHERE ?=10; // Can't bind column name as a parameter
If SQLite doesn't accept table names as parameters, I don't think there is a solution for your problem...
Take into account that:
Parameters that are not assigned values using sqlite3_bind() are treated as NULL.
so in the case of your query, the table name would be NULL which of course is invalid.
I was looking for something like this too and couldn't find it either. In my case, the expected table names were always among a fixed set of tables (so those were easy to validate). The field names on the other hand weren't so I ended up filtering the string, pretty much removing everything that was not a letter, number, or underscore (I knew my fields would fit this parameters). That did the trick.