I don't understand how I could get a result of 4 for difference on the following:
col_a col_b
201 E. Rudisill 2535 E 10th St.
6039 Bunt Drive 408 W. Petit Ave.
difference(upper(a), upper(b)) returned 4 for both rows.
How is this possible? They do not sound anything alike?
SOUNDEX converts an alphanumeric string to a four-character code that is based on how the string sounds when spoken. May be the above string starts with numbers the soundex() return '0000'.
Similar Question : Soundex with numbers as String parameter
Related
I have a column named "edited_address" in a df named CB_Edit. In "edited_address" column there are a multitude of addresses. Some addresses include a variable ("L#") which I want to completely remove from all of them which possess it. For instance:
edited_address:
100 S Smith Street
200 S Smith L100 Street
300 S Smith Street
400 S L1 Smith Street
500 S Smith Street L999
600 N Jacobs Blvd
900 L53 Cascades Street
I want to remove the "L#" from the column. There are two problems. The first is that the L followed by the number range anywhere from 0-9999. The second is that the L# can be anywhere in the column cell. Let me know what I can do, thank you!
As a side, there are numerous outliers for this problem, here are some examples. These numeric ranges are from 1-9999 and have a second letter from A-Z. These second letters sometimes also possess their own number range:
L1A thru L9999Z
as well as
L1A1 thru L9999Z9999
I have tried several gsub functions, expecting to capture everything in the range. That did not happen.
We can use the following approach with sub(), for a base R option:
df$edited_address <- gsub("^\\s+|\\s+$", "", sub("\\s*\\bL\\d{1,4}\\b\\s*", " ", df$edited_address))
df
edited_address
1 100 S Smith Street
2 200 S Smith Street
3 300 S Smith Street
4 400 S Smith Street
5 500 S Smith Street
For your updated question I would match on this regex pattern:
\s*\bL\d{1,4}(?:[A-Z]\d{0,4})?\b\s*
I need to add a comma to a string using Snowflake
EX: Austin TX --> Austin, TX
I already tried (b2_loc ||', '|| (RIGHT(b2_loc, 2))) AS b2_loc which gave me Austin TX, TX
Sounds like a job for String Functions (Regular Expressions)
with cities(city) as (
select * from values
('Austin TX'),
('Nashville TN'),
('Minneapolis MN'),
('St. Louis MO')
)
select regexp_replace(city, '(.*) (.*)', '\\1, \\2') as city
from cities;
CITY
Austin, TX
Nashville, TN
Minneapolis, MN
St. Louis, MO
A good site to learn more about regular expressions Regular Expressions Info
I would inclined to use Dave's answer. But you answer can be fixed:
select
column1 as b2_loc
,(b2_loc ||', '|| (RIGHT(b2_loc, 2))) AS wrong
,trim(substring(b2_loc,0, length(b2_loc)-2)) ||', '|| (RIGHT(b2_loc, 2)) AS correct
from values
('Austin TX')
;
B2_LOC
WRONG
CORRECT
Austin TX
Austin TX, TX
Austin, TX
the reason this happens is, you are get all of the input string then attaching the comma and the last two values.. thus you don't want all of the input, just the minus 2 part of it, and then the whitespace also trimmed. Unfortunately you cannot use -2 in the substring to make it right side relative (like you can is some other languages) so LENGTH also needs to be used, then TRIM to remove whitespace.
Assuming state abbreviations are always 2 characters preceded by a single space, you could use insert
set str='Austin TX';
select insert($str,length($str)-2,0,',')
Alternatively, you could also reverse the string, insert your comma, and reverse it back
select reverse(insert(reverse($str),4,0,','));
I would like to check if a specific column in one of my tables meets the following conditions:
String must contain at least three characters
String must contain at least two different numbers [e.g. 123 would work but 111 would not]
Characters which are allowed in the string:
Numbers (0-9)
Uppercase letters
Lowercase letters
Underscores (_)]
Dashes (-)
I have some experience with Regex but am having issues with Snowflake's syntax. Whenever I try using the '?' regex character (to mark something as optional) I receive an error. Can someone help me understand a workaround and provide a solution?
What I have so far:
SELECT string,
LENGTH(string) AS length
FROM tbl
WHERE REGEXP_LIKE(string,'^[0-9]+{3,}[-+]?[A-Z]?[a-z]?$')
ORDER BY length;
Thanks!
Your regex looks a little confusing and invalid, and it doesn't look like it quite meets your needs either. I read this expression as a string that:
Must start with one or more digits, at least 3 or more times
The confusing part to me is the '+' is a quantifier, which is not quantifiable with {3,} but somehow doesn't produce an error for me
Optionally followed by either a dash or plus sign
Followed by an uppercase character zero or one times (giving back as needed)
Followed by and ending with a lowercase character zero or one times (giving back as needed)
Questions
You say that your string must contain 3 characters and at least 2 different numbers, numbers are characters but I'm not sure if you mean 3 letters...
Are you considering the numbers to be characters?
Does the order of the characters matter?
Can you provide an example of the error you are receiving?
Notes
Checking for a second digit that is not the same as the first involves the concept of a lookahead with a backreference. Snowflake does not support backreferences.
One thing about pattern matching with regular expressions is that order makes a difference. If order is not of importance to you, then you'll have multiple patterns to match against.
Example
Below is how you can test each part of your requirements individually. I've included a few regexp_substr functions to show how extraction can work to check if something exists again.
Uncomment the WHERE clause to see the dataset filtered. The filters are written as expressions so you can remove any/all of the regexp_* columns.
select randstr(36,random(123)) as r_string
,length(r_string) AS length
,regexp_like(r_string,'^[0-9]+{3,}[-+]?[A-Z]?[a-z]?$') as reg
,regexp_like(r_string,'.*[A-Za-z]{3,}.*') as has_3_consecutive_letters
,regexp_like(r_string,'.*\\d+.*\\d+.*') as has_2_digits
,regexp_substr(r_string,'(\\d)',1,1) as first_digit
,regexp_substr(r_string,'(\\d)',1,2) as second_digit
,first_digit <> second_digit as digits_1st_not_equal_2nd
,not(regexp_instr(r_string,regexp_substr(r_string,'(\\d)',1,1),1,2)) as first_digit_does_not_appear_again
,has_3_consecutive_letters and has_2_digits and first_digit_does_not_appear_again as test
from table(generator(rowcount => 10))
//where regexp_like(r_string,'.*[A-Za-z]{3,}.*') // has_3_consecutive_letters
// and regexp_like(r_string,'.*\\d+.*\\d+.*') // has_2_digits
// and not(regexp_instr(r_string,regexp_substr(r_string,'(\\d)',1,1),1,2)) // first_digit_does_not_appear_again
;
Assuming the digits need to be contiguous, you can use a javascript UDF to find the number in a string with with the largest number of distinct digits:
create or replace function f(S text)
returns float
language javascript
returns null on null input
as
$$
const m = S.match(/\d+/g)
if (!m) return 0
const lengths = m.map(m=> [...new Set (m.split(''))].length)
const max_length = lengths.reduce((a,b) => Math.max(a,b))
return max_length
$$
;
Combined with WHERE-clause, this does what you want, I believe:
select column1, f(column1) max_length
from t
where max_length>1 and length(column1)>2 and column1 rlike '[\\w\\d-]+';
Yielding:
COLUMN1 | MAX_LENGTH
------------------------+-----------
abc123def567ghi1111_123 | 3
123 | 3
111222 | 2
Assuming this input:
create or replace table t as
select * from values ('abc123def567ghi1111_123'), ('xyz111asdf'), ('123'), ('111222'), ('abc 111111111 abc'), ('12'), ('asdf'), ('123 456'), (null);
The function is even simpler if the digits don't have to be contiguous (i.e. count the distinct digits in a string). Then core logic changes to:
const m = S.match(/\d/g)
if (!m) return 0
const length = [...new Set (m)].length
return length
Hope that's helpful!
I have to parse some tables from an ASCII text file. Here's a partial sample:
QSMDRYCELL 11.00 11.10 11.00 11.00 -.90 11 11000 1.212
RECKITTBEN 192.50 209.00 192.50 201.80 5.21 34 2850 5.707
RUPALIINS 150.00 159.00 150.00 156.25 6.29 4 80 .125
SALAMCRST 164.00 164.75 163.00 163.25 -.45 80 8250 13.505
SINGERBD 779.75 779.75 770.00 773.00 -.89 8 95 .735
SONARBAINS 68.00 69.00 67.50 68.00 .74 11 3050 2.077
The table consists of 1 column of text and 8 columns of floating point numbers. I'd like to capture each column via regex.
I'm pretty new to regular expressions. Here's the faulty regex pattern I came up with:
(\S+)\s+(\s+[\d\.\-]+){8}
But the pattern captures only the first and the last columns. RegexBuddy also emits the following warning:
You repeated the capturing group
itself. The group will capture only
the last iteration. Put a capturing
group around the repeated group to
capture all iterations.
I've consulted their help file, but I don't have a clue as to how to solve this.
How can I capture each column separately?
In C# (modified from this example):
string input = "QSMDRYCELL 11.00 11.10 11.00 11.00 -.90 11 11000 1.212";
string pattern = #"^(\S+)\s+(\s+[\d.-]+){8}$";
Match match = Regex.Match(input, pattern, RegexOptions.MultiLine);
if (match.Success) {
Console.WriteLine("Matched text: {0}", match.Value);
for (int ctr = 1; ctr < match.Groups.Count; ctr++) {
Console.WriteLine(" Group {0}: {1}", ctr, match.Groups[ctr].Value);
int captureCtr = 0;
foreach (Capture capture in match.Groups[ctr].Captures) {
Console.WriteLine(" Capture {0}: {1}",
captureCtr, capture.Value);
captureCtr++;
}
}
}
Output:
Matched text: QSMDRYCELL 11.00 11.10 11.00 11.00 -.90 11 11000 1.212
...
Group 2: 1.212
Capture 0: 11.00
Capture 1: 11.10
Capture 2: 11.00
...etc.
If you want to know what the warning is appearing for, it's because your capture group matches multiple times (8, as you specified) but the capture variable can only have one value. It is assigned the last value matched.
As described in question 1313332, retrieving these multiple matches is generally not possible with a regular expression, although .NET and Perl 6 have some support for it.
The warning suggests that you could put another group around the whole set, like this:
(\S+)\s+((\s+[\d\.\-]+){8})
You would then be able to see all the columns, but of course they would not be separated. Because it's generally not possible to capture them separately, the more common intention is to capture all of it, and the warning helps remind you of this.
Unfortunately you need to repeat the (…) 8 times to get each column separately.
^(\S+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)\s+([-.\d]+)$
If code is possible, you can first match those numeric columns as a whole
>>> rx1 = re.compile(r'^(\S+)\s+((?:[-.\d]+\s+){7}[-.\d]+)$', re.M)
>>> allres = rx1.findall(theAsciiText)
then split the columns by spaces
>>> [[p] + q.split() for p, q in allres]
I have a vcf file like this:
http://www.1000genomes.org/node/101
Here's the example from that site:
##fileformat=VCFv4.0
##fileDate=20090805
##source=myImputationProgramV3.1
##reference=1000GenomesPilot-NCBI36
##phasing=partial
##INFO=<ID=NS,Number=1,Type=Integer,Description="Number of Samples With Data">
##INFO=<ID=DP,Number=1,Type=Integer,Description="Total Depth">
##INFO=<ID=AF,Number=.,Type=Float,Description="Allele Frequency">
##INFO=<ID=AA,Number=1,Type=String,Description="Ancestral Allele">
##INFO=<ID=DB,Number=0,Type=Flag,Description="dbSNP membership, build 129">
##INFO=<ID=H2,Number=0,Type=Flag,Description="HapMap2 membership">
##FILTER=<ID=q10,Description="Quality below 10">
##FILTER=<ID=s50,Description="Less than 50% of samples have data">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype Quality">
##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Read Depth">
##FORMAT=<ID=HQ,Number=2,Type=Integer,Description="Haplotype Quality">
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT NA00001 NA00002 NA00003
20 14370 rs6054257 G A 29 PASS NS=3;DP=14;AF=0.5;DB;H2 GT:GQ:DP:HQ 0|0:48:1:51,51 1|0:48:8:51,51 1/1:43:5:.,.
20 17330 . T A 3 q10 NS=3;DP=11;AF=0.017 GT:GQ:DP:HQ 0|0:49:3:58,50 0|1:3:5:65,3 0/0:41:3
20 1110696 rs6040355 A G,T 67 PASS NS=2;DP=10;AF=0.333,0.667;AA=T;DB GT:GQ:DP:HQ 1|2:21:6:23,27 2|1:2:0:18,2 2/2:35:4
20 1230237 . T . 47 PASS NS=3;DP=13;AA=T GT:GQ:DP:HQ 0|0:54:7:56,60 0|0:48:4:51,51 0/0:61:2
20 1234567 microsat1 GTCT G,GTACT 50 PASS NS=3;DP=9;AA=G GT:GQ:DP 0/1:35:4 0/2:17:2 1/1:40:3
After the header lines, each line has fields that contain genotypes starting with the 10th field. The 10th field is below the NA0001 heading; the 11th field is genotype NA0002, etc. I have a file with 123 different genotypes, so going from position 10 to 133 (NA0001 until NA0123). What is shown in these fields can be 0/0, 0/1, 0/2 .... till 8/9 for instance. Now I want to replace all the non-equal ones. So I would like to keep 0/0, 1/1, 2/2, etc. And replace 0/1, 0/2, 1/2, 4/5, 4/6 etc by ./.
I would like to write this in a C script. Thought about using sed y/regexp/replacement/ but no idea how to write all those unequal values in a regular expression. And on other positions in the file there could also be these values, so really only positions 10 till 133 should be replaced. And it needs to be replaced; I will be needing the rest of the file with the new values.
Hope it is clear. Anyone any idea how to do this?
This regex should do what you want: \s(\d)[|\/](?!\1)\d: Replace matches with ./.:
Breakdown:
\s(\d) matches a space followed by a single digit, capturing the digit in capture group #1
[|\/] matches a pipe or slash (since it seems that the VCF format allows either)
(?!\1)\d uses a negative lookahead to ensure that the next character is not the same as capture group #1, and matches the digit
Caveats:
I matched a leading space and trailing : to try to ensure it matches only the intended values. I couldn't work out a good way to limit it to fields 10 and after.
Example using perl:
perl -pe 's#\s(\d)[|/](?!\1)\d:# ./.:#g' testfile.vcf > testfile_afterchange.vcf
Note: I used # as the delimiter to avoid having to escape the / characters in the regex.