How do I define an expression (or any other alternative method) when I want to select a value from a dynamic drop down list?
ex:
drop down list has these values:
Plan A (X)
Plan B (Y)
Plan C (Z)
where, X, Y and Z are all random numbers.
Is there a way where I can define my selection to select the text up to "Plan #" and ignore the "(X)"?
You could try using wildcards, those are usually supported by SilkTest.
For your example this would be
"Plan A*"
Related
I have a requirement where the string from a column has a value "/Date(-34905600000)/". The value within brackets could be in any one of the following patters
"/Date(-34905600000)/"
"/Date(1407283200000)/"
"/Date(1636654411000+0000)/"
I need to extract all inside the parenthesis for examples 1 and 2 including the "-" if any. For the 3rd example, it should be only the numbers inside the parenthesis before "+" ie 1636654411000.
I tried the following and not getting the results as the output is coming along with the parenthesis.
select REGEXP_substr("/Date(-34905600000)/", '\\([[:alnum:]\-]+\\)')
from table A;
select REGEXP_substr("/Date(-34905600000)/", '\\((.*?)\\)') from table
A;
select REGEXP_substr("/Date(-34905600000)/", '[0-9]+') from table A;
Using regexp_replace() instead you could do:
regexp_replace(colA, '(\\/Date\\()([-0-9]*)(.*)', '\\2')
That splits the string into three substitution groups and then only keeps the second. I often end up doing regexp_replace() with substitution groups like this when regexp_substr() fails me.
if you want the REGEXP_SUBSTR to sub-matches you need to use the 'e' <regex_parameters> option, and then you can use 1 as the to match your first grouping, thus:
SELECT column1,
REGEXP_substr(column1, 'Date\\(([-+]?[0-9]+)',1,1,'e')
FROM VALUES
('"/Date(-34905600000)/"'),
('"/Date(1407283200000)/"'),
('"/Date(1636654411000+0000)/"');
gives:
COLUMN1
REGEXP_SUBSTR(COLUMN1, 'DATE\(([-+]?[0-9]+)',1,1,'E')
"/Date(-34905600000)/"
-34905600000
"/Date(1407283200000)/"
1407283200000
"/Date(1636654411000+0000)/"
1636654411000
I am quite sure the regexp is greedy by default, but otherwise you can force the match to the timezone or paren with
'Date\\(([-+]?[0-9]+)[-+\\)]'
I'm trying to get rows where a column of type text[] contains a value similar to some user input.
What I've thought and done so far is to use the 'ANY' and 'LIKE' operator like this:
select * from someTable where '%someInput%' LIKE ANY(someColum);
But it doesn't work. The query returns the same values as that this query:
select * from someTable where 'someInput' = ANY(someColum);
I've got good a result using the unnest() function in a subquery but I need to query this in WHERE clause if possible.
Why doesn't the LIKE operator work with the ANY operator and I don't get any errors? I thought that one reason should be that ANY operator is in the right-hand of query, but ...
Is there any solution to this without using unnest() and if it is possible in WHERE clause?
It's also important to understand that ANY is not an operator but an SQL construct that can only be used to the right of an operator. More:
How to use ANY instead of IN in a WHERE clause with Rails?
The LIKE operator - or more precisely: expression, that is rewritten with to the ~~ operator in Postgres internally - expects the value to the left and the pattern to the right. There is no COMMUTATOR for this operator (like there is for the simple equality operator =) so Postgres cannot flip operands around.
Your attempt:
select * from someTable where '%someInput%' LIKE ANY(someColum);
has flipped left and right operand so '%someInput%' is the value and elements of the array column someColum are taken to be patterns (which is not what you want).
It would have to be ANY(someColum) LIKE '%someInput%' - except that's not possible with the ANY construct which is only allowed to the right of an operator. You are hitting a road block here.
Related:
Is there a way to usefully index a text column containing regex patterns?
Can PostgreSQL index array columns?
You can normalize your relational design and save elements of the array in separate rows in a separate table. Barring that, unnest() is the solution, as you already found yourself. But while you are only interested in the existence of at least one matching element, an EXISTS subquery will be most efficient while avoiding duplicates in the result - Postgres can stop the search as soon as the first match is found:
SELECT *
FROM tbl
WHERE EXISTS (
SELECT -- can be empty
FROM unnest(someColum) elem
WHERE elem LIKE '%someInput%'
);
You may want to escape special character in someInput. See:
Escape function for regular expression or LIKE patterns
Careful with the negation (NOT LIKE ALL (...)) when NULL can be involved:
Check if NULL exists in Postgres array
An admittedly imperfect possibility might be to use ARRAY_TO_STRING, then use LIKE against the result. For example:
SELECT *
FROM someTable
WHERE ARRAY_TO_STRING(someColum, '||') LIKE '%someInput%';
This approach is potentially problematic, though, because someone could search over two array elements if they discover the joining character sequence. For example, an array of {'Hi','Mom'}, connected with || would return a result if the user had entered i||M in place of someInput. Instead, the expectation would probably be that there would be no result in that case since neither Hi nor Mom individually contain the i||M sequence of characters.
My question was marked duplicate and linked to a question out of context by a careless mod. This question comes closest to what I asked so I leave my answer here. (I think it may help people for who unnest() would be a solution)
In my case a combination of DISTINCT and unnest() was the solution:
SELECT DISTINCT ON (id_) *
FROM (
SELECT unnest(tags) tag, *
FROM someTable
) x
WHERE (tag like '%someInput%');
unnest(tags) expands the text array to a list of rows and DISTINCT ON (id_) removes the duplicates that result from the expansion, based on a unique id_ column.
Update
Another way to do this without DISTINCT within the WHERE clause would be:
SELECT *
FROM someTable
WHERE (
0 < (
SELECT COUNT(*)
FROM unnest(tags) AS tag
WHERE tag LIKE '%someInput%'
)
);
Please check this out.
This answer was exactly what I was looking for. It also provides for some useful tips (and examples) in case you need more flexibility.
It basically explains the ANY(), the #> and the && operators.
"If you want to search multiple values, you can use #> operator"
"#> means contains all the values in that array. If you want to search if the current array contains any values in another array, you can use &&"
I am trying to check a string array for existence of a converted integer number. This sits inside of a procedure where:
nc_ecosite is an integer variable
current_consite is a string array
ecosite is an integer
current_ecosite_nc is double
IF to_char(nc_ecosite, '999') IN
(select current_consite from current_site_record
where current_ecosite_nc::integer = nc_ecosite) THEN
ecosite := nc_ecosite;
The result always comes from the ELSIF that follows the first IF. This occurs when nc_ecosite is in the array (from checks). Why is ecosite not being populated with nc_ecosite when values are matching?
I am working with Postgres 9.3 inside pgAdmin.
I found the following to provide the desired result:
IF nc_ecosite in
(select (unnest(string_to_array(current_consite, ',')))::integer
from current_site_record
where current_ecosite_nc::integer = nc_ecosite) THEN
ecosite := nc_ecosite::integer;
The immediate reason for the problem is that to_char() inserts a leading blank for your given pattern (legacy reasons - to make space for a potential negative sign). Use the FM Template Pattern Modifier to avoid that:
to_char(nc_ecosite, 'FM999')
Of course, it would be best to operate with matching data types to begin with - if at all possible.
Barring that, I suggest this faster and cleaner statement:
SELECT INTO ecosite nc_ecosite -- variable or column??
WHERE EXISTS (
SELECT 1 FROM current_site_record c
WHERE current_ecosite_nc::integer = nc_ecosite
AND to_char(nc_ecosite, 'FM999') = ANY(current_consite)
);
IF NOT FOUND THEN ... -- to replace your ELSIF
Make sure you don't run into naming conflicts between parameters, variables and column names! A widespread convention is to prepend variable names with _ (and never use the same for column names). But you better table-qualify column names in all queries anyway. You did not make clear which is a column and which is a variable ...
I might be able to optimize the statement further if I had the complete function and table definition.
Related:
Remove blank-padding from to_char() output
Variables for identifiers inside IF EXISTS in a plpgsql function
Naming conflict between function parameter and result of JOIN with USING clause
I have a table X with one column of type jsonb.
Jsonb contains json array - "cities":["aaaa","bbbb","cccc"].
Postgresql 9.4 provides jsonb operators to get json array elements using '->'
There is another table Y with column cities.
Y
a b cities
aaaa
bbbb
cccc
I want to display
select Y.a, Y.b from Y, X only if X.jsonb->cities is present in Y.cities.
This is done with a lateral join over the json_array_elements (or in this case json_array_elements_text since y.cities is presumably text-typed) function. You didn't provide a full sample schema, so I'll hand-wave some untested SQL to give you the idea.
select *
from x
cross join json_array_elements_text(x.cities) AS x_cities(city)
inner join y on (x_cities.city = y.cities);
If you're going to use json you're going to need to get very good with lateral joins.
In general I'm seeing a lot of people using json where it's completely unnecessary and a simple relational modelling would be more appropriate. Think about whether you really need to do this. In this case it seems like if you must use an array, a native PostgreSQL text[] array would be better, but you should probably model it with a join-table instead.
What is the best way to validate fields in a row and if invalid, correct it to the right form?
The simplest example would be checking phone number field (can come in variant formats -> 111-111-1111, (111) 111-1111 etc), and we would ideally want to validate these and standardize to one form (lets say: 1111111111). One way to do this is to use filter rows and then use a regex, or we can use data validator. But this will only tell us what data is invalid but not actually format it for us. We can then use Javascript modified value step to write a js script to do this. But I am guessing there is a better way (or a built in integration that I haven't come across) that would do these basic validations. Or is it recommended to just dump rows containing invalid fields in a separate csv file and then use a script to parse it separately?
g'day
i use the excellent 'replace in string' step to handle this circumstance
you can cumulatively apply rules for removing bad char from strings within the single step - it is really easy to use for single-char fixes like what you have described, and best of all, it also allows you to search based on regex as well - in a single step you have documented your standardisation and produced the clean output
in your case, i would create two 'rules' to replace ( and ) with nothing - however, the - is a little trickier; you need a rule for each removal of a single char, so you would need to know the maximum number of - in a single data field, then add this many lines to your 'replace in string' step
if this is unpalatable, consider the 'user defined java expression' and a call to replace, eg: ( (t0 != null) ? t0.replace("-","") : t0 )
as i stated, each 'fix' is applied in sequential order - the In stream field is the input field-name, whereas the Outstream field is left blank instructing d.i. to modify the field itself - here's a more complex example where i search for regex and replace them with nothing, escape for the case where i escape a " double-quote:
In stream field Out stream field use RegEx Search Replace with
sc_srcuri N {Internal.Transformation.Filename.Directory}
re_s_sciname Y ["] \\"
re_s_sciname Y .[\x08]
re_s_sciname Y .[\x08]
re_s_sciname Y .[\x08]
re_s_sciname Y [*]
re_s_sciname Y \s*$
re_s_sciname Y ^\s*
notice i am removing up to three 'delete' control-codes [\x08] from this particular string?