(Postgres 10.10)
I have the following fields in my_table:
loval INTEGER
hival INTEGER
valcount INTEGER
values INTEGER[]
I need to set values to an array containing valcount random integers each between loval and hival inclusive. So for:
loval: 3
hival: 22
valcount: 6
I'm looking to set values to something like:
{3, 6, 6, 13, 17, 22}
I know how to do this with an inefficient "loop through the cursor" solution, but I'm wondering if Postgres has a way to do a looping computation inline.
Note: I looked at generate_series, but I don't think it produces what I need.
generate_series() is indeed the solution:
update my_table
set "values" = array(select (random() * (hival - loval) + loval)::int
from generate_series(1, valcount));
Online example
Note that values is a reserved keyword, it's not a good idea to use that as a column name.
Related
I made a BigQuery query which involves generating an array of random numbers for each row. I use the random numbers to decide which elements to include from an array that exists in my source table.
I had a lot of trouble getting the arrays of random numbers to not repeat themselves over every single row. I found a workaround, but is this expected behavior? I'll post two "methods" (one with desired results, one with bad results) below. Note that both methods work fine if you don't use an array, but just generate a single random number.
Method 1 (BAD Results):
SELECT
(
SELECT
ARRAY(
SELECT AS STRUCT
RAND() AS random
FROM UNNEST(GENERATE_ARRAY(0, 10, 1)) AS _time
) AS random_for_times
)
FROM UNNEST(GENERATE_ARRAY(0, 10, 1))
Method 2 (GOOD results):
SELECT
(
SELECT
ARRAY(
SELECT AS STRUCT
RAND() AS random
FROM UNNEST(GENERATE_ARRAY(0, 10, 1)) AS _time
) AS random_for_times
FROM (SELECT NULL FROM UNNEST([0]))
)
FROM UNNEST(GENERATE_ARRAY(0, 10, 1))
Example Results - Method 1 (BAD):
Row 1
0.5431173080158003
0.5585452983410205
...
Row 2
0.5431173080158003
0.5585452983410205
...
Example Results - Method 2 (GOOD):
Row 1
0.49639706531271377
0.1604380522058521
...
Row 2
0.7971869432989377
0.9815667330115473
...
EDIT: See below for some alternative examples that are similar, after Yun Zhang's theory about subqueries. Your solution was useful for the problem I posted, but note that there are still some cases I am finding baffling. Also, although I agree that you are probably correct about the subqueries being tied to the problem: shouldn't a subquery (especially one without a FROM clause) be less likely to have its results re-used than selecting a "normal" value? People talk about performance issues with subqueries sometimes, because they are supposedly calculated one time for each row, even if the results may be the same.
Do you agree that this seems like it may be a bug?
The below examples show that it is not necessarily even creating an array of randoms that is the problem -- even performing a sub-select that just happens to have an unrelated array in it can cause problems with RAND(). The problem goes away by eliminating the sub-select, by choosing just the random value from the sub-select, or by including a value inside the array which varies by row. Weird !!!
BAD
SELECT
(SELECT AS STRUCT RAND() AS r, ARRAY(SELECT 1) AS a)
FROM UNNEST(GENERATE_ARRAY(0, 5, 1)) AS u
FIX #1 - No subselect
SELECT
STRUCT(RAND() AS r, ARRAY(SELECT 1) AS a)
FROM UNNEST(GENERATE_ARRAY(0, 5, 1)) AS u
FIX #2 - Select only r
SELECT
(SELECT AS STRUCT RAND() AS r, ARRAY(SELECT 1) AS a).r
FROM UNNEST(GENERATE_ARRAY(0, 5, 1)) AS u
Fix #3 - Array contains "u"
SELECT
(SELECT AS STRUCT RAND() AS r, ARRAY(SELECT u) AS a).r
FROM UNNEST(GENERATE_ARRAY(0, 5, 1)) AS u
Haven't understood why first query didn't work but I have a simpler version that works for you:
SELECT (
SELECT array_agg(RAND()) AS random
FROM UNNEST(GENERATE_ARRAY(0, 10, 1)) AS _time
) AS random_for_times
FROM UNNEST(GENERATE_ARRAY(0, 10, 1))
Update: I later realized that the problem is with ARRAY(subquery), as long as you can avoid using it for your case (like in my query above), you should be fine.
I have table called arraytable -> create table arraytable(id int, somearray int[][])
INSERT INTO arraytable(id, somearray) values(1,array[[3,5],[4,12]]);
INSERT INTO arraytable(id, somearray) values(2,array[[7,15],[13,47],[15,27],[18,97]]);
INSERT INTO arraytable(id, somearray) values(3,array[[56,1],[67,78],[105,78]]);
I have no idea how to select second index values of array elements in all rows according to particular first index values of array elements;
First, I want to select 6 array elements that have first index values smaller than 67 which would look like:
[4,12],[7,15],[13,47],[15,27],[18,97],[56,1]
And now I need to select second index values of these which would look like:
12, 15, 47, 27, 97, 1.
This is so ugly I'm sure there is a better way to do this but since nobody has answered this question, I'll post this answer bearing in mind that I don't think it's a good solution and a stable one. Don't use this in production! It's merely an exercise for me with my very limited knowledge about how multidimensional arrays work in Postgres.
It's just for the sake of answering:
with x as (
select foo.id, goo.nr, goo.first_ind, foo.somearray
from (
select *, somearray[1:array_upper(somearray,1)][1] AS first_indexes
from arraytable
) foo,
unnest(foo.first_indexes) WITH ORDINALITY goo(first_ind,nr)
where goo.first_ind < 67
)
select string_agg(z.second_ind::text, ', ' order by x.id, x.nr)
from x
join (
select *
from (
select id, first_ind, somearray[1:array_upper(somearray,1)][2:3] AS second_indexes
-- [2:3] should actually be [2] but for some reason it wouldn't work? so this is specific to data with 2 indexes
from x
) y,
unnest(y.second_indexes) WITH ORDINALITY goo(second_ind,nr)
) z on
x.id=z.id
and x.nr=z.nr
and x.first_ind=z.first_ind;
Output:
string_agg
--------------------
5, 12, 15, 47, 27, 97, 1
(1 row)
Also, {3,5} should be taken into consideration so 5 should be present in the output.
Is there any way to combine data types in SQL Server? i.e. have the value of a cell to be both text and number where the text is always the same?
I have a table called Contract. The ContractID field should have the value of: 'TCwxyz' where 'TC' are string characters and 'wxyz' are integers.
I have the following but it doesn't seem to be working:
CREATE TYPE TenantContracts AS CHAR(6)
CHECK (SUBSTRING(VALUE,1,2)='TC'
AND (SUBSTRING(VALUE,2,4) AS SMALLINT)
Any assistance would be appreciated.
Constraints are added to the table definition, you don't need to create a type.
ALTER TABLE Contract
ADD CONSTRAINT chk_ContractID CHECK (
SUBSTRING(ContractID, 1, 2) = 'TC'
AND ISNUMERIC(SUBSTRING(ContractID, 3, 4)) = 1
)
This solution will accept a few incorrect values, for instance TC1.01. I'd just use this for the virtue of simplicity though, rather than trying to determine if the last 4 digits are an integer, which gets surprisingly tricky (T-sql - determine if value is integer).
Edit: If you did want to make a more robust integer check, perhaps best would be to simply check individually if each of the last 4 characters is numeric:
ALTER TABLE Contract
ADD CONSTRAINT chk_ContractID CHECK (
SUBSTRING(ContractID, 1, 2) = 'TC'
AND ISNUMERIC(SUBSTRING(ContractID, 3, 1)) = 1
AND ISNUMERIC(SUBSTRING(ContractID, 4, 1)) = 1
AND ISNUMERIC(SUBSTRING(ContractID, 5, 1)) = 1
AND ISNUMERIC(SUBSTRING(ContractID, 6, 1)) = 1
)
I'm trying to run this query:
SELECT
USER_KEY, CHAR_KEY, CONVERT(VARCHAR,substring(char_data, 9, 16)) as CHAR_NAME
FROM CHAR_DATA0
WHERE CONVERT(VARCHAR,substring(char_data, 9, 16)) LIKE '%BrightSide08'
Which returns me nothing. (I don't understand why)
But changing the query to
SELECT
USER_KEY, CHAR_KEY, CONVERT(VARCHAR,substring(char_data, 9, 16)) as CHAR_NAME
FROM CHAR_DATA0
WHERE CONVERT(VARCHAR,substring(char_data, 9, 16)) LIKE '%BrightSide08%'
Note that the only change is ...LIKE '%BrightSide08%'
This query now returns 1 row with the data:
21045 300434 BrightSide08
examples:
(I only need the wild card to be at the beginning because)
I want the following:
0BrightSide08
1BrightSide08
But not:
0BrightSide082
1BrightSide083
This is char_data
0x0600700701003800427269676874536964653038000000000401040024002900870000006126001E0000000000000000000000007526211E0000000000000000000000006B26021E0000000000000000000000007F26031E0000000000000000000000008C26041E0000000000000000000000009A26051E0000000000000000000000009F1F000014FE180079704700A83F0000EA47193000000000000000000000F102FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF16220B00000000000000000000000000E82210000000000000000000000000006722060000000000000000000000000097221800000000000000000000000000832202000000000000000000000000000000DC055802DC055C025802D007370000000600891300009B3300004D6400004D640000BCAC050076D71F00E462362D1C1300006E600A139A58000020060000000000000000000000000000DC1701000000000000000000000000004712022B0000000000000000000000004B1203320000000000000000000001004B1204130000000000000000000001004B1205320000000000000000000001003D120600000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4712080C0000000000000000000000000106090500000000000000000000010047120A190000000000000000000001004B120B0A000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC1210E09000000000000000000000202C1210E09000000000000000000000202210610000000000000000000000000000A3111000000000000000000000000004D221213000000000000000000004C004D221213000000000000000000004C00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC1210E09000000000000000000000202C1210E0900000000000000000000020236121800000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4D221213000000000000000000004C004D221213000000000000000000004C00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFDA221D06000000000000000000004701DA221D06000000000000000000004701FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5A3D200500000000000000000000000057222105000000000000000000008201572221050000000000000000000082012F22231400000000000000000000F4012F22231400000000000000000000F401DA221D06000000000000000000004701DA221D06000000000000000000004701FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5A3D200500000000000000000000000057222105000000000000000000008201572221050000000000000000000082012F22231400000000000000000000F4012F22231400000000000000000000F401FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFB373011000000000000000000000000FB373011000000000000000000000000953C3202000000000000000000000000953C32020000000000000000000000001438341D0000000000000000000000001438341D000000000000000000000000B5383690000000000000000000000000B5383690000000000000000000000000FB373011000000000000000000000000FB373011000000000000000000000000953C3202000000000000000000000000953C32020000000000000000000000001438341D0000000000000000000000001438341D000000000000000000000000B5383690000000000000000000000000B5383690000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF31000000C6340000000000000200000000000000040000000000000000000000000000000000000000000000000000000D0000009C00000069CF7F0000000000FFFF000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Actually, I spoke too soon in my comment. Defining an explicit length for the portion of the substring you're going after does prevent the volatility of varchar without length from interfering with your query:
DECLARE #x TABLE(y VARBINARY(MAX));
INSERT #x VALUES(
0x0600700701003800427269676874536964653038000000000401040024002900870000006126001E0000000000000000000000007526211E0000000000000000000000006B26021E0000000000000000000000007F26031E0000000000000000000000008C26041E0000000000000000000000009A26051E0000000000000000000000009F1F000014FE180079704700A83F0000EA47193000000000000000000000F102FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF16220B00000000000000000000000000E82210000000000000000000000000006722060000000000000000000000000097221800000000000000000000000000832202000000000000000000000000000000DC055802DC055C025802D007370000000600891300009B3300004D6400004D640000BCAC050076D71F00E462362D1C1300006E600A139A58000020060000000000000000000000000000DC1701000000000000000000000000004712022B0000000000000000000000004B1203320000000000000000000001004B1204130000000000000000000001004B1205320000000000000000000001003D120600000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4712080C0000000000000000000000000106090500000000000000000000010047120A190000000000000000000001004B120B0A000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC1210E09000000000000000000000202C1210E09000000000000000000000202210610000000000000000000000000000A3111000000000000000000000000004D221213000000000000000000004C004D221213000000000000000000004C00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC1210E09000000000000000000000202C1210E0900000000000000000000020236121800000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4D221213000000000000000000004C004D221213000000000000000000004C00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFDA221D06000000000000000000004701DA221D06000000000000000000004701FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5A3D200500000000000000000000000057222105000000000000000000008201572221050000000000000000000082012F22231400000000000000000000F4012F22231400000000000000000000F401DA221D06000000000000000000004701DA221D06000000000000000000004701FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5A3D200500000000000000000000000057222105000000000000000000008201572221050000000000000000000082012F22231400000000000000000000F4012F22231400000000000000000000F401FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFB373011000000000000000000000000FB373011000000000000000000000000953C3202000000000000000000000000953C32020000000000000000000000001438341D0000000000000000000000001438341D000000000000000000000000B5383690000000000000000000000000B5383690000000000000000000000000FB373011000000000000000000000000FB373011000000000000000000000000953C3202000000000000000000000000953C32020000000000000000000000001438341D0000000000000000000000001438341D000000000000000000000000B5383690000000000000000000000000B5383690000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF31000000C6340000000000000200000000000000040000000000000000000000000000000000000000000000000000000D0000009C00000069CF7F0000000000FFFF000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
);
SELECT CONVERT(VARCHAR, SUBSTRING(y, 9, 16)) FROM #x
WHERE CONVERT(VARCHAR, SUBSTRING(y, 9, 16)) LIKE '%BrightSide08';
-- 0 rows
SELECT CONVERT(VARCHAR, SUBSTRING(y, 9, 16)) FROM #x
WHERE CONVERT(VARCHAR(12), SUBSTRING(y, 9, 16)) LIKE '%BrightSide08';
-- 1 row
Now, whether you should be using 12 or something else depends on what all of the possible values might be and where they may be embedded in this specific slot in your varbinary value. If you can give some more specific examples we can help further, but in the meantime, it pays to be specific rather than wishy-washy when declaring varchar.
The most likely reason is that there is some unseen character after the 08. One way this could occur is if the field is defined as a char; it would then be padded with spaces.
One way to see if there are any such values is to append characters to delimit the value:
select '|'+char_data+'|'
. . .
You have another problem in your query. You are using varchar without a length. Bad, bad, bad. In fact, the convert doesn't seem to be needed at all. You can just do:
substring(char_data, 9, 16) LIKE '%BrightSide08'
But this is equivalent to:
left(char_data, 25) LIKE '%BrightSide08'
Or, because you are looking for values at the end of the field:
right(char_data, 12) = 'BrightSide08'
For example, if I have these 2 Documents:
id: 1
multifield: 2, 5
id: 2
multifield: 2, 5, 9
Then say I have a set that I'm querying with, which is {2, 5, 7}. What I would want is document 1 returned because 2 and 5 are both contained in the set. But document 2 should not be returned because 9 is not in the set.
Both the multivalued field and my set are of arbitrary length. Hopefully that makes sense.
Figured this out. This was the inspiration, specifically the answer suggesting to use Function Queries.
Using the same data in the question, I will add a calculated field to my documents which contains the number of values in my multivalued field.
id: 1
multifield: 2, 5
nummultifield: 2
id: 2
multifield: 2, 5, 9
nummultifield: 3
Then I'll use an frange with some function queries. For each item in my set, I'll use the termfreq function which will return 1 or 0. I will then sum up all of these values. Finally, if that sum equals the calculated field nummultifield, then I know that for that document, every value in the document is present in the set. Remember my set is 2,5,7 so my function query will look something like this:
fq={!frange l=0 u=0}sub( nummultifield, sum( termfreq(multifield,2), termfreq(multifield,5), termfreq(multifield,7)))
If we fill in the values for Document 1 and 2, it will look like this:
Document 1: sub( 2, sum( 1,1,0 ) ) = 0 ' in my range of {0,0} so Doc 1 is returned
Document 2: sub( 3, sum( 1,1,0 ) ) = 1 ' not in the range of {0,0} so not returned
I've tested it out and it works great. You need to make sure you don't duplicate any values in multifield or you'll get weird results. Incidentally, this trick of using frange could be used whenever you want to fake a boolean result from one or more function queries.
Faceting may be the what you are looking for.
http://wiki.apache.org/solr/SolrFacetingOverview
http://www.lucidimagination.com/devzone/technical-articles/faceted-search-solr
how to search for more than one facet in solr?
I adapted this from the Lucid Imagination link.
Choose all documents that have values 2 or 5 or 7:
http://localhost:8983/solr/select?q=*
&facet=on
&facet.field=multifield
&fq=multifield:2
&fq=multifield:5
&fq=multifield:7
Incomplete: I dont know any options to exclude all other values.