i have 2 tables; table 1: contain a wildcard paths and table 2: files with full path;
i want to select all files that match wild card path
example:
table1:
| type | path |
| sys | /etc/* |
| protected | /etc/* |
| sys | /sys/* |
| log | /log/* |
table2:
| file | path |
| f1.cmd | /etc/folder/name |
| f2.cmd | /log/folder/name |
| f3.cmd | /etc/folder/name |
| f4.cmd | /sys/folder/name |
my ultimate goal is: to create a VIEW that has all data from table2 and add one more column type to tell me which type does this file belongs to.
so that i can select all files that is of type = sys for example
** what i tried:**
step 1: get list of all paths of wanted type.
select array_agg(replace(path,'*','%')) from
table1 where type = 'sys'
group by type
this will result with something like {"etc\\%","sys\\%"}
step 2 select files using like any
select * from symbols where path like any ( array['etc\\%', 'sys\\%'] )
this successfully returned all files with paths like one i need.
now quesiton is how can i combine both queries into one :D ?
or is there an easier way using JOIN for example.
thanks
You could get the table1.type from each row in table2 by checking if table1.path is a substring of table2.path:
with table1(type, path) as (
values ('sys', '/etc/*'),
('sys', '/sys/*'),
('log', '/log/*'),
('etc', '/etc/*')
),
table2(file, path) as (
values ('f1.cmd', '/etc/folder/name'),
('f2.cmd', '/log/folder/name'),
('f3.cmd', '/etc/folder/name'),
('f4.cmd', '/sys/folder/name')
)
select *,
(select type
from table1
where position(replace(path, '*', '') in table2.path) > 0
limit 1) as type
from table2;
file | path | type
--------+------------------+------
f1.cmd | /etc/folder/name | sys
f2.cmd | /log/folder/name | log
f3.cmd | /etc/folder/name | sys
f4.cmd | /sys/folder/name | sys
(4 rows)
Related
I am currently trying to work through this and I'm unsure as to how to proceed. I have the below data
ID
name
value
One
a,b,c
10,20,30
I would like to turn it into
| ID | properties.name | properties.value |
|:---- |:------: | -----: |
| One | a | 10 |
| | b | 20 |
| | c | 30 |
The below query looked like it was working but instead of having an array it created a nested record with 2 array fields.
SELECT ID
name
, value
, array (
select as struct
split(name, ',') as name
, split(value, ',') as value
) as properties
FROM `orders`
Consider below approach
select id, array(
select as struct name, value
from unnest(split(name)) name with offset
join unnest(split(value)) value with offset
using(offset)
) as properties
from `orders`
if applied to sample data in your question - output is
I would like to have something like a procedure that takes a query definition as input and output a set of tables containing the individual elements of the query.
Searching the internet for this yields me numerous results in various programming languages, but not in tsql itself. Is there such a resource around?
An example in order to illustrate what I mean by parser:
Input example (any query, really:)
'select t1.col1,t2.col2
from table1 t1
inner join table2.col2
on t1.t2ref=t2.key'
The output, of course, will be a multitude of data. I mentioned tables, but it could be in any form eg an xml. Here is a VERY SIMPLISTIC and arbitrary example of decomposition for the query above:
tables_used:
+----+-----------+--------+------------+
| id | object_id | name | alias used |
+----+-----------+--------+------------+
| 1 | 43252345 | table1 | t1 |
| 2 | 6542625 | table2 | t2 |
+----+-----------+--------+------------+
columns_used:
+----------+-------------+
| table_id | column name |
+----------+-------------+
| 1 | col1 |
| 1 | t2ref |
| 2 | key |
| 2 | col2 |
+----------+-------------+
joins_used:
+-----+-----+-------+-----------------+
| tb1 | tb2 | type | on |
+-----+-----+-------+-----------------+
| 1 | 2 | inner | t1.t2ref=t2.key |
+-----+-----+-------+-----------------+
I use PostgreSQL 10.1 and:
CREATE TABLE human
(
id ... NOT NULL,
gender ...,
height ...,
weight ...,
eye ...,
hair ...,
...
);
I have an input form through which I insert the data. I wish an elegant and proper way by which I can SELECT which columns required to be DISPLAYED in that form, something like weight ... DISPLAYED, or eye ... NOT DISPLAYED, .
One way is to correspond NULL with DISPLAYED (when NOT NULL then display it, or when NULL then do not display it) and use information_schema which (corresponding) makes me no so happy:
Another way is to:
CREATE TABLE human_column
(
id ... NOT NULL,
characteristic character varying(...),
is_displayed boolean
);
where characteristic data are the names of the columns of human table.
Is there a better way to add a direct foreign attribute to the columns of a table? (In 51.7. pg_attribute there is a column named attoptions. Would it be used?)
specifying "options" for columns to define if they will be "displayed" or not seems a little overhead. Imagine you keep such list in human_column. To modify it you would need to update it with new is_displayed values. Then you would need to build column list to be selected in query.
When you create a view, you do the same (specify a list of columns to be displayed) and then you can just query the view, without need to dynamically build the query. Also you can always check the current list of included columns from catalog or information_schema.
The only "not cosy" feature of a view - you can't change columns in it, thus you have to drop and create it again.
drop/create view on demand looks cheaper to me then dynamically building query with list of columns on each select still.
demo:
db=# create view v as select oid,datname from pg_database;
CREATE VIEW
db=# select * from v;
oid | datname
-------+-----------
13505 | postgres
16384 | t
1 | template1
13504 | template0
16419 | o
(5 rows)
checking list of columns:
db=# select column_name,ordinal_position,column_default,is_nullable,data_type,character_maximum_length from information_schema.columns where table_name = 'v';
column_name | ordinal_position | column_default | is_nullable | data_type | character_maximum_length
-------------+------------------+----------------+-------------+-----------+--------------------------
oid | 1 | | YES | oid |
datname | 2 | | YES | name |
(2 rows)
same for original table:
db=# select column_name,ordinal_position,column_default,is_nullable,data_type,character_maximum_length from information_schema.columns where table_name = 'pg_database';
column_name | ordinal_position | column_default | is_nullable | data_type | character_maximum_length
---------------+------------------+----------------+-------------+-----------+--------------------------
datname | 1 | | NO | name |
datdba | 2 | | NO | oid |
encoding | 3 | | NO | integer |
datcollate | 4 | | NO | name |
datctype | 5 | | NO | name |
datistemplate | 6 | | NO | boolean |
datallowconn | 7 | | NO | boolean |
datconnlimit | 8 | | NO | integer |
datlastsysoid | 9 | | NO | oid |
datfrozenxid | 10 | | NO | xid |
datminmxid | 11 | | NO | xid |
dattablespace | 12 | | NO | oid |
datacl | 13 | | YES | ARRAY |
(13 rows)
I have a table named VPX_EVENT_ARG where a column, ARG_DATA, contains XML values.
Table 1
+----------+-------------------+----------+
| EVENT_ID | ARG_TYPE | ARG_DATA |
+----------+-------------------+----------+
| 7121001 | vim.vm.ConfigSpec | XML1 |
| 7121002 | vim.vm.ConfigSpec | XML2 |
| 7121003 | vim.vm.ConfigSpec | XML3 |
+----------+-------------------+----------+
XML1, XML2 and XML3 are XML values. They are too long to type in the table. Here are the real values. Actually they appear in one line.
XML1, for example,
<obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="5.5" xsi:type="VirtualMachineConfigSpec"><changeVersion>2015-09-24T10:02:53.866694Z</changeVersion><files><vmPathName>ds:///vmfs/volumes/54e5d10c-c527b7f3-7eea-a0d3c1f01404/CommVault VM Test/CommVault VM Test.vmx</vmPathName></files><deviceChange><operation>remove</operation><device xsi:type="VirtualDisk"><key>2003</key><deviceInfo><label>Hard disk 4</label><summary>20,971,520 KB </summary></deviceInfo><backing xsi:type="VirtualDiskFlatVer2BackingInfo"><fileName>ds:///vmfs/volumes/54e5d10c-c527b7f3-7eea-a0d3c1f01404/CommVault VM Test/CommVault VM Test_2.vmdk</fileName><diskMode>persistent</diskMode><split>false</split><writeThrough>false</writeThrough><thinProvisioned>false</thinProvisioned><uuid>6000C29b-e652-b5fe-76fa-18f6de988807</uuid><contentId>5bd085f0f9391346751e1e7efffffffe</contentId><digestEnabled>false</digestEnabled></backing><controllerKey>1000</controllerKey><unitNumber>3</unitNumber><capacityInKB>20971520</capacityInKB><shares><shares>1000</shares><level>normal</level></shares><storageIOAllocation><limit>-1</limit><shares><shares>1000</shares><level>normal</level></shares></storageIOAllocation></device></deviceChange></obj>
I will separate XML1 into multiple lines so that it will be easier to read.
<obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="5.5" xsi:type="VirtualMachineConfigSpec"><changeVersion>2015-09-24T10:02:53.866694Z</changeVersion><files><vmPathName>ds:///vmfs/volumes/54e5d10c-c527b7f3-7eea-a0d3c1f01404/CommVault VM Test/CommVault VM Test.vmx</vmPathName></files>
<deviceChange>
<operation>remove</operation>
<device xsi:type="VirtualDisk">
<key>2003</key>
<deviceInfo>
<label>Hard disk 4</label>
<summary>20,971,520 KB </summary>
</deviceInfo>
<backing xsi:type="VirtualDiskFlatVer2BackingInfo">
<fileName>ds:///vmfs/volumes/54e5d10c-c527b7f3-7eea-a0d3c1f01404/CommVault VM Test/CommVault VM Test_2.vmdk
</fileName>
<diskMode>persistent</diskMode>
<split>false</split>
<writeThrough>false</writeThrough>
<thinProvisioned>false</thinProvisioned>
<uuid>6000C29b-e652-b5fe-76fa-18f6de988807</uuid>
<contentId>5bd085f0f9391346751e1e7efffffffe</contentId>
<digestEnabled>false</digestEnabled>
</backing>
<controllerKey>1000</controllerKey>
<unitNumber>3</unitNumber>
<capacityInKB>20971520</capacityInKB>
<shares><shares>1000</shares><level>normal</level></shares><storageIOAllocation><limit>-1</limit><shares><shares>1000</shares><level>normal</level></shares></storageIOAllocation>
</device>
</deviceChange>
</obj>
I would like to extract XML1 into tables using an MSSQL query.
DECLARE #xml XML
SET #xml = (SELECT ARG_DATA FROM VPX_EVENT_ARG WHERE ARG_ID = 1 AND EVENT_ID = 7121001); --EVENT_ID is fixed.
WITH XMLNAMESPACES('urn:vim25' AS NS)
SELECT
'diskUnit' = ref.value('./NS:device[1]/NS:unitNumber[1]', 'INT'),
'operation' = ref.value('./NS:operation[1]', 'NVARCHAR(100)'),
'newSizeKB' = ref.value('./NS:device[1]/NS:capacityInKB[1]', 'BIGINT')
FROM #xml.nodes('/NS:obj/NS:deviceChange') data(ref)
Well, I get this as a result.
Table 2
+------+-----------+-----------+
| unit | operation | newSizeKB |
+------+-----------+-----------+
| 1 | edit | 24117248 |
| 2 | edit | 108003328 |
| 3 | add | 20971520 |
+------+-----------+-----------+
You can see that Table 2 is just the result of the first row in Table 1. Not even the first row, it is just ARG_DATA on the first row which fixes EVENT_ID as well. I wish anyone can help.
Question 1: I would like to consolidate multiple rows of XML values into one table without fixing EVENT_IDs and put EVENT_IDs into a column too. Please assume that VPX_EVENT_ARG table contains hundreds of rows.
Table3
+----------+------+-----------+-----------+
| EVENT_ID | unit | operation | newSizeKB |
+----------+------+-----------+-----------+
| 7121001 | 1 | edit | 24117248 |
| 7121001 | 2 | edit | 108003328 |
| 7121001 | 3 | add | 20971520 |
| 7121002 | 1 | edit | 1048576 |
| 7121002 | 3 | edit | 52428800 |
| 7121003 | 3 | edit | 125829120 |
| 7121003 | 5 | remove | 83886080 |
+----------+------+-----------+-----------+
Question 2: Is there a way to use the query without setting XML? I need to set XML for the query, then I can use nodes().
SET #xml = (SELECT ARG_DATA FROM VPX_EVENT_ARG WHERE ARG_ID = 1 AND EVENT_ID = 7121001);
.
.
FROM #xml.nodes('/NS:obj/NS:deviceChange') data(ref)
I wonder if it can be done like this.
FROM (SELECT ARG_DATA FROM VPX_EVENT_ARG WHERE ARG_ID = 1 AND EVENT_ID = 7121001).nodes('/NS:obj/NS:deviceChange') data(ref)
Error: Incorrect syntax near '.'. Expecting AS, ID, or QUOTED_ID.
I really want to use such a result as Table3 to join the table I have queried before. I have hard time dealing with XML but stuck at this so long. By the way, I cannot update the table; it is restricted.
Of couse, you can. Use cross apply on your nodes:
;WITH XMLNAMESPACES('urn:vim25' AS NS)
SELECT
v.EVENT_ID,
'diskUnit' = ref.value('./NS:device[1]/NS:unitNumber[1]', 'INT'),
'operation' = ref.value('./NS:operation[1]', 'NVARCHAR(100)'),
'newSizeKB' = ref.value('./NS:device[1]/NS:capacityInKB[1]', 'BIGINT')
FROM VPX_EVENT_ARG AS v
CROSS APPLY v.ARG_DATA.nodes('/NS:obj/NS:deviceChange') data(ref)
If your column is not XML, we need a subquery to convert it:
;WITH XMLNAMESPACES('urn:vim25' AS NS)
SELECT
v.EVENT_ID,
'diskUnit' = ref.value('./NS:device[1]/NS:unitNumber[1]', 'INT'),
'operation' = ref.value('./NS:operation[1]', 'NVARCHAR(100)'),
'newSizeKB' = ref.value('./NS:device[1]/NS:capacityInKB[1]', 'BIGINT')
FROM
(
SELECT
c.EVENT_ID,
c.ARG_TYPE,
CONVERT(xml,c.ARG_DATA) AS ARG_DATA,
c.ARG_ID
FROM VPX_EVENT_ARG AS c
) AS v
CROSS APPLY v.ARG_DATA.nodes('/NS:obj/NS:deviceChange') data(ref)
Sorry I know that's a rubbish Title but I couldn't think of a more concise way of describing the issue.
I have a (MSSQL 2008) table that contains telephone numbers:
| CustomerID | Tel1 | Tel2 | Tel3 | Tel4 | Tel5 | Tel6 |
| Cust001 | 01222222 | 012333333 | 07111111 | 07222222 | 01222222 | NULL |
| Cust002 | 07444444 | 015333333 | 07555555 | 07555555 | NULL | NULL |
| Cust003 | 01333333 | 017777777 | 07888888 | 07011111 | 016666666 | 013333 |
I'd like to:
Remove any duplicate phone numbers
Rearrange the telephone numbers so that anything beginning with "07" is the first phone number. If there are multiple 07's, they should be in the first fields. The order of the numbers apart from that doesn't really matter.
So, for example, after processing, the table would look like:
| CustomerID | Tel1 | Tel2 | Tel3 | Tel4 | Tel5 | Tel6 |
| Cust001 | 07111111 | 07222222 | 01222222 | 012333333 | NULL | NULL |
| Cust002 | 07444444 | 07555555 | 015333333 | NULL | NULL | NULL |
| Cust003 | 07888888 | 07011111 | 016666666 | 013333 | 01333333 | 017777777 |
I'm struggling to figure out how to efficiently achieve my goal (there are 600,000+ records in the table). Can anyone help?
I've created a fiddle if it'll help anyone play around with the scenario.
You can break up the numbers into individual rows using UNPIVOT, then reorder them based on the occurence of the '07' prefix using ROW_NUMBER(), and finally recombine it using PIVOT to end up with the 6 Tel columns again.
select *
FROM
(
select CustomerID, Col, Tel
FROM
(
select *, Col='Tel' + RIGHT(
row_number() over (partition by CustomerID
order by case
when Tel like '07%' then 1
else 2
end),10)
from phonenumbers
UNPIVOT (Tel for Seq in (Tel1,Tel2,Tel3,Tel4,Tel5,Tel6)) seqs
) U
) P
PIVOT (MAX(TEL) for Col IN (Tel1,Tel2,Tel3,Tel4,Tel5,Tel6)) V;
SQL Fiddle
Perhaps using cursor to collect all customer id and sorting the fields...traditional sorting technique as we used to do in school c++ ..lolz...like to know if any other method possible.
If you dont get any then it is the last way . It will take a long time for sure to execute.