Select dbms_metadata.get_ddl without select_catalog_role - database

I am trying to execute the query
SELECT dbms_metadata.get_ddl('TABLE','PRODUCT','INVENTORY') FROM DUAL;
which the giving me error -
ORA-31603: object "PRODUCT" of type TABLE not found in schema "INVENTORY"
ORA-06512: at "SYS.DBMS_METADATA", line 6781
Use of select_catalog_role and select any dictionary is not allowed by DBA policy on service account. So I must provide granular role/previliage to service account so I can execute the select metadata query.

Create a fake DBMS_METADATA on the DEBEZIUM schema that calls a definer's rights function on INVENTORY that calls the real DBMS_METADATA. This workaround assumes that Debezium calls the package DBMS_METADATA instead of using the fully-qualified name SYS.DBMS_METADATA. Oracle uses schema objects before public synonyms, so this custom package let's you hijack some calls to the real SYS.DBMS_METADATA.
Yes, this is an overly complicated mess that should only be used as a last resort. Perhaps instead of actually using this code, you can just show it to the DBA to convince them that you need the right privilege.
As INVENTORY, create a package that mimics part of SYS.DBMS_METADTATA, and grant it to DEBEZIUM:
create or replace package inventory.dbms_metadata_without_privs authid definer is
/*
This package exists because DEBEZIUM cannot be granted SELECT_CATALOG_ROLE because
[enter reason here]
Note that this package is "DEFINER" instead of the "CURRENT_USER" of the real DBMS_METADATA.
*/
FUNCTION get_ddl (
object_type IN VARCHAR2,
name IN VARCHAR2,
schema IN VARCHAR2 DEFAULT NULL,
version IN VARCHAR2 DEFAULT 'COMPATIBLE',
model IN VARCHAR2 DEFAULT 'ORACLE',
transform IN VARCHAR2 DEFAULT 'DDL')
RETURN CLOB;
PROCEDURE set_transform_param (
transform_handle IN NUMBER,
name IN VARCHAR2,
value IN BOOLEAN DEFAULT TRUE,
object_type IN VARCHAR2 DEFAULT NULL,
flags IN NUMBER DEFAULT 0);
end dbms_metadata_without_privs;
/
create or replace package body inventory.dbms_metadata_without_privs is
FUNCTION get_ddl (
object_type IN VARCHAR2,
name IN VARCHAR2,
schema IN VARCHAR2 DEFAULT NULL,
version IN VARCHAR2 DEFAULT 'COMPATIBLE',
model IN VARCHAR2 DEFAULT 'ORACLE',
transform IN VARCHAR2 DEFAULT 'DDL')
RETURN CLOB IS
BEGIN
RETURN SYS.DBMS_METADATA.get_ddl(object_type => object_type,
name => name,
schema => schema,
version => version,
model => model,
transform => transform);
END;
PROCEDURE set_transform_param (
transform_handle IN NUMBER,
name IN VARCHAR2,
value IN BOOLEAN DEFAULT TRUE,
object_type IN VARCHAR2 DEFAULT NULL,
flags IN NUMBER DEFAULT 0) IS
BEGIN
SYS.DBMS_METADATA.set_transform_param(transform_handle => transform_handle,
name => name,
value => value,
object_type => object_type,
flags => flags);
END;
end dbms_metadata_without_privs;
/
grant execute on inventory.dbms_metadata_without_privs to debezium;
As DEBEZIUM, create a fake DBMS_METADATA that simply calls the package in INVENTORY:
CREATE OR REPLACE NONEDITIONABLE PACKAGE DEBEZIUM.dbms_metadata AUTHID CURRENT_USER AS
/*
This fake DBMS_METADATA exists because DEBEZIUM cannot be granted SELECT_CATALOG_ROLE because
[enter reason here]
Yes, this is a stupid idea for many reasons...
*/
--------------------
-- PUBLIC CONSTANTS
--
SESSION_TRANSFORM CONSTANT BINARY_INTEGER := -1;
FUNCTION get_ddl (
object_type IN VARCHAR2,
name IN VARCHAR2,
schema IN VARCHAR2 DEFAULT NULL,
version IN VARCHAR2 DEFAULT 'COMPATIBLE',
model IN VARCHAR2 DEFAULT 'ORACLE',
transform IN VARCHAR2 DEFAULT 'DDL')
RETURN CLOB;
PROCEDURE set_transform_param (
transform_handle IN NUMBER,
name IN VARCHAR2,
value IN BOOLEAN DEFAULT TRUE,
object_type IN VARCHAR2 DEFAULT NULL,
flags IN NUMBER DEFAULT 0);
END DBMS_METADATA;
/
CREATE OR REPLACE NONEDITIONABLE PACKAGE BODY DEBEZIUM.dbms_metadata AS
FUNCTION get_ddl (
object_type IN VARCHAR2,
name IN VARCHAR2,
schema IN VARCHAR2 DEFAULT NULL,
version IN VARCHAR2 DEFAULT 'COMPATIBLE',
model IN VARCHAR2 DEFAULT 'ORACLE',
transform IN VARCHAR2 DEFAULT 'DDL')
RETURN CLOB
IS
BEGIN
RETURN inventory.dbms_metadata_without_privs.get_ddl(object_type => object_type,
name => name,
schema => schema,
version => version,
model => model,
transform => transform);
END;
PROCEDURE set_transform_param (
transform_handle IN NUMBER,
name IN VARCHAR2,
value IN BOOLEAN DEFAULT TRUE,
object_type IN VARCHAR2 DEFAULT NULL,
flags IN NUMBER DEFAULT 0)
IS
BEGIN
inventory.dbms_metadata_without_privs.set_transform_param(transform_handle => transform_handle,
name => name,
value => value,
object_type => object_type,
flags => flags);
END;
END DBMS_METADATA;
/
As DEBEZIUM, you can now run these statements:
select dbms_metadata.get_ddl('TABLE', 'PRODUCT', 'INVENTORY') from dual;
begin dbms_metadata.set_transform_param(DBMS_METADATA.SESSION_TRANSFORM, 'DEFAULT'); end;
/
Most of the above code was taken straight from the DBMS_METADATA package. If you need to implement more functionality from that package, you can use the below query to copy and paste the procedure definitions. (Oddly, DBMS_METADATA does not work correctly on DBMS_METADATA.)
select *
from all_source
where owner = 'SYS'
and name = 'DBMS_METADATA'
and type = 'PACKAGE'
order by line;

Related

Internal vs external data type in oracle 19c

I read long Oracle documentation , still could not understand what is difference and actual meaning of Internal vs External datatype. In oracle 19c I read internal vs external datatype.
Can somebody suggest, how below code is not compatible with oracle19c..it is just header. I think some datatype issue is there but not able to figure out.
I am not able to understand how below package specification is not compatible with oracle19c. This file have been scanned invalid. Oracle docs is not helpful
CREATE OR REPLACE PACKAGE ABC_ECC_DATASECURITY_PKG_PUB AS
/* $Header: ABCCCDSS.pls 120.0.12020000.2 2020/11/17 21:49:12 mgarg noship $ */
PROCEDURE GetFilterAttributeValues
( p_dataset_key IN VARCHAR2,
p_user_id IN NUMBER,
p_org_id IN NUMBER,
p_resp_id IN NUMBER,
p_resp_app_id IN NUMBER,
p_sec_group_id IN NUMBER,
p_params IN ecc_sec_field_values DEFAULT NULL,
x_return_status OUT NOCOPY VARCHAR2,
x_return_message OUT NOCOPY VARCHAR2,
x_sec_filter OUT NOCOPY CLOB
);
END ABC_ECC_DATASECURITY_PKG_PUB;
/
COMMIT;
EXIT;
Can somebody suggest me oracle internal and external datatype and any help in above specification?
I read this in oracle docs.
https://docs.oracle.com/en/database/oracle/oracle-database/19/lnoci/data-types.html#GUID-D69455D9-CE01-44CC-B5A9-E541C7774805
here it is mentioned internal vs external datatype.
https://docs.oracle.com/en/database/oracle/oracle-database/19/lnoci/data-types.html#GUID-027FB2E2-593C-43F1-9184-DFEF7A984A27
Your package is fine, if you leave out the user defined datatype ecc_sec_field_values:
CREATE OR REPLACE PACKAGE ABC_ECC_DATASECURITY_PKG_PUB AS
PROCEDURE GetFilterAttributeValues
( p_dataset_key IN VARCHAR2,
p_user_id IN NUMBER,
p_org_id IN NUMBER,
p_resp_id IN NUMBER,
p_resp_app_id IN NUMBER,
p_sec_group_id IN NUMBER,
x_return_status OUT NOCOPY VARCHAR2,
x_return_message OUT NOCOPY VARCHAR2,
x_sec_filter OUT NOCOPY CLOB
);
END ABC_ECC_DATASECURITY_PKG_PUB;
/
Package ABC_ECC_DATASECURITY_PKG_PUB compiled
However, if you include ecc_sec_field_values, I'm getting an error:
CREATE OR REPLACE PACKAGE ABC_ECC_DATASECURITY_PKG_PUB AS
PROCEDURE GetFilterAttributeValues
( p_dataset_key IN VARCHAR2,
p_user_id IN NUMBER,
p_org_id IN NUMBER,
p_resp_id IN NUMBER,
p_resp_app_id IN NUMBER,
p_sec_group_id IN NUMBER,
p_params IN ecc_sec_field_values DEFAULT NULL,
x_return_status OUT NOCOPY VARCHAR2,
x_return_message OUT NOCOPY VARCHAR2,
x_sec_filter OUT NOCOPY CLOB
);
END ABC_ECC_DATASECURITY_PKG_PUB;
/
Error(9,35): PLS-00201: identifier 'ECC_SEC_FIELD_VALUES' must be declared
I guess this is a TYPE declared in some other file. Look for CREATE OR REPLACE TYPE ecc_sec_field_values or something.
BTW, the COMMIT seems dangerous, as the original author seems not to understand the commit behaviour of CREATE statements...

Mapping a user defined function with SSRS package parameters

I have created the following User Defined Function :
CREATE FUNCTION MyTable
(
#dd_YearNo INT,
#txt_WeekNoFrom INT,
#txt_WeekNoTo INT,
#dd_Group VARCHAR(MAX),
#dd_Team VARCHAR(MAX),
#dd_SalesPerson NVARCHAR(MAX)
)
RETURNS TABLE
AS
RETURN
SELECT *
FROM T1 (NOLOCK)
WHERE YearNo IN (#dd_YearNo)
AND (WeekNo BETWEEN #txt_WeekNoFrom AND #txt_WeekNoTo )
AND LEFT(Team,2) IN (#dd_Group)
AND Team IN (#dd_Team)
AND SalesPerson IN(#dd_SalesPerson)
I am using it as a my data set n SSRS like below :
How to map the parameters I created for my SSRS package to this user defined query. This is waht I get since I validate it :
These are the parameters I am using in my SSRS package and I am getting them from different queries :
I get the following error :
Invalid Object MyTable

another case of "Table options do not contain an option key 'connector'"

I have read the link Table options do not contain an option key 'connector'
it said we should set the format.
But My Scene is datagen->hive.
Here's my completed example(it's wrong Now)
drop table if exists datagen;
CREATE TABLE datagen (
f_sequence INT,
f_random INT,
f_random_str STRING,
ts AS localtimestamp,
WATERMARK FOR ts AS ts
) WITH (
'connector' = 'datagen',
-- optional options --
'rows-per-second'='5',
'fields.f_sequence.kind'='sequence',
'fields.f_sequence.start'='1',
'fields.f_sequence.end'='50',-- 这个地方限制了一共会产生的条数
'fields.f_random.min'='1',
'fields.f_random.max'='50',
'fields.f_random_str.length'='10'
);
SET table.sql-dialect=hive;
drop table if exists hive_table;
CREATE TABLE hive_table (
f_sequence INT,
f_random INT,
f_random_str STRING
) PARTITIONED BY (dt STRING, hr STRING, mi STRING) STORED AS parquet TBLPROPERTIES (
'partition.time-extractor.timestamp-pattern'='$dt $hr:$mi:00',
'sink.partition-commit.trigger'='partition-time',
'sink.partition-commit.delay'='1 min',
'sink.partition-commit.policy.kind'='metastore,success-file'
);
Flink SQL> insert into hive_table select f_sequence,f_random,f_random_str ,DATE_FORMAT(ts, 'yyyy-MM-dd'), DATE_FORMAT(ts, 'HH') ,DATE_FORMAT(ts, 'mm') from datagen;
[INFO] Submitting SQL update statement to the cluster...
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.table.api.ValidationException: Table options do not contain an option key 'connector' for discovering a connector.
Is the solution from above link suitable for this case?
Need your help,Thanks~!
please use SET table.sql-dialect=default; before call insert into hive_table..., the statement insert into hive_table... using datagen connector which hive dialect should not support.

Default value for TIMESTAMP column

This DDL:
CREATE TABLE example (
my_stamp TIMESTAMP_NTZ NOT NULL DEFAULT CURRENT_TIMESTAMP(0)
)
Yields this error:
SQL compilation error: Default value data type does not match data type for column MY_STAMP
Changing CURRENT_TIMESTAMP(0) to CURRENT_TIMESTAMP makes the error "go away".
Yet, this command returns successfully:
select CURRENT_TIMESTAMP(0), CURRENT_TIMESTAMP;
This would also work, I believe. You just need to cast the output of the function to the same timestamp data type as the my_stamp column:
CREATE TABLE example (
my_stamp TIMESTAMP_NTZ NOT NULL DEFAULT CURRENT_TIMESTAMP(0)::TIMESTAMP_NTZ
)

Oracle 10g Datamasking

I have Oracle 10g database. I want to mask my record of tables. It doesn't really need to make sense, it doesn't need to be readable. Just needs to be masked. For example:
select *
from customer;
LAST_NAME FIRST_NAME ADDRESS
-------------- -------------- --------------------
Doe John 10 someroad st
i convert to this :
LAST_NAME FIRST_NAME ADDRESS
-------------- -------------- --------------------
Ahd Uiea 55 xxxx ue
I need open source software that can do this work. What should i use?
You can use ORA_HASH or DBMS_CRYPTO package to full fill your requirements. Giving solution using DBMS_CRYPTO:
--Source data:
create table customer(last_name varchar2(50),first_name varchar2(50), address varchar2(200));
--Encrypt Function(Script Source):
CREATE OR REPLACE FUNCTION encrypt_value (p_in IN varchar2, p_key IN raw) RETURN raw IS l_enc_val raw (2000);
l_mod number := dbms_crypto.ENCRYPT_AES128 + dbms_crypto.CHAIN_CBC + dbms_crypto.PAD_PKCS5;
BEGIN l_enc_val := dbms_crypto.encrypt ( UTL_I18N.STRING_TO_RAW (p_in, 'AL32UTF8'), l_mod, p_key );
RETURN l_enc_val;
END;
--Function Implementation:
select encrypt_value(last_name,'AABBCC'),encrypt_value(first_name,'AABBCC'), encrypt_value(address,'AABBCC') from customer;
If you're using the Enterprise version of Oracle, you can use a Virtual Private Database (VPD) for this.
A VPD allows you to fine-grained access control (based on the account used to connect to the database). It can:
return only a subset of rows
use column masking to display sensitive columns as NULL values
It achieves this by appending a custom WHERE clause to every query run against the table. There's no way to circumvent it, and no need to adapt existing applications (for using a custom-built view etc.)
To create a VPD for your customer table, you need to:
create a function for generating the WHERE clause
create a policy for your database table
enable the policy
Function
CREATE OR REPLACE FUNCTION hide_address (
v_schema IN VARCHAR2,
v_objname IN VARCHAR2)
RETURN VARCHAR2 AS
result VARCHAR2 (200);
BEGIN
result := '1=0'; -- evaluates to FALSE for every account
RETURN (result);
END hide_address;
Creating a policy
BEGIN
DBMS_RLS.ADD_POLICY(
object_schema => 'scott',
object_name => 'customer',
policy_name => 'hide_address_policy',
policy_function => 'hide_address',
sec_relevant_cols =>' address',
sec_relevant_cols_opt => dbms_rls.ALL_ROWS);
END;
After enabling the policy, every query trying to access CUSTOMER.ADDRESS will return NULL. Depending on your requirements, you might want to add a view to access the table that returns a random address instead of NULL:
select name,
(case when address is NULL
then dbms_random.string('', 15)
else address end) as address
from
customer

Resources