There is this table in my Oracle database that is used to store audit information.
When I first did a SELECT * on that table, the audit timestamps were all on the same day, within the same hour (e.g. 18/10/2013 15:06:45, 18/10/2013 15:07:29); the next time I did it, the previous entries were gone, and the table then only contained entries with the 16:mm:ss timestamp.
I think something is acting on that table, such that every interval the table contents is/may be backed up to somewhere - I don't know where, and then the table is cleared. However, as I'm not familiar with databases, I'm not sure what is doing this.
I'd like to know how I can find out what is acting on this table, so that I can in turn retrieve the previous data I need.
EDIT:
What I've tried thus far...
SELECT * FROM DBA_DEPENDENCIES WHERE REFERENCED_NAME='MY_AUDIT_TABLE';
I got back four results, but all of which were (based on my programming skills) talking about putting data into the table, none about backing it up anywhere.
SELECT * FROM MY_AUDIT_TABLE AS OF TIMESTAMP ...
This only gives me a snapshot at a certain time, but since the table is being updated very frequently, it does not make sense for me to query every second.
The dba_dependencies view will give you an idea on what procedures, function etc will act on the table
SELECT * FROM DBA_DEPENDENCIES WHERE REFERENCED_NAME='MY_AUDIT_TABLE';
where MY_AUDIT_TABLE is the audit table name
if the table's synonym is used in the database then
SELECT * FROM DBA_DEPENDENCIES WHERE REFERENCED_NAME='MY_AUDIT_TABLE_SYNONYM';
where MY_AUDIT_TABLE_SYNONYM is the synonym for MY_AUDIT_TABLE
Or if any triggers are acting on the table
Select * from dba_triggers where table_name='MY_AUDIT_TABLE';
for external script to process the table
you can request DBA to turn on DB Fine grained audit for the table
Then query view DBA_FGA_AUDIT_TRAIL with timestamp between 15:00:00 and 16:00:00 to check the external call(OS_PROCESS column will give Operating System Process ID) or what SQL(SQL_TEXT) is executing on the table
Related
I will try to simplify my problem:
let's say that I have 2 databases, let's call them DBA and DBB,
I have this table on DBA
shopping
id - name - amount
and on my DBB I have this other table:
shopping_hist
id - name - amount
every end of the month, I generate a dump from table shopping on DBA and copy its data
on table shopping_hist on DBB, it's possible to create a trigger that for every insert on Shopping, it will also make an insert on Shopping_hist, since they are not even on the same database?
I know that if they were on the same database, even if not on the same schema, it would be possible, but I'm not finding anything to automate this when it's for distinct databases
I am working on tracking the changes in data along with few audit details like user who made the changes.
Streams in Snowflake gives delta records details and few audit columns including METADATA$ROW_ID.
Another table i.e. information_schema.query_history contain query history details including query_id, user_name, DB name, schema name etc.
I am looking for a way so that I can join query_id & METADATA$ROW_ID so that I can find the user_name corresponding to each change in data.
any lead will be much appreciated.
Regards,
Neeraj
The METADATA$ROW_ID column in a stream uniquely identifies each row in the source table so that you can track its changes using the stream.
It isn't there to track who changed the data, rather it is used to track how the data changed.
To my knowledge Snowflake doesn't track who changed individual rows, this is something you would have to build into your application yourself - by having a column like updated_by for example.
Only way i have found is to add
SELECT * FROM table(information_schema.QUERY_HISTORY_BY_SESSION()) ORDER BY start_time DESC LIMIT 1
during reports / table / row generation
Assuming that you have not changed setting that you can run more queries at same time in one session , that gets running querys id's , change it to CTE and do cross join to in last part of select to insert it to all rows.
This way you get all variables in query_history table. Also remember that snowflake does keep SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY ( and other data ) up to one year. So i recommend weekly/monthly job which merges data into long term history table. That way you an also handle access to history data much more easier that giving accountadmin role to users.
We use Oracle database and now I face a problem.
We need to copy a subset of columns from table A to a new table B:
Table A
Name|Birth Date|Location|Office
Table B
Name|Location
And table A will change quite often (several times in a month). And it's managed by another team.
What is the best way to keep synchronized the table B from table A ?
Thank you very much.
Instead of a new table - think of a view or a materialized view.
View won't even occupy any space, it is just a stored query:
create or replace view v_b as
select name, location
from some_user.table_a;
It would always be "synchronized", you'd instantly see all committed data that belongs to some_user.
A materialized view occupies space, acts as if it was another table - you can even create indexes on it. Set it to refresh in a scheduled manner (for example, every night) or on demand or whenever some_user commits changes made in their table_a.
If I were you, I wouldn't create another table; (materialized) view seems to be a more appropriate solution.
I am trying to use change tracking to copy data incrementally from a SQL Server to an Azure SQL Database. I followed the tutorial on Microsoft Azure documentation but I ran into some problems when implementing this for a large number of tables.
In the source part of the copy activity I can use a query that gives me a change table of all the records that are updated, inserted or deleted since the last change tracking version. This table will look something like
PersonID Age Name SYS_CHANGE_OPERATION
---------------------------------------------
1 12 John U
2 15 James U
3 NULL NULL D
4 25 Jane I
with PersonID being the primary key for this table.
The problem is that the copy activity can only append the data to the Azure SQL Database so when a record gets updated it gives an error because of a duplicate primary key. I can deal with this problem by letting the copy activity use a stored procedure that merges the data into the table on the Azure SQL Database, but the problem is that I have a large number of tables.
I would like the pre-copy script to delete the deleted and updated records on the Azure SQL Database, but I can't figure out how to do this. Do I need to create separate stored procedures and corresponding table types for each table that I want to copy or is there a way for the pre-copy script to delete records based on the change tracking table?
You have to use a LookUp activity before the Copy Activity. With that LookUp activity you can query the database so that you get the deleted and updated PersonIDs, preferably all in one field, separated by comma (so its easier to use in the pre-copy script). More information here: https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity
Then you can do the following in your pre-copy script:
delete from TableName where PersonID in (#{activity('MyLookUp').output.firstRow.PersonIDs})
This way you will be deleting all the deleted or updated rows before inserting the new ones.
Hope this helped!
In the meanwhile the Azure Data Factory provides the meta-data driven copy task. After going through the dialogue driven setup, a metadata table is created, which has one row for each dataset to be synchronized. I solved this UPSERT problem by adding a stored procedure as well as a table type for each dataset to be synchronized. Then I added the relevant information in the metadata table for each row like this
{
"preCopyScript": null,
"tableOption": "autoCreate",
"storedProcedure": "schemaname.UPSERT_SHOP_SP",
"tableType": "schemaname.TABLE_TYPE_SHOP",
"tableTypeParameterName": "shops"
}
After that you need to adapt the sink properties of the copy task like this (stored procedure, table type, table type parameter name):
#json(item().CopySinkSettings).storedProcedure
#json(item().CopySinkSettings).tableType
#json(item().CopySinkSettings).tableTypeParameterName
If the destination table does not exist, you need to run the whole task once before adding the above variables, because auto-create of tables works only as long as no stored procedure is given in the sink properties.
I am coding in ASP.NET C# 4. The database is SQL Server 2012.
I have a table that has 2000 rows and 10 columns. I want to load this table in memory and if the table is updated/inserted in any way, I want to refresh the in-memory copy from the DB.
I looked into SQL Server Change Tracking, and while it does what I need, it appears I have to write quite a bit of code to select from the change functions -- more coding than I want to do for a simple scenario that I have.
What is the best (simplest) solution for this problem? Do I go with CacheDependency?
I currently have a similar problem: I'm implementing a rest service that returns a table with 50+ columns and I want to cache the data on the client to reduce trafic.
I'm thinking about this implementation:
All my tables have the fields
ID AutoIncrement (primary key)
Version RowVersion (a numeric value that will be incremented
every time the record is updated)
To calculate a "fingerprint" of the table I use the select
select count(*), max(id), sum(version) from ...
Deleting records changes the first value, inserting the second value and updating the third value.
So if one of the three values changes, i have to reload the table.