I have a permanent table which needs to be dropped. Its current retention is set to 1.
This table will no longer be used and will never be required to be restored again.
How do I drop this table without retaining it for Fail Safe?
I am just trying to save on the Fail Safe storage cost. If possible.
Thanks,
T
It's not possible to skip fail-safe for permanent tables.
https://docs.snowflake.com/en/user-guide/data-failsafe.html#what-is-fail-safe
Based on the KB article here:
https://community.snowflake.com/s/article/Change-permanent-table-to-transient-table
There is no way to skip the Fail-Safe as Gokhan mentioned previously.
Just want to share in case it can be useful.
Related
As we all know, in snowflake one mini-partition maybe rewrite/replace when user update the row in the mini-partition. How snowflake handle the mini-partition in time-travel table space when user update?
I think snowflake need to mark the row which has updated in the old time-travel mini-partition, and it will create a new time-travel mini-partition for the rows which has updated, considering it implement time-travel and stream.
I want to know that Because: if time-travel micro-partition can rewriteable, maybe is not safety for my customer's requirement. If it is not rewriteable, maybe the query for time-travel is not efficient for my customer's requirement too.
I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
I’m not sure why you think that how Snowflake works “under the covers” would affect safety or performance in any way that would be of interest to your customer - but, given that, Snowflake is insert-only at the micro partition level. Basically, if a user “updates” a record then the new values are inserted and the old version is marked as no longer current
If I have a Snowflake table containing data and I drop it, what happens to that data? Is it physically purged at any point, or is the storage simply released to the pool of available memory? If the latter, does the option exist to do an explicit purge?
What happens to Snowflake data after it's past the deletion fail-safe point?
When the table leaves Fail-safe, it is purged.
If I have a Snowflake table containing data and I drop it, what happens to that data?
This is covered in detail in the Usage Notes of the DROP TABLE comment.
Is it physically purged at any point, or is the storage simply released to the pool of available memory?
Based on the above, it's physically purged after it's fail-safe expires.
If the latter, does the option exist to do an explicit purge?
Can you add some more context as to what you are looking for specifically? Is this GDPR related? Do you need a guarantee it's physically deleted by X days?
I have a legacy database and I was able to use ALL_TAB_MODIFICATIONS to easily find candidates for deletion - tables, that are no longer used by the application using that database, and then continuing with more detailed review of these tables and possibly deleting them
(I know it isn't guaranteed, hence I call them just "candidates for deletion").
My question is, is there anything similar I could use to find columns which aren't used?
That means that all newly inserted data has NULL, and it is never UPDATEd to get a not null value.
If there isn't a similar view provided by Oracle, what other approach would you recommend to find them?
This isn't needed because of storage reasons, but because the database is open also for reporting purposes and we have had cases where reports have been created based on old columns and thus providing wrong results.
Why not invest a little and get an exact result?
The idea is to store the contant of the tables say at the begin of the month and repeat this at the end of teh month.
From the difference you can see with table columns were changed by updates or with inserts. You'd probably tolerate changes caused by deletes.
You'll only need twice the space of your DB and invest a bit in the reporting SQLs.
Please note also that a drop of a column - even if not actively used - may invalidate your application in case wenn the column is referenced or select * from is used.
Suppose I have a table which contains relevant information. However, the data is only relevant for, let's say, 30 minutes.
After that it's just database junk, so I need to get rid of it asap.
If I wanted, I could clean this table periodically, setting an expiration date time for each record individually and deleting expired records through a job or something. This is my #1 option, and it's what will be done unless someone convince me otherwise.
But I think this solution may be problematic. What if someone stops the job from running and no one notices? I'm looking for something like a built-in way to insert temporary data into a table. Or a table that has "volatile" data itself, in a way that it automagically removes data after x amount of time after its insertion.
And last but not least, if there's no built-in way to do that, could I be able to implement this functionality in SQL server 2008 (or 2012, we will be migrating soon) myself? If so, could someone give me directions as to what to look for to implement something like it?
(Sorry if the formatting ends up bad, first time using a smartphone to post on SO)
As another answer indicated, TRUNCATE TABLE is a fast way to remove the contents of a table, but it's aggressive; it will completely empty the table. Also, there are restrictions on its use; among others, it can't be used on tables which "are referenced by a FOREIGN KEY constraint".
Any more targeted removal of rows will require a DELETE statement with a WHERE clause. Having an index on relevant criteria fields (such as the insertion date) will improve performance of the deletion and might be a good idea (depending on its effect on INSERT and UPDATE statements).
You will need something to "trigger" the DELETE statement (or TRUNCATE statement). As you've suggested, a SQL Server Agent job is an obvious choice, but you are worried about the job being disabled or removed. Any solution will be vulnerable to someone removing your work, but there are more obscure ways to trigger an activity than a job. You could embed the deletion into the insertion process-- either in whatever stored procedure or application code you have, or as an actual table trigger. Both of those methods increase the time required for an INSERT and, because they are not handled out of band by the SQL Server Agent, will require your users to wait slightly longer. If you have the right indexes and the table is reasonably-sized, that might be an acceptable trade-off.
There isn't any other capability that I'm aware of for SQL Server to just start deleting data. There isn't automatic data retention policy enforcement.
See #Yuriy comment, that's relevant.
If you really need to implement it DB side....
Truncate table is fast way to get rid of records.
If all you need is ONE table and you just need to fill it with data, use it and dispose it asap you can consider truncating a (permanent) "CACHE_TEMP" table.
The scenario can become more complicated you are running concurrent threads/jobs and each is handling it's own data.
If that data is just existing for a single "job"/context you can consider using #TEMP tables. They are a bit volatile and maybe can be what you are looking for.
Also you maybe can use table variables, they are a bit more volatile than temporary tables but it depends on things you don't posted, so I cannot say what's really better.
I have a production Oracle database which contains a large amount of data backed up in tables which were made during previous work. The tables are independent of each other and the rest of the database.
I want to remove these backups, preferably in one shot. I know in more recent versions of Oracle dropped tables don't actually get dropped until purged from from the recycle bin. I will take of that.
Is it safe to DROP them all at once? Is there a performance penalty during the DROP operation? Is there a chance to run out of resources during the operation?
What is the safest way to do this?
It's probably safe to drop them all at once.
In general, dropping a table is very quick regardless of the size of the table. DROP doesn't really change any data, Oracle just changes the data dictionary to mark the space as available. I've dropped lots of tables with hundreds of gigabytes or more of data and never had a problem. (Your datafiles may not be sized properly anymore, but that's another issue.)
Other than dependencies and locks, the only time I've ever seen a drop take a (relatively) long time was because of delayed block cleanout. Basically, if you update, delete, or insert (without append) a lot of data, Oracle may write some transaction data to the blocks. The reason for this is to make COMMIT instantaneous, but it means that the next query that even reads from the table may have to clean up the old transaction entries.
But your chances of running into that problem are small. If you have very large tables they were probably created with direct path inserts, or someone else has already queried the table and cleaned out the blocks. Even in the worst case, if your system was good enough to write the data it will probably be good enough to get rid of it (although you could run into ORA-01555 snapshot too old if the transactions are too old, or out of archive log space from the extra redo from delayed block cleanout, etc.).
If the tables have no dependents and are not in use, its safe to drop them all at once. If you are worry about the new recyclebin feature, you can do "drop table table_name purge" and it'll bypass the recyclebin and get purge without having to purge them from the recyclebin.