In Snowflake:
I want to clone the source
Do operations on the clone
Turn clone into the source
Drop the original source
When reading the documents, I’m interpreting a clone is a unique, independent object that’s identical to the source. So, in my head, I can create a clone and drop the source and be OK. Anyone ever do this in a prod environment?
Thanks for any guidance. We've tested the theory and it doesn't look like there are any side effects with the exception of losing time travel and file loading history on the old source; but we're OK with that.
Yes you can create a clone and drop the "source" of the clone. You might also be able to achieve the same effect by using a transaction too but with simpler code:
begin transaction;
[do operation 1 on source table];
[do operation 2 on source table];
commit;
I guess your use case may be same as the use case I had.
As Simon said it is possible, but with clone, you need to be careful about the access control privileges. Please also note that you will lose load history of the source table. Also if a table is cloned, historical data for the table clone begins at the time/point when the clone was created
Access Control Privileges for Cloned Objects
A cloned object does not retain any granted privileges on the source object itself (i.e. clones do not automatically have the same privileges as their sources). A system administrator or the owner of the cloned object must explicitly grant any required privileges to the newly-created clone.
However, if the source object is a database or schema, for child objects contained in the source, the clone replicates all granted privileges on the corresponding child objects:
For databases, contained objects include schemas, tables, views, etc.
For schemas, contained objects include tables, views, etc.
Related
We have different clients and each client has its own database. Now for different clients our roles are named something like write_role_clientID. Now for any new client I want to clone an older DB but the roles should be for that clientID. I know I can get a DDL to create a DB but that doesn't include any privileges/grants. If somehow I can get the DDL for a clone because in a clone the privileges are also inherited by the objects then I might be able to manipulate the sql using code.
SELECT GET_DDL('database', 'your_cloned_database') should work fine over a cloned database. However, I'd be cautious using clones for new clients as they'll have access to data from the old client unless you go through every table and TRUNCATE all the tables (and be cautious around Time Travel!).
It's probably easier to do something like SELECT REPLACE(GET_DDL('database', 'original_database'), 'old_client_id', 'new_client_id') (you could probably even just EXECUTE that, but YMMV).
Is it possible to clone schemas selectively in Snowflake?
For e.g.
Original:
DB_OG
--schema1
--schema2
--schema3
Clone:
DB_Clone
--schema1
--schema3
The CREATE <object> … CLONE statement does not support applying a filter or pattern or multiple objects, and its behaviour is to recursively clone every object underneath:
For databases and schemas, cloning is recursive:
Cloning a database clones all the schemas and other objects in the database.
There are a few explicit ways to filter the clone:
Clone the whole database, then follow up with DROP SCHEMA commands to remove away unnecessary schema
Create an empty database and selectively clone only the schemas required from the source database into it
Both of the above can also be automated by logic embedded within a stored procedure that takes a pattern or a list of names as its input and runs the appropriate SQL commands.
Currently the elimination of certain schemas and cloning all the other schema's of a database is not supported.
If the use case has schemas that are not required, are the recently created schemas, you could use the AT | BEFORE clause to eliminate the schemas(clone till a particular timestamp, that will eliminate the schemas that are created post the mentioned timestamp).
Ref: https://docs.snowflake.com/en/sql-reference/sql/create-clone.html#notes-for-cloning-with-time-travel-databases-schemas-tables-and-streams-only
Other options include dropping the schemas post the cloning operation or cloning only the required schemas
Question
From SSMS, I can use "Generate Scripts" for views, procedures, and functions, and then save each as a "Single file per object."
How can I generate the correct order to execute the files later, given their dependencies on other views?
Backstory
I am attempting to get my database schema under revision control.
For views (and procedures and functions), the page below suggests saving one per file. On an update, all would be dropped then all would be recreated.
Unfortunately, the page never mentions details of how it handles dependencies between views.
Versioning Databases – Views, Stored Procedures, and the Like
One File per Object
My strategy is to script every view, stored procedure, and function
into a separate file, then commit the files to source control. If
someone needs to add a new view, they script the view into a file and
commit the file to source control. If someone needs to modify a view,
they modify the view's script file and commit again. If you need to
delete a view from the database, delete the file from source control.
It's a relatively simple workflow.
The magic happens when a developer, tester, or installer updates from
source control and runs a tool that updates their local database. The
tool uses a three step process:
The tool applies new schema changes by comparing the available schema change files to the SchemaChangeLog records in the database.
The tool will DROP all stored procedures, views, and functions in the database.
The tool will run all of the scripts needed to add the views, stored procedures, and functions back into the database.
Due to an employee quitting, I've been given a project that is outside my area of expertise.
I have a product where each customer will have their own copy of a database. The UI for creating the database (licensing, basic info collection, etc) is being outsourced, so I was hoping to just have a single stored procedure they can call, providing a few parameters, and have the SP create the database. I have a script for creating the database, but I'm not sure the best way to actually execute the script.
From what I've found, this seems to be outside the scope of what a SP easily can do. Is there any sort of "best practice" for handling this sort of program flow?
Generally speaking, SQL scripts - both DML and DDL - are what you use for database creation and population. SQL Server has a command line interface called SQLCMD that these scripts can be run through - here's a link to the MSDN tutorial.
Assuming there's no customization to the tables or columns involved, you could get away with using either attach/reattach or backup/restore. These would require that a baseline database exist - no customer data. Then you use either of the methods mentioned to capture the database as-is. Backup/restore is preferrable because attach/reattach requires the database to be offline. But users need to be sync'd before they can access the database.
If you got the script to create database, it is easy for them to use it within their program. Do you have any specific pre-requisite to create the database & set permissions accordingly, you can wrap up all the scripts within 1 script file to execute.
We are in the process of a multi-year project where we're building a new system and a new database to eventually replace the old system and database. The users are using the new and old systems as we're changing them.
The problem we keep running into is when an object in one system is dependent on an object in the other system. We've been using views, but have run into a limitation with one of the technologies (Entity Framework) and are considering other options.
The other option we're looking at right now is replication. My boss isn't excited about the extra maintenance that would cause. So, what other options are there for getting dependent data into the database that needs it?
Update:
The technologies we're using are SQL Server 2008 and Entity Framework. Both databases are within the same sql server instance so linked servers shouldn't be necessary.
The limitation we're facing with Entity Framework is we can't seem to create the relationships between the table-based-entities and the view-based-entities. No relationship can exist in the database between a view and a table, as far as I know, so the edmx diagram can't infer it. And I cannot seem to create the relationship manually without getting errors. It thinks all columns in the view are keys.
If I leave it that way I get an error like this for each column in the view:
Association End key property [...] is
not mapped.
If I try to change the "Entity Key" property to false on the columns that are not the key I get this error:
All the key properties of the
EntitySet [...] must be mapped to all
the key properties [...] of table
viewName.
According to this forum post it sounds like a limitation of the Entity Framework.
Update #2
I should also mention the main limitation of the Entity Framework is that it only supports one database at a time. So we need the old data to appear to be in the new database for the Entity Framework to see it. We only need read access of the old system data in the new system.
You can use linked server queries to leave the data where it is, but connect to it from the other db.
Depending on how up-to-date the data in each db needs to be & if one data source can remain read-only you can:
Use the Database Copy Wizard to create an SSIS package
that you can run periodically as a SQL Agent Task
Use snapshot replication
Create a custom BCP in/out process
to get the data to the other db
Use transactional replication, which
can be near-realtime.
If data needs to be read-write in both database then you can use:
transactional replication with
update subscriptions
merge replication
As you go down the list the amount of work involved in maintaining the solution increases. Using linked server queries will work best if its the right fit for what you're trying to achieve.
EDIT: If they're the same server then as suggested by another user you should be able to access the table with servername.databasename.schema.tablename Looks like it's an entity-framework issues & not a db issue.
I don't know about EntityToSql but I know in LinqToSql you can connect to multiple databases/servers in one .dbml if you prefix the tables with:
ServerName.DatabaseName.SchemaName.TableName
MyServer.MyOldDatabase.dbo.Customers
I have been able to click on a table in the .dbml and copy and paste it into the .dbml of the alternate project prefix the name and set up the relationships and it works... like I said this was in LinqToSql, though have not tried it with EntityToSql. I would give it shot before you go though all the work of replication and such.
If Linq-to-Entities cannot cross DB's then Replication or something that emulates it is the only thing that will work.
For performance purposes you probably want either Merge replication or Transactional with queued (not immediate) updating.
Thanks for the responses. We're going to try adding triggers to the old database tables to insert/update/delete records in the new tables of the new database. This way we can continue to use Entity Framework and also do any data transformations we need.
Once the UI functions move over to the new system for a particular feature, we'll remove the table from the old database and add a view to the old database with the same name that points to the new database table for backwards compatibility.
One thing that I realized needs to happen before we can do this is we have to search all our code and sql for ##Identity and replace it with scope_identity() so the triggers don't mess up the Ids in the old system.