I am using switch table sometime on SQL Server 2008 to transfer quickly data between 2 tables, but I do not know how it works, and what's the difference between this an just a insert into T2 select * from T1.
Someone can explain in detail how it works and the difference?
Furthermore I notice that if both tables are not clones this statement does not work. For example if I have one index in one table and not in the other one it fails.
Inserting data using the statement
insert into T2 select * from T1
Loads data by inserting data from T1 into T2.
When a partition is transferred (switched), the data is not physically moved; only the metadata about the location of the data changes.
Before you can switch partitions, several general requirements must be met:
General Requirements for Switching Partitions:
When a partition is transferred, the data is not physically moved; only the metadata about the location of the data changes. Before you can switch partitions, several general requirements must be met:
Both tables must exist before the SWITCH operation. The table from which the partition is being moved (the source table) and the table that is receiving the partition (the target table) must exist in the database before you perform the switch operation.
The receiving partition must exist and it must be empty. Whether you are adding a table as a partition to an already existing partitioned table, or moving a partition from one partitioned table to another, the partition that receives the new partition must exist and it must be an empty partition.
The receiving nonpartitioned table must exist and it must be empty. If you are reassigning a partition to form one nonpartitioned table, the table that receives the new partition must exist and it must be an empty nonpartitioned table.
Partitions must be on the same column. If you are switching a partition from one partitioned table to another, both tables must be partitioned on the same column.
Source and target tables must share the same filegroup. The source and the target table of the ALTER TABLE...SWITCH statement must reside in the same filegroup, and their large-value columns must be stored in the same filegroup. Any corresponding indexes, index partitions, or indexed view partitions must also reside in the same filegroup. However, the filegroup can be different from that of the corresponding tables or other corresponding indexes.
This info and more is located here:
https://technet.microsoft.com/en-us/library/ms191160(v=sql.105).aspx
Related
I have a huge table which is partitioned by date.
We have 8 partitions all on different file groups, with one of these file groups being PRIMARY.
I would like to replace the PRIMARY file group with a new file group called 'FG_odsvr_misc', and remove PRIMARY from the partition schema.
How would i achieve this without creating a new table with a new partition function?
The boundaries look like below -
The partition function is as below -
CREATE PARTITION FUNCTION [fn_odstable1](numeric(9,0))
AS RANGE LEFT FOR VALUES (20151231, 20161231, 20171231, 20181231, 20191231, 20201231, 20211231)
The partition scheme is as below -
CREATE PARTITION SCHEME [sch_odstable1] AS PARTITION [fn_odstable1]
TO ([FG_odsvr_pre_2016], [FG_odsvr_2016], [FG_odsvr_2017], [FG_odsvr_2018], [FG_odsvr_2019], [FG_odsvr_2020], [FG_odsvr_2021], [PRIMARY])
Ok. The partition you have on the PRIMARY filegroup is the so-called "Permanent Partition"
From Dan Guzman's Table Partitioning Best Practices:
You might not be aware that each partition scheme has a permanent
partition that can never be removed. This is the first partition of a
RANGE RIGHT function and the last partition of a RANGE LEFT one. Be
mindful of this permanent partition when creating a new partition
scheme when multiple filegroups are involved because the filegroup on
which this permanent partition is created is determined when the
partition scheme is created and cannot be removed from the scheme.
. . .
Consider mapping partitions containing data outside the expected range
to a dummy filegroup with no underlying files. This will guarantee
data integrity much like a check constraint because data outside the
allowable range cannot be inserted. If you must accommodate errant
data rather than rejecting it outright, instead map these partitions
to a generalized filegroup like DEFAULT or one designated specifically
for that purpose.
http://www.dbdelta.com/table-partitioning-best-practices/
Since this is a RANGE LEFT partition scheme you can move all the data off of PRIMARY onto a new filegroup by splitting the rightmost partition at a boundary point greater than the greatest value present in your table.
ALTER PARTITION SCHEME sch_odstable1 NEXT USED [FG_odsvr_2022];
ALTER PARTITION FUNCTION fn_odstable1() SPLIT RANGE (20221231);
The rightmost partition will still be on PRIMARY though. You'll just need to create your future partitions before you need them to keep that partition empty. If you want to you can create a new Partition Scheme
alter database current add filegroup no_files_cant_be_used
CREATE PARTITION SCHEME [sch_odstable2] AS PARTITION [fn_odstable1]
TO ([FG_odsvr_pre_2016], [FG_odsvr_2016], [FG_odsvr_2017], [FG_odsvr_2018], [FG_odsvr_2019], [FG_odsvr_2020], [FG_odsvr_2021], [FG_odsvr_2022], no_files_cant_be_used)
And then create a matching table on the new scheme, ALTER TABLE SWITCH to move all the partitions to the new table, and then rename the tables.
I have a flat file which has following columns
Device Name
Device Type
Device Location
Device Zone
Which I need to insert into SQL Server table called Devices.
Devices table has following structure
DeviceName
DeviceTypeId (foreign key from DeviceType table)
DeviceLocationId (foreign key from DeviceLocation table)
DeviceZoneId (foreign key from DeviceZone table)
DeviceType, DeviceLocation and DeviceZone tables are already prepopulated.
Now I need to write ETL which reads flat file and for each row get DeviceTypeId, DeviceLocationId and DeviceZoneId from corresponding tables and insert into Devices table.
I am sure this is not new but its being a while I worked on such SSIS packages and help would be appreciated.
Load the flat content into a staging table and write a stored procedure to handle the inserts and updates in T-SQL.
Having FK relationships between the destination tables, can probably make a lot of trouble with a single data flow and a multicast.
The problem is that you have no control over the order of the inserts so the child record could be inserted before the parent.
Also, for identity columns on the tables, you cannot retrieve the identity value from one stream and use it in another without using subsequent merge joins.
The simplest way to do that, is by using Lookup Transformation to get the ID for each value. You must be aware that duplicates may lead to a problem, you have to make sure that the value is not found multiple times in the foreign tables.
Also, make sure to redirect rows that have no match into a staging table to check them later.
You can refer to the following article for a step by step guide to Lookup Transformation:
An Overview of the LOOKUP TRANSFORMATION in SSIS
I have gone through a bunch of documentation for PostgresSQL 10 partitioning but I am still not clear on whether existing tables can be partitioned. Most of the posts mention about partitioning existing tables using PostgreSQL 9.
Also, in the official PostgresSQL website : https://www.postgresql.org/docs/current/static/ddl-partitioning.html, it mentions 'It is not possible to turn a regular table into a partitioned table or vice versa'.
So, my question is can existing tables be partitioned in PostgreSQL 10?
If the answer is YES, my plan is :
Create a partitions
Alter the existing table to include the range so new data goes into the new partition. Once that is done, write a script which loops over the master table and moves the data into the right partitions.
Then, truncate the master table and enforce that nothing can be inserted into it.
If the answer is NO, my plan is to make the existing table the first partition
Create a new parent table and children(partitions).
Perform light transaction which will rename the existing table to a partition table name and the new parent to the actual table name.
Are there better ways to partition existing tables in PostgreSQL 10/9?
I have a huge table with around 110 partitions. I wish to archive the oldest partition and drop the FileGroup. Following is the strategy I adopted.
Created an exact empty table tablename_archive and met all partitioning requirements.
Perform Partition switch
ALTER TABLE tablename SWITCH PARTITION 1 TO tablename_archive PARTITION 1
After verifying the switch (partition swap) , I dropped the archived table.
Merged the Partition function using the first boundary value as follows
ALTER PARTITION FUNCTION YMDatePF2 () MERGE RANGE ('2012-01-01 00:00:00.000')
Although there is no data now on FG, when I try to drop the File or FG it errors out saying.
The file 'XXXXXXXX' cannot be removed because it is not empty.
The filegroup 'XXXXXXXX' cannot be removed because it is not empty.
Is there any change I need to make it to Partition scheme too, after merging the function.
Please let me know if you need any more details.
You can never remove the first (or only) partition from a RANGE RIGHT partition function (or conversely, the last (or only) partition of a RANGE LEFT function). The first (or last if RANGE LEFT) filegroup from the underlying partition schemes can never be removed from the schemes either. Remember you have one more partition, and partition scheme filegroup mapping, than partition boundaries.
If your intent was to archive January 2012 data, you should have switched partition 2 rather than 1 because the first partition contained data less than '2012-01-01 00:00:00.000'. Now that the second partition has been merged, the first partition (and the first filegroup) contains data less than '2012-02-01T00:00:00.000', which includes January 2012 data.
With a RANGE RIGHT sliding window, it is best to plan to keep the first filegroup empty. You could used the PRIMARY filegroup or a dummy one with no files for that purpose. See Table Partitioning Best Practices.
Using SQL Server 2014 Partitioned Tables;
According to the below post:
Can I move rows between partitions by updating the partition key?
I understand that we can move rows between partitions by changing the partition key.
What is the performance cost of moving rows between partitions by updating the partition key? Is it equivalent to having two tables, deleting a row from the 1st table and insert it into the second table? OR is it more efficient within partitions?
A link to an online article would be very helpful. I can't find the right Google search key for this question. Hopefully this post will help others.
It would be the former.
If you update any other column than the partition key, then the record stays in the same bucket, the column value gets changed and the transaction log gets a delete/insert (=update) to track the changes.
If you update the partition key, then that record no longer maps to that partition. So it will delete the old record and insert the new record (into a new partition per the partitioning rules). Note that the transaction log entry for this change requires writing the whole record as it is truly deleted in one partition then rewritten wholly into another, and not just a one column change.