SQL Server 2016 what are the advantages of using Horizontal Partitions.
I want to make multiple similar tables into 1 large table.
Ex : Progress_YearWeekNo is name of table which is created every week dynamicaly,i want to make it 1 common Progress table with YearWeekNo column on which i will be creating partition.
It will be huge data like 10M records in one table.
Below are my questions?
1.query to select from particular partition.
2.mark only 1 partition as ACTIVE and remaining as history.
As weekly new tables are created dynamically , there are more than 200 tables .So if i combine all tables and partition it will be more manageable table than having more tables of similar type.
Related
Once a day I have to synchronize table between two databases.
Source: Microsoft SQL Server
Destination: PostgreSQL
Table contains up to 30 million rows.
For the first time i will copy all table, but then for effectiveness my plan is to insert/update only changed rows.
In this way if I delete row from source database, it will not be deleted from the destination database.
The problem is that I don’t know which rows were deleted from the source database.
My dirty thoughts right now tend to use binary search - to compare the sum of the rows on each side and thus catch the deleted rows.
I’m at a dead end - please share your thoughts on this...
In SQL Server you can enable Change Tracking to track which rows are Inserted, Updated, or Deleted since the last time you synchronized the tables.
with TDS FDWs (Foreign Data Wrapper), map the source table with a temp table in pg, an use a join to find/exclude the rows that you need.
I have a source table in a Sybase database (ORDERS) and a Source table in an MSSQL Database (DOCUMENTS). I need to query the Sybase database and for each row found in the ORDERS table get the matching row(s) by order number from the DOCUMENTS table.
I originally wrote the SSIS package using a lookup transformation, simple, except that it could be a one-to-many relationship, where 1 order number will exist in the ORDERS table but more than 1 documents could exist in the DOCUMENTS table. The SSIS lookup will only match on first.
My 2nd attempt will be to stage the rows from the ORDERS table into a staging table in MSSQL and then loop through the rows in this table using a FOR EACH LOOP CONTAINER and get the matching rows from the DOCUMENTS table, inserting the DOCUMENTS rows into another staging table. After all rows from ORDERS have been processed I will write a query to join the two staging tables to give me my result. A concern with this method is that I will be opening and closing the DOCUMENTS database connection many times, which will not be very efficient (although there will probably be less than 200 records).
Or could you let me know of any other way of doing this?
I am working on a heavy record set database in MS SQL 2016. So I want to use row table partition feature to improve speed.
As we know partition feature is working on partition column of a table. Let's say [Date Column] of a table. In our scenario, have many tables that need to partition because of heaver record set in 5 to 7 tables. Each table not have that [Date column]. Also not possible to add that column in each table.
So is there any way I can select partition column of another table or something else.
The best option is to add a common column to all tables that you will then use to partition by.
You must already have a way of relating the different tables to each other so you can use this to tag each table with the correct Partition column.
This column could be as simple as an int with YYYYMM as values for monthly partitions.
You also need to make sure your queries are "Partition Aware".
This means that you should include this column in your WHERE Clause and also your JOIN Clauses for any queries.
Use Query Plans to make sure you are getting Partition Elimination on your queries.
If you can't change the model (but can add partitions???) then you could implement the partitioning with different columns in each table provided you have a single column in each table that you can partition on named ranges - but if you have 1-many relationships then it is unlikely that the child tables keys will be consecutive relative to the parent table. Note that this approach will make your "partition aware" queries more complex to craft.
I have a database with 51 tables all with the same schema (one table per state). Each table has a couple million rows and about 50 columns.
I've normalized the columns into 6 other tables, and now I want to import all of the data from those 51 tables into the 6 new tables. The column names are all the same, and so I'm hoping I can automate the process of importing all the data.
I'm assuming what I'll need to do is:
Select the names of all the lists that have the raw schema
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'raw'
Iterate over all the results
Grab all rows from that table, and SELECT INTO the appropriate cols into the appropriate tables
Delete row from raw table
Is there anything I'm missing? Also, is there any way to have this run on the SQL Server so I don't have to have my SQL Server Management Studio open the whole time?
Yes, obviously, you can automate it with t-sql. But I recommened you to use SSIS in this case. As you say, structure of all tables are the same than you can make some ETL process and then you just change table name in the source. Consecuently, you will have the folowwing advantages:
Solve issue with couple of clicks
Low risk of errors
You will able to use the number of data transformations
In my current environment there are hundreds of DB tables with 20% master tables. The values of some of the columns is a subset of a column from the master table. Unfortunately it is not documented.
Wondering if the master tables and dependent tables are known, how to craft a SQL statement to get which column of the dependent table is a subset of the master table column.
Thanks,
cabear