Is it possible for a memory optimized table in SQL server to have part of its data in memory and the rest on disk?
I have a requirement to load the last 3 months' data into memory, and the rest
of it isn't really required to be in memory because it will not be
queried.
Is there any way to do this with memory optimized tables? If not, is there any alternate way to do this?
Use a view to union an in-memory table with a standard table (a partitioned view). Run a maintenance process to move data from the in-memory table to the standard table as needed.
You can add check-constraints to the standard table to help eliminate it from the query if that data will not be touched.
The database can have few tables on disk and few in memory,but it is not possible to have ,some of table data in disk and some data in memory
I have a requirement to load the last 3 months' data into memory, and the rest of it isn't really required to be in memory because it will not be queried
why not archive the table regularly ,so it retains only three months data and optimize it for In-memory usage..
Related
I just started my new job and after looking at the DBM I was shocked. Its a huge mess.
Now first thing I wanted to do is get some consistency in the order of table columns. We publish new database versions over a .dacpac. My co-worker told me that changing the order of a column would force MSSQL to create a temporary table which stores all the data. MSSQL then creates a new table and inserts all the data into that table.
So lets say my server only runs 2GB of RAM and has 500MB storage left on the harddrive. The whole database weights 20GB. Is it possible that changing the order of columns will cause trouble (memory related)? Is the statement of my co-worker correct?
I couldnt find any good source for my question.
Thanks!
You should NOT "go one table by one".
You should leave your tables as they are, if you don't like the order of columns of some table just create a view reordering your columns as you want.
Not only changing order of columns will cause your tables to be recreated, all the indexes will be recreated, you'll get problems with FK constraints.
And after all, you'll gain absolutely nothig but do damage only. You'll waste server resources, make your tables temporarily inaccessible and the columns will not be stored as you defind anyway, internally they will be stored in "var-fix" format (divided into fixed-length and variable-length)
I have a table that is 13 TB big (due to historical reasons).
I want to reload the data because I have corrupt and duplicate data in that table.
The question is what is the fastest way to load data to an empty, partitioned table (partitioned by month)?
My thoughts:
Fill table by filling partition slices. I create two or three (depending on I/O cap) temp tables and load the data via SSIS OLEDB (openrowset bulk) to three temp tables at once. Afterwards I switch in the partitions and go on with the next three.
Insert latest and oldest data at once via "normal" insert (I don't think the clustered index will like that)
???
So what would be the best and fastest way?
The biggest speed gain you can achieve is to drop all the indexes on the target table, load the data (probably using choice 1) and then rebuild all your indexes (clustered index first). Keep the source data and the target mdf/ndf on separate physical drives (RAID hopefully).
You can script out the indexes by right clicking on the index and selecting Script index create to. Make sure that if the index is partitioned that the portioning info is included.
The largest gain would be in using the Bulk Insert Function, this is designed to deal in sets rather than row based operations I don't know where you're importing the data from but I imagine it would be a backup if you're dealing with corruption.
I have an application that logs to a DB2 database. Each log is stored in a daily table, meaning that I have several tables, one per each day.
Since the application is running for quite some time, I dropped some of the older daily tables, but the disk space was not reclaimed.
I understand this is normal in DB2, so I goggled and found out that the following command can be used to reclaim space:
db2 alter tablespace <table space> reduce max
Since the tablespace that store the daily log tables is called USERSPACE1, I executed the following command successfully:
db2 alter tablespace userspace1 reduce max
Unfortunately the disk space used by DB2 instance is still the same...
I've read somewhere that the REORG command can be executed, but what I've seen it is used to reorganize tables. Since I dropped the tables, how can I use REORG?
Is there any other way to do this?
Thanks
Reduce the size of a tablespace is very complex. The extents (set of contiguous pages; unit of tablespace allocation) of the tables are not distributed sequentially for a same table. When you reorg a table, the rows will be organized in pages, and the new pages will be written normally at the end of the tablespace. Sometimes, the high watermark will be increased, and your tablespace will be bigger.
You need to reorg all tables from a tablespace in order to "defrag" all tables. Then, you have to perform a new reorg in order to use the previous space, because it should be an empty space in the tablespace.
However, there are many criteria that impacts the organization of the tables in a tablespace: New extents are created (new rows, rows overflow due to updates); compression could be activated after reorg.
What you can do is to assign few or just one table per tablespace; however, you will waste a lot of space (overhead, empty pages, etc.)
The command that you are using is an automatic way to do that, but it does not always work as desired: http://www-01.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.admin.dbobj.doc/doc/c0055392.html
If you want to see the distribution of the tables in your tablespace, you can use db2dart. Then, you can have an idea of which table to reorg (move).
Sorry guys,
The command that I mentioned on the original post works after all, but the space was retrieved very slowly.
Thanks for the help
I am wondering, when selecting rows and inserting them into a temp table, is the data actually copied or just referenced?
For example:
SELECT * INTO #Temp FROM SomeTable
If the table is very large, is this going to be a costly operation?
From my tests it seems to execute about as fast as a simple SELECT, but I'd like a better insight about how it actually works.
Cheers.
Temporary tables are allocated in tempdb. SQL server will generally try to keep the tempdb pages in memory, but large tables may end up written out to disk.
And yes, the data is always copied. So, for instance, if an UPDATE occurs on another connection between selecting into your temporary table and a later usage, the temporary table will contain the old value(s)
What would be more efficient in storing some temp data (50k rows in one and 50k in another) to perform come calculation. I'll be doing this process once, nightly.
How do you check the efficiency when comparing something like this?
The results will vary on which will be easier to store the data, in disk (#temp) or in memory (#temp).
A few excerpts from the references below
A temporary table is created and populated on disk, in the system database tempdb.
A table variable is created in memory, and so performs slightly better than #temp tables (also because there is even less locking and logging in a table variable). A table variable might still perform I/O to tempdb (which is where the performance issues of #temp tables make themselves apparent), though the documentation is not very explicit about this.
Table variables result in fewer recompilations of a stored procedure as compared to temporary tables.
[Y]ou can create indexes on the temporary table to increase query performance.
Regarding your specific case with 50k rows:
As your data size gets larger, and/or the repeated use of the temporary data increases, you will find that the use of #temp tables makes more sense
References:
Should I use a #temp table or a #table variable?
MSKB 305977 - SQL Server 2000 - Table Variables
There can be a big performance difference between using table variables and temporary tables. In most cases, temporary tables are faster than table variables. I took the following tip from the private SQL Server MVP newsgroup and received permission from Microsoft to share it with you. One MVP noticed that although queries using table variables didn't generate parallel query plans on a large SMP box, similar queries using temporary tables (local or global) and running under the same circumstances did generate parallel plans.
More from SQL Mag (subscription required unfortunately, I'll try and find more resources momentarily)
EDIT: Here is some more in depth information from CodeProject