I run a database that has a tables with several NTEXT columns that contain short lived data. We would normally only keep a few weeks of data in those tables but in an effort to reduce disk usage this was reduced down to only 72 hours.
While I expected to see some increase in backup performance, I was not expecting such a large increase. The database dropped from 105 GB of storage to 99 GB with the change in data retention. Before the change backups would roughly take 60 minutes. After the change it dropped to 40 minutes to backup.
I assumed that with a 6% reduction of storage that it would be an equal reduction in backup time, but appears to have shaved off a third of the time required for backup.
Because the majority of the data that was removed was NTEXT, does this have a much larger impact on backup performance than other data types?
I have done some searching but I haven't been able to find any connection between the two things.
Edit: I left out that these are full backups running.
Related
I have an issue where a full backup on a MSSQL server randomly takes ~2.5 the time to complete. Is there any option in SQL Management Studio or Stored Procedure that would tell me what cause this giant slowdown.
I have many MSSQL server running and that particular one is internal only and backup happen at 5am in the morning while there is no one in the office until 7:30-8:00. Backup is typically taking a steady 14 minutes 20 seconds (plus or less 10 seconds) but once or twice per week it suddenly takes upward of 45 minutes.
The backup size is growing but it's really minor like 40-50 mb per day while the backup size is sitting currently at around 21 gb. All daily transaction are stable in size too. When i have a slow backup the transaction of the previous day is not any different is size than the day before neither the other "normal" days.
The only logs i see simply give me start time and end time of the maintenance plan which is useless as it's the total runtime.
I have an Access database for which I am the only user. It's the first database I've built. It has 16 related tables, around 40 select queries, and a dozen or so update/delete queries. It is already 512MB and will at least double in size as additional data is added to tables and more queries & reports are created over the next 12 months. The largest table (which is accessed by most of the queries) is around 800k rows by 11 fields. This table will most likely grow to over 2M lines over the useful life of the database (c. 12 months).
Queries that ran in under 30 sec a month ago are starting to run slower as the tables have grown, some queries which include calculations now taking 10 min or so to complete (and yes, I am using stacked queries as much as possible).
Does anyone have solid advice one way or the other as to the performance boost I could expect from splitting?
Thanks
No you wouldn't, only query optimizing and careful indexing can speed up the query time.
That said, you should split it anyway (create a backup and run the wizard) if for nothing else to ease backup of your data and to make that independent of your ongoing development of the frontend.
I'm running some stored procedures in SQL Server 2012 under Windows Server 2012 in a dedicated server with 32 GB of RAM and 8 CPU cores. The CPU usage is always below 10% and the RAM usage is at 80% because SQL Server has 20 GB (of 32 GB) assigned.
There are some stored procedures that are taking 4 hours some days and other days, with almost the same data, are taking 7 or 8 hours.
I'm using the least restrictive isolation level so I think this should not be a locking problem. The database size is around 100 GB and the biggest table has around 5 million records.
The processes have bulk inserts, updates and deletes (in some cases I can use truncate to avoid generating logs and save some time). I'm making some full-text-search queries in one table.
I have full control of the server so I can change any configuration parameter.
I have a few questions:
Is it possible to improve the performance of the queries using
parallelism?
Why is the CPU usage so low?
What are the best practises for configuring SQL Server?
What are the best free tools for auditing the server? I tried one
from Microsoft called SQL Server 2012 BPA but the report is always
empty with no warnings.
EDIT:
I checked the log and I found this:
03/18/2015 11:09:25,spid26s,Unknown,SQL Server has encountered 82 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.HLSQLSERVER\MSSQL\DATA\templog.ldf] in database [tempdb] (2). The OS file handle is 0x0000000000000BF8. The offset of the latest long I/O is: 0x00000001fe4000
Bump up max memory to 24 gb.
Move tempdb off the c drive and consider mult tempdb files, with auto grow at least 128 Mbps or 256 Mbps.
Install performance dashboard and run performance dashboard report to see what queries are running and check waits.
If you are using auto grow on user data log and log files of 10%, change that to something similar to tempdb growth above.
Using performance dashboard check for obvious missing indexes that predict 95% or higher improvement impact.
Disregard all the nay Sayers who say not to do what I'm suggesting. If you do these 5 things and you're still having trouble post some of the results from performance dashboard, which by the way is free.
One more thing that may be helpful, download and install the sp_whoisactive stored proc, run it and see what processes are running. Research the queries that you find after running sp_whoisactive.
query taking hours but using low CPU
You say that as if CPU would matter for most db operations. HINT: They do not.
Databases need IO. RAM sin some cases helps mitigate this, but at the end it runs down to IO.
And you know what I see in your question? CPU, Memory (somehow assuming 32gb is impressive) but NO WORD ON DISC LAYOUT.
And that is what matters. Discs, distribution of files to spread the load.
If you look into performance counters then you will see latency being super high on discs - because whatever "pathetic" (in sql server terms) disc layout you have there, it simply is not up to the task.
Time to start buying. SSD are a LOT cheaper than discs. You may say "Oh, how are they cheaper". Well, you do not buy GB - you buy IO. And last time I checked SSD did not cost 100 times the price of discs - but they have 100 times or more the IO. and we talk always of random IO.
Then isolate Tempdb on separate SSD - tempdb either does no a lot or a TON and you want to see this.
Then isolate the log file.
Make multiple data files, for database and tempdb (particularly tempdb - as many as you have cores).
And yes, this will cost money. But at the end - you need IO and like most developers you got CPU. Bad for a database.
We have a lot of sql server databases in Simple Recovery Mode.
We need to move these databases on a different datacenter. These databases can not be closed (they are serving 24x7 mission critical services).
We are thinking to use log shipping to create a remote standby database and then switching the remote copy to be primary.
How we can estimate the amount o transaction log written daily by each of these databases?
We need to know this amount to estimate netowork bandwith and size of transaction log backup.
Regards
Giovanni
You don't have to estimate: set the database to full recovery and start taking log backups. You'll have to do this anyways to do log shipping and you should really be doing this anyways if the db is that important to you. Once you're taking the log backups, you'll know how big the log backups are over any given prior of time so you can estimate bandwidth based on that.
I wrote a stored procedure that do the following:
read num_of_bytes_written from sys.dm_io_virtual_file_stats
random insert and delete rows from tables
read num_of_bytes_written from sys.dm_io_virtual_file_stats
calculate difference in megabytes between the two values of
num_of_bytes_written
make compressed transaction log backup
check the difference between log backup size and the num of bytes
written
After 1000 executions I'll calculate the ratio between log size and backup size
regards
Giova
One of our servers is an SQL Server 2008 R2 database (full recovery) of less than 100GB in size that is backed up daily with transaction log backups taken every 15 minutes. The Log file growth is set to 500MB.
But no matter what I do, I cannot get the VLF count down to a respectable figure (e.g. 50) and is now constantly sitting between 500-700.
I've used the expand, backup logs, shrink, expand, backup, shrink, etc... technique and whilst it works to drop the VLF count from 700 to 500, that is as far as it will go.
The database is partially replicated (Transactional - continuous). We rely heavily on doing reports off the subscription database and up-to-date reports are run 24-7, so we cannot change the replication model. We do nightly index rebuilds/reorganising and update statistics weekly.
There aren't any long running transactions according SYSPROCESSES.
I am at a bit of a loss. Is there any cure apart from temporarily performing a Backup -> Simple -> Full operation?
TIA