I have a serious problem with a postgres 8.3 database running on CentOs, a closed service use this database to store analog variables on a especific table from 5 and 5 minutes about 2 years, this information is precious from my company.
A nice day the postgres process stopped and the Hard drive from server was full. We order more 2 compatible hd with the server, it will take 30 days to arrive, but the application have to work until this drives arrive.
Today the application insert around 400 MB per day on Database.
I just symlinked some folders to a external 1 TB HD USB 2.0.
The actual situation is:
df -h
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda1 ext4 1010G 1009G 224M 100% /
/dev/sda2 ext4 99M 17M 78M 18% /boot
and the $PGDATA folder have
/var/pgsql/data 1005G
The main table have aroud 600GB.
I just make a Backup.tar from the database and delete a lot of older rows, and trying run the VACUUM command, on the middle of process a error about disk space occurs and don't finish the VACUUM.
I've already tried run VACUUM, REINDEX, CLUSTER, and I have the same error.
I'm thinking on DROP TABLE and CREATE TABLE, it's the best option? free the disk space??
Someone have a tip or a solution for this case?
Thanks, and sorry for the english.
Related
It takes about 3 days for the transaction log (tempdb) to get to the size of a production database (~130 GB). It has come up recently (~2 weeks ago) and there had been no problems up to that point. I'm not the owner of the database nor the author or developer of the solution that is utilizing the database.
We have several databases of the same type across several plants, so I did some comparisons, and the problem happens to be only in one plant.
According to SQL Activity Monitor (Data File I/O tab), templog.ldf is being utilized almost all the time in the sense of write operations - it grows 100 MB every 3-5 minutes.
When I do bcc opentran against tempdb I see one transaction open since when I restarted the instance last time. So it's been open since that time which is almost 3 days.
When I query sys.sysprocesses I actually see two SPIDs of the same age (started 3 days ago) and with the flag open_tran and both with the suspended status.
This is weird because I don't see the same thing in other databases of the same type.
Both queries show up in the Active Expensive Queries (SQL Activity Monitor) and they are not letting go... (this might be because of what the query is for both: WAITFOR DELAY #delay)
I set up an Extended Events session and was looking for anything tempdb, physical files growth, size changed, write ops related and I was trying to correlate the output with the moment when the templog file expands by 100 MB.
database_log_file_size_changed event appears to be associated with almost every session there is in the tempdb. I guess the log file is being utilized by almost each session AND what is interesting, the two old, 3-day sessions do not appear in the EE output (as if they did not affect the log file at all).
I'm gonna go to the 3rd party that owns the solution, and ask them what the problem might be, but I'm starting to wonder if this might have anything to do with the physical structure of the file or maybe the volume.
templog shares the same volume with other dbs and other log files - no problems with any of them up to that point
I'm not able to go to the Properties of the tempdb - Property Size is not available for Database (tempdb) error - even a few minutes after the restart.
I'm not able to run dbcc checkdb against tempdb either as I'm getting:
Object ID 3 (object 'sys.sysrscols'): DBCC could not obtain a lock on this object because the lock request timeout period was exceeded. This object has been skipped and will not be processed.
I have no more ideas on how to investigate the tempdb and the log growing quickly issue. I'd be grateful for any hints.
Regards,
I'm doing a database migration from Oracle tables to SQL Server tables. Two of the three tables were successful during the first try, mostly because they didn't have that many rows as the third (about 3,5 million rows with around 30 columns). I took me around 15 attempts to accomplish the migration process, because Dbeaver use all the RAM available (around 14 GB).
Migration using 10.000/100.000 segments, CPU use to 100% for many minutes, Dbeaver crashed because the JVM use all assigned memory.
After increasing the JVM memory to 14 GB, migration crashed because the system didn't have more RAM available.
I did change the segment size many times with no results. I ended up using the 'direct Query' and after 1,5 hours it finished successfully.
The question is: Why Dbeaver keeps using RAM without GC cleaning it?
How can I change the beauvoir of the GC to be more 'eager'?
Thanks.
We work for a small company that cannot afford to pay SQL DBA nor consultation.
What started as a small project has now become a full scale system with a lot of data.
I need someone to help me sort out performance improvements. I realise no-one will be able to help directly and nail this issue completely, but I just want to make sure I have covered my tracks.
OK, the problem is basically we are experiencing time-outs with our queries on cached data. I have increased the time-out time with c# code but I can only go so far when it's becoming ridiculous.
The current setup is a database that has data inserted every 5 / 10 seconds, constantly! During this process we populate tables from csv files. Over night we run data caching processes that reduces the overload on the "inserted" tables. Originally we were able to convert 10+ million rows into say 400000 rows, but as users want more filtering we had to include more data rows and of course increases the number of data cached tables from 400000 to 1-3 million rows.
On my SQL Development Server (which does not have data inserted every 5 seconds) it used to take 30 seconds to run queries on data cache table with 5 million rows, now with indexing and some improvements it's now 17 seconds. The live server has standard SQL Server and used to take 57 seconds, now 40 seconds.
We have 15+ instances running with same number of databases.
So far we have outlined the following ways of improving the system:
Indexing on some of the data cached tables - database now bloated and slows down overnight processes.
Increased CommandTimeout
Moved databases to SSD
Recent improvements likely:
Realised we will have to move csv files on another hard disk and not on the same SSD drive SQL Server databases reside.
Possibly use file-groups for indexing or cached tables - not sure if SQL Server standard will cover this.
Enterprise version and partition table data - customer may pay for this but we certainly can't afford this.
As I said I'm looking for rough guidelines and realise no-one may be able to help fix this issue completely. We're are a small team and no-one has extended SQL Server experience. Customer wants answers and we've tried everything we know. Incidentally they had a small scale version in Excel and said they found no issues so why are we?!?!?
Hope someone can help.
Using Mac OS X 10.10 (Yosemite) and Homebrew (0.9.5), I've installed PostgreSQL 9.4.1. Basically all current versions as of the time of this posting...
I've successfully managed to import a brutally large database (56M records - 15 hours to import), and written an app to work with it locally. I found an app, pgAdmin3, that lets me do all sorts of admin tasks with the GUI. All is good.
Every time I restart the computer, once it was due to a kernel panic from a USB firewire audio interface, once was from a power failure, and two were from user initiated restarts - each time after reboot, the database is empty. Database users/roles are still there, tables and relations are there, however, all the tables are empty. 0 records. Nothing. pgAdmin3 even shows that the table has space allocated for some number of millions of records, with an "undefined" count. Refreshing the count, reveals 0 records. Browsing the data reveals 0 records. My Mac application (using libpq) connects to the database successfully, and gets 0 results from any/all of its SELECT statements. Is it redundant to mention that each re-import of the database takes 15 hours? (rhetorical question)
Any and all suggestions appreciated.
Mac OS X 10.0.2 (Yosemite)
Homebrew 0.9.5
PostgreSQL 9.4.1 on x86_64-apple-darwin14.1.0, compiled by Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn), 64-bit
The behaviour you describe would be consistent with your tables being UNLOGGED.
An unlogged table is halfway between a normal table and a temp table. It's like a normal table in that:
It's usable from multiple sessions
It isn't cleared when a session exits
It's stored in the main tablespace, not the temp tablespace
but it's like a temp table in that:
It isn't crash safe. The contents of UNLOGGED tables are preserved when PostgreSQL shuts down and restarts, but only if it's shut down cleanly. On unclean shutdown, UNLOGGED tables may be truncated, so they still exist but contain zero rows.
Writing to it doesn't produce write-ahead log records in pg_xlog
It can't be replicated to other nodes over streaming or log-based replication
Writing to an unlogged table causes about half the disk I/O as a regular table
See the manual.
At a guess, your tables are defined as UNLOGGED and you aren't shutting PostgreSQL down cleanly when your Mac shuts down. So it's truncating the unlogged tables, since it can't be sure they're in a consistent state.
Shut PostgreSQL down cleanly (e.g. pg_ctl -m fast -w stop, or a launchd configuration that makes sue to do that). Or don't use unlogged tables. Or be willing to re-load your data when you restart.
Since you wondered what unlogged tables are for:
They're great for data loading and transform, where you want to load a bunch of data, then extract and transform it for loading into other tables.
Reporting tables
Work queues and task queues for anything where the whole system can just be restarted, so you don't need to make the lists of in-progress tasks etc crash-safe
Craig Ringer's suggestion about UNLOGGED tables is great, the other thing to check is that you have committed whatever operation loaded the data.
I'm running some stored procedures in SQL Server 2012 under Windows Server 2012 in a dedicated server with 32 GB of RAM and 8 CPU cores. The CPU usage is always below 10% and the RAM usage is at 80% because SQL Server has 20 GB (of 32 GB) assigned.
There are some stored procedures that are taking 4 hours some days and other days, with almost the same data, are taking 7 or 8 hours.
I'm using the least restrictive isolation level so I think this should not be a locking problem. The database size is around 100 GB and the biggest table has around 5 million records.
The processes have bulk inserts, updates and deletes (in some cases I can use truncate to avoid generating logs and save some time). I'm making some full-text-search queries in one table.
I have full control of the server so I can change any configuration parameter.
I have a few questions:
Is it possible to improve the performance of the queries using
parallelism?
Why is the CPU usage so low?
What are the best practises for configuring SQL Server?
What are the best free tools for auditing the server? I tried one
from Microsoft called SQL Server 2012 BPA but the report is always
empty with no warnings.
EDIT:
I checked the log and I found this:
03/18/2015 11:09:25,spid26s,Unknown,SQL Server has encountered 82 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.HLSQLSERVER\MSSQL\DATA\templog.ldf] in database [tempdb] (2). The OS file handle is 0x0000000000000BF8. The offset of the latest long I/O is: 0x00000001fe4000
Bump up max memory to 24 gb.
Move tempdb off the c drive and consider mult tempdb files, with auto grow at least 128 Mbps or 256 Mbps.
Install performance dashboard and run performance dashboard report to see what queries are running and check waits.
If you are using auto grow on user data log and log files of 10%, change that to something similar to tempdb growth above.
Using performance dashboard check for obvious missing indexes that predict 95% or higher improvement impact.
Disregard all the nay Sayers who say not to do what I'm suggesting. If you do these 5 things and you're still having trouble post some of the results from performance dashboard, which by the way is free.
One more thing that may be helpful, download and install the sp_whoisactive stored proc, run it and see what processes are running. Research the queries that you find after running sp_whoisactive.
query taking hours but using low CPU
You say that as if CPU would matter for most db operations. HINT: They do not.
Databases need IO. RAM sin some cases helps mitigate this, but at the end it runs down to IO.
And you know what I see in your question? CPU, Memory (somehow assuming 32gb is impressive) but NO WORD ON DISC LAYOUT.
And that is what matters. Discs, distribution of files to spread the load.
If you look into performance counters then you will see latency being super high on discs - because whatever "pathetic" (in sql server terms) disc layout you have there, it simply is not up to the task.
Time to start buying. SSD are a LOT cheaper than discs. You may say "Oh, how are they cheaper". Well, you do not buy GB - you buy IO. And last time I checked SSD did not cost 100 times the price of discs - but they have 100 times or more the IO. and we talk always of random IO.
Then isolate Tempdb on separate SSD - tempdb either does no a lot or a TON and you want to see this.
Then isolate the log file.
Make multiple data files, for database and tempdb (particularly tempdb - as many as you have cores).
And yes, this will cost money. But at the end - you need IO and like most developers you got CPU. Bad for a database.