I am not a database administrator. Our DBA left the organization on bad terms about a month ago and I am a application developer and didn't knew anything about database administration about a week ago, so please bear with me.
We have two servers, Server A (primary server) and Server B (secondary server). Mirroring was configured between A and B.
A was configured to replicate our DB tables to C (reporting server). Replication was configured on between A and C
A went down due to hardware failure (hard drives corrupted); B now is our only server.
A was down so our replication broke between A and C.
B was acting as our principal server now. Note that we were already low on disk space (different configurations of both servers), unusual behavior was observed that the disk space was being filled up so quickly.When the disk space was full the error encountered was “the transaction log for database ‘DBName’ is full due to ‘REPLICATION’.”
Note that the replication between B and C was not configured.
We faced a downtime of 18 hours
During that downtime started replacing hard drives of A and reconfigured it from scratch (windows server installation, SQL server installation)
We then copied the .mdf and .ldf files from B to our fresh A and restored them.
We are now on our A (primary) back again.
We bought a new server (D) to make work as our secondary server and restored the latest back up (DBName.bak) of fresh working A and the latest transaction log file backup (DBName.trn) taken after the full database backup. This was done to configure mirroring between A and D successfully.
We configured mirroring between A (primary) and D (secondary) successfully.
Now we are trying to configure replication between A (reporting) and C (reporting server) for that we need to make sure that our transnational log backup job is running which takes the backup and then shrinks the ldf files, the job runs after every 15 mins.This job is not running successfully. We notice an error with message ID 9003
Error :
The log scan number (7796674:12164:1) passed to log scan in database ' DBName ' is not valid. This error may indicate data corruption or that the log file (.ldf) does not match the data file (.mdf). If this error occurred during replication, re-create the publication. Otherwise, restore from backup if the problem results in a failure during startup
We have read several blogs and articles proposing different solutions but the scenario doesn’t match to mine. I am confused to apply any of those solutions.
Currently we don’t have any replication configured. So the best solution a could read out of blogs were to apply sp_repldone which will remove or mark all the pending replication transactions as replicated from the transactions log files.
Given the above scenario should we be applying the above solution.
Related
Following a SAN issue a SQL database was marked Suspect. Due to the extent of the inconsistencies recovery was from a valid backup & log backups. No other system or other user databases had issues and CHECKDBs succeeded. The recovered database also had a successful CHECKDB and the application was re-enabled.
However the daily backup have been failing on the problem database. CHECKDB continues to succeed with no errors. Full, Copy_Only backups produce the same error (have also tried continue_after_error)
Msg 3203, Level 16, State 1, Line 3
Read on "mydb.mdf" failed:
23(failed to retrieve text for this error. Reason 15105)
Msg 3203, Level 16, State 1, Line 3
BACKUP DATABASE is terminating abnormally.
I also see in the System Event log
The device, \Device\Harddisk2\DR2, has a bad block.
The server itself has since been restarted and SQL Server came back online with no errors. CHECKDB continues to report no errors for any of the databases - but the position is worsening with no valid backup now for over a week.
Other forums suggest this error may be due to file access/permissions or not enough disk space for the backup to complete but this is not the case, having tried backing up to several different locations under different credentials with the same outcome.
I’m putting together a process to export all the DB objects and bulk copy all the data out into a clean database. Another option I’ve considered is detaching/stopping sql and copying the mdf,ndf, ldf files to another server, but reluctant to stop SQL Server at the moment without securing the data first.
Would welcome any thoughts, further checks I might be able to perform whilst the DB is online to establish what the bad block might relate to.
Screenshot 1 - shows running the backup gets 70% through.1
Just to say that we have concluded the mdf file is beyond repair. To share the scenario again
With the bad sector in the mdf file
the T-Log backups succeeded
the Database was still accessible/functioning
and CHECKDB were appearing good
However
* Full & Diff backups failed
* the MDF file could not be copied when the DB was detached
* the DB could still be be reattached in situ
Due to some poor file management & the delayed identification of this whole issue
the log chain become broken (due to limited log backups retention)
the only solution was to restoring an old backup and painful copy out of the data
I have inherited a SQL Server 2008 live server with a hot swap-able backup server which has transaction logs shipped every 15 minutes from multiple production servers. One of the production servers keeps getting out of sync, when I came it the last successful log restore on the backup was over a year ago... So clearly my predecessor wasn't watching this, I restored the database and ensured that the logs synced correctly at the next 15 minute interval. However, every couple of days or so (it is random sometimes an hour sometimes 3 days) it gets back out of sync and I have to spend 10 minutes of my morning restoring the database.
Basically I am wondering what I need to be looking at to figure out why these keep getting out of sync? I ran a query I found on sqlauthority.com which shows me the trn's and their license numbers for the primary database and when I try to just restore the transaction logs on the backup database with the file which is supposed to be the next license to get it back in sync it says the license number is too recent, so I try the next file and it says is out of sync and can't be restored.
Any help would be appreciated :)
Upon digging into the job history further I found that my LS_Restore job was succeeding with errors, which were that the sql server agent service account didn't have access to the folder the trn files were stored which it was trying to restore the database from.
I researched what the original developer had setup as the sql server agent service account and then granted that user the necessary permissions, the logs are now restoring as needed.
I hope this information helps someone in the future!!
I have a database that has lost one of its .ndf files and have been unable to get at the data. The .ndf file in question was added last Thursday and placed in a temporary storage location by a colleague (d'oh!). There is no backup available from this database since prior to this .ndf being created. I have seen numerous solutions to similar problems when the .ndf in question is its own filegroup, but in this case it actually is in a filegroup with an additional file which I want to try and get data from. I am pretty sure what I am trying to do is not possible but there is always a chance right?
The database setup
PRIMARY: Data.mdf -200mb
Data Filegroup 1: Data_1.ndf - 2.9gb
Data_2.ndf - 64gb (newly added file that is now lost - I believe it is just preallocated space)
LOG: Log.ldf - 128mb
When we logged into the VM this morning (hosted in Azure), we were presented with an unexpected shutdown notification from Windows (it seems there was a powerloss/shutdown at 1am) and our application was not reaching the database. Looking in SQL Server Management Studio I could see that it was Recovery Pending status. Trying to bring the db online lead me to an error about Data_2.ndf not being found (located at D:\SQL\Data\Data_2.ndf).
When I accessed the D drive (temporary storage drive) I was presented with a wonderful blank Windows Explorer window - completely blank drive.
I was able to set the Data_2.ndf file offline and bring the database itself online, however I am not able to query any of the data (as all tables were in Data Filegroup 1) due to the filegroup being offline. The other 3 files (mdf, ndf, ldf) are all online.
Is there any way out of this? Any way to perhaps recover any remaining data from Data_1.ndf or is it completely toast?
(This was a hastily stood up development server and there was no backup/recovery strategy for it, as "Azure never crashes" :)).
(Edit:formatting)
You are hosed. Its a miracle you can bring up your database. Are you sure you can retrieve data - have you tried doing selects? You probably will receive more extensive answers on the Database Administrators group.
I have a server which is running SQL Server 2008R2 SP1 and there are more than 200 online databases with their specific login and I want to move all of these databases to another machine with the same software SQL Server 2008R2 SP1.
I can afford downtime up to 8 hours so I'm not looking for complex solution like database mirroring and logshipping.
I have tried Backup & Restore, it work fine but trouble me in two ways :
1- I couldn't transfer login to new server so all of my users will have to create their user again which will my users to get angry.
(I have tried to script all of the logins from the security section but since I have realize the hash algorithm between two system maybe different, I cannot transfer my logins in this way)
2 - I have to backup & restore each database one by one which is so time consuming.
I have also tried copy wizard, but I have encountered with following error message at the final stage :
"SQL Server schedule job" job failed the job was invoked by user sa step to run was step 1.
Since mysource server is running based on windows server 2003, I cannot use the powershell V3 solution.
I have done many searches but all I found was a solution to transfer one database to another server.
But since I'm looking for a solution to transfer mass databases, this situation is being hair pulling and so difficult So I wil be very appreciated if you can provide me an easy and practical solution.
for moving logins between servers
https://support.microsoft.com/en-us/kb/918992
[moving logins][1]
then backup and restore all databases in that migraton day using pre built restore script if the db size is small , later restore the differential backups and make it online. if the size is an issue then implement log shipping or mirroring
I really need some help here.
I'm the owner of a SQL Server Database application that lost three days data! I can't understand how or why.
So here is the set-up.
SQL Server 2005 32bit standard edition database on Windows 2000 server. (Database B)
Database is in simple recovery mode
The database is connected as a subscriber to another database(SQL Server 2005 64bit enterprise edition on Win2k3 enterprise) using SQL Server continuous Merge Replication. (Database A)
DatabaseB was rebooted on night X as part of scheduled reboot. When the database came back up it was used as normal for a couple of days and data was created into it perfectly fine.
But then yesterday Day X + 4 it lost a lot of data.
Database B is on a server with another instance of SQL Server and they both started to run out of memory(conflicting with each other).
Here is the sequence of events from the event log when I think this happened.
AppDomain 2 (DatabaseB.dbo[runtime].1) is marked for unload due to memory pressure.
AppDomain 2 (DatabaseB.dbo[runtime].1) unloaded.
BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG is deprecated.
The simple recovery model should be used to automatically truncate the transaction log. (on DatabaseB)
AppDomain 3 (DatabaseB.dbo[runtime].2) created.
I know the data is missing because of my audit logs and that a user had taken a screen shot of some of the data before it was deleted.
So here is my dilema...how could this have happened?
How can several days data go missing from DatabaseB?? (it subsequently is missing from the publication db also!)
Did the truncate with the Appdomain down cause the data to be flushed from the log?
Any and all theories considered. If anyone needs more data I can add it.
Help!
This isn't the answer you want to hear, but in a nutshell, SQL Server doesn't "lose" data. Someone deleted it. If you had the database in full recovery mode, you could use a product like Quest LiteSpeed to read the logs and identify exactly how it was deleted, but in simple mode...sorry, sir, but you're out of luck.
Merge replication is implemented with triggers, so it doesn't need full recovery. Is it possible that someone disabled all triggers in the db? its easy to do DISABLE TRIGGER [database] This would at least account for the subscriber losing data.
Those appdomain lines in the log don't mean that much, its the SQL CLR telling you its unloading assemblies to free up some memory. & then reloading them later on.
Truncating the log removes inactive parts that have been committed to disk, having the recovery model set to simple means there's no point in truncating the log, as the message suggests.
None of this explains why data went missing on both the servers though. There has to be something else that caused this.
How did you verify that for the 4 days when everything was 'created perfectly fine' that it actually was? do you have backups from these days? can you see records with time stamps from those days?
Is it possible there's a ghost in the machine that did a restore without telling you?