I have the scenario where I am using a FILESTREAM enabled table to upload and download files from.
I need the files available in two instances, database A and database B. Files can be uploaded to both A and B and must be synced to the other party. There is no restriction on uniqueness, identical files might be uploaded to the databases. Note the table to be replicated is used only for the files and nothing else.
How reliable is a Merge replication in this case? Some of the files can be up to 2GB in size. Is the replication revertible, i.e. if it fails midway while streaming the files to the other database will all the changes, caused by the replication, be rolled back?
Related
Background:
I have a medium-sized database (900GB) that needs to be copied onto another server (driven via code, not scheduled). Currently we take a backup (to .bak), move it to a staging server, and restore it to the target server. The target server does not have enough space to hold the backup file, and the restored instance simultaneously, thus the staging server. These transfers (backup to staging, restore from staging) happen over SMB2. The staging server needs to go away due to business requirements, however. It is worth mentioning the target server will be taken offline (and used offline) after the transfer, so I'm not sure the mirroring or replication options are valid.
I have identified two options -- one is to backup the database to the primary server, and open up firewall rules/smb to serve the backup file to the target server over SMB. ("RESTORE FROM \x.x.x.x\blah\db.bak"). Security isn't a fan, though.
The ideal solution (and one that could easily be implemented in every other database I've worked with), is to quiesce the database and transfer the datafiles (in the case of ms-sql, mdf and ldf files). However, upon research I see there is no such functionality available out of the box. I know I can take the database offline to copy the mdf/ldf safely, but that's not an acceptable solution (database must remain online).
I have read LOTS of posts and Microsoft documentation regarding VSS / shadow copy, but I have also read lots of conflicting information about the reliability of using VSS/sqlwriter to copy the mdf/ldf file to the target server, and simply re-attaching the database.
I am looking for documentation or advice (or even backup software that can be programmatically driven via an API) to accomplish this goal of transferring the database without requiring a secondary holding place. Currently I'm researching how to drive this copying process with Powershell, using VSS(vssadmin/vshadow from sdk), but I'm not confident in what I'm reading, and it's not even clear to me if VSS/sqlwriter is a supportable method to copying online LDF/MDF files. Any advice is appreciated.
Thanks,
I'm writing an mvc/sql server application that needs to associate documents (word, pdf, excel, etc) with records in the database (supporting sql server 2005). The consensus is it's best to keep the files in the file system and only save a path/reference to the file in the database. However, in my scenario, an audit trail is extremely important. We already have a framework in place to record audit information whenever a change is made in the system so it would be nice to use the database to store documents as well. If the documents were stored in their own table with a FK to the related record would performance become an issue? I'm aware of the potential problems with backups/restores but would db performance start to degrade at some point if the document tables became very large? If it makes any difference I would never expect this system to need to service anywhere near 100 concurrent requests, maybe tens of requests.
Storing the files as blob in database will increase the size of the db and will definitely affect the backups which you know and is true.
There are many things of consideration whether the db and code server are same.
Because it happens to be code server requests and gets data from db server and then from code server to client.
If the file sizes are too large I would say go for the file system and save file paths in db.
Else you can keep the files as blog in db, it will definitely be more secure, as well as safe from virus, etc.
We are working on generating a database that will be distributed to several third parties. We will also re-generate this database on an on-going basis, redistribute it, and those third parties will need to overwrite their existing database copy with the new version.
I'm trying to decide whether I want to detach the database temporarily, make a copy of the .mdf, and send that copy out, or whether I should just do a full backup of the database, and send the .bak out.
The primary difference I can see is that to distribute the .mdf, you must detach the database temporarily, so that you can copy it.
What are the other pros/cons of each format?
Are there security implications with distributing one over the other?
Is it easier to initially import one format over the other?
Thank you.
Neither. The proper way to distribute database changes is via upgrade scripts, otherwise those third parties using the database will loose the actual data contained in the database.
For the case when the data is never changed by the third parties (ie. the database is read only at those sites) then distribution by backup file is feasible. MDF is completely out of the question, first and foremost because MDF is not the entire database: at least the LDF is required in addition to recreate a coherent database. Simply attaching the MDF w/o a corresponding LDF will result in most cases, in a corrupt database. In addition to being incorrect, MDF distribution is inefficient (BAK files are smaller than the corresponding MDF because they do not contain unallocated pages) and also MDF manipulation requires placing the database offline during the file copy.
What would be the best strategy for a SQL server database copy/move from a source location separated by the Internet to a destination? (preferably automated)
The destination will not need to alter data but will require query access.
In this case both locations are not connected by VPN.
They are on different domains/networks.
The frequency of getting the copy from the source will be approximately twice a month.
Assuming both sites are using MS SQL Server 2008 (not Express/Standard editions).
Can include strategy with differential backups if suitable
Prefer to use a MS SQL Server solution rather than custom programmatic solution
Initial Database size and the incremental changes might be approximately ~150MB
Possible solutions I can think of:
Snapshot Replication with FTP - the only problem is trying to Automate the FTP connection in retrieval. Also provides flexibility with Articles that can be chosen. But the tables require Primary Keys (PK)
Log Shipping (Recovery in Standby mode) - the problem here is copying and retrieving the Transaction Logs from the folder; as they are on different networks.
For a single 30MB database twice a month, I would use an SSIS package which can handle either backup/restores or dumping /reading tables and the associated FTP
The fact of no direct connection limits your options immensely (replication, log shipping, mirroring etc)
Our EPOS system copies data by compressing the database into a zip file, and manually copying to each till, using shared directories.
Each branched is liked to the main location, using VPN which can be problematic, but is required for the file sharing to work correctly.
Since our database system currently does not support replication, is there another solution for copying data or should we migrate our software to another database?
Replication is the "right" way to go, so if migrating to another database is an option (is it really?), that's the best route.
You might consider a utility that queries all the tables for raw data (in CSV?), sending that to files. Then at least you don't have to take the database down to do the backup.