Restore database from .frm and .ibd - database

I had a WordPress website running a long while ago. I used WAMP server. Then, I uninstalled WAMP. Today I would like to work again on my website; all that remains of my database is this (I cannot find the backup I once made):
So, there seem to be some data left.
However, here is what I get in phpMyAdmin:
It says my tables don't exist (no table were found for wp_db). Is there any way I can restore the database?

Related

MAMP Change Document Root Requires re-import of sql database

I develop on a Mac (High Sierra) with MAMP (basic, not Pro), and I have several different Drupal sites in ~/Sites/<sitename>.
Currently, if I change Document Root to work on a different site, I get errors on the new site until I delete the database .frm and .ibd files, as well as the log/data files, and re-import the database from a SQL dump.
Errors are:
2019-11-04T20:01:50.231324Z 2 [Warning] InnoDB: Cannot open table
/cache_bootstrap from the internal data dictionary of InnoDB
though the .frm file for the table exists. Please refer to
http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for
how to resolve the issue.
Files I have to delete:
/Applications/MAMP/db/mysql57/<sitename>/ (entire contents)
ibdata1
ib_logfile0
ib_logfile1
Re-importing the SQL database each time I want to work on a different site is getting really old. It is not a crisis, since this is just local development and it's not impacting the server, but it is seriously annoying. Has anyone else seen this?

Make SQL Server LocalDb portable

I'm setting up a new machine and to my surprise it is really difficult to get the localdb working on a new machine. Everything in my solution explorer populated, but nothing shows up in my (localdb) project after I fetch from source (using git).
What is the best way to get my localdb project from my old machine to this new one?
In an ideal world, what i'd like to do is have it set ups so then when some pulls the SSDT project down from our source they can hit deploy and completely populate the localdb project on their machine. Has anyone done this before, know how to do this?
--Disclaimer--
I feel like when talking about localdb it's important make some clarifications, because it seems that when other people post questions like this they get a lot of responses that apply to SQL Server databases.
What a localdb project is:
A localdb SQL Server project is a special server instance that runs only
when its connected to and acts partly like a SQL Server and
contains special instances of databases that only run when a
connection is made to them. A localdb can be used for certain types
of production, but they are most often used as a means to test other
databases.
What a localdb project is not:
A localdb project is NOT a SQL Server database.
I'm well aware that I could back up every DB in my project and manually recover them on the new machine, but that is not what I'm looking to do.
You can achieve this by detaching and attaching process.
Step 1: Find the location of the localDB database.
Right click on the database name and select properties
From the Data File property, you can find the database current location.
Step 2: Detach the current database
EXEC sp_detach_db 'aspnet-IdentityApplication-E2BBF1E6-123-4567-8910-07BC0413419B', 'true';
Step 3: Copy and Paste the localDB to the different location (The location where you want to put the localDB database)
Step 4: Attach the database with the new location
CREATE DATABASE [aspnet-IdentityApplication-E2BBF1E6-123-4567-8910-07BC0413419B]
ON (FILENAME = 'D:\Test\aspnet-IdentityApplication-E2BBF1E6-123-4567-8910-07BC0413419B.mdf'),
(FILENAME = 'D:\Test\aspnet-IdentityApplication-E2BBF1E6-123-4567-8910-07BC0413419B_log.ldf')
FOR ATTACH;

Automating import of data-tier application (SQL database) from Azure with a Master Key

When I extract a data-tier application from a Microsoft Azure SQL database that has a Master Key, I was unable to import it into SQL server on my local PC.
You will find others had this issue here: SSMS 2016 Error Importing Azure SQL v12 bacpac: master keys without password not supported
However the steps provided as the answer did not work on my installation.
Steps are
1. Disable auditing on the server (or database)
2. Drop the database master key with DROP MASTER KEY command.
Microsoft Tech Support verified this solution did not work on my installation of SQL Server and after actually taking remote control of my PC and trouble shooting, they were unable to determine why this was occurring.
I needed to find a way to remove the Master Key from the bacpac file. I have a Powershell script to remove the Master Key from the BACPAC file but it requires extracting, renaming files and running scripts from Windows Powershell to get the db imported.
Does anyone have a program or set of scripts which would automate the process of removing the Master Key and importing a SQL DB from Azure with a single command?
I am new to this forum. Please do not be harsh with this post. I am trying to do the best I can to help others to save the many hours I spent coming up with this.
I have cobbled together a T-SQL script which calls a Windows Powershell script (also cobbled from multiple sources) to extract a data-tier application (database) from Microsoft Azure SQL database and import it into a database on my local SQL Server by running ONE command. Over the months I found some of the code that is in my scripts from other blogs etc. I am not able to provide the credit due to those folks as I didn't keep track of where I got the info. If you are reading this and you see your code, please take credit. I apologize for not being able to give you the credit for your work.
There may be configuration settings on your PC and your local SQL server that need adjustments as this entire solution requires pretty much full access to your computer. If you run into trouble with compatibility, let me know and I will do the best I can to let you know how my system is configured in case it will help you.
I am using Windows 10 Pro and Microsoft SQL Server Developer (64-bit) v12.0.5207.0
I have placed the two files that do all the work on GitHub here: https://github.com/Wingloader/Auto-Azure-BACPAC-Download.git
GetNewBacpac-forGitHub.sql
GetAzureDB-forGitHub.ps1
WARNING: The Powershell script file will store your SQL sa password and your Azure SQL login in clear text!
If you don't want to do this, don't use this solution.
My computer is owned and controlled solely by me so I am able to open up the security in my system and I am willing to assume the responsibility of safeguarding it.
The basic steps of my solution are are accomplished as follows: (steps 1 and 2 are optional as I like to keep a version of the DB I am working with as of the point in time I pull down a clean production copy of my Azure DB)
Back up the current DB as MyLocalDB.bak.
Restore that backup from step 1 to a new DB with the previous day stamped at the end of the DB name (e.g., MyLocalDB20171231)
Delete the original MyLocalDB database (needed so we can recreate the DB with the original name later on)
Pull down the production database from Azure and create a new database with the name MyLocalDB.
The original DB is deleted in step 3 so that the restored DB can use the original name (important when you have data connections referring to that DB name)
In Step 4, the work of extracting the data-tier application DB from Azure is initiated by this line in the T-SQL:
EXEC MASTER..xp_cmdshell '%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe -File C:\Git\GetUpdatedAzureDB\GetAzureDB.ps1"'
The Powershell script does the following:
The target for the extract is a file named today.bacpac (hardcoded). The first thing to do is delete that file if it already exists.
Extract the DB from Azure into the today.bacpac file.
Note: my DB on Azure has a Master Key for encryption. This will need to be removed from the files prior to importing the bacpac file into your local DB or it will fail (this may not be required in SQL 2017 according to my previous conversations with MS Support). If you do not use a Master Key, you can either strip out the code that does this step or just leave it alone. It won't remove anything if it isn't there. It would just add a little overhead to the program.
Open the today.bacpac file (zip file) and remove the MasterKey node from the Origin.xml file.
Modify the Model.xml file to updates the SHA hash length. This is required in order for the file not to appear to have been tampered with when SQL opens the bacpac file.
Re-zips the files back into a new file today-patched.bacpac
Runs this line of code (from Powershell) to import the bacpac file into SQL Server
&C:\Program Files (x86)\Microsoft SQL Server\140\DAC\bin\SqlPackage.exe" /Action:Import /SourceFile:"C:\Git\GetUpdatedAzureDB\today-patched.bacpac" /TargetConnectionString:"Data Source=MyLocalSQLServer;User ID=sa; Password=MySAPassword; Initial Catalog=MyLocalDB; Integrated Security=false;"
After editing the two files to provide updated paths, usernames and passwords, run the SQL script. You do not need to edit the scripts again. You can run the SQL script again without modification and it will create a new copy of your Azure DB.
Done!

SSMS 2016 Error Importing Azure SQL v12 bacpac: master keys without password not supported

I have done this dozens of times but just recently ran into this error. Here are the steps I have gone through to get here:
Create a copy of my Azure SQL v12 database on the same server as the original
Export the copy-version (completely inactive from user interaction) to blob storage
Download the .bacpac file from blob storage to my local drive
In SSMS (October 2016 release) my local sql server instance, right click Databases and choose 'Import Data Tier Application'
Choose my recently downloaded bacpac file and start the import
It only takes a few seconds for it to bomb out and I get the error:
Error SQL72014: .Net SqlClient Data Provider: Msg 33161, Level 15, State 1, Line 1 Database master keys without password are not supported in this version of SQL Server
Error SQL72045: Script execution error. The executed script: CREATE MASTER KEY;
I followed the same process for the same database 1.5 months ago any everything worked fine...Is anyone else experiencing this??? I have SSDT version 14.0.61021.0 installed - not sure if that matters or not. I'm also running SQL Server 2016 Developer Edition (v13.0.1722.0).
I had the same problem. After speaking to Azure support they found out the issue was caused because a blank database master key is created to encrypt the storage credentials for the auditing (auditing is an optional setting).
Note that database auditing settings are inherited from the server settings.
Anyway, the work around they came up with was:
Disable auditing on the server (or database)
Drop the database master key with DROP MASTER KEY command.
Then the export works as expected. Hopefully Azure will fix this issue soon so that auditing and export can work together.
Update 21st March 2017 Better work-around From MS
As the fix will take some time to be deployed, they also suggested an
alternative solution, which will not require any additional steps
(like disabling auditing or the steps form the blog) on your side to
avoid this issue. After auditing is enabled, please update the master
key and set the password. Setting a password for the existing master
key will mitigate the issue. Also, setting the password will not
impact auditing and it will keep working. The syntax to add the
password is as follows:
-- execute in the user database
ALTER MASTER KEY ADD ENCRYPTION BY PASSWORD = ‘##############’;
The link also has a PowerShell script you can use to remove the offending SQL Statement from the .bacpac file.
Okay the way I ended up resolving this issue was to do the following:
Create a copy of the SQL Azure database on the same server as the original
In that database run the following script:
ALTER MASTER KEY REGENERATE WITH ENCRYPTION BY PASSWORD = [password here];
Follow steps 2-5 in the original post
I haven't seen documentation on this, but apparently when you create an Azure SQL database it creates a database master key (DMK) without a password, and in SQL Server 2016 this is not okay. Hopefully this helps somebody else out.
Note: I was able to do this since I only wanted the data from the original database to refresh my local development copy - I haven't fully researched the implications of this.
You need to:
remove the master key object from the bacpac
remove the credential object from the bacpac
There is this script published on Microsoft blog, run it, and will do all the above.
C:\PS> .\RemoveMasterKey.ps1 -bacpacPath "C:\BacPacs\Test.bacpac
Fix corrupt bacpac created by removing master key.
The Sugestion to run the script RemoveMasterKey, also created a corrupted bacpac file in my case by inserting  into the model.xml file in multiple places.
There is a way to edit the bacpac by extractig the files, Removing the offending characters in the model.xml file and then generating a new checksum for the Origin.xml file
once done Zipping the files back up with the extension .bacpac allows you to import the backpac.
Fix found at:
http://inworksllc.com/editing-sql-database-azure-bacpac-files/
checksum generator:
https://github.com/gertd/dac/tree/master/drop/debug
Steps provided:
1) Update the zip file with the modified model.xml
2) Rename the zip file with the bacpac extension
3) Run dacchksum.exe /i:database.bacpac (where database.bacpac is the name the bacpac file)
4) Update Origin.xml in the bacpac file with the new value provided by dacchksum.exe

TFS 2012 Restoration-based Move - Unexpected Outcome

The goal: decom the old server where TFS/SQL was originally installed, and run TFS/SQL on new server. To add insult to injury, the old server I will reference here is SBS 2011 - if you know anything about that environment, you may understand why it is slated for decom.
I performed a restoration-based move last week. While it was successful with respect to functionality, I now have what I would describe as a dual data + application tier implementation. Today, I have TFS/SQL installed on two servers, both with TFS Version: 11.0.60610.1 (Tfs2012.Update3) and SQL Version: 2008R2. Both servers in the same domain.
My curiosity lies in the behavior of the Tfs_Configuration db. I restored both the Tfs_ db as well as the Tfs_Configuration db (via .BAK files) to the new SQL server, but I still see activity happening on the old server here "c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\Web Services_tfs_data", but no updated/recent files in the same location on the new server, suggesting the Tfs_Configuration db really did not move/restore properly.
In the TFS console on the new server, I see the URLs in the “Application Tier Summary” section referring to the old server, but the Machine Name is the new server. I also see in the "Application Tiers" section, a reference to the old server Machine name. Yet, in verifying change logs, the Tfs_ db is now resident on the new server and accepting Visual Studio commits/check-ins. There is a Tfs_Configuration db on the new server, but it seems to be the default install copy and not my restored db.
In the various guides I have read, I do understand the web.config file holds the instructional set for the catalog, etc. here "appSettings … add key="applicationDatabase" value="Data Source=instance name;Initial Catalog=Tfs_Configuration;Integrated Security=True". I was expecting to change that entry once it migrated to the new server, but rather it is still parked on the old server.
I have turned off the TFS and SQL services on the old server as a trial to see if the new installation would pick up the load, but as you might expect, TFS then goes into an unavailable state to the users.
The primary questions are:
Why did the Tfs_Configuration db not restore to the new server in the same fashion as the Tfs_ db?
How can I move that Tfs_Configuration db and turn off that old SBS 2011 unit?
Any tips or tricks are welcomed and appreciated.
Thank you.
What you did is a non trivial operation (see Move Team Foundation Server from One Hardware Configuration to Another).
Typical missing steps:
Changing URLs
Cleaning caches
Changing server ID if you want to keep both instances live
Changing accounts in case you used local user accounts
I have completed this process successfully. It required a triangulation between the three servers. The essential aspects involved solid SQL backups, coupled with the settings.xml from healthy TFS Console backups.
It was certainly a process that took a lot of planning and anticipating snags.
All-in-all, it was a great exercise in watching the data flow and understanding the roles of the configuration and collection DBs. Thank you for responding to my inquiry.

Resources