Am trying to restore the database from python 3.7 in Windows using below script.
Drop database functions correctly as expected.
The restore database doesn't work as expected, database always shows "Restoring...." and never completes.
Database files are there in the specified path, but database is not usable.
How to fix this?
import pyodbc
try:
pyconn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=MY-LAPTOP\\SQLEXPRESS;DATABASE=master;UID=sa;PWD=sa123')
cursor = pyconn.cursor()
pyconn.autocommit = True
sql = "IF EXISTS (SELECT 0 FROM sys.databases WHERE name = 'data_test') BEGIN DROP DATABASE data_test END"
pyconn.cursor().execute(sql)
sql = """RESTORE DATABASE data_test FROM DISK='G:\\dbbak\\feb-20-2020\\data_test_backup_2020_02_20_210010_3644975.bak' WITH RECOVERY,
MOVE N'Omnibus_Data' TO N'd:\\db\\data_test.mdf',
MOVE N'Omnibus_Log' TO N'd:\\db\\data_test_1.ldf';"""
print(sql)
pyconn.cursor().execute(sql)
while pyconn.cursor().nextset():
pass
pyconn.cursor().close()
except Exception as e:
print(str(e))
You're not using a single cursor, and so your program is exiting before the restore is complete, aborting it in the middle.
Should be something like:
conn = pyodbc.connect(' . . .')
conn.autocommit = True
cursor = conn.cursor()
cursor.execute(sql)
while cursor.nextset():
pass
cursor.close()
0
After hours I found solution. It must be performed no MASTER, other sessions must be terminated, DB must be set to OFFLINE, then RESTORE and then set to ONLINE again.
def backup_and_restore():
server = 'localhost,1433'
database = 'myDB'
username = 'SA'
password = 'password'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE=MASTER;UID='+username+';PWD='+ password)
cnxn.autocommit = True
def execute(cmd):
cursor = cnxn.cursor()
cursor.execute(cmd)
while cursor.nextset():
pass
cursor.close()
execute("BACKUP DATABASE [myDB] TO DISK = N'/usr/src/app/myDB.bak'")
execute("ALTER DATABASE [myDB] SET SINGLE_USER WITH ROLLBACK IMMEDIATE;")
execute("ALTER DATABASE [myDB] SET OFFLINE;")
execute("RESTORE DATABASE [myDB] FROM DISK = N'/usr/src/app/myDB.bak' WITH REPLACE")
execute("ALTER DATABASE [myDB] SET ONLINE;")
execute("ALTER DATABASE [myDB] SET MULTI_USER;")
Related
I have been able to read/write from Databricks into SQL Server table using JDBC driver. However this time I have to execute a command before I write to a SQL Server.
I need to execute this command on SQL server: SET IDENTITY_INSERT <sqlserver_table_name> ON
How to do this from Databricks ? Any help/pointers are appreciated. Thanks.
You can't do this with the JDBC Spark Connector (or the SQL Server Spark Connector), but it's trivial when using JDBC directly in Scala or Java. When using JDBC directly you have explicit control of the session, and you can issue multiple batches in the same session, or multiple statements in the same batch. EG
%scala
import java.util.Properties
import java.sql.DriverManager
val jdbcUsername = dbutils.secrets.get(scope = "kv", key = "sqluser")
val jdbcPassword = dbutils.secrets.get(scope = "kv", key = "sqlpassword")
val driverClass = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
// Create the JDBC URL without passing in the user and password parameters.
val jdbcUrl = s"jdbc:sqlserver://xxxxxx.database.windows.net:1433; . . ."
val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword)
val stmt = connection.createStatement()
val sql = """
SET IDENTITY_INSERT <sqlserver_table_name> ON
"""
stmt.execute(sql)
//run additional batches here with IDENTITY_INSERT ON
connection.close()
And you can always use the Spark Connector to load a staging table, then use JDBC to run a stored procedure or ad-hoc SQL batch to load the staging data into the target table.
I have DEV and STAGING servers. DEV is a different box than the others. This code works perfectly on DEV. On STAGING it always returns zero for both hCount and vCount.
-- BEGIN EDIT
I tried the suggestions in the comments to no avail. The stored procedure does include a call to an extended system stored procedure
EXEC Master.dbo.xp_fileexist #FileName, #File_Exists OUT
I did a bit of research on xp_fileexist and found this, "Before you begin implementation of this extended stored procedure into your applications, you will need to be aware of the security issues. When executing xp_fileexist (as well as other extended stored procedures), you will have the same rights and permissions as whichever NT account is configured to start SQL Server (MSSQLSERVER service). This account is generally either an administrator or system account. In either case, you pose a substantial security risk if you don't lock down the extended stored procedure to not allow your non-sa users to execute it."
I'm guessing this is where the problem lies. I'm looking into it.
-- END EDIT
When I run the stored procedure on the STAGING box in SQL Server 2017 separately from the web server, it returns the correct results:
exec [uDocCounts] 1, 'all', #Path, 0, 3, ''
My guess is the issue is with the communication between the web server and SQL Server.
EDIT: This deals with files/documents uploaded to server from application.
To further complicate matters, every page and stored procedure work fine on STAGING.
STAGING server:
Windows Server 2016 Version 1607
IIS Version 10.0.14393.0
SQL Server 2017 (v14.0.2027.2)
DEV server:
Windows 10 Pro
IIS Express
SQL Server 2017 (v14.0.1000.169)
Code:
// COUNT THE DOCUMENTS
var query = "uDocCounts"; // Stored procedure name
SqlCommand com = new SqlCommand(query, con);
com.CommandType = CommandType.StoredProcedure;
com.Parameters.AddWithValue("#UserID", uID);
com.Parameters.AddWithValue("#Type", ty);
com.Parameters.AddWithValue("#Path", MySession.Current.DocPath);
com.Parameters.Add("#countHidden", SqlDbType.Int).Direction = ParameterDirection.Output;
com.Parameters.Add("#countVisible", SqlDbType.Int).Direction = ParameterDirection.Output;
com.Parameters.Add("#words", SqlDbType.VarChar, size: 128).Direction = ParameterDirection.Output;
var iError = com.Parameters.Add("#ReturnVal", SqlDbType.Int);
iError.Direction = ParameterDirection.ReturnValue;
com.ExecuteNonQuery();
var intError = iError.Value;
if ((int)intError == 0)
{
var hCount = com.Parameters["#countHidden"].Value.ToString();
var vCount = com.Parameters["#countVisible"].Value.ToString();
var words = com.Parameters["#words"].Value.ToString();
lblCounts.Text = "There are " + hCount + " hidden documents and " + vCount + " visible ones";
lblCounts.Visible = true;
}
From the STAGING SQL Server 2016 :
conn = pyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};SERVER=localhost\<redacted>;DATABASE=master;UID=<redacted>;PWD=<redacted>')
cursor = conn.cursor()
query = """SELECT <redacted> FROM <redacted> WHERE <redacted>"""
row = cursor.execute(query).fetchone()
dummyName = row[0]
cursor.close()
cursor = conn.cursor()
query = """SELECT <redacted> FROM <redacted> WHERE <redacted>"""
print query
row = cursor.execute(query)
print row.fetchone()
This code properly connects to the db and executes the first query on the first db. However, when it executes the second query on the other db, it doesn't return any data and I get a popup window saying python.exe has stopped working when I try to fetch any rows, after which my program crashes. I checked and the query I'm trying to execute is a valid query that works properly from the master db and from the same account I'm connected to in the code.
The problem was we were using an old version of pyodbc. I updated it to the new version and now it works perfectly.
I am trying to do a full backup for an MSSQL database using go.
However, I tested the connection to the MSSQL server and it works like charm!
By executing also a query into the code SELECT ##VERSION
It gives me the version and that has been connected to mssql using the credentials in the code below.
By running a query BACKUP DATABASE name_db TO DISK = '/tmp/database.bak' it fails with the following error
Error with Backing up the DBmssql: BACKUP DATABASE is terminating abnormally
I am sure that I am missing something, any ideas/thoughts would be much appreciated.
BTW, I am on Ubuntu 16.04 and the mssql is installed there, I am using go version 1.9.
This is my code:
package main
import (
_ "github.com/denisenkom/go-mssqldb"
"database/sql"
"context"
"log"
"fmt" )
var server = "server_url"
var port = 1433
var user = "user"
var password = "pass"
var db *sql.DB
func main() {
var err error
// Create connection string
connString := fmt.Sprintf("server=%s;user id=%s;password=%s;
port=%d", server, user, password, port)
// Create connection pool
db, err = sql.Open("sqlserver", connString)
if err != nil {
log.Fatal("Error creating connection pool: "
+ err.Error())
}
log.Printf("Connected!\n")
defer db.Close()
Backup()
//Restore()
}
func Backup(){
ctx := context.Background()
err := db.PingContext(ctx)
if err != nil {
log.Fatal("Error pinging database :( " + err.Error())
}
var result string
err = db.QueryRowContext(ctx, "BACKUP DATABASE tempdb
TO DISK= '/tmp/Backup.bak' WITH NOFORMAT, NOINIT,
SKIP, NOREWIND, NOUNLOAD, STATS = 10;").Scan(&result)
//err = db.QueryRowContext(ctx, "select ##version").Scan(&result)
if err != nil {
log.Fatal("Error with Backing up the DB", err.Error())
}
fmt.Printf("%s\n",result)
}
I have found the solution which is very silly.. You can find it in the 1st reply.
That was obvious though, the code is clean and working as expected.
The only issue was from MSSQL, where it have a specific value set for the timeout queries.
That's why I succeeded executing other SQL queries like: SELECT ##VERSION from the example above in the Q.
Though, I set the value of the timeout for remote queries to 0 this SQL query:
EXEC SP_CONFIGURE 'remote query timeout', 0
reconfigure
EXEC sp_configure
The 0 in remote query timeout means that the timeout is unlimited.
The issue was from MSSQL and os , try query first on SQL Server Management Studio to explane where problem come from.
now, change this :
(
err = db.QueryRowContext(ctx, "BACKUP DATABASE tempdb
TO DISK= '/tmp/Backup.bak' WITH NOFORMAT, NOINIT,
SKIP, NOREWIND, NOUNLOAD, STATS = 10;").Scan(&result)
)
to
(
err = db.QueryRowContext(ctx, "BACKUP DATABASE tempdb
TO DISK= '/tmp/Backup.bak' WITH NOFORMAT, NOINIT,
SKIP, NOREWIND, NOUNLOAD, STATS = 10;").Err()
)
becouse db.QueryRowContext expected to return at most one row
and backup Query dont return any row
Do any sql ninja's know how to execute a script (.sql) from within another script? Specifically for the Sql Server 2005+ platform. E.g. psudeo code follows:
ScriptA contents
RESTORE DATABASE [MyDbName]
FROM
DISK = N'path' WITH FILE = 1
.......
GO
Exec c:\scripts\ScriptB.sql
ScriptB contents
USE [MyDbName]
GO
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[MyTableName]') AND type in (N'U'))
CREATE TABLE [dbo].[MyTableName]
........
GO
You can turn SQLCMD mode on from SSMS or run your script in SQLCMD.exe.
SQLCMD supports script variables and script references.
---script1-----
create table ....
:r path/to/another/script (which can reference others)
---------------
hope this helps
The following article provides examples of how you can achieve this.
http://www.sqlservercentral.com/articles/Administering/scriptrunner/292/
Here is a free tool that can run multiple scripts but not from within an existing script.
http://www.xsqlsoftware.com/Product/ProductDetails.aspx?ProdID=ScriptExecutor
Here is an example of executing a script from within another script by calling the isql utility using the xp_cmdshell stored procedure.
http://www.sqlservercentral.com/articles/Administering/scriptscheduling/450/
Should be what you need to get going but drop me a line if you need further assistance.
Cheers, John
Core MS SQL doesn't do this, but some tools have some macro capabilities (i.e. done client site).
SQLCMD supports ":r" to import another file.
As far as i can see you dont have to combine the two scripts. You can just execute the first and then the other.
You can do this by making a vbscript that loads the .sql files and then uses an ADODB.Connection to execute the two scripts.
This vbscript should do it:
Set con = CreateObject("ADODB.Connection")
con.ConnectionString = "*** Database connection string here ***"
con.Open
Set fso = CreateObject("Scripting.FileSystemObject")
Set sqlFile = fso.OpenTextFile("scriptA.sql", 1)
con.Execute sqlFile.ReadAll
sqlFile.Close
Set sqlFile = fso.OpenTextFile("scriptB.sql", 1)
con.Execute sqlFile.ReadAll
sqlFile.Close
Set fso = Nothing
Set con = Nothing