PostgreSQL "Database does not exist" but it does - database

I have been trying to create a database in PostgreSQL for several days and have run into several issues but seem to be stuck.
I have manually created a database in PostgreSQL called postgres_development because bundle exec rake db:create wasn't working.
Now, I am trying to run bundle exec rake db:migrate but it is not recognizing that I have a database called postgres_development.
This is my Rakefile.
require 'rake'
require 'rspec/core/rake_task'
require 'active_support'
require 'active_support/core_ext'
require_relative 'config'
namespace :db do
desc "Drop, create, and migrate the database"
task :reset => [:drop, :create, :migrate]
desc "Create #{APP_NAME} databases"
task "create" do
puts "Creating #{APP_NAME} development and test databases if they don't exist..."
system("#SET PGPASSWORD=#{DB_PASSWORD}; createdb --username=#{DB_USERNAME} --password=#{DB_PASSWORD} #{DB_NAME} && #SET PGPASSWORD=#{DB_PASSWORD}; createdb --username=#{DB_USERNAME} --password=#{DB_PASSWORD} #{TEST_DB_NAME}")
end
desc "Drop #{APP_NAME} databases"
task "drop" do
puts "Dropping #{APP_NAME} development and test databases..."
system("dropdb #{DB_NAME} && dropdb #{TEST_DB_NAME}_test")
end
desc "Migrate the database"
task "migrate" do
ActiveRecord::Migrator.migrations_paths << File.dirname(__FILE__) + 'db/migrate'
ActiveRecord::Migration.verbose = true
ActiveRecord::MigrationContext.new("/db/migrate/").migrate
end
desc "Populate the database with sample data"
task "seed" do
require APP_ROOT.join('db', 'seeds.rb')
end
end
namespace :generate do
desc "Create a database migration\n rake generate:migration NAME=create_people"
task :migration do
unless ENV.has_key?('NAME')
raise "Must specify NAME for migration, e.g. rake generate:migration NAME=create_people"
end
migration_name = ENV['NAME']
class_name = migration_name.camelize
timestamp = Time.now.strftime('%Y%m%d%H%M%S')
filename = "#{timestamp}_#{migration_name}.rb"
path = APP_ROOT.join('db', 'migrate', filename)
if File.exist?(path)
raise "ERROR! File '#{path}' already exists"
end
puts "Creating migration at #{path}"
File.open(path, 'w+') do |f|
f.write("class #{class_name} < ActiveRecord::Migration\n\tdef change\n\n\tend\nend")
end
end
end
desc 'Start IRB with application environment loaded'
task "console" do
exec "irb -r./config"
end
desc "Run the specs"
RSpec::Core::RakeTask.new(:spec)
task :default => :specs
# Will this not work?
#desc "Run the specs"
#task 'specs' do
# exec "rspec spec"
#end
And this is my config.rb in the same folder, postgres which I renamed from activerecord-template in a failed attempt to get it to connect to my database.
require 'pathname'
require 'pg'
require 'active_record'
require 'logger'
## Load all files and configure the db
APP_ROOT = Pathname.new(File.expand_path(File.dirname(__FILE__)))
APP_NAME = APP_ROOT.basename.to_s
DB_PATH = APP_ROOT.join('db', APP_NAME + "_development.db").to_s
DB_NAME = APP_NAME + "_development.db"
TEST_DB_NAME = APP_NAME + "_test.db"
DB_USERNAME = 'postgres'
DB_PASSWORD = '****'
if ENV['DEBUG']
ActiveRecord::Base.logger = Logger.new(STDOUT)
end
Dir[APP_ROOT.join('models', '*.rb')].each do |model_file|
filename = File.basename(model_file).gsub('.rb', '')
autoload ActiveSupport::Inflector.camelize(filename), model_file
end
ActiveRecord::Base.establish_connection :adapter => 'postgresql',
:database => DB_NAME,
:host => 'localhost',
:username => DB_USERNAME,
:password => DB_PASSWORD
Any thoughts on what is going on here would be greatly appreciated!

That seems obvious: You created a database postgres_development, then you try to connect to a database with a different name, namely postgres_development.db.
How is that supposed to work?

in your Rakefile in line 28, add a . in front of the /, i.e. changing it to ActiveRecord::MigrationContext.new("./db/migrate/").migrate
db/migrate/20181108113458_create_people.rb add a [4.2] at the end of the first line ie. changing it to class CreatePeople < ActiveRecord::Migration[4.2]

Related

Unable to connect to Pokertracker postgreSQL database

I am following this tutorial to connect to a PostgreSQL database stored on my computer.
I am using Windows 10 and I have installed PostgreSQL 9.3
I run the following code:
import psycopg2
class PostgreSQL():
def __init__(self):
self.conecta()
def conecta(self):
#establishing the connection
conn = psycopg2.connect(
database="PokerTracker 4 Database", user='postgres', password='dbpass', host='localhost', port= '5432')
#Creating a cursor object using the cursor() method
cursor = conn.cursor()
#Executing an MYSQL function using the execute() method
cursor.execute("select version()")
# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print("Connection established to: ",data)
conn.close()
if __name__=='__main__':
PostgreSQL()
I receive the following error from the cmd while running the script:
C:\Users\USUARIO\Desktop\OMAHA>python postgresql.py
Traceback (most recent call last):
File "C:\Users\USUARIO\Desktop\OMAHA\postgresql.py", line 18, in <module>
PostgreSQL()
File "C:\Users\USUARIO\Desktop\OMAHA\postgresql.py", line 4, in __init__
self.conecta()
File "C:\Users\USUARIO\Desktop\OMAHA\postgresql.py", line 7, in conecta
conn = psycopg2.connect(
File "C:\Users\USUARIO\AppData\Local\Programs\Python\Python39\lib\site-packages\psycopg2\__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError
What am I doing wrong? I copied the code from the tutorial.
If someone is trying to connect to Pokertracker database, the error was the database name was uncorrect. Go on pokertracker to File->Open User Data Folder. Open config folder, open pokertracker.cfg and there you have all the info.
In my case this are the parameters:
import psycopg2
class PostgreSQL():
def __init__(self):
self.conecta()
def conecta(self):
hostname = 'localhost'
username = 'postgres'
password = 'dbpass'
database = 'PT4DB'
myConnection = psycopg2.connect( host=hostname, user=username, password=password, dbname=database )
cursor = myConnection.cursor()
cursor.execute("""SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'""")
for table in cursor.fetchall():
print(table)
myConnection.close()
if __name__=='__main__':
PostgreSQL()
There you have the notes table. You can't edit the notes, they are protected by the system of Pokertracker. cheers

Pyodbc - SQL Server database restore incomplete

Am trying to restore the database from python 3.7 in Windows using below script.
Drop database functions correctly as expected.
The restore database doesn't work as expected, database always shows "Restoring...." and never completes.
Database files are there in the specified path, but database is not usable.
How to fix this?
import pyodbc
try:
pyconn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=MY-LAPTOP\\SQLEXPRESS;DATABASE=master;UID=sa;PWD=sa123')
cursor = pyconn.cursor()
pyconn.autocommit = True
sql = "IF EXISTS (SELECT 0 FROM sys.databases WHERE name = 'data_test') BEGIN DROP DATABASE data_test END"
pyconn.cursor().execute(sql)
sql = """RESTORE DATABASE data_test FROM DISK='G:\\dbbak\\feb-20-2020\\data_test_backup_2020_02_20_210010_3644975.bak' WITH RECOVERY,
MOVE N'Omnibus_Data' TO N'd:\\db\\data_test.mdf',
MOVE N'Omnibus_Log' TO N'd:\\db\\data_test_1.ldf';"""
print(sql)
pyconn.cursor().execute(sql)
while pyconn.cursor().nextset():
pass
pyconn.cursor().close()
except Exception as e:
print(str(e))
You're not using a single cursor, and so your program is exiting before the restore is complete, aborting it in the middle.
Should be something like:
conn = pyodbc.connect(' . . .')
conn.autocommit = True
cursor = conn.cursor()
cursor.execute(sql)
while cursor.nextset():
pass
cursor.close()
0
After hours I found solution. It must be performed no MASTER, other sessions must be terminated, DB must be set to OFFLINE, then RESTORE and then set to ONLINE again.
def backup_and_restore():
server = 'localhost,1433'
database = 'myDB'
username = 'SA'
password = 'password'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE=MASTER;UID='+username+';PWD='+ password)
cnxn.autocommit = True
def execute(cmd):
cursor = cnxn.cursor()
cursor.execute(cmd)
while cursor.nextset():
pass
cursor.close()
execute("BACKUP DATABASE [myDB] TO DISK = N'/usr/src/app/myDB.bak'")
execute("ALTER DATABASE [myDB] SET SINGLE_USER WITH ROLLBACK IMMEDIATE;")
execute("ALTER DATABASE [myDB] SET OFFLINE;")
execute("RESTORE DATABASE [myDB] FROM DISK = N'/usr/src/app/myDB.bak' WITH REPLACE")
execute("ALTER DATABASE [myDB] SET ONLINE;")
execute("ALTER DATABASE [myDB] SET MULTI_USER;")

SoapUI - How to connect database and prepare data before load test?

Recently I've discovered that you can make a JDBC request Test Step in SoapUI (doc 1, doc 2). And I have a load test that fails under certain conditions, i.e. I need to manually execute SQL script in order to prepare data each time before I run this load test.
I'm not sure that it's possible, but if it is, how can I automate my initialization step?
ps. If I simply add JDBC Request test step to the load test then this step executes multiple times and this is not what I want. I think I need to query database from setup script:
Possible, Setup Script will be run before test is executed - for example you can set a groovy script like:
import groovy.sql.Sql
// db connection
def DBurl = 'jdbc:oracle:thin:#11.111.1.11:1521:SID'
def DBuser = 'user'
def DBpassword = 'password'
def DBdriver = 'oracle.jdbc.pool.OracleDataSource'
def DBsql = Sql.newInstance(DBurl, DBuser, DBpassword, DBdriver)
// your sql
try{
DBsql.execute('''
[SQL U WANT TO EXECUTE]
''' )
} catch (Exception e) {
log.error e.getMessage()
}

Powershell error setting Environment variable for an SSIS package execution

My Powershell script executes an SSIS package, but first over-rides an Environment variable. The Alter method on the EnvironmentInfo object fails with a generic error message: "Operation 'Alter' on object [EnvironmentInfo[#Name='MyVariable']' failed during execution."
I also tried removing the environment variable and changing the Project parameter, but received the same error on the Alter method for the Project object.
I suspect this is either 1) a shortcoming of using the 32-bit version of SQL Server 2012, or 2) a permissions issue.
I've made sure the executing Windows Account has full privileges on the SSISDB database and the SSIS Catalog project, and the child folder, environment, etc.
Any ideas on the error or how I can get more details? I don't see anything in the Windows Event Logs.
Here's my code:
# Variables
$ServerInstance = "MyServer"
$32bitSQLServer = "false" #use #null for 32-bit
$SSISNamespace = "Microsoft.SqlServer.Management.IntegrationServices"
$FolderName = "MyFolder"
$ProjectName = "MyProject"
$PackageName = "MyPackage.dtsx"
$EnvironmentName = "MyEnvironment"
$VariableName = "MyVariable"
$VariableValue = Read-Host "What is the new environment variable value? "
# Create a connection to the server - Have to use Windows Authentication in order to Execute the Package
$sqlConnectionString = `
"Data Source=" + $ServerInstance + ";Initial Catalog=master;Integrated Security=SSPI;"
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection $sqlConnectionString
# Create the Integration Services object
[Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.IntegrationServices")
$integrationServices = New-Object $SSISNamespace".IntegrationServices" $sqlConnection
# Get the Integration Services catalog
$catalog = $integrationServices.Catalogs["SSISDB"]
# Get the folder
$folder = $catalog.Folders[$FolderName]
# Get the project
$project = $folder.Projects[$ProjectName]
# Get the environment
$environment = $folder.Environments[$EnvironmentName]
# Get the environment reference
$environmentReference = $project.References.Item($EnvironmentName, $FolderName)
$environmentReference.Refresh()
# Get the package
$package = $project.Packages[$PackageName]
# Set the Environment Variable
$environment.Variables[$VariableName].Value = $VariableValue
$environment.Alter()
# Execute the package
Write-Host "Running Package " $PackageName "..."
$result = $package.Execute($32bitSQLServer, $environmentReference)
# Alternate approach, also not working
# $project.Parameters[$VariableName].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Literal,$VariableValue)
# $project.Alter()
# $result = $package.Execute($32bitSQLServer, $environmentReference)
Write-Host "Done."
Just alter the Parameter on the package and don't worry about the environment variable. I am sure it doesn't change the value stored in the package on the server, just the object held in your$package variable. Something like this:
$package.Parameters[$VariableName].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Literal, $VariableValue)
$package.Alter()
$execution = $integrationServices.Catalogs['SSISDB'].Executions[$package.Execute($false, ($package.Parent.References[$package_name, $folder_Name]))]
do {
$execution.Refresh()
} until ($execution.Completed)
So with some digging I found the answer to my error. Turned out I had multiple versions (11.0.0.0, 12.0.0.0, 13.0.0.0) of the assembly Microsoft.SqlServer.Management.IntegrationServices in my GAC (Windows\assembly). Examining the schema of the SSISDB catalog, it was version 12.0.5000.0, which meant I needed the 12.0.0.0 version of the assembly. The code I was using:
[Reflection.Assembly]::LoadWithPartialName( "Microsoft.SqlServer.Management.IntegrationServices")
was loading the wrong version (probably 13.0.0.0), so I needed to explicitly load the assembly version matching this installation of SSIS, which was 12.0.0.0:
[System.Reflection.Assembly]::Load("Microsoft.SqlServer.Management.IntegrationServices, Version=12.0.0.0, Culture=neutral, processorArchitecture=MSIL, PublicKeyToken=89845dcd8080cc91")

Troubleshooting DSC configuration

I'm looking for an advice on how to troubleshoot DSC configuration. I'm using Azure DSC Extension and the logs are:
VERBOSE: [2017-09-08 02:56:48Z] [VERBOSE] [sql-sql-0]:
[[xSqlServer]ConfigureSqlServerWithAlwaysOn] The file "MSDBLog" has been
modified in the system catalog. The new path will be used the next time the
database is started.
VERBOSE: [2017-09-08 02:56:50Z] [ERROR] Exception setting "StartupParameters":
"STARTUPPARAMETERS: unknown property."
VERBOSE: [2017-09-08 02:56:50Z] [VERBOSE] [sql-sql-0]:
[[xSqlServer]ConfigureSqlServerWithAlwaysOn] Stopping SQL server instance
'MSSQLSERVER' ...
VERBOSE: [2017-09-08 02:57:07Z] [VERBOSE] [sql-sql-0]:
[[xSqlServer]ConfigureSqlServerWithAlwaysOn] Starting SQL server instance
'MSSQLSERVER' ...
From there it just freezes then times out.
This is the only part of my DSC configuration that uses xSQLServer:
xSqlServer ConfigureSqlServerWithAlwaysOn
{
InstanceName = $env:COMPUTERNAME
SqlAdministratorCredential = $Admincreds
ServiceCredential = $SQLCreds
MaxDegreeOfParallelism = 1
FilePath = "F:\DATA"
LogPath = "G:\LOG"
DomainAdministratorCredential = $DomainFQDNCreds
DependsOn = "[xSqlLogin]AddSqlServerServiceAccountToSysadminServerRole"
}
This is the part of MicrosoftAzure_xSqlServer.psm1 that contains mentions of "startup":
function Alter-SystemDatabaseLocation([string]$FilePath, [string]$LogPath,[PSCredential]$ServiceCredential )
{
$permissionString = $ServiceCredential.UserName+":(OI)(CI)(F)"
icacls $FilePath /grant $permissionString
icacls $LogPath /grant $permissionString
Invoke-Sqlcmd "Use master"
Invoke-sqlCmd "ALTER DATABASE tempdb MODIFY FILE (NAME = tempdev, FILENAME = '$FilePath\tempdb.mdf');"
Invoke-sqlCmd "ALTER DATABASE tempdb MODIFY FILE (NAME = templog, FILENAME = '$LogPath\templog.ldf');"
Invoke-sqlCmd "ALTER DATABASE model MODIFY FILE (NAME = modeldev, FILENAME = '$FilePath\model.mdf');"
Invoke-sqlCmd "ALTER DATABASE model MODIFY FILE (NAME = modellog, FILENAME = '$LogPath\modellog.ldf');"
Invoke-sqlCmd "ALTER DATABASE msdb MODIFY FILE (NAME = MSDBData, FILENAME = '$FilePath\msdbdata.mdf');"
Invoke-sqlCmd "ALTER DATABASE msdb MODIFY FILE (NAME = MSDBLog, FILENAME = '$LogPath\msdblog.ldf');"
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SqlWmiManagement')| Out-Null
$smowmi = New-Object Microsoft.SqlServer.Management.Smo.Wmi.ManagedComputer
$sqlsvc = $smowmi.Services | Where-Object {$_.Name -like 'MSSQL*'}
$OldStartupParameters = $sqlsvc.StartupParameters
$params = '-d'+$FilePath+'\master.mdf;-e'+$LogPath+'\ERRORLOG;-l'+$LogPath+'\mastlog.ldf'
$sqlsvc[1].StartupParameters = $params
$sqlsvc[1].Alter()
}
What should be my next steps to understand the problem?
If it makes any difference, I'm trying to create SQL Server Always On Availability group on Windows Server 2016 and SQL Server 2016 SP1, by using templates and DSC code that work with Windows Server 2012 R2 and SQL Server 2014.
my advice is to use existing working code ;) I don't think it makes sense to copy paste the blog post. But I'll leave links to relevant files here.
Links:
1. DSC modules
2. ARM Template. Note: Arm template that deploys sql needs to have vnet and domain already in place. Also, you will get a hard time trying to deploy this template outside of that framework, so probably just copy out dsc extension\dsc scripts and use in your deployments.
3. Parameters file example
The xSqlServer (the new one) is not capable of moving sql to another disk, so you are stuck with custom script or this old xSql module. Also, please note, DSC modules inside packages are modified. These DSC configurations won't work with not modified versions of modules (xSqlCreateVirtualDataDisk, xDatabase, xSQLServerAlwaysOnAvailabilityGroupListener maybe some other modules, I can't recall at this point).
PS. working that configuration out and patching relevant parts wasn't exactly a pleasant journey...
PPS. that repo also contains dsc configuration for ADDS that can also run in parallel (compared to official example).
try this.
$sqlsvc = $smowmi.Services | Where-Object {$_.Name -eq 'MSSQLSERVER'}
$OldStartupParameters = $sqlsvc.StartupParameters
$params = '-d'+$FilePath+'\master.mdf;-e'+$LogPath+'\ERRORLOG;-l'+$LogPath+'\mastlog.ldf'
$sqlsvc[0].StartupParameters = $params
$sqlsvc[0].Alter()

Resources