I prefer to connect to databases with DSNs. I'm not a fan of putting user names and passwords in code or config files nor do I appreciate the trusted connection approach.
When I google how to MANAGE DSNs with Python, I wind up with some variation of the below.
import ctypes
ODBC_ADD_DSN = 1 # Add data source
ODBC_CONFIG_DSN = 2 # Configure (edit) data source
ODBC_REMOVE_DSN = 3 # Remove data source
ODBC_ADD_SYS_DSN = 4 # add a system DSN
ODBC_CONFIG_SYS_DSN = 5 # Configure a system DSN
ODBC_REMOVE_SYS_DSN = 6 # remove a system DSN
def create_sys_dsn(driver, **kw):
"""Create a system DSN
Parameters:
driver - ODBC driver name
kw - Driver attributes
Returns:
0 - DSN not created
1 - DSN created
"""
nul = chr(32)
attributes = []
for attr in kw.keys():
attributes.append("%s=%s" % (attr, kw[attr]))
if (ctypes.windll.ODBCCP32.SQLConfigDataSource(0, ODBC_ADD_SYS_DSN, driver, nul.join(attributes))):
return True
else:
print(ctypes.windll.ODBCCP32.SQLInstallerError)
return False
if __name__ == "__main__":
if create_sys_dsn("SQL Server",SERVER="server name", DESCRIPTION="SQL Server DSN", DSN="SQL SERVER DSN", Database="ODS", Trusted_Connection="Yes"):
print ("DSN created")
else:
print ("DSN not created")
When you run this, I wind up with this as output:
<_FuncPtr object at 0x0000028E274C8930>
I have two issues.
I'm not used to working with the OS through an API and I can't find much documentation or usage examples that aren't in C++. That said, the code runs, it just never returns true.
I can't figure out how to get it to kick out error information so I can diagnose the problem. This code leaves no footprint in the logs so nothing shows up in event viewer as something being wrong.
How can I troubleshoot this? Am I even doing the right thing by taking this route? What exactly IS best practice for connecting to databases with Python?
There are actually TWO answers to this question.
You can do it by calling PowerShell scripts from Python.
You don't. You use trusted connections in your connection string.
I went with option 2.
Related
I need to avoid storing plain text passwords in config files, and so I'm storing the Postgres password externally (in AWS Secrets Manager).
Similarly to the solution provided here:
Encrypted database password in Play + Slick + HikariCP application, I've been able to override dbConfig and supply the password to my DAO classes like this:
trait MyDaoSlick extends MyTableDefinitions with HasDatabaseConfig[MyPostgresDriver] {
protected val dbConfigProvider: DatabaseConfigProvider
override protected val dbConfig: DatabaseConfig[MyPostgresDriver] = secretDbConfig(dbConfigProvider)
def secretDbConfig(dbConfigProvider: DatabaseConfigProvider): DatabaseConfig[MyPostgresDriver] = {
DatabaseConfig.forConfig[MyPostgresDriver]("", dbConfigProvider.get[MyPostgresDriver].config
.withValue("db.user", ConfigValueFactory.fromAnyRef(getUN))
.withValue("db.password", ConfigValueFactory.fromAnyRef(getPWD)))
}
}
This works great for regular DB queries, however evolutions bypass this and still expect the username and the password to be in application.conf, which kind of defeats the purpose of the password being a secret.
Any advice on how evolutions could get the DB credentials from a function?
I ran into the same issue, and I managed to resolve it like this:
Create a custom application loader, as shown here: https://www.playframework.com/documentation/2.7.x/ScalaDependencyInjection#Advanced:-Extending-the-GuiceApplicationLoader
Inside the custom loader's builder, append the DB configuration parameters for Slick:
val extra = Seq(
"slick.dbs.default.db.url" -> secrets.url,
"slick.dbs.default.db.user" -> secrets.user,
"slick.dbs.default.db.password" -> secrets.pass
)
Nothing else needs to be changed, as you've basically added the configuration needed for anything Slick, evolutions included.
On older versions of Play, we used to do this inside GlobalSettings.onLoadConfig, but, at some point, that has been deprecated in favour of DI. More details here: https://www.playframework.com/documentation/2.7.x/GlobalSettings
I have an application that doesn't appear to be responding to SikuliX's (v 1.1.2) .focus(). It is one of three custom WinForms applications I have running. I found this link pertaining to my exact situation, but the suggestions did not help.
I have been able to get the code to work for Chrome, SQL Server, and other random applications I have running at the time. The problem seems to come in when I have more than one type of application running. If my applications are named "Version Launcher", "Device 1", and "Alternative", I am able to switch to "Version Launcher", but "Device 1" and "Alternative" aren't found.
class myDevice:
def startApp(self):
#my_app = App("Chrome") # works
#my_app = App("Visual Studio") # works
#my_app = App("Version Selector") # works
#my_app = App("Device 1") # does NOT work
my_app = App("Alternative") # does NOT work
my_app.focus(); wait(1)
my_device = myDevice()
my_device.startApp()
In order to rule out a bad name (perhaps, on some level, the application is not really named "Device 1"), I'd like to build a list of every application that Sikulix can detect at runtime. Has anyone ever tried such a thing? I've searched all over the documentation and cannot find any features that allows for this sort of querying.
Raimund Hocke, the maintainer of the SikuliX project, answered my question over on launchpad.
https://answers.launchpad.net/sikuli/+question/664004
In short, SikuliX uses the 'tasklist' command in Windows to grab the list of available applications.
I am attempting to use multiple datasources in a Grails 2.4.4 project. According to the docs, this should be possible:
http://www.grails.org/doc/2.4.4/guide/conf.html#multipleDatasources
My primary dataSource (the one I want to use for all domain classes) is using H2 at the moment, as configured by the default DataSource.groovy configuration. My second, read-only datasource is SQL Server, and I tried to declare it as follows at the top level of my DataSource.groovy config (shared by all environments):
ds {
pooled = true
dialect = "org.hibernate.dialect.SQLServer2008Dialect"
driverClassName = "net.sourceforge.jtds.jdbc.Driver"
url = "jdbc:jtds:sqlserver://myserver:1433/mydb;domain=mydomain;useNTLMv2=true;user=myuser"
dbCreate = "none"
}
(Don't let the URL throw you off - I'm just having to use Windows Auth with JTDS. I've tested this via third-party clients as well.)
I inject this into my service class and use it, and everything appears to hook up well:
def dataSource_ds
def serviceMethod(){
Sql ds = new Sql(dataSource_ds)
String query = "SELECT ... "
def results = ds.rows(query)
println "Results are ${results.size()}"
return "Some value"
}
But when I try to access this from an IntegrationSpec-backed Integration Test, I noticed that I was getting "schema not found" errors for valid schemas referred to by my query string, such as "dbo". And the stack trace of any errors from this setup looks like this:
org.h2.jdbc.JdbcSQLException: Schema "DBO" not found; SQL statement:
...
at org.h2.message.DbException.getJdbcSQLException(DbException.java:329)
at org.h2.message.DbException.get(DbException.java:169)
at org.h2.message.DbException.get(DbException.java:146)
at org.h2.command.Parser.readTableOrView(Parser.java:4774)
at org.h2.command.Parser.readTableFilter(Parser.java:1083)
at org.h2.command.Parser.parseSelectSimpleFromPart(Parser.java:1689)
at org.h2.command.Parser.parseSelectSimple(Parser.java:1796)
at org.h2.command.Parser.parseSelectSub(Parser.java:1683)
at org.h2.command.Parser.parseSelectUnion(Parser.java:1526)
at org.h2.command.Parser.parseSelect(Parser.java:1514)
at org.h2.command.Parser.parsePrepared(Parser.java:404)
at org.h2.command.Parser.parse(Parser.java:278)
at org.h2.command.Parser.parse(Parser.java:250)
at org.h2.command.Parser.prepareCommand(Parser.java:217)
at org.h2.engine.Session.prepareLocal(Session.java:414)
at org.h2.engine.Session.prepareCommand(Session.java:363)
...
Now why would THIS datasource be trying to use the H2 driver?
In case it's relevant, my Integration test looks like this:
void "serviceMethod" () {
when: "service method is called"
String response = myService.serviceMethod()
then: "we should get the appropriate text back"
response.equals("Some value")
}
If, in the Service class, I hard-code the connection using a constructor of the Groovy Sql object, the integration test works fine, and any stack traces go through the JTDS driver.But when I try to use the injected datasource, things are strange.
Any idea what I'm doing wrong here?
Just to close the loop on this and hopefully save someone pain on this oversight in the future:
Grails uses an in-memory database when running tests. Make sure to read up on the other differences between integration tests and production here:
http://www.grails.org/doc/latest/guide/testing.html#integrationTesting
This feature makes the use of external (read-only) datasources during any tests pretty interesting, but some of that is to be expected (a test which depends on an external datasource is not a very good test in the long run). I hope to refactor my app and its testing approach at some point (e.g., to use a simple DAO and mock that during the test), because I don't really care about asserting the contents of the external datasource from my app's tests.
Summary
I create a PKCS#10 CSR with certreq and have set the option Exportable=TRUE. This successfully creates a key under the location REQUEST. I also have a valid certificate with key in MY. If I try to access any one of them the CryptoAPI reports error code 0x80090016.
Running under different access rights could not solve this problem so far.
Goal
My goal is to get both the keys in MY and REQUEST. If I call CryptAcquireContextA() on any of those, it fails.
System
Windows 7 x64
Sample Source Code
My complete code looks like this:
hStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, 0, CERT_SYSTEM_STORE_LOCAL_MACHINE, "REQUEST");
pCert = CertFindCertificateInStore(hStore, X509_ASN_ENCODING, 0, CERT_FIND_SUBJECT_STR_A, "CERTIFICATE_SUBJECT", NULL);
CertGetCertificateContextProperty(pCert, CERT_KEY_PROV_INFO_PROP_ID, NULL, &len);
pinfo = (CRYPT_KEY_PROV_INFO *) malloc(len);
CertGetCertificateContextProperty(pCert, CERT_KEY_PROV_INFO_PROP_ID, pinfo, &len);
provname = wide_to_asc(pinfo->pwszProvName);
contname = wide_to_asc(pinfo->pwszContainerName);
if(!CryptAcquireContextA(&hCryptProv, contname, provname, pinfo->dwProvType, 0)) {
err = GetLastError();
fprintf(stderr, "Error: 0x%x\n", err);
}
CryptGetUserKey(hCryptProv, pinfo->dwKeySpec, &hUserkey);
This code is mostly copied from the OpenSSL capi engine. Since the engine failed, I created the smallest possible code to search the error.
The error
If I run this, it fails with the output Error: 0x80090016. This means one of three things according to Microsoft:
Key container does not exist.
You do not have access to the key container.
The Protected Storage Service is not running.
What have I done so far?
Started service "Protected Storage"
Verified container exists with MMC & Certificate Snap-In for Local Computer
Ran the same code on the User store in user context - it worked
File System Permissions
After some googling, I tried to change permissions on the file system. I found the files by looking at the contname variable of my code and searching for the file. I changed permissions on them (more accurate, I changed permissions on the parent folder). While this fixed the issue for MY, it seems I cannot change it for REQUEST.
One note here is that my container for MY seems to be here:
%APPDATA%\Microsoft\Crypto\RSA\S-1-5-21-1650336054-1974872081-316617838-545102
For REQUEST I found it under a different address:
%ALLUSERSPROFILE%\Microsoft\Crypto\RSA\MachineKeys
I am not sure on the workings here so I cannot explain why it would put them in different locations (one being user centric, the other one a system folder). The MY store was created with a regular administrator prompt and the command certreq -new inf_file.inf cert-csr.csr and after I received my certificate, I issued certreq -accept cert.pem. Then I created a new csr with the same command.
Different privilege levels
I tried to execute my program with the following privileges:
my local user account
admin prompt (cmd->start as administrator)
nt authority\system (whoami output)
To recieve a service prompt, I executed psexec.exe –ids cmd.exe according to a tip from MaaSters Center
Final words
Any help or guidance on how to further narrow this problem down will be greatly appreciated.
I was finally able to solve this problem and it is a lot simpler than I thought. I was sure that I would receive an unambiguous container name and don't need to be more specific but CryptAcquireContext actually requires me to pass a flag CRYPT_MACHINE_KEYSET.
So my function call has to look like this:
CryptAcquireContextA(&hCryptProv, contname, provname, pinfo->dwProvType, CRYPT_MACHINE_KEYSET)
Unfortunately this is not supported by the OpenSSL engine, so you would have to alter it yourself in the engine.
See MSDN: "A key container created without this flag by a user that is not an administrator can be accessed only by the user creating the key container and the local system account."
Complete details here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa379886(v=vs.85).aspx
I should preface by saying my experience with scripting or programming in OOP languages is limited.
I'm working on a method for programatically creating and executing SSIS packages using PowerShell. Unfortunately, most of the resources available for PowerShell and SSIS are for calling PS from SSIS, not the other way around.
I have, however, found a number of resources for VB/C# for creating SSIS packages.
Example resource here.
I've succeeded in converting most of the code by calling the DTS/SSIS assemblies, but it's failing now on converting the TaskHost object to a mainpipe.
Sample code:
[Void][Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.ManagedDTS')
[Void][Reflection.Assembly]::LoadWithPartialName('Microsoft.Sqlserver.DTSPipelineWrap')
# Create the Package and application, set its generic attributes
$Package = New-Object Microsoft.SqlServer.Dts.Runtime.Package
$Package.CreatorName = $CreatorName
$App = New-Object Microsoft.SqlServer.Dts.Runtime.Application
# Set connection info for our package
$SourceConn = $package.Connections.Add("OLEDB")
$SourceConn.Name = "Source Connection"
$SourceConn.set_ConnectionString("Data Source=$SourceServer;Integrated Security=True")
$TargetConn = $package.Connections.Add("OLEDB")
$TargetConn.Name = "Target Connection"
$TargetConn.set_ConnectionString("Data Source=$TargetServer;Integrated Security=True")
# Build the tasks
# Data Flow Task - actually move the table
[Microsoft.SQLServer.DTS.Runtime.Executable]$XferTask = $Package.Executables.Add("STOCK:PipelineTask")
$XferTaskTH = [Microsoft.SqlServer.Dts.Runtime.TaskHost]$XferTask
$XferTaskTH.Name = "DataFlow"
$XferTaskTH.Description = "Dataflow Task Host"
$DataPipe = [Microsoft.SQLServer.DTS.pipeline.Wrapper.MainPipeClass]($XferTaskTH.InnerObject)
Everything works fine til the last line, when I get the error:
Cannot convert the
"System.__ComObject" value of type
"System.__ComObject#{}" to
type
"Microsoft.SqlServer.Dts.Pipeline.Wrapper.MainPipeClass"
Any assistance or ideas are welcome!
Microsoft.SqlServer.DTSPipelineWrap makes heavy use of COM instances.
This forum post suggested using CreateWRapperOfType method:
http://social.technet.microsoft.com/Forums/en-US/ITCG/thread/0f493a31-fbf0-46ac-a6a5-8a10af8822cf/
You could try this:
$DataPipe = [System.Runtime.InteropServices.Marshal]::CreateWrapperOfType($XferTaskTH.InnerObject, [Microsoft.SQLServer.DTS.pipeline.Wrapper.MainPipeClass])
Doesn't error out and produces an object--I'm not sure of what type.
You could always just compile the working .NET version you referenced above into an exe, and allow it to accept parameters as needed in order to create the SSIS packages. Then, use Powershell to call the executable with the parameters as needed.