I have followed the various tutorials from philly and others to setup django with django-mssql-backend but have had no luck. I think the connection is working but when it is trying to parse the tables I get a collation error that it cannot get past. The specs of what I'm running are as follows:
django-mssql-backend: 2.8.1
django: 3.2
pyodbc: 4.0.30
'server':{
'ENGINE': 'sql_server.pyodbc',
'NAME': 'database',
'USER': 'username',
'PASSWORD': 'password',
'HOST': 'hostname of server',
'PORT': '',
'OPTIONS': {
'driver': 'ODBC Driver 17 for SQL Server',
'unicode_results': True,
},
}
When I attempt to run the migration class creator or:
python manage.py inspectdb --database=server
I get the following error in the output:
# This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
# * Rearrange models' order
# * Make sure each model has one field with primary_key=True
# * Make sure each ForeignKey and OneToOneField has `on_delete` set to the desired behavior
# * Remove `managed = False` lines if you wish to allow Django to create, modify, and delete the table
# Feel free to rename the models, but don't rename db_table values or field names.
from django.db import models
# Unable to inspect table 'ADObjectMemberships'
# The error was: __new__() missing 1 required positional argument: 'collation'
# Unable to inspect table 'ADObjects'
# The error was: __new__() missing 1 required positional argument: 'collation'
I am sure this is possible because I spooled up a different venv with django 1.8 installed and the old django-pyodbc-azure module installed and it connects to the tables and pulls their information. The biggest problem I have with it is that it stops with ~15 tables left in the DB and throws a memory error no matter what I do to fix it.
Any thoughts or help on the issue is greatly appreciated!
Confirmed through independent testing that the issue does appear to be a bug in version 3.2 and will put in a bug report. However I did come out with a workaround for now as follows:
Create new temporary virtual environment
Install django==3.0 pyodbc==4.0 django-mssql-backend==1.8
Create the database entry for the SQL Server in settings.py
run python manage.py inspectdb --database=yourentry > yourentry.py
Once you have created all the models for the existing database you want to use in your website you can grab each of the yourentry.py files and copy them to a submodel folder and import them into the main models.py file.
When the virtual environment is no longer needed it can be deleted.
Related
I'm trying to connect to MongoDB in a IBMCloud instance and I'm using MongoID. I ever use some configurations and try very things. I'm getting the error:
MONGODB | Error running ismaster on address:port: system lib
Here is my mongoid.yml that I'm trying to connect to mongoDB:
development:
clients:
default:
database: databasename
hosts:
- address:port
- address:port
options:
user: 'admin'
password: 'some-password'
auth_mech: :scram
auth_source: admin
connect: :replica_set
replica_set: replset
ssl: true
ssl_ca_cert: ./some-certificate.pem
ssl_verify: true
options:
log_level: :debug
What do I need to add to connect with the mongo database in IBMCloud?
"system lib" is coming from OpenSSL and indicates a problem with the CA certificate, such as the path not referring to an existing file. Note that in the configuration you specified a relative path which could cause problems.
I created https://jira.mongodb.org/browse/RUBY-1946 to add the exception class to the reported message which would clarify where the error is originating in the future.
Upgrade to the current version of Ruby driver (2.10.2 as of this writing) which provides more extensive diagnostics. Set driver log level to debug (https://docs.mongodb.com/mongoid/master/tutorials/mongoid-configuration/#logging). If you are having trouble identifying the problem then, post the
updated error and log messages here.
I am trying to connect a phoenix application to MS SQL. After looking around online I came across a couple of adapters called mssql_ecto & mssqlex.
I have added them to the project following the instructions in the readme, installed odbc and checked that the db is online but I am now getting the following error..
[error] Mssqlex.Protocol (#PID<0.13069.0>) failed to connect: ** (Mssqlex.Error) odbc_not_started
My app is configured as below..
config :my_app, MyApp.Repo,
adapter: MssqlEcto,
username: "<my_username>",
password: "<my_password>",
database: "test",
hostname: "<my_server>.database.windows.net",
pool_size: 10
My environment is as follows..
MacOS Sierra 10.12.5
Elixir version: 1.4.4
There is an issue in the mssql_ecto repo already, and I have tried the suggestions from there, but it still has not worked.
If anyone has managed to connect their phoenix/elixir application to MSSQL on macOS and could provide some instructions it would be greatly appreciated (even if it was done in a completely different way to my approach 😉 ).
Side note: Tried it in node with the same db and was able to connect to and query the db.
I've been working through the same issue this week, connecting to an Azure SQL Database, and this is what I've got working.
You are seeing the first of 2 main problems I ran into:
Erlang installed without ODBC
Mssqlex adapter is missing required configuration options
#1: Erlang installed without ODBC
If you've installed Elixir and Erlang using brew, or something like it, you are getting an installation of Erlang that does not include erlang-odbc. In order to remedy this, you will need to install Erlang from source.
I recommend that you install both Erlang and Elixir from source using the method documented here:
http://www.lambdacat.com/how-to-install-elixir-on-mac/
Just make sure that you remove the Kerl option, --without-odbc, before you build Erlang.
Now that you have Erlang installed correctly, the odbc_not_started error should go away... to be replaced by connection errors to the database.
#2: Mssqlex adapter is missing required configuration options
As of this moment, the mssqlex adapter does not support the following configuration options that Azure is looking for:
Encrypt
TrustServerCertificate
Connection Timeout
To get around this issue, I forked the mssqlex library and added those options in a branch. I am not committing to maintaining this so feel free to fork it yourself.
In order to use the modified mssqlex, replace the mix dependency with the github location and branch name:
{:mssqlex, git: "https://github.com/tvanhouten/mssqlex.git", branch: "azure-settings", override: true}
The override: true is important because mssql_ecto has a dependency on the mssqlex version in hex.
Now you can update your configuration:
config :my_app, MyApp.Repo,
adapter: MssqlEcto,
username: "<my_username>",
password: "<my_password>",
database: "test",
hostname: "<my_server>.database.windows.net",
pool_size: 10,
encrypt: "yes",
trust_server_certificate: "no",
connection_timeout: "30
Other "Gotchas" to Look Out For
Azure requires TCP connection:
In order to connect to an Azure SQL Database, you will need to make sure that the hostname: option starts with tcp:. So, all together, it will look like
hostname: "tcp:.database.windows.net"
Mssql_ecto does not support Ecto 2.2+:
If you're getting errors related to Ecto or Ecto.SubQuery, you probably need to make sure that you are using a version of Ecto that is supported by mssql_ecto. At this point in time, that means Ecto 2.1.x. Again, this requires a slight change to your mix dependencies:
{:ecto, ">= 2.1.0 and < 2.2.0"}
Dependency cache and lock issues:
If everything seems to be in order according to the above but it just doesn't seem like the new dependencies are being updated and compiled as expected, do the following:
Run mix deps.clean --all
Delete the mix.lock file from your root project directory
Run mix deps.get
Run mix deps.compile
That should do it!
I'm new to Heroku, so I tried following the instructions literally and I'm lost with this error.
So, when I included the settings.py configurations written in "Getting Started with Django on Heroku" I can't run the local server anymore unless I comment out South from my Installed Apps.
This is the error:
from south.db import DEFAULT_DB_ALIAS
File "/home/alejandro/Proyectos/olenv/local/lib/python2.7/site-packages/south/db/__init__.py", line 83, in <module>
db = dbs[DEFAULT_DB_ALIAS]
KeyError: 'default'
This is the relevant settings in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbolib',
'USER': 'alejandro',
'PASSWORD': 'zzzzz'
}
}
# --- HEROKU --- #
# Parse database configuration from $DATABASE_URL
import dj_database_url
DATABASES['default'] = dj_database_url.config()
Maybe it is supposed to be this way since South is a tool for production and therefore it conflicts with Heroku. Maybe not, since Im new to Heroku, any clarification will help.
Edit
When I comment out South and try to run syncdb, I ge this error:
File "/home/alejandro/Proyectos/olenv/local/lib/python2.7/site-packages/django/db/backends/dummy/base.py", line 15, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
requirements.txt:
Django==1.6
South==0.8.4
argparse==1.2.1
dj-database-url==0.3.0
dj-static==0.0.5
django-crispy-forms==1.4.0
django-debug-toolbar==1.2
django-endless-pagination==2.0
django-extensions==1.3.5
django-toolbelt==0.0.1
gunicorn==18.0
psycopg2==2.5.2
pystache==0.5.4
six==1.6.1
sqlparse==0.1.11
static==1.0.2
wsgiref==0.1.2
When you comment out South in your INSTALLED_APPS, does it let you run python manage.py syncdb locally?
I think the problem is that you're overwriting your default database with your dj_database_url lines in your settings.py. When you're on Heroku, there's an environment variable named DATABASE_URL, which is what dj_database_url uses to config your database. However, if you don't have a similar environment variable locally, you won't be configuring your database correctly. The reason South is throwing an error is because it tries to connect to your database upon initialization (note how the error comes from South's __init__). If you tried to actually use your database locally after commenting out South, I'm guessing you'd get an error.
There are two ways to fix this. The first, and easiest, is to make a DATABASE_URL variable on your local machine. Given the settings you listed above, set your DATABASE_URL to 'postgres://alejandro:zzzzz#localhost/dbolib'. The second, and more difficult, is to add something in your settings to essentially check if you're on Heroku and, if not, skip over the dj_database_url configuration. Basically, nest your dj_database_url lines in your settings inside an if statement that would return True if on Heroku and False if not.
EDIT:
It gets a little messy, but if you want to keep it all in one file, you could do something like the following:
First, try to set your default database using dj_database_url. This means moving the following code ABOVE the DATABASES setting in your settings:
import dj_database_url
DATABASES = {}
DATABASES['default'] = dj_database_url.config()
This code sets your default database equal to the result from dj_database_url. However, there may not be a result. You'll want to test for this (which would indicate local development) and set a database accordingly. So, AFTER THE PREVIOUS CODE, add the following code:
if len(DATABASES['default']) == 0:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbolib',
'USER': 'alejandro',
'PASSWORD': 'zzzzz'
}
}
This checks to see if you have anything in your DATABASES['default'] and, if not, sets your default database using your local settings.
This is a little messy, but it'll work. The better option may be to use different settings files, which adds another layer of complexity. Truly, the best option is to set your DATABASE_URL environment variable on all of your machines so that dj_database_url works as it should.
I'm following the Heroku Django tutorial. I believe I followed it exactly. I ran no additional commands besides what they asked for.
However, when I get to the part where I sync the Celery and Kombu tables (under the "Running a Worker" section), I get a bug.
Typing in their command python hellodjango/manage.py syncdb, gives me the following:
...
File "/Users/Alex/Coding/getcelery/venv/lib/python2.7/site-packages/django/db/backends/dummy/base.py", line 15, in complain
raise ImproperlyConfigured("You haven't set the database ENGINE setting yet.")
django.core.exceptions.ImproperlyConfigured: You haven't set the database ENGINE setting yet.
Anybody run into this problem before? Should I be doing something that's not explicit in the tutorial?
Any hints would be greatly appreciated!
Your output is from running the syncdb locally. Enabling the database addon will set DATABASE_URL in your config, and hence the environment of the dynos (see heroku config). What it won't do is set DATABASE_URL locally - you'll need to do that yourself (or sort some other local database)
Its likely because your DATABASE dictionary is undefined. Can you attempt to add this code which should read your database from the environment variable then the CELERY db can be setup from it:
import os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
I'm new to PostgreSQL and this is my first symfony project with PostgreSQL. When I try to run insert-sql in symfony, it tells me my database, salon, doesn't exist. But it does exist, as you can see below.
ason#ve:~/salon$ ./symfony doc:insert-sql
>> doctrine creating tables
PDO Connection Error: SQLSTATE[08006] [7] FATAL: database "SALON" does not exist
jason#ve:~/salon$ sudo -u postgres createdb salon;
createdb: database creation failed: ERROR: database "salon" already exists
It seems that any symfony command I run will tell me that salon doesn't exist (I've tried others). For this reason, my guess is that this is a symfony problem as opposed to a PostgreSQL or PHP problem.
all:
doctrine:
class: sfDoctrineDatabase
param:
dsn: pgsql:host=localhost;dbname=SALON
username: salon
password: foobar123
Any idea what I'm doing wrong?
Boy, do I feel dumb. If I use dbname=salon instead of dbname=SALON, it works.