I'm trying to set up a Debezium SQL Server Connector against a SQL Server instance that is controlled by DBAs at my workplace. I've been able to start up Zookeeper and Kafka Server without issue, and Kafka Connect itself works with sample Connectors, but when attempting to start a Debezium SQL Server Connector instance I've been getting the error "Couldn't obtain database name".
[2022-07-12 16:36:04,269] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:117)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.runtime.rest.errors.BadRequestException: Connector configuration is invalid and contains the following 1 error(s):
Unable to connect. Check this and other connection properties. Error: Couldn't obtain database name
Here is my debezium config:
name=Dbz-SqlServer-connector
connector.class=io.debezium.connector.sqlserver.SqlServerConnector
database.hostname=MyDbHost
database.port=1433
database.user=MyUsername
database.password=MyPassword
database.dbname=MyDatabase
database.server.name=MyDbHost
table.include.list=dbo.CdcTest
database.history.kafka.bootstrap.servers=localhost:9092
database.history.kafka.topic=dbhistory.CdcTest
I've tried this in a .properties file passed to a standalone Connect instance, and as a JSON POST to a distributed Connect instance. I have tried all of the same steps on both my local Windows machine as well as on a linux VM, with the same results.
Confluent and Docker are not options for me in this situation.
for SQL Server login credentials, I am using a local account on the SQL Server instance that does have access to the database in question. I found the source code for debezium's connectors on their github and was able to find that specific error message within the code:
private static final String GET_DATABASE_NAME = "SELECT name FROM sys.databases WHERE name = ?";
...
public String retrieveRealDatabaseName(String databaseName) {
try {
return prepareQueryAndMap(GET_DATABASE_NAME,
ps -> ps.setString(1, databaseName),
singleResultMapper(rs -> rs.getString(1), "Could not retrieve exactly one database name"));
}
catch (SQLException e) {
throw new RuntimeException("Couldn't obtain database name", e);
}
}
I'm not completely familiar with Java but it appears that basically something is going wrong when the connector is trying to run "SELECT name FROM sys.databases WHERE name = 'MyDatabase'". When I run this against the database myself, logged in with the same account I'm using, it seems to work just fine, so I'm really not sure where to go from here. It is fair to say that since I'm not in full control of the SQL Server environment that I'm using, there may be some permissions issues that I'm not aware of, but from what I'm able to test it seems like it should be working.
I would greatly appreciate any help at all, whether just suggestions on settings/configs to check or a full-blown solution.
Thank you!
Update: I've built a simple console app to run that sys.databases query against MyDbHost, master database, as the relevant account, and it's working just fine so I feel like that confirms that my connection info is correct and account permissions are also correct. Seems like this is an issue within the Debezium connector.
It turned out that my problem was a mistake in the connector's config setting. I misunderstood which specific pieces of data to put into database.hostname and database.server.name, and one I corrected those fields the connector works.
Related
I started in a large company as an IT consultant. One of my tasks is to manage an application that has a SQL database.
I have very limited knowledge of SSIDB and SQL Server Management Studio - but I am willing to learn.
The SQL database is updated with external data. This can be done by users directly in the application, but it also happens through a scheduled job. The job runs from SQL manager. The job has only two steps, one of which is to execute a dtsx package.
The dtsx package is set up retrieves data from an external FTP server and merges the data into the database. The job was made by my predecessor and has run flawlessly for a very long time.
Now we are in the situation where the FTP, supplying the data, has been changed.
I have therefore been inside the Connection managers and changed to the new FTP server.
Running the jobs however we still get the following error message:
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”
I have checked the Connection managers and the ChunkSize is unchanged from when the job was working correctly. ChunkSize is set to 1000, both in the Connection manager, but also in the dtsx package itself.
When I have searched for the problem, it is mentioned that it may have something to do with the connection to the FTP server. So I have checked the connection to the FTP server from the server where the SQL database is located - and there is a connection. In addition to this, I have also made sure that there is a firewall rule that allows traffic between the two servers. This is ensured across protocols and port 20-22
When the job itself is run, however, no traffic leaves the server. So I believe the problem is with the job itself.
Edit: after having done a validation of the package i have gotten the following.
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”.
: at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties(Sting parameterName, object parameterValue)
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties()
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.ValidatePackageWithReference(Int64 validationId, Int64 infold, Int64 projectId, String packageName, Int64 versionId, Nullable'1 referenceId, Project isserverProject, Boolean OfflineMode)
I hope my description is comprehensive enough - otherwise please do write follow-up questions.
ps english is not my first language. sorry if something didn't turn out too well.
I have a Python package that I am able to run successfully on an Azure Data Science Virtual Machine. However, when I push it to Azure as a Function, I cannot successfully make a database connection. I was getting an error that the ODBC Driver 13 for SQL Server was not supported, so I changed the driver to ODBC Driver 17 for SQL Server and now I am NOT getting an error, but no data is being returned for a query that I know should return data.
Is there any other reason that data would not be returned? Firewall issues? do I need to add a binding? Do I need to separate out the connection string to feed each part (e.g., Driver, UID, PWD) into pyodbc.connect() separately? Right now I am feeding it in like this:
setting = os.environ("CONNECTIONSTRING")
conn = pyodbc.connect(setting)
This query works fine returning data when I run it on the VM using this code, just not as a Function.
(Note, this is different from my previous post regarding reading the Azure App Setting. That problem has been solved).
There are many parts where this could be breaking.
I'd suggest start by having a Profiler or Extended Events trace on your SQL Server to verify whether a connection is even being established. If not then you need to work through the the various points of connectivity to find out where it breaks. The identity, firewall, NSGs etc might all come into play here.
Once you see a connection then you can play with permissions to ensure that your query then returns your data.
Without a full picture of your infrastructure and settings it is hard to pin it down further.
Turns out it was not a database connectivity issue like I thought it was; it was a code error.
I Created SQL Server Database in Azure which is serverless and tried to access it using my SQL Server Management Studio in my local but I couldn't get it work.
It always gives me this message:
I tried to whitelist also my IP in Azure but still I get the same result.
Is there a possible way to make it connect?
Is the database currently online or paused?
I'll repeat the text from #David Browne's link:
If a serverless database is paused, then the first login will resume the database and return an error stating that the database is unavailable with error code 40613. Once the database is resumed, the login must be retried to establish connectivity. Database clients with connection retry logic should not need to be modified.
So;
Assuming the database is paused, this is normal operation
Please read docs
You need to retry after the database starts OR manually pre-start it using the Powershell provided in the link below
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-serverless#connectivity
And yes, you also need to whitelist your IP address as you have already done.
Obviously this flavour of SQL is unsuitable for some types of applications - there is more information in the link - I suggest you read the whole thing.
I am trying to connect Jmeter to a geo redundant database with two hosts and I am struggling with finding the right Database Url format.
This is how my connection string looks like:
jdbc:oracle:thin:#(DESCRIPTION=(ENABLE=BROKEN)(FAILOVER=on)(CONNECT_TIMEOUT=5sec)(TRANSPORT_CONNECT_TIMEOUT=3sec)(RETRY_COUNT=3)(LOAD_BALANCE=on)(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=HostName)(PORT=port)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=HostName2)(PORT=port)))(CONNECT_DATA=(SERVICE_NAME=ServiceName)))
Database Connection Configuration is as following:
JDBC Driver Class: oracle.jdbc.OracleDriver Username: username
Password: password
For the Database URL I tried different formats and I keep getting the error:
Cannot load JDBC driver class 'oracle.jdbc.OracleDriver'
Note that the ojdbc.jar file is in the /lib folder as per the Jmeter documentation. Also, the ports are the same for both hosts.
Any suggestion is welcome. :)
I don't think you will be able to establish the connection to Oracle RAC using JMeter's JDBC Connection Configuration as it doesn't allow full flexibility therefore you will not be able to properly instantiate the PoolDataSourceFactory
So I would recommend switching to JSR223 Test Elements and Groovy language where you will have the full freedom when it comes to setting up the connection, executing queries, accessing results, etc. The relevant code would be something like:
def prop = new Properties()
prop.put('oracle.jdbc.thinForceDNSLoadBalancing','true')
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource()
pds.setConnectionProperties(prop)
pds.setConnectionFactoryClassName('oracle.jdbc.pool.OracleDataSource'); pds.setUser('johndoe')
pds.setPassword('secret')
String dbURL =
'jdbc:oracle:thin:#(DESCRIPTION=(ENABLE=BROKEN)(FAILOVER=on)(CONNECT_TIMEOUT=5sec)' +
'(TRANSPORT_CONNECT_TIMEOUT=3sec)(RETRY_COUNT=3)(LOAD_BALANCE=on)(ADDRESS_LIST=(LOAD_BALANCE=on)' +
'(ADDRESS=(PROTOCOL=TCP)(HOST=HostName)(PORT=port)))(ADDRESS_LIST=(LOAD_BALANCE=on)' +
'(ADDRESS=(PROTOCOL=TCP)(HOST=HostName2)(PORT=port)))(CONNECT_DATA=(SERVICE_NAME=ServiceName)))'
pds.setURL(dbURL)
More information: Configuring Fast Connection Failover for JDBC Clients
It appears to be working with a connection string containing only host 1.
The Database URL is in the form:
jdbc:oracle:thin:#<hostname>:<port>/<serviceName>
Additionally, I got the error because the .jar file's path was not added to the classpath (click on Test Plan, in the bottom select browse next to Add directory or jar to classpath and select your odbc jar).
Another thing that was wrong was the Validation query, it should be "select 1 from dual" and also the query should not contain any semicolon at the end.
I hope this help people with the same issue.
(I am a sql noob and I just can not figure this out on my own)
For some time now I have been trying to establish a connection to a SQL database in codename one but to no avail. First I tried connecting to a MariaDB database from one.com. All that's needed for the connection is
Database db = Display.getInstance().openOrCreate("databaseName");
if I am not mistaken, but I am guessing this implies that I have somehow already established a connection to the database. This is not the case however so it creates a new .sql file, right? I can recall that you can connect to a database in the services tab in Netbeans. I chose the MySQL(Connector/ J Driver) which should work with MariaDB, or should it? I entered all my data and i says that it can not establish connection to the database.
the error i get
So I thought I might as well try using localhost. I used XAMPP to host a database and connected in the netbeans services tab.
connected?
Now testing was needed to see if this works. I started the SQL journey with this https://www.codenameone.com/manual/files-storage-networking.html#_sql and integrated the part after "You can probably integrate this code into your app as a debugging tool". I changed database name to "mybase" (it's existance can be confirmed in picture 2). Ran the app, opened the dialog, entered "select ID from customers" and got: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: customers) It does not get past the first call to "executeQuery". The customers table definitely exists so what am I missing to establish connection?
I really need instructions to connect to the localhost database and ideally also to the one hosted by my webhost provider.
Thanks,
Jona
The Database class is to access the SQLite DB on the mobile device. To connect to external databases, you'd have to do something different, such as a ConnectionRequest or Socket I think.