Error detecting index schema from datasource: "Data source type 'powerquery is not supported
enter image description here
Make sure that you are connecting after requesting access to Power Query connectors. You will be provided instructions on how to use them from the portal.
Power Query connector support is currently in a gated public preview.
For instructions please visit:
https://learn.microsoft.com/azure/search/search-how-to-index-power-query-data-sources
Related
I'm trying to set up a Debezium SQL Server Connector against a SQL Server instance that is controlled by DBAs at my workplace. I've been able to start up Zookeeper and Kafka Server without issue, and Kafka Connect itself works with sample Connectors, but when attempting to start a Debezium SQL Server Connector instance I've been getting the error "Couldn't obtain database name".
[2022-07-12 16:36:04,269] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:117)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.runtime.rest.errors.BadRequestException: Connector configuration is invalid and contains the following 1 error(s):
Unable to connect. Check this and other connection properties. Error: Couldn't obtain database name
Here is my debezium config:
name=Dbz-SqlServer-connector
connector.class=io.debezium.connector.sqlserver.SqlServerConnector
database.hostname=MyDbHost
database.port=1433
database.user=MyUsername
database.password=MyPassword
database.dbname=MyDatabase
database.server.name=MyDbHost
table.include.list=dbo.CdcTest
database.history.kafka.bootstrap.servers=localhost:9092
database.history.kafka.topic=dbhistory.CdcTest
I've tried this in a .properties file passed to a standalone Connect instance, and as a JSON POST to a distributed Connect instance. I have tried all of the same steps on both my local Windows machine as well as on a linux VM, with the same results.
Confluent and Docker are not options for me in this situation.
for SQL Server login credentials, I am using a local account on the SQL Server instance that does have access to the database in question. I found the source code for debezium's connectors on their github and was able to find that specific error message within the code:
private static final String GET_DATABASE_NAME = "SELECT name FROM sys.databases WHERE name = ?";
...
public String retrieveRealDatabaseName(String databaseName) {
try {
return prepareQueryAndMap(GET_DATABASE_NAME,
ps -> ps.setString(1, databaseName),
singleResultMapper(rs -> rs.getString(1), "Could not retrieve exactly one database name"));
}
catch (SQLException e) {
throw new RuntimeException("Couldn't obtain database name", e);
}
}
I'm not completely familiar with Java but it appears that basically something is going wrong when the connector is trying to run "SELECT name FROM sys.databases WHERE name = 'MyDatabase'". When I run this against the database myself, logged in with the same account I'm using, it seems to work just fine, so I'm really not sure where to go from here. It is fair to say that since I'm not in full control of the SQL Server environment that I'm using, there may be some permissions issues that I'm not aware of, but from what I'm able to test it seems like it should be working.
I would greatly appreciate any help at all, whether just suggestions on settings/configs to check or a full-blown solution.
Thank you!
Update: I've built a simple console app to run that sys.databases query against MyDbHost, master database, as the relevant account, and it's working just fine so I feel like that confirms that my connection info is correct and account permissions are also correct. Seems like this is an issue within the Debezium connector.
It turned out that my problem was a mistake in the connector's config setting. I misunderstood which specific pieces of data to put into database.hostname and database.server.name, and one I corrected those fields the connector works.
I'm creating a Spring Boot application and I'm using Intellij's embedded h2 database.
I have added the following lines in my application.properties file:
spring.datasource.url=jdbc:h2:~/testdb;MV_STORE=false;AUTO_SERVER=TRUE
This is my data source configuration
Although the connection is successful and I can query the database using Intellij's query console, the tables do not appear in the Database tab.
Succeeded
DBMS: H2 (ver. 2.1.210 (2022-01-17))
Case sensitivity: plain=upper, delimited=exact
Driver: H2 JDBC Driver (ver. 2.1.210 (2022-01-17), JDBC4.2)
Ping: 16 ms
When I refresh the connection or go to the schemas tab of the data source configuration, I get the following error:
[42S02][42102] org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "INFORMATION_SCHEMA_CATALOG_NAME" not found; SQL statement: select CATALOG_NAME from INFORMATION_SCHEMA.INFORMATION_SCHEMA_CATALOG_NAME [42102-210].
By going to the advanced tab of the data source and clicking on expert options, we are presented with a checkbox labeled "Introspect using JDBC metadata"
By checking that box, the tables successfully appear in the Database tab
Regarding why this works, this is taken from the official documentation:
https://www.jetbrains.com/help/datagrip/data-sources-and-drivers-dialog.html
Introspect using JDBC metadata
Switch to the JDBC-based introspector. Available for all the databases.
To retrieve information about database objects (DB metadata), DataGrip uses the following introspectors:
A native introspector (might be unavailable for certain DBMS). The native introspector uses DBMS-specific tables and views as a source of metadata. It can retrieve DBMS-specific details and produce a more precise picture of database objects.
A JDBC-based introspector (available for all the DBMS). The JDBC-based introspector uses the metadata provided by the JDBC driver. It can retrieve only standard information about database objects and their properties.
Consider using the JDBC-based intorspector when the native introspector fails or is not available.
The native introspector can fail, when your database server version is older than the minimum version supported by DataGrip.
You can try to switch to the JDBC-based introspector to fix problems with retrieving the database structure information from your database. For example, when the schemas that exist in your database or database objects below the schema level are not shown in the Database tool window.
I would like to connect my SQL Database on Siteground Hosting to Power BI through the SQL Server connection. However, I need to get the following information which I am not sure where to get:
Siteground Database Credentials in the format: servername:portname
Username & Password (which I have)
And finally, when I inserted the IP Address in item no.1, the error received is as below:
Based on the error above, I figured it could be either:
(i) the IP address I have inserted is wrong; or
(ii) I need to allow remote access to my database on Siteground CPanel:
If item (ii) is the reason for the error, in order to allow PowerBI to access the database, I would need to insert the IP Address of my PowerBI Online which I do not know how to get
I need assistance basically connecting my PowerBI application to my SQL Database created on my hosting account Siteground
Any help is much appreciated.
Your Siteground control panel suggest that the database is MySQL - your are trying to connect to a SQL Server database. They are not the same thing which is probably why it doesn't work.
Try using the MySQL connector for Power BI
First, as Martin Cairney told you, the connector that you are using is wrong - Microsoft SQL Server and MySQL are completely different products, which are not compatible. First change the connector.
The list of IP addresses is something dynamic. There is no guarantee, that Power BI will try to connect to your database from a single IP address only, nor that this IP address will stay the same for a long time. If you go this way, you should maintain this on a weekly basis and monitor for changes. The list of IP addresses can be downloaded from these json files:
Public: https://www.microsoft.com/en-us/download/details.aspx?id=56519
US Gov: http://www.microsoft.com/en-us/download/details.aspx?id=57063
Germany: http://www.microsoft.com/en-us/download/details.aspx?id=57064
China: http://www.microsoft.com/en-us/download/details.aspx?id=57062
They are updated weekly.
Better solution is to move your database to Azure (e.g. Azure MySQL service or MySQL installed in Virtual Machine in Azure) or to migrate to another data source.
I Created SQL Server Database in Azure which is serverless and tried to access it using my SQL Server Management Studio in my local but I couldn't get it work.
It always gives me this message:
I tried to whitelist also my IP in Azure but still I get the same result.
Is there a possible way to make it connect?
Is the database currently online or paused?
I'll repeat the text from #David Browne's link:
If a serverless database is paused, then the first login will resume the database and return an error stating that the database is unavailable with error code 40613. Once the database is resumed, the login must be retried to establish connectivity. Database clients with connection retry logic should not need to be modified.
So;
Assuming the database is paused, this is normal operation
Please read docs
You need to retry after the database starts OR manually pre-start it using the Powershell provided in the link below
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-serverless#connectivity
And yes, you also need to whitelist your IP address as you have already done.
Obviously this flavour of SQL is unsuitable for some types of applications - there is more information in the link - I suggest you read the whole thing.
I am currently setting up our second Azure Search service. I am making it identical to our existing one, just in a different region.
I'm using the portal Import Data function to set up my index. For the Data Source, I have configured it to connect to my Azure SQL Database and table, which definitely has Integrated Change Tracking turned on. Further, it's the exact same database and table that I'm connected to and indexing from in my existing Azure Search service.
The issue is that when I get to the "Create an Indexer" step, I get the message that says "Consider enabling integrated change tracking on your database..." In other words, it doesn't think I have change tracking on this database. I definitely do, and our other Azure Search Service recognizes this just fine on the exact same database.
Any idea what's going on here? How can I get this Data Source to be recognized as having Change Tracking turned on, and why isn't it doing so when all is working as expected in our existing Search service with identical set up?
We will investigate. In the meantime, please try creating your datasource and indexer programmatically using the REST API or .NET SDK.
When I was experiencing this problem, I tried creating the search service via "Add Azure Search" in Azure portal > SQL database.
Using that wizard I was able to create the search data source, index & indexer.
Update: I opened a ticket with Azure support, and when trying to get more information to provide to them, I tried to reproduce the problem (create a data source via REST API), but the expected failure did not happen ("Change tracking not enabled for table..." despite it being enabled). This makes me think there was something wrong with internal Azure code that was fixed in the meantime.