I am trying to connect a Nifi QueryDatabaseTable processor to Salesforce in order to retrieve data to load into a Marklogic data hub. I found a couple of shareware JDBC drivers for Salesforce but have not been able to successfully connect to the SF Connected App endpoint I've been given. Can anyone advise whether this is a good approach for automated data retrieval from SF? Should I be able connect to a SF Connected App URL via JDBC driver like the one at ascendix?
TIA!
I ended up finding a free Salesforce JDBC driver (not the ascendix driver, which did not work for me) that seems to work fine for simple select queries to pull data from a Salesforce Connected App. So, the JDBC approach for this solution works using a Nifi QueryDatabaseTable processor which produces an Avro dataset that is easily split into records and converted to JSON for ingest to MarkLogic.
Yes, the RelierSoft driver works for me, though the 0.4 version (Oct 2019) of the JAR is packaged with an old version of the apache commons.csv library that causes errors for some latest NiFi services, like CSVReader. So, I repackaged it without the old commons.csv library classes to make that service work for me.
Related
I have a project to ingest metadata from the Snowflake data warehouse into the Azure Data Catalog (ADC). The ADC does not natively support this, so I must use the ADC API or ODBC.
Before I proceed on implementing the API solution, I thought it wise to ask if anyone has suggestions on this. A Google search returned nothing of note.
Unfortunately ODBC will not work either. I am moving onto a solution to load via API.
The API or ODBC are the best choices.
I use Datagrip for SQL development. Recently I've had to work with Salesforce SQL (aka SOQL), but there doesn't seem to be an IDE as useful as Datagrip.
So: I'd like to add Salesforce as a data source to Datagrip, just as I would a Postgres or MySQL database.
Is this possible to do, with an extension or otherwise?
I know https://jetforcer.com/ exists, but it seems to fall short on two counts:
(a) it's apparently just a UI that lives in JetBrains apps, so I don't think it'd fit as seamlessly in Datagrip workflow as I'd like
(b) It costs $60. I don't need to do any actual Salesforce development, so paying $60 just to run queries against Salesforce isn't ideal.
DataGrip can support any database that have JDBC driver.
First of all, you need to find JDBC driver for the database. In your case that can be:
http://reliersoft.com/download/salesforce-jdbc-driver/
Then follow the instructions of generic-JDBC data source configuration:
https://www.jetbrains.com/help/datagrip/connecting-to-a-database.html#vertica_as_unsupported_dbms
URL should be like jdbc:sforce://<hostname>
(More options are available in driver's docs http://reliersoft.com/salesforce-jdbc-driver-documentation/)
After that you can run queries and view data.
If something does not work as expected, please file an issue in our bug tracker
Is there an existing way to get a message from RabbitMQ and to use it to create an SQL query such as an "INSERT INTO"? For example, take the data that a webservice retrieves to a client (that data must go through Rabbit as well) and insert into a MS-SQL Server the data that the client is supposed to get.
Pretty much any language can talk to RabbitMQ. There is also a powershell module built for it.
For your needs, I've found the following link which demonstrates how you can set up rabbitmq with sql server.
I would suggest creating a sandbox example on your local machine using the link as a guide so that you can get familiar with RabbitMQ and its setup.
Hopefully that will provide you with the knowledge you need to solve your current problem.
I am working on a project where i need to display the database mssql server's performance metrics for example memory consumed/free, storage free space etc. I have researched for this purpose and one thing came up was DOGSTATSD.
Datadog provides the library for .net project to get custom metrics but that was not the solution for me because the metrics works on datadog website. I have to display the all (in graph or whatever suited) data, received from MSSQL SERVER. There will be multiple servers/instances.
Is there a way to do that, our WebApp connected with multiple databases and we receive/display information.
I cannot use already available tools for the insights.
You can easily get all needed data via querying dmv and other resources inside SQL Server. Good start is here.
We have been using a web application framework to build apps that need to be able to query a SQL Server database and get the results as XML.
In the past, the framework provided that capability. But that capability is now deprecated.
So we were thinking, the framework allows us to easily query a REST service over HTTP, so why not use a SQL Server HTTP Endpoint. However, we then read that HTTP Endpoints are deprecated, as of SQL Server 2008. Not a platform on which to design an architecture for the future.
Azure (formerly SQL Data Services) was going to offer similar services, but now only supports the TDS protocol, not http. So no REST to be found in Azure.
The suggested alternative is to develop a custom app using WCF Data Services (formerly ADO.NET Data Services). But that would mean a whole additional app to develop, deploy, and maintain, presumably with its own authentication setup separate from SQL Server's, and its own source code repository... using a technology we have no experience with, therefore with its own pretty deep learning curve.
Can you suggest any other way to query a SQL Server database via REST/HTTP, that is not deprecated, and that would return results as XML?
Thanks for any help.
Read here: Creating an OData API for StackOverflow including XML and JSON in 30 minutes. Basically, the road forward is for REST to be offered by app layer (WCF powering EF that provides the OData mapping). IMHO straight HTTP access into the engine was a very bad idea to start with, nobody liked the HTTPEndpoints of SQL Server 2005 and they were as misguided as it gets. One cannot map the HTTP error model, security, type system into SQL and expect a smooth interoperability. Having the HTTP layer live in a dedicated app pushes the responsibility of handling the HTTP ecosystem into a component specialized in that (WCF), and the logic of mapping the REST model to the DB model ina component specialized in that job (EF).
It sounds like you may be wedded to an MS stack but if you're not, you can use restSQL in a Java EE container (Tomcat, WebLogic, etc.) on top of MySQL or PostgreSQL. restSQL has a full HTTP API with JSON or XML encoding. It offers two twists: updatable composite views and hierarchical composite views. The framework is extensible to other databases and addition of SQL Server is in its supported evolution. Check out http://restsql.org.
Another option is something like Dreamfactory. They have a SOAP to REST solution that allows you to connect to any database or service. I have used their free hosted solution in the past for projects. They also have an open source solution available. The cool thing about the service is that they use Swagger 3.0 to create service definitions in a nice front-end solution so you can test and create new endpoints.
I have used the OpenAPI 3.0 definitions to connect to 3rd party SOAP and REST services as well. They also support stored procedures and server-side scripting in the SQL Server environments.
Anyways, might be another option for you.