I typically use pyodbc when running jupyter notebooks from my machine, but this does not work on Azure ML. My assumption is that this is being caused by Azure ML not knowing if I'm on my company's network as I typically need a VPN to the server if I'm not in office. The only solutions I can find online involve copying the data over on Azure Data Factory however I need to avoid this if possible as there are many tables I will need to experiment with, but nothing is intended to be long term and I'm unsure what I will even end up using.
Ideally there is a way to make pyodbc work but any other suggestions are welcome. I have researched integration runtimes but was unsure if that would solve my problem here.
The only solutions I can find online involve copying the data over on
Azure Data Factory however I need to avoid this if possible as there
are many tables I will need to experiment with, but nothing is
intended to be long term and I’m unsure what I will even end up using.
Ideally there is a way to make pyodbc work but any other suggestions
Unfortunately, the on-Prem SQL Server is not supported as a Data Source in Azure ML.
Only the Data sources available below are supported:-
Approach1)
You can copy your data from the on-premises SQL database to Azure SQL via copy tool in Azure Data factory and connect to Azure SQL via Azure Machine learning by directly connecting to it via Datasource like below:-
You can also use Self-hosted integration run time to connect to your SQL server on-prem in your data factory:-
Click on Option 2 to download the Integration runtime and set it in your local machine with the Registration keys mentioned above:-
Approach2)
If there’s a large data You can automate your entire copy process from the on-prem SQL server to Azure SQL by using the Azure DevOps pipeline.
References:-
https://learn.microsoft.com/en-us/answers/questions/775844/unable-to-connect-sql-server-to-azure-ml-pipeline By Ramr-msft
How To: Azure Data Factory CI/CD with Azure DevOps pipelines — The YAML WAY! | by Raghavendra Bharadwaj | Servian
Related
My client has his data stored on SQL Server hosted on an on-premise network. I established a VPN connection from Google to the network, but I don't know how to follow from here. My final goal is to process his data using cloud functions. Any suggestions?
PS: I read that Shared VPC can be used to accomplish this, but I don't have a proper organization for this purpose :/
Edit: I followed the suggestions on the comments but now I'm missing to extract the data since pyodbc is not pre-installed on Cloud Functions. Any ideas oh how to query an on-prem database on SQL Server through Cloud Functions?
To simulate a device I want to use a Logic App to fill a Azure Database for PostgreSQL with test data.
I found the two connectors PostgreSQL and SQL server but unfortunately both don't work for me. The PostgreSQL connector does only support onpremise databases and neither the autodetection nor the manually input of the connectionstring works with SQL Server.
Would great if somebody could give me a hint how to write data into a Azure Database for PostgreSQL from a Logic app.
Unfortunately, there is no connector available for this as of today.
The current alternatives would be
Use Azure Functions (which can be called from Logic Apps if required)
Create Custom APIs and a Custom Logic App Connector
Also, there is a feature request on UserVoice that you may upvote to gain traction.
I have built a CRM for my company using MS Access 2016, and we are looking to be able to use it remotely. I have currently split the database and have the back-end residing on our company server/shared drive. As I see it, we could put the back-end on SQL Server, but that would mean finding a way to host it. We recently moved to Google Apps for Work and I have been looking at the Google Cloud platform (e.g. Cloud SQL), but I don't know if it will integrate nicely with MSSQL. There's also Sharepoint, but we are are having difficulty even finding it to buy on Microsoft.com, and are trying to move away from Microsoft Office products anyway.
What would be the most stable, cost-effective, and fastest way to allow our employees to use our Access DB remotely?
My suggestion would be to migrate the database into SQL Server as step one. Then step two host the database in Microsoft Azure using the SQL DB service.
You can find more details and pricing here: https://azure.microsoft.com/en-us/services/sql-database/
Be sure you check the single database pricing. Unless you want the elastic scaling and performance.
I'm not sure about the cost part, but the fastest and most stable (IMHO) would be to use Remote Desktop. Either to a user's workstation or to a server set up for it.
We have migrated our backend to SQL Server from MS Access for obvious reasons. Our users access the front end off a VPS Hosted in the cloud. Connection string ODBC. It is robust and fast.
we are currently using a VPS. Looks cheaper and easy to access ;
https://clients.databasemart.com/aff.php?aff=275
We have two Sql 2008 R2 database. First and main one is on-premise which is used by our stock management and accounting systems.
The second is for our web site and it is on a 3th party hosting firm and that database is updated manually by a standard XML file operation
(creating XML from on-premise database, sending it to web server, reading from XML file & insert/update/delete web database)
we need to get rid of that manual XML operation and sync that two databases automatically, but problem is our hosting firm does not allow "Replication" or "Linked Server" facilities on their "Sql Server" system. That's way we can not sync databases with these "Sql Server" facilities.
I am trying find out sync alternatives for that scenario without changing the hosting system.
What options can we use ?
Is Microsoft Sync Framework can be used for this scenario? ( I'm not sure if it is just for the Microsoft Azure Sql system)
Thanks…
if you're fine coding, then Sync Framework can do this.
Sync Framework can sync SQL Compact, Express, Server, Azure or LocalDB
But bear in mind that Sync Framework will need to create some Sync-related objects on your databases (triggers, stored procedures, tables, etc...)
It sounds like you need a hosting provider that provides these services.
It is typical for a hosting provider to provide an on-demand or always VPN connection to the production servers from the client location to the servers at the hosting location. Such a service is often needed for support of online systems. If you had such a connection then having SQL Server do syncs would be no problem.
Any major (Rack Space, Peer1, etc) provider would be able to set up such a system.
It probably won't be as inexpensive as your current provider.
Well if changing a provider is not an option, at list in the short term and you need something right now, you can always automate parts or all your current process.
For that you can at least use:
PowerShell (or even plain old cmd.exe) script(batch) and bcp.exe to export and import your files on both ends and PowerShell and ftp to transfer your files from one server to another.
SQLServer Integration Services on premises to export and send files via ftp to your hosting location. And SSIS to grab and import those files at your hosting location.
I have a simple ms sql 2012 express database & a simple VB Express 2010 application as its front end.
Can I simply host my MS SQL database on Azure & update the application to interrogate the hosted database? This would mean, the application would still be local, but the DB would be hosted. My offices are distributed & if we could do this it would save quite a bit of hassle.
Do I need to use a virtual machine, cloud services or straight hosting? Its difficult for me to understand the subtleties each service.
I've had a look at the azure services information and the documentation is pitched a little high for me. I just wanted to know if this is relatively simple thing to do & if anyone has has any pointers to How To's or Noob guides to azure.
It's absolutely possible and in fact, super easy to set up.
Check out this simple step-by-step instruction on how to set up SQL Server DB on Windows Azure - https://www.windowsazure.com/en-us/develop/net/common-tasks/sql-azure-management/
You can even connect to it from your SQL Server Management Studio just like you would connect to a local database or a database on your network.
Ideally - yes, you can change the connection string used by the VB desktop app to reference the Windows Azure SQL Database instance instead of the local database. That said, SQL Database is not 100% compatible with SQL Server. Be sure to check out the differences at http://msdn.microsoft.com/en-us/library/windowsazure/ff394115.aspx
Can I simply host my MS SQL database on Azure & update the application
to interrogate the hosted database?
Yes, your SQL Azure database is accessible via a connection string, but you have to make sure that you create a firewall rule in SQL Azure to allow your locally hosted or remotely hosted applications to access the database. Everything is turned off by default.
Do I need to use a virtual machine, cloud services or straight
hosting?
I'm not exactly sure what you're asking here, but a VM may be overkill, as that gives you basically a place to host a VM in Azure. You control the O/S, patching, etc. A cloud service (a web or worker role) is a little less maintenance heavy as the O/S, patches, updates, etc. are maintained by Azure and you just worry about your application. Straight hosting - are you talking about hosting on 1&1 or GoDaddy or something? If so, that's really no different than your VB app running locally and accessing your Azure database. For this option, just make sure you have your firewall rules set properly, otherwise you won't be able to access the database.
Hopefully this helps. Good luck!
Azure has some specific requirements that need to be met (like every table has to have a Clustered Index), and it does not support some functionality that is available on "normal" SQL Server (like replication, FILESTREAM data, or data compression).
Other than that, as long as your app meets the requirements, there should not be a reason for it not to work on Azure. The majority of the functionality not supported by Azure is most likely out6 of the scope of what a "noob" would implement, so you probably are safe.
However, Azure might be overly expensive for your needs. It might be cheaper to go for hosting the database somewhere. This would mean you would only have to change the connection string of the application to point to the new database server (though there could be additional limitations imposed by your hosting provider).
Good luck with whichever option you end up taking!