what is the alternative of XP_CMDshell ? it is working fine in sql server
but it is not support in sql azure
so what is alternative for sql azure ?
I assume you need command line access to access BCP since you are trying to create and upload/download data into a flat or xml file.
SQL Azure does not provide access to a command prompt. Think of SQL Azure as just a virtualized service, without host infrastructure (Virtualized or physical). Since BCP is actually a command line based utility you won't have access to it from Azure. You'll want to setup BCP, which is part of the command line utilities, on a machine that does have access to command line. That machine might be either on-premise or Azure IaaS. You can install just the BCP component.
http://www.microsoft.com/en-us/download/details.aspx?id=36433
Alternatively if you don't want to setup physical infrastructure to complete this, Azure Data Factory would be able to complete the task of moving data between flat files and databases.
Related
We have a legacy AS400 host system that runs our warehouse. We want it to securely trigger jobs on the SQL server that runs the rest of the business to enable automatic and timely data exchange between the two systems.
Currently we have the IBM iSeries Access for Windows client software installed on an old Windows SQL 2005 server. This allows the AS400 to run remote commands on a Windows box. The AS400 uses this client software to run batch files that execute dtexec on the SQL 2005 server box which then runs SSIS packages that exchanges data between the AS400 and the production SQL servers. The SQL 2005 box is just acting as a go between the AS400 and the production SQL servers. The two problems with this setup are:
The AS400 client software runs as a system process and not a network account so the SQL login info has to be included in the plain text in the SSIS files.
When dtexec is run it adds two entries into the Windows event log. We have several process that poll for new data every few minutes or in some cases seconds so the event log is consistently purging older entries which leaves about 4 hours worth of events. To say this makes it hard to troubleshoot errors on this box would be an understatement.
We are decommissioning the old SQL 2005 server box so I have to replicate this functionality on a newer server. I want to address these two issues before doing so.
I have tried using sqlcmd but since the AS400 Client software is running under a local system process it can't connect to the production SQL servers which are setup with Windows Authentication Mode. I tried using psexec which works if I leave a RDP session for the user open to but would still need the user name and password to be in plain text. I have been testing out having the batch file write a blank file to a directory and having the SQL job triggered by an WMI event. In testing this works but is unreliable when the job needs to be quickly triggered multiple times in a row, like most of our jobs need to be.
I have also looked into using the C# and .NET to store the password using the Windows Security classes and/or data protection API (DPAPI) but I am at best a intermediate .NET programmer so this is a worse case solution for me.
This has to be a common issue with integrating older host systems with SQL Server so I assuming there has to be a solution that allows for the Host system to call the SQL server in a secure method. If not a Job directly, run a stored procedure that can trigger the Job. Any alternative that does not require a the password/user name saved in plain text somewhere and not using dtexec would be appreciated.
LocalSystem Account "has extensive privileges on the local computer, and acts as the computer on the network."
This means you can grant it access to your remote SQL Servers by creating a login for the "computer account". If you server is named MyServer and the domain is MyDomain then:
use msdb
create login [MyDomain\MyServer$] from windows
create user [MyDomain\MyServer$] for login [MyDomain\MyServer$]
alter role SQLAgentOperatorRole add member [MyDomain\MyServer$]
And a process running as LocalSystem on the other server will be able to connect via Windows Integrated Auth and run SQL Agent jobs.
This year We moved from hosted servers to Azure VM's, we run two production servers (SQL and IIS). A vital component of our business is bulk transfer of data file. We take customers data from our SQL Server and then write it out to a file (XLS, CSV, XML, PDF, Word, etc.) and then either email these files to customers or in most cases, push them into their FTP server. We also have a few import procedures where we retrieve data files. All of this is currently done with SSIS packages.
We're examining a move to Azure Data Factory as a replacement for SSIS so that we can possibly move to either Azure SQL (if we can work out Broker Services limitations) or an Azure SQL Managed Instance.
I've done some preliminary work with ADF but I saw a couple of posts about lack of FTP support. Is it possible to create/deliver files to FTP and retrieve/consume files from FTP using ADF? Also, almost all of these jobs are automated and we use SQL Agent to run the packages. What is the Azure equivalent for scheduling these jobs to run?
There is automation in ADF but the scheduler is per pipeline. Azure Automation is more powerful and can automate more than one pipeline (Azure Data Factory v2), if needed.
Automation with Azure Data Factory (ADF)
You can receive files from FTP into an Azure Data Factory pipeline: Copy data from FTP server by using Azure Data Factory The idea is that you receive a file via FTP to submit to a particular pipeline activity, and that activity pushes data to an Azure data source. It might be possible to reverse the flow, and send data out.
The Azure SQL Database Managed Instance is the most on-premise like database (PaaS) service but SQL Server deployed on an Azure VM still has more functionality.
We are developing and SSIS service to import some data in Excel and CSV files in Azure. For uploading the files we have chosen Azure File Storage and we are running the SSIS packages on a VM. For picking up the files from file storage, we have mapped the File Storage as mapped network drive on the VM. This works file when we manually trigger the SSIS jobs. However, this fails when running as SQL Server Agent job. As far as I understand, the mapped drives are per user and they do not work for service account used for SQL Server Agent. Is there a way by which we can access the file storage in SSIS packages as SQL Agent Jobs?
I found this page but this is for basic windows network file sharing. Does not work for us as we also need to use the Shared Access Key for Azure File Storage.
I solved the problem using this solution.
Add credential via cmd not via the GUI
We have two Sql 2008 R2 database. First and main one is on-premise which is used by our stock management and accounting systems.
The second is for our web site and it is on a 3th party hosting firm and that database is updated manually by a standard XML file operation
(creating XML from on-premise database, sending it to web server, reading from XML file & insert/update/delete web database)
we need to get rid of that manual XML operation and sync that two databases automatically, but problem is our hosting firm does not allow "Replication" or "Linked Server" facilities on their "Sql Server" system. That's way we can not sync databases with these "Sql Server" facilities.
I am trying find out sync alternatives for that scenario without changing the hosting system.
What options can we use ?
Is Microsoft Sync Framework can be used for this scenario? ( I'm not sure if it is just for the Microsoft Azure Sql system)
Thanks…
if you're fine coding, then Sync Framework can do this.
Sync Framework can sync SQL Compact, Express, Server, Azure or LocalDB
But bear in mind that Sync Framework will need to create some Sync-related objects on your databases (triggers, stored procedures, tables, etc...)
It sounds like you need a hosting provider that provides these services.
It is typical for a hosting provider to provide an on-demand or always VPN connection to the production servers from the client location to the servers at the hosting location. Such a service is often needed for support of online systems. If you had such a connection then having SQL Server do syncs would be no problem.
Any major (Rack Space, Peer1, etc) provider would be able to set up such a system.
It probably won't be as inexpensive as your current provider.
Well if changing a provider is not an option, at list in the short term and you need something right now, you can always automate parts or all your current process.
For that you can at least use:
PowerShell (or even plain old cmd.exe) script(batch) and bcp.exe to export and import your files on both ends and PowerShell and ftp to transfer your files from one server to another.
SQLServer Integration Services on premises to export and send files via ftp to your hosting location. And SSIS to grab and import those files at your hosting location.
I'm trying to run a SQL job (sql server 2008) every day that needs to read a file stored on another server. I have set up a share and I can access it using my user through windows explorer.
Then I set up the sql job to "run as" that same user, but it doesn't seem to be able to access the file.
I set up Delegation in Active Directory (windows 2003). This didn't solve the issue.
How can i access the file in the share?
Does that user have access to sqlcmd or whichever command line tools you're using to run the SQL?...Does that user have access to all the sql server obujects in the sql?