Working with SQL Server 2019 Enterprise CU18 Polybase (On-premise). I have a working connection from SQL Server to Oracle that is defined as follows:
CREATE EXTERNAL DATA SOURCE [OracleDataSource]
WITH (LOCATION = 'oracle://ExaData1:1521', CREDENTIAL = [OracleCredential])
However, we have an HA/DR pair for our exadata system. ExaData1 and ExaData2. If we did this in a linked server, the connetion would look like this:
(DESCRIPTION=(ENABLE=BROKEN)(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=ExaData1.domain.com)(PORT=1521))(ADDRESS=(PROTOCOL=tcp)(HOST=ExaData2.domain.com)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=OracleTNSName)))
Yet, I can't seems to define it this way in Polybase (at least I haven't figure out how to yet). I need to figure out how to list both possible servers in the External Data Source so when the DB fails over, it will follow it naturally.
The other option is to figure out how to use the TNS Name, rather than the server name for the Location, but I've not figured that out either. Currently, when I define a table, it looks like this, which is based upon the previously defined data source, listing the TNS name as part of the DATA_SOURCE properties:
CREATE EXTERNAL TABLE [Polybase].[sample]
(
[SID] [nvarchar](8) NULL,
[MESSAGE_DATE] [datetime2](7) NULL,
[MESSAGE_ID] [nvarchar](3) NULL
)
WITH (DATA_SOURCE = [OracleDataSource],LOCATION = N'[OracleTNSNAME.domain.com].OracleSchema.SAMPLE')
Anyone have suggestions or options as I'm not finding anything in the MS documenation.
Appreciate any and all support.
Have you tried with this?
oracle//server1:1521; AlternateServers=(server2:1521,server3:1521,server4:1521); LoadBalancing=true
The only other option to connect is by using ExaData SCAN name (configured by Oracle database administrator, which resolves to any of the nodes using a single name). From Oracle it looks like this:
(DESCRIPTION =
(CONNECT_TIMEOUT=90) (RETRY_COUNT=20)(RETRY_DELAY=3) (TRANSPORT_CONNECT_TIMEOUT=3)
( ADDRESS = (PROTOCOL = TCP)(HOST=sales-scan.mycluster.example.com)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=oltp.example.com)))
Then from SQL Server you connect to sales-scan.mycluster.example.com
Related
I'm unable to access a local CSV file from SQL Server 2019 Polybase. This is a simple 3-columned text file. I have also created a local system DSN (from ODBC32 UI).
I got the sample code from here. However, the driver in the link (cdata) is not free. Any assistance in solving this issue will be greatly appreciated.
create master key encryption by password = 'Polybase2CSV';
create database scoped credential csv_creds
with identity = 'username', secret = 'password';
create external data source csv_source
with (
location = 'odbc://localhost',
connection_options = 'DSN=CustomerDSN', -- this is the DSN name
-- PUSHDOWN = ON | OFF,
credential = csv_creds
);
CREATE EXTERNAL TABLE Customer
(
CUSTOMERID int,
CUSTOMERNAME varchar(250),
DEPARTMENT varchar(250)
) WITH (
LOCATION='customer.txt',
DATA_SOURCE=csv_source
);
This requires a few steps to make it work successfully. As prerequisites, you'll need to make sure SQL Server 2019 has been updated to CU4 (KB4548597) to fix a few known issues. For a free solution, you will need to install the 64-bit version Microsoft Access Database Engine 2016 Redistributable. This will install the 64-bit version of the ODBC drivers.
With these two things in place, you can now create the external data source. I recommend disabling PUSHDOWN. I've seen it cause some problems with this particular driver.
If you want to directly connect to the CSV file that contains a header row, you can create the external data source by simply specifying the Access Text Driver and the folder that will contain the files:
CREATE EXTERNAL DATA SOURCE MyODBC
WITH
(
LOCATION = 'odbc://localhost',
CONNECTION_OPTIONS = 'Driver=Microsoft Access Text Driver (*.txt, *.csv);Dbq=F:\data\files\',
PUSHDOWN = OFF
);
To use the data source, you need to create an external table definition that reflects the file format. The LOCATION parameter will be the name of the file to load. You can wrap the file name and driver name in braces to avoid issues with special characters. It's important to make sure the column names you define for this table match the names in the header row. Because you're using CU4, if a data type doesn't match the driver's expectations, you'll get an error indicating which data types were expected.
CREATE EXTERNAL TABLE dbo.CsvData
(
Name nvarchar(128),
Count int,
Description nvarchar(255)
)
WITH
(
LOCATION='[filename.csv]',
DATA_SOURCE = [MyODBC]
)
If you want to define the column names, data types, etc., in the ODBC Data Sources (64-bit ) UI, choose the Microsoft Access Text Driver. You can then select the folder, file types, and definition of the text file format. Make sure to use the 64-bit data sources. Once you're done defining the format details, you'll see a schema.ini file is created in the folder which contains those details.
For the external data source, you'll specify the name of the DSN:
CREATE EXTERNAL DATA SOURCE MyODBC
WITH
(
LOCATION = 'odbc://localhost',
CONNECTION_OPTIONS = 'DSN=LocalCSV',
PUSHDOWN = OFF
);
The EXTERNAL TABLE is created the same way as before, with the column names and data types matching the definition you declared in the DSN.
To create a data source directly you'll need to buy that driver. That is option 1, but since it out the window. You have two more options. Import that data directly to SQL Server or if you really want to use PolyBase. Load that Data into Staging SQL Tables then create External Tables referencing that staging table.
My assumptions: CSV data isn't stale. structure/schema will remain constant.
Create a staging table. Use
Import-DbaCSV -Path "D:\CustomerTest\Customer.csv"`
-SqlInstance ServerName`
-Database DBName`
-Table "Customer"
Then recycle your code to either connect to PolyBase or Use the data directly.
CREATE DATABASE SCOPED CREDENTIAL csv_creds
WITH IDENTITY = 'username', SECRET = 'password';
CREATE EXTERNAL DATA SOURCE csv_source
WITH ( LOCATION = 'sqlserver://SERVERNAME:PORTNUMBER',
PUSHDOWN = ON,
CREDENTIAL = csv_creds);
You can then periodically run the PS function to load data into the table as needed.
I have severe troubles to set up an MS access application that uses linked tables to an SQL Server 2012 Database.
The problem is that SQL Queries have problems to interpret German dates: e.g. "31.12.2019" doesn't work, "01.01.2019" works. So I suspect that it is a problem with localization. E.g.
select * from table where date >= [Forms]![someForm]![fromDate]
[Forms]![someForm]![fromDate] is a string in a form, edited by a date picker.
I was able to solve the problem by using the ODBC Microsoft SQL Server Setup Wizzard, and selecting "Ländereinstellungen verwenden" (engl. use country specific settings).
(Sorry the following screenshot is in German).
I would like to specify this in a classic ODBC connection string: e.g
DRIVER=ODBC Driver 13 for SQL Server;SERVER=.\SqlExpress2012;Trusted_Connection=Yes;APP=Microsoft Office;DATABASE=suplattform;?country-specific=yes?
However I did not find such an parameter in any documentation. Is this possible?
Best regards
Michael
Also, specify the data type of the parameter - and date is a reserved word in Access SQL:
parameters [Forms]![someForm]![fromDate] DateTime;
select * from table where [date] >= [Forms]![someForm]![fromDate]
I am trying to copy table structure with data from one server to other. I already have a linked server in place in my source server. When I try to execute the below query I am getting an error.
SELECT Appt.C1 AS AppointmentId
,Appt.C2
INTO [190.28.111.187].[WH_AC].[dbo].[ApptDet]
FROM [whse].[Vw_ApptDet] Appt
INNER JOIN #DealerList DL ON DL.DealerId = Appt.DealerId
Error:
Msg 117, Level 15, State 1, Line 4
The object name '190.28.111.187.WH_AC.dbo.ApptDet' contains more than the maximum number of prefixes. The maximum is 2.
I am not sure where I am missing. Please suggest me how do I perform this.
From the documentation on the INTO clause:
You cannot create new_table on a remote server; however, you can populate new_table from a remote data source. [...]
So you will have to reverse your statement (ie execute it from the remote server), or do it another way.
You can retrive the data using this type of code style [190.28.111.187].[WH_AC].[dbo].[ApptDet]. in case of manipulation activity the SQL do not support. I would recommend to use openrowset to achieve this concept.
OPENROWSET('SQLNCLI', 'Server=YourServername;Trusted_Connection=yes;',
To identify server name run the code
Select ##servername.
I would use Integration Services (SSIS) for this purpose.
If you need to do it that way (which I do next, so good question).
Just create the table first (on the remote server)
CREATE TABLE dbo.[TEST](
[id] [int] NOT NULL,
[description] [varchar](50) NULL,
)
Then
insert into [server].[db].dbo.[TEST]
select * from [TEST]
I need to move data to a linked server where I don't sa rights next week so you had me worried. I can of course create a table but not a linked server on the remote.
We have a few SQL Server boxes and instances running, we wish our code to identify if it is running on Development, Test or Production server instance of SQL Server. Based on that we would like to code to take a certain path.
We have few choices of hard coding the SQL Server Name and Instance like
Case when ##SERVERNAME= ABC/XYZ then DEV and so on.
Check specific letters in Name or INSTANCE WHEN (CHARINDEX('DEV', ##SERVERNAME, 1) > 0 ) then DEV and so on.
Select type from a table:
select #vcServerType = vcServerType
from master.dbo.tbl_ServerDetails
when #vcServerType = 'DEV' and so on.
But all rely on assumption that DEV will be found or server name will contains certain letters, creating a table in database that will exist in all instances..
Should have been so much easier if only there exists a property to set/identify at instance level.. any suggestions (apart from above)
IS there a way in SQL server, where we can set some property or Tag them as DEV, TEST or Production?
There is a tab named "Extended Properties" on the Database Level in it's properties.
You can set them with sys.sp_addextendedproperty
And read them for instance in ADO.NET like the answer to this question Shows.
Edit: I didn't really see that you asked about a tag on the instance-level or maybe on Server Level. Sorry for not directly addressing that. But you should be able to achieve what you wish for through the properties on DB-Level, shouldn't you?
Following up the advise above, yes extended properties are fabulous solution.
use master
go
EXEC sp_addextendedproperty
#name = 'Environment'
, #value = 'Development'
SELECT [value] AS [EnvironmentType]
FROM [sys].[extended_properties] AS [ep]
WHERE [ep].[class_desc] = 'DATABASE'
AND [ep].[name] = 'Environment';
Is any way to check a database is already exists or not in PostgreSQL?
Iam a new one in PostgreSQL.
And also need to check it from my java application via jdbc driver.
There is no IF NOT EXISTS option for CREATE DATABASE in PostgreSQL. See CREATE DATABASE.
You can check pg_database to see if the DB exists, and only attempt to create it if it does not.
You would need to do this via dblink, because of the limitation Frank points out in the comments:
regress=> SELECT create_database_if_not_exists('test');
ERROR: CREATE DATABASE cannot be executed from a function or multi-command string
CONTEXT: SQL statement "CREATE DATABASE test"
PL/pgSQL function create_database_if_not_exists(text) line 6 at EXECUTE statement
I found an alternate solution for this problem. By using the following query :
ResultSet res = st.executeQuery("select count(*) from pg_catalog.pg_database where datname = 'sample'") ;
res.next();
int count = res.getInt("count");
System.out.println("Count : " + count);
if(count == 0) {
st.executeUpdate("CREATE DATABASE sample");
}
Its work fine
Provided you do not have many databases made in postgreSQL
Here is a simple way to see what databases you have and do not have
Can use pgadmin ,and expand database column and see what databases you have and dont have.
As shown by the yellow box, the databases created in pgadmin, by me, can try this