I'm trying to find out what protocol the SnowFlake JDBC library uses to communicate with SnowFlake. I see hints here and there that it seems to be using HTTPS as the protocol. Is this true?
To my knowledge, other JDBC libraries like for example for Oracle or PostgreSQL use the lower level TCP protocol to communicate with their database servers, and not the application-level HTTP(S) protocol, so I'm confused.
My organization only supports securely routing http(s)-based communication. Can I use this snowflake jdbc library then?
I have browsed all documentation that I could find, but wasn't able to answer this question.
My issue on GitHub didn't get an answer either.
Edit: Yes, I've seen this question, but I don't feel that it answers my question. SSL/TLS is an encryption, but that doesn't specify the data format.
It looks like the jdbc driver uses HTTP Client HttpUtil.initHttpClient(httpClientSettingsKey, null);, as you can see in here
The HTTP Utility Class is available here
Putting an excerpt of the session open method here in case the link goes bad/dead.
/**
* Open a new database session
*
* #throws SFException this is a runtime exception
* #throws SnowflakeSQLException exception raised from Snowflake components
*/
public synchronized void open() throws SFException, SnowflakeSQLException {
performSanityCheckOnProperties();
Map<SFSessionProperty, Object> connectionPropertiesMap = getConnectionPropertiesMap();
logger.debug(
"input: server={}, account={}, user={}, password={}, role={}, database={}, schema={},"
+ " warehouse={}, validate_default_parameters={}, authenticator={}, ocsp_mode={},"
+ " passcode_in_password={}, passcode={}, private_key={}, disable_socks_proxy={},"
+ " application={}, app_id={}, app_version={}, login_timeout={}, network_timeout={},"
+ " query_timeout={}, tracing={}, private_key_file={}, private_key_file_pwd={}."
+ " session_parameters: client_store_temporary_credential={}",
connectionPropertiesMap.get(SFSessionProperty.SERVER_URL),
connectionPropertiesMap.get(SFSessionProperty.ACCOUNT),
connectionPropertiesMap.get(SFSessionProperty.USER),
!Strings.isNullOrEmpty((String) connectionPropertiesMap.get(SFSessionProperty.PASSWORD))
? "***"
: "(empty)",
connectionPropertiesMap.get(SFSessionProperty.ROLE),
connectionPropertiesMap.get(SFSessionProperty.DATABASE),
connectionPropertiesMap.get(SFSessionProperty.SCHEMA),
connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE),
connectionPropertiesMap.get(SFSessionProperty.VALIDATE_DEFAULT_PARAMETERS),
connectionPropertiesMap.get(SFSessionProperty.AUTHENTICATOR),
getOCSPMode().name(),
connectionPropertiesMap.get(SFSessionProperty.PASSCODE_IN_PASSWORD),
!Strings.isNullOrEmpty((String) connectionPropertiesMap.get(SFSessionProperty.PASSCODE))
? "***"
: "(empty)",
connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY) != null
? "(not null)"
: "(null)",
connectionPropertiesMap.get(SFSessionProperty.DISABLE_SOCKS_PROXY),
connectionPropertiesMap.get(SFSessionProperty.APPLICATION),
connectionPropertiesMap.get(SFSessionProperty.APP_ID),
connectionPropertiesMap.get(SFSessionProperty.APP_VERSION),
connectionPropertiesMap.get(SFSessionProperty.LOGIN_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.NETWORK_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.QUERY_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.TRACING),
connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE),
!Strings.isNullOrEmpty(
(String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE_PWD))
? "***"
: "(empty)",
sessionParametersMap.get(CLIENT_STORE_TEMPORARY_CREDENTIAL));
HttpClientSettingsKey httpClientSettingsKey = getHttpClientKey();
logger.debug(
"connection proxy parameters: use_proxy={}, proxy_host={}, proxy_port={}, proxy_user={},"
+ " proxy_password={}, non_proxy_hosts={}, proxy_protocol={}",
httpClientSettingsKey.usesProxy(),
httpClientSettingsKey.getProxyHost(),
httpClientSettingsKey.getProxyPort(),
httpClientSettingsKey.getProxyUser(),
!Strings.isNullOrEmpty(httpClientSettingsKey.getProxyPassword()) ? "***" : "(empty)",
httpClientSettingsKey.getNonProxyHosts(),
httpClientSettingsKey.getProxyProtocol());
// TODO: temporarily hardcode sessionParameter debug info. will be changed in the future
SFLoginInput loginInput = new SFLoginInput();
loginInput
.setServerUrl((String) connectionPropertiesMap.get(SFSessionProperty.SERVER_URL))
.setDatabaseName((String) connectionPropertiesMap.get(SFSessionProperty.DATABASE))
.setSchemaName((String) connectionPropertiesMap.get(SFSessionProperty.SCHEMA))
.setWarehouse((String) connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE))
.setRole((String) connectionPropertiesMap.get(SFSessionProperty.ROLE))
.setValidateDefaultParameters(
connectionPropertiesMap.get(SFSessionProperty.VALIDATE_DEFAULT_PARAMETERS))
.setAuthenticator((String) connectionPropertiesMap.get(SFSessionProperty.AUTHENTICATOR))
.setOKTAUserName((String) connectionPropertiesMap.get(SFSessionProperty.OKTA_USERNAME))
.setAccountName((String) connectionPropertiesMap.get(SFSessionProperty.ACCOUNT))
.setLoginTimeout(loginTimeout)
.setAuthTimeout(authTimeout)
.setUserName((String) connectionPropertiesMap.get(SFSessionProperty.USER))
.setPassword((String) connectionPropertiesMap.get(SFSessionProperty.PASSWORD))
.setToken((String) connectionPropertiesMap.get(SFSessionProperty.TOKEN))
.setPasscodeInPassword(passcodeInPassword)
.setPasscode((String) connectionPropertiesMap.get(SFSessionProperty.PASSCODE))
.setConnectionTimeout(httpClientConnectionTimeout)
.setSocketTimeout(httpClientSocketTimeout)
.setAppId((String) connectionPropertiesMap.get(SFSessionProperty.APP_ID))
.setAppVersion((String) connectionPropertiesMap.get(SFSessionProperty.APP_VERSION))
.setSessionParameters(sessionParametersMap)
.setPrivateKey((PrivateKey) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY))
.setPrivateKeyFile((String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE))
.setPrivateKeyFilePwd(
(String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE_PWD))
.setApplication((String) connectionPropertiesMap.get(SFSessionProperty.APPLICATION))
.setServiceName(getServiceName())
.setOCSPMode(getOCSPMode())
.setHttpClientSettingsKey(httpClientSettingsKey);
// propagate OCSP mode to SFTrustManager. Note OCSP setting is global on JVM.
HttpUtil.initHttpClient(httpClientSettingsKey, null);
SFLoginOutput loginOutput =
SessionUtil.openSession(loginInput, connectionPropertiesMap, tracingLevel.toString());
isClosed = false;
authTimeout = loginInput.getAuthTimeout();
sessionToken = loginOutput.getSessionToken();
masterToken = loginOutput.getMasterToken();
idToken = loginOutput.getIdToken();
mfaToken = loginOutput.getMfaToken();
setDatabaseVersion(loginOutput.getDatabaseVersion());
setDatabaseMajorVersion(loginOutput.getDatabaseMajorVersion());
setDatabaseMinorVersion(loginOutput.getDatabaseMinorVersion());
httpClientSocketTimeout = loginOutput.getHttpClientSocketTimeout();
masterTokenValidityInSeconds = loginOutput.getMasterTokenValidityInSeconds();
setDatabase(loginOutput.getSessionDatabase());
setSchema(loginOutput.getSessionSchema());
setRole(loginOutput.getSessionRole());
setWarehouse(loginOutput.getSessionWarehouse());
setSessionId(loginOutput.getSessionId());
setAutoCommit(loginOutput.getAutoCommit());
// Update common parameter values for this session
SessionUtil.updateSfDriverParamValues(loginOutput.getCommonParams(), this);
String loginDatabaseName = (String) connectionPropertiesMap.get(SFSessionProperty.DATABASE);
String loginSchemaName = (String) connectionPropertiesMap.get(SFSessionProperty.SCHEMA);
String loginRole = (String) connectionPropertiesMap.get(SFSessionProperty.ROLE);
String loginWarehouse = (String) connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE);
if (loginDatabaseName != null && !loginDatabaseName.equalsIgnoreCase(getDatabase())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Database",
loginDatabaseName,
getDatabase()));
}
if (loginSchemaName != null && !loginSchemaName.equalsIgnoreCase(getSchema())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Schema",
loginSchemaName,
getSchema()));
}
if (loginRole != null && !loginRole.equalsIgnoreCase(getRole())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP, "Role", loginRole, getRole()));
}
if (loginWarehouse != null && !loginWarehouse.equalsIgnoreCase(getWarehouse())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Warehouse",
loginWarehouse,
getWarehouse()));
}
// start heartbeat for this session so that the master token will not expire
startHeartbeatForThisSession();
}
Related
I'm trying to connect to my local sql server database at my home network using the code that I found from this site Querying SQL Server with Google Apps Script via JDBC about 3 years ago which was marked being correct. However, I get the error message "We're sorry, a server error occurred. Please wait a bit and try again.". This error is from line 2 where the connection string is defined. I retried several times, but I always get the same error. When I searched this error, it seems like it could be too many things and I was not able to find any answers for my issue. Thanks.
This was the code that was marked being correct:
function readAzure() {
var conn = Jdbc.getConnection("jdbc:sqlserver://XYZ.database.windows.net:1433;databaseName=MYDATABSENAME","USERNAME","PASSWORD");
var stmt = conn.createStatement();
var rs = stmt.executeQuery("select * from helloworld");
var doc = SpreadsheetApp.create('azure');
var cell = doc.getRange('a1');
var row = 0;
while(rs.next()) {
cell.offset(row, 0).setValue(rs.getString(1));
cell.offset(row, 1).setValue(rs.getString(2));
row++;
}
rs.close();
stmt.close();
conn.close();
}
I also found another connection string code and when I try this format I get the same error.
var conn = Jdbc.getConnection("jdbc:sqlserver://IP-address:1433;" + "databaseName=DBName;user=username;password=password;");
Based on this documentation, you need to ensure that your database accepts connections from any of Apps Script's IP addresses. These are the address ranges you'll need to whitelist.
64.18.0.0 - 64.18.15.255
64.233.160.0 - 64.233.191.255
66.102.0.0 - 66.102.15.255
66.249.80.0 - 66.249.95.255
72.14.192.0 - 72.14.255.255
74.125.0.0 - 74.125.255.255
173.194.0.0 - 173.194.255.255
207.126.144.0 - 207.126.159.255
209.85.128.0 - 209.85.255.255
216.239.32.0 - 216.239.63.255
Also, from the documentation link above, there is a sample code that you can follow to set up external database via JDBC.
Here is a code that demonstrates how to write a single record to the database as well as a batch of 500 records.
// Replace the variables in this block with real values.
var address = 'database_IP_address';
var user = 'user_name';
var userPwd = 'user_password';
var db = 'database_name';
var dbUrl = 'jdbc:mysql://' + address + '/' + db;
// Write one row of data to a table.
function writeOneRecord() {
var conn = Jdbc.getConnection(dbUrl, user, userPwd);
var stmt = conn.prepareStatement('INSERT INTO entries '
+ '(guestName, content) values (?, ?)');
stmt.setString(1, 'First Guest');
stmt.setString(2, 'Hello, world');
stmt.execute();
}
// Write 500 rows of data to a table in a single batch.
function writeManyRecords() {
var conn = Jdbc.getConnection(dbUrl, user, userPwd);
conn.setAutoCommit(false);
var start = new Date();
var stmt = conn.prepareStatement('INSERT INTO entries '
+ '(guestName, content) values (?, ?)');
for (var i = 0; i < 500; i++) {
stmt.setString(1, 'Name ' + i);
stmt.setString(2, 'Hello, world ' + i);
stmt.addBatch();
}
var batch = stmt.executeBatch();
conn.commit();
conn.close();
var end = new Date();
Logger.log('Time elapsed: %sms for %s rows.', end - start, batch.length);
}
For more information, just read through to this documentation and check this thread and related SO question.
Hello I´ve a problem when I try to monitor which one of a cluster oam servers is online and offline I use the the getServerDiagnosticInfo() method of AccessClient class from aSDK, but the Hashtable that returns only contains Keys (name and port of server) and Values that contains another HashTable (ObKeyMapVal a subtype of HashTable) but I think that this object must contains the health, server port, server name and number of connections as mentioned in the API doc but when I print the size and contents of it only prints "0" and [] (its empty)
snippet:
try{
AccessClient ac = AccessClient.createDefaultInstance("/dir",AccessClient.CompatibilityMode.OAM_10G);
Hashtable info = ac.getServerDiagnosticInfo();
Set<?> servers = info.keySet();
Collection<?> serverInfo = info.values();
System.out.println("Num of servers: " + servers.size());
Iterator it = servers.iterator();
Object servidor = null;
Object dato = null;
while(it.hasNext()){
servidor = it.next();
System.out.println("Server: " + servidor);
dato = info.get(servidor);
System.out.println("Data: " + dato);
ObKeyValMap ob = (ObKeyValMap) dato;
System.out.println("Size: " + ob.keySet().size());
System.out.println("Is Empty: " + ob.keySet().isEmpty());
System.out.println("Properties: " + ob.keySet());
}
ac.shutdown();
} catch (oracle.security.am.asdk.AccessException e) {
e.printStackTrace();
} catch (Exception e){
e.printStackTrace();
}
And got the next output:
Num of servers: 2
Server: myserver1.com5575
Data: {}
Size: 0
Is Empty: true
Properties: []
Server: myserver2.com5575
Data: {}
Size: 0
Is Empty: true
Properties: []
Thanks for your help !!!
Once you get the OAM Server Host and Port using getServerDiagnosticInfo(). Try to do telnet ( I am not Java Expert, following link may help How to connect/telnet to SPOP3 server using java (Sockets)?) , if the server is up the telnet session will be established.
I am implementing a server for communication using the OMA DM 1.2 SyncML protocol and reffers to the OMA Device Management Security document. I am having issues with authentication. The client sends a challenge to the server as:
<Chal>
<Meta>
<Format xmlns="syncml:metinf">b64</Format>
<Type xmlns="syncml:metinf">syncml:auth-md5</Type>
<NextNonce xmlns="syncml:metinf">RLLe7tWM313qHMq9ooUZUPJX0RqU9mEZuyoVF+jXhqQ=</NextNonce>
</Meta>
</Chal>
I then calculate the md5-digest to return to the device using the java code, where nonce is the Base64 string in "NextNonce" in challenge above:
MessageDigest digest = MessageDigest.getInstance("MD5");
String usrPwd = username + ":" + password;
String usrPwdHash = Base64.encodeBase64String(digest.digest(usrPwd.getBytes("utf-8")));
String usrPwdNonce = usrPwdHash + ":" + nonce;
String usrPwdNonceHash = Base64.encodeBase64String(digest.digest(usrPwdNonce.getBytes("utf-8")));
return usrPwdNonceHash;
Then this hash is returned to the device as:
<Cred>
<Meta>
<ns2:Type>syncml:auth-md5</ns2:Type>
<ns2:Format>b64</ns2:Format>
</Meta>
<Data>QpbMtvvfNGRIavJ0jqcxaw==</Data>
</Cred>
But the device returns with a status 401 and a new challenge. Is there something wrong with how i calculate the md5-hash or must there be some other issue?
Found my error. The nonce should be the decoded Base64 string value, not the Base64 string.
nonce = new String(Base64.decodeBase64("RLLe7tWM313qHMq9ooUZUPJX0RqU9mEZuyoVF+jXhqQ="), "utf-8");
Iam building a cloud application with google apps.While establishing a connection with cloud database instance iam getting an error . I have also downloaded and configured appengine-api-1.0-sdk-1.6.5 in Tomcat7.0 server library , i have also installed the rdbms package , but still the error is same .The code is as given below
<%# page import="com.google.appengine.api.rdbms.*, java.io.*,java.sql.*"%>
<%
Connection c = null;
DriverManager.registerDriver(new AppEngineDriver());
c = DriverManager.getConnection("jdbc:google:rdbms://ctsteaching:ctsdatabase/ctsdb");
String fname = request.getParameter("fname");
String content = request.getParameter("content");
if (fname == "" || content == "") {
out.println("<html><head></head><body>You are missing either a message or a name! Try again! Redirecting in 3 seconds...</body></html>");
} else {
String statement ="INSERT INTO entries (guestName, content) VALUES(?,?)";
PreparedStatement stmt = c.prepareStatement(statement);
stmt.setString(1,fname);
stmt.setString(2,content);
int success = 2;
success = stmt.executeUpdate();
if(success == 1) {
out.println("<html><head></head><body>Success! Redirecting in 3 seconds...</body> </html>");
} else if (success == 0)
{
out.println("<html><head></head><body>Failure! Please try again! Redirecting in 3 seconds...</body></html>");
}
}
%>
The error is as given below
com.google.apphosting.api.ApiProxy$CallNotFoundException: The API package 'rdbms' or call 'OpenConnection()' was not found.
com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.makeSyncCall(RdbmsApiProxyClient.java:95)
com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.openConnection(RdbmsApiProxyClient.java:73)
com.google.cloud.sql.jdbc.internal.SqlProtoClient.openConnection(SqlProtoClient.java:58)
com.google.cloud.sql.jdbc.Driver.connect(Driver.java:66)
com.google.cloud.sql.jdbc.Driver.connect(Driver.java:26)
java.sql.DriverManager.getConnection(Unknown Source)
java.sql.DriverManager.getConnection(Unknown Source)
org.apache.jsp.abcd.test_jsp._jspService(test_jsp.java:70)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
It looks like you are using the wrong jar file. You need to set some extra jvm_flags when developing locally.
https://developers.google.com/cloud-sql/docs/developers_guide_java#with_eclipse
I have used Oracle Advanced Security to encrypt data during data transfer. I have successfully configured ssl with below parameters and I have restarted the instance. I am retrieving data from a Java class given below. But I could read the data without decrypting, the data is not getting encrypted.
Environment:
Oragle 11g database
SQLNET.AUTHENTICATION_SERVICES= (BEQ, TCPS, NTS)
SSL_VERSION = 0
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
SSL_CLIENT_AUTHENTICATION = FALSE
WALLET_LOCATION =
(SOURCE =
(METHOD = FILE)
(METHOD_DATA =
(DIRECTORY = C:\Users\kcr\Oracle\WALLETS)
)
)
SSL_CIPHER_SUITES= (SSL_RSA_EXPORT_WITH_RC4_40_MD5)
Java class:
try{
Properties properties = Utils.readProperties("weka/experiment/DatabaseUtils.props");
// Security.addProvider(new oracle.security.pki.OraclePKIProvider()); //Security syntax
String url = "jdbc:oracle:thin:#(DESCRIPTION =\n" +
" (ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))\n" +
" (CONNECT_DATA =\n" +
" (SERVER = DEDICATED)\n" +
" (SERVICE_NAME = sal)\n" +
" )\n" +
" )";
java.util.Properties props = new java.util.Properties();
props.setProperty("user", "system");
props.setProperty("password", "weblogic");
// props.setProperty("javax.net.ssl.trustStore","C:\\Users\\kcr\\Oracle\\WALLETS\\ewallet.p12");
// props.setProperty("oracle.net.ssl_cipher_suites","SSL_RSA_EXPORT_WITH_RC4_40_MD5");
// props.setProperty("javax.net.ssl.trustStoreType","PKCS12");
//props.setProperty("javax.net.ssl.trustStorePassword","welcome2");
DriverManager.registerDriver(new OracleDriver());
Connection conn = DriverManager.getConnection(url, props);
/*8 OracleDataSource ods = new OracleDataSource();
ods.setUser("system");
ods.setPassword("weblogic");
ods.setURL(url);
Connection conn = ods.getConnection();*/
Statement stmt = conn.createStatement();
ResultSet rset = stmt.executeQuery("select * from iris");
///////////////////////////
while(rset.next()) {
for (int i=1; i<=5; i++) {
System.out.print(rset.getString(i));
}
}
Are you expecting that your SELECT statement would return encrypted data and that your System.out.print calls would result in encrypted output going to the screen? If so, that's not the way advanced security works-- Advanced Security allows you to encrypt data over the wire but the data is unencrypted in the SQLNet stack. Your SELECT statement, therefore, would always see the data in an unencrypted state. You would need to do a SQLNet trace or use some sort of packet sniffer to see the encrypted data flowing over the wire.
You'll find the documentation in "SSL With Oracle JDBC Thin Driver".
In particular you should probably use PROTOCOL = TCPS instead of PROTOCOL = TCP. I'd also suggest using a stronger cipher suite (and avoid the anonymous ones, since with them you don't verify the identity of the remote server).