I try to send a connection message to a server but I get a response from the server with the status 400 (Bad request).
I understand from RFC that the ":scheme" and ":path" pseudo-header fields MUST be omitted, so I put in the header just :method and :authority
nghttp2_nv hdrs[] = {
MAKE_NV2(":method", "CONNECT"),
MAKE_NV(":authority", authority, authoritylen),
};
nghttp2_submit_request(session, NULL, hdrs, ARRLEN(hdrs), NULL, NULL);
What I did wrong?
Related
I'm trying to find out what protocol the SnowFlake JDBC library uses to communicate with SnowFlake. I see hints here and there that it seems to be using HTTPS as the protocol. Is this true?
To my knowledge, other JDBC libraries like for example for Oracle or PostgreSQL use the lower level TCP protocol to communicate with their database servers, and not the application-level HTTP(S) protocol, so I'm confused.
My organization only supports securely routing http(s)-based communication. Can I use this snowflake jdbc library then?
I have browsed all documentation that I could find, but wasn't able to answer this question.
My issue on GitHub didn't get an answer either.
Edit: Yes, I've seen this question, but I don't feel that it answers my question. SSL/TLS is an encryption, but that doesn't specify the data format.
It looks like the jdbc driver uses HTTP Client HttpUtil.initHttpClient(httpClientSettingsKey, null);, as you can see in here
The HTTP Utility Class is available here
Putting an excerpt of the session open method here in case the link goes bad/dead.
/**
* Open a new database session
*
* #throws SFException this is a runtime exception
* #throws SnowflakeSQLException exception raised from Snowflake components
*/
public synchronized void open() throws SFException, SnowflakeSQLException {
performSanityCheckOnProperties();
Map<SFSessionProperty, Object> connectionPropertiesMap = getConnectionPropertiesMap();
logger.debug(
"input: server={}, account={}, user={}, password={}, role={}, database={}, schema={},"
+ " warehouse={}, validate_default_parameters={}, authenticator={}, ocsp_mode={},"
+ " passcode_in_password={}, passcode={}, private_key={}, disable_socks_proxy={},"
+ " application={}, app_id={}, app_version={}, login_timeout={}, network_timeout={},"
+ " query_timeout={}, tracing={}, private_key_file={}, private_key_file_pwd={}."
+ " session_parameters: client_store_temporary_credential={}",
connectionPropertiesMap.get(SFSessionProperty.SERVER_URL),
connectionPropertiesMap.get(SFSessionProperty.ACCOUNT),
connectionPropertiesMap.get(SFSessionProperty.USER),
!Strings.isNullOrEmpty((String) connectionPropertiesMap.get(SFSessionProperty.PASSWORD))
? "***"
: "(empty)",
connectionPropertiesMap.get(SFSessionProperty.ROLE),
connectionPropertiesMap.get(SFSessionProperty.DATABASE),
connectionPropertiesMap.get(SFSessionProperty.SCHEMA),
connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE),
connectionPropertiesMap.get(SFSessionProperty.VALIDATE_DEFAULT_PARAMETERS),
connectionPropertiesMap.get(SFSessionProperty.AUTHENTICATOR),
getOCSPMode().name(),
connectionPropertiesMap.get(SFSessionProperty.PASSCODE_IN_PASSWORD),
!Strings.isNullOrEmpty((String) connectionPropertiesMap.get(SFSessionProperty.PASSCODE))
? "***"
: "(empty)",
connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY) != null
? "(not null)"
: "(null)",
connectionPropertiesMap.get(SFSessionProperty.DISABLE_SOCKS_PROXY),
connectionPropertiesMap.get(SFSessionProperty.APPLICATION),
connectionPropertiesMap.get(SFSessionProperty.APP_ID),
connectionPropertiesMap.get(SFSessionProperty.APP_VERSION),
connectionPropertiesMap.get(SFSessionProperty.LOGIN_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.NETWORK_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.QUERY_TIMEOUT),
connectionPropertiesMap.get(SFSessionProperty.TRACING),
connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE),
!Strings.isNullOrEmpty(
(String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE_PWD))
? "***"
: "(empty)",
sessionParametersMap.get(CLIENT_STORE_TEMPORARY_CREDENTIAL));
HttpClientSettingsKey httpClientSettingsKey = getHttpClientKey();
logger.debug(
"connection proxy parameters: use_proxy={}, proxy_host={}, proxy_port={}, proxy_user={},"
+ " proxy_password={}, non_proxy_hosts={}, proxy_protocol={}",
httpClientSettingsKey.usesProxy(),
httpClientSettingsKey.getProxyHost(),
httpClientSettingsKey.getProxyPort(),
httpClientSettingsKey.getProxyUser(),
!Strings.isNullOrEmpty(httpClientSettingsKey.getProxyPassword()) ? "***" : "(empty)",
httpClientSettingsKey.getNonProxyHosts(),
httpClientSettingsKey.getProxyProtocol());
// TODO: temporarily hardcode sessionParameter debug info. will be changed in the future
SFLoginInput loginInput = new SFLoginInput();
loginInput
.setServerUrl((String) connectionPropertiesMap.get(SFSessionProperty.SERVER_URL))
.setDatabaseName((String) connectionPropertiesMap.get(SFSessionProperty.DATABASE))
.setSchemaName((String) connectionPropertiesMap.get(SFSessionProperty.SCHEMA))
.setWarehouse((String) connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE))
.setRole((String) connectionPropertiesMap.get(SFSessionProperty.ROLE))
.setValidateDefaultParameters(
connectionPropertiesMap.get(SFSessionProperty.VALIDATE_DEFAULT_PARAMETERS))
.setAuthenticator((String) connectionPropertiesMap.get(SFSessionProperty.AUTHENTICATOR))
.setOKTAUserName((String) connectionPropertiesMap.get(SFSessionProperty.OKTA_USERNAME))
.setAccountName((String) connectionPropertiesMap.get(SFSessionProperty.ACCOUNT))
.setLoginTimeout(loginTimeout)
.setAuthTimeout(authTimeout)
.setUserName((String) connectionPropertiesMap.get(SFSessionProperty.USER))
.setPassword((String) connectionPropertiesMap.get(SFSessionProperty.PASSWORD))
.setToken((String) connectionPropertiesMap.get(SFSessionProperty.TOKEN))
.setPasscodeInPassword(passcodeInPassword)
.setPasscode((String) connectionPropertiesMap.get(SFSessionProperty.PASSCODE))
.setConnectionTimeout(httpClientConnectionTimeout)
.setSocketTimeout(httpClientSocketTimeout)
.setAppId((String) connectionPropertiesMap.get(SFSessionProperty.APP_ID))
.setAppVersion((String) connectionPropertiesMap.get(SFSessionProperty.APP_VERSION))
.setSessionParameters(sessionParametersMap)
.setPrivateKey((PrivateKey) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY))
.setPrivateKeyFile((String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE))
.setPrivateKeyFilePwd(
(String) connectionPropertiesMap.get(SFSessionProperty.PRIVATE_KEY_FILE_PWD))
.setApplication((String) connectionPropertiesMap.get(SFSessionProperty.APPLICATION))
.setServiceName(getServiceName())
.setOCSPMode(getOCSPMode())
.setHttpClientSettingsKey(httpClientSettingsKey);
// propagate OCSP mode to SFTrustManager. Note OCSP setting is global on JVM.
HttpUtil.initHttpClient(httpClientSettingsKey, null);
SFLoginOutput loginOutput =
SessionUtil.openSession(loginInput, connectionPropertiesMap, tracingLevel.toString());
isClosed = false;
authTimeout = loginInput.getAuthTimeout();
sessionToken = loginOutput.getSessionToken();
masterToken = loginOutput.getMasterToken();
idToken = loginOutput.getIdToken();
mfaToken = loginOutput.getMfaToken();
setDatabaseVersion(loginOutput.getDatabaseVersion());
setDatabaseMajorVersion(loginOutput.getDatabaseMajorVersion());
setDatabaseMinorVersion(loginOutput.getDatabaseMinorVersion());
httpClientSocketTimeout = loginOutput.getHttpClientSocketTimeout();
masterTokenValidityInSeconds = loginOutput.getMasterTokenValidityInSeconds();
setDatabase(loginOutput.getSessionDatabase());
setSchema(loginOutput.getSessionSchema());
setRole(loginOutput.getSessionRole());
setWarehouse(loginOutput.getSessionWarehouse());
setSessionId(loginOutput.getSessionId());
setAutoCommit(loginOutput.getAutoCommit());
// Update common parameter values for this session
SessionUtil.updateSfDriverParamValues(loginOutput.getCommonParams(), this);
String loginDatabaseName = (String) connectionPropertiesMap.get(SFSessionProperty.DATABASE);
String loginSchemaName = (String) connectionPropertiesMap.get(SFSessionProperty.SCHEMA);
String loginRole = (String) connectionPropertiesMap.get(SFSessionProperty.ROLE);
String loginWarehouse = (String) connectionPropertiesMap.get(SFSessionProperty.WAREHOUSE);
if (loginDatabaseName != null && !loginDatabaseName.equalsIgnoreCase(getDatabase())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Database",
loginDatabaseName,
getDatabase()));
}
if (loginSchemaName != null && !loginSchemaName.equalsIgnoreCase(getSchema())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Schema",
loginSchemaName,
getSchema()));
}
if (loginRole != null && !loginRole.equalsIgnoreCase(getRole())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP, "Role", loginRole, getRole()));
}
if (loginWarehouse != null && !loginWarehouse.equalsIgnoreCase(getWarehouse())) {
sqlWarnings.add(
new SFException(
ErrorCode.CONNECTION_ESTABLISHED_WITH_DIFFERENT_PROP,
"Warehouse",
loginWarehouse,
getWarehouse()));
}
// start heartbeat for this session so that the master token will not expire
startHeartbeatForThisSession();
}
We have an issue with identity server configuration endpoint generating invalid JSON, I cant show too much but the screenshot below shows the call to the .well-known/openid-configuration endpoint . The one with 7ee gives us this error, another environment that works shows valid JSON.
"#t": "2022-08-24T08:59:41.1177158Z",
"#mt": "{msg} {#dt}",
"#l": "Error",
"msg": "Exception caught while processing request",
"dt": {
"StackTrace": " at Newtonsoft.Json.JsonTextReader.ParseReadNumber(ReadType readType, Char firstChar, Int32 initialPosition)\r\n at Newtonsoft.Json.JsonTextReader.ParseValue()\r\n at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)\r\n at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)\r\n at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)\r\n at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)\r\n at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)\r\n at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)\r\n at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)",
"Details": "Input string '7ee' is not a valid number. Path '', line 1, position 3.",
"CallingMethod": "Invoke",
"$type": "ErrorLogDetails"
},
Has anyone experienced this before and help point me in the right direction, many thanks.
This was a hard one, and really outside of our domain, we have a F5 load balancer that didn't have the right cors policy, unsure how this caused the extra characters, but it did. Adding this to the load balancer policy for that site corrected the issue.
when HTTP_REQUEST priority 200 {
unset -nocomplain cors_origin
if { ( [HTTP::header Origin] contains "example.com" ) } {
if { ( [HTTP::method] equals "OPTIONS" ) and ( [HTTP::header exists "Access-Control-Request-Method"] ) } {
# CORS preflight request - return response immediately
HTTP::respond 200 "Access-Control-Allow-Origin" [HTTP::header "Origin"] \
"Access-Control-Allow-Methods" [HTTP::header "Access-Control-Request-Method"] \
"Access-Control-Allow-Headers" [HTTP::header "Access-Control-Request-Headers"] \
"Access-Control-Max-Age" "86400" \
"Access-Control-Allow-Credentials" "true"
} else {
# CORS GET/POST requests - set cors_origin variable
set cors_origin [HTTP::header "Origin"]
}
}
}
when HTTP_RESPONSE {
set cors_origin [HTTP::header "example.com"]
# CORS GET/POST response - check cors_origin variable set in request
if { [info exists cors_origin] } {
HTTP::header remove Access-Control-Allow-Origin
HTTP::header remove Access-Control-Allow-Credentials
HTTP::header remove Vary
HTTP::header insert "Access-Control-Allow-Origin" example.com
HTTP::header insert "Access-Control-Allow-Credentials" "true"
HTTP::header insert "Vary" "Origin"
}
}
I'm trying to send a list from server to client.
The list looks like this (it's a csv file).
201,8,0040000080
205,8,1f421d25721e
but when sending I get this error:
TypeError: must be string or buffer, not list
I tried 2 options:
Iterate through the list and send each string to the server, but got this as a result:
201 ---> 2,0,1
Tried casting each line, e.g str(line), and then send it, but got this:
201,8,0040000080 ---> [,',2,0,1,',",", ,',8,',",", ,',0,0,4,0,0,0,0,0,8,0,',]
how can I solve this? I just want to send the data from the client to server as is. For the record, the Client code:
import socket
import csv
clientSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
with open('can_data.csv', 'rb') as csv_file:
csv_reader = csv.reader(csv_file)
for line in csv_reader:
clientSock.sendto(str(line), (self.address, self.port))
Server code:
with open('output.csv', 'wb') as new_file:
csv_writer = csv.writer(new_file)
while True:
data, addr = s.recvfrom(1024)
csv_writer.writerow(data)
Both sides need to agree on a serialization format. str(line) may work when paired with ast.literaleval() but repr(line) would be the better choice as repr tries for a more precise representation that str. You could also move to a serialization protocol like pickle or json.
Assuming this is python 3, I also moved the csv to text and used utf-8 encoding on the wire.
client:
import socket
import csv
address = 'localhost'
port = 5555
clientSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
with open('can_data.csv', 'r') as csv_file:
csv_reader = csv.reader(csv_file)
for line in csv_reader:
clientSock.sendto(repr(line).encode('utf-8'), (address, port))
server:
import socket
import csv
import ast
address = 'localhost'
port = 5555
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((address, port))
with open('output.csv', 'w') as new_file:
csv_writer = csv.writer(new_file)
while True:
data, addr = sock.recvfrom(1024)
csv_writer.writerow(ast.literal_eval(data.decode('utf-8')))
I am implementing a server for communication using the OMA DM 1.2 SyncML protocol and reffers to the OMA Device Management Security document. I am having issues with authentication. The client sends a challenge to the server as:
<Chal>
<Meta>
<Format xmlns="syncml:metinf">b64</Format>
<Type xmlns="syncml:metinf">syncml:auth-md5</Type>
<NextNonce xmlns="syncml:metinf">RLLe7tWM313qHMq9ooUZUPJX0RqU9mEZuyoVF+jXhqQ=</NextNonce>
</Meta>
</Chal>
I then calculate the md5-digest to return to the device using the java code, where nonce is the Base64 string in "NextNonce" in challenge above:
MessageDigest digest = MessageDigest.getInstance("MD5");
String usrPwd = username + ":" + password;
String usrPwdHash = Base64.encodeBase64String(digest.digest(usrPwd.getBytes("utf-8")));
String usrPwdNonce = usrPwdHash + ":" + nonce;
String usrPwdNonceHash = Base64.encodeBase64String(digest.digest(usrPwdNonce.getBytes("utf-8")));
return usrPwdNonceHash;
Then this hash is returned to the device as:
<Cred>
<Meta>
<ns2:Type>syncml:auth-md5</ns2:Type>
<ns2:Format>b64</ns2:Format>
</Meta>
<Data>QpbMtvvfNGRIavJ0jqcxaw==</Data>
</Cred>
But the device returns with a status 401 and a new challenge. Is there something wrong with how i calculate the md5-hash or must there be some other issue?
Found my error. The nonce should be the decoded Base64 string value, not the Base64 string.
nonce = new String(Base64.decodeBase64("RLLe7tWM313qHMq9ooUZUPJX0RqU9mEZuyoVF+jXhqQ="), "utf-8");
i am trying to call this rest api method
#HttpGet
global static String retrievingNotificationSettings(){
RestRequest req=RestContext.request;
RestResponse res=RestContext.response;
String accountId = req.requestURI.substring(req.requestURI.lastIndexOf('/')+1);
if(req.headers.get('Content-Type').equals('application/xml'))
{
Map<String, SObjectField> fields = Notification_Settings__c.SObjectType.getDescribe().fields.getMap();
Schema.sObjectField T ;
Notification_Settings__c ss;
return 'hello World';
}
if(req.headers.get('Content-Type').equals('application/json'))
return System.JSON.serialize(note);
return null;
My request is
GET /services/apexrest/v.9/notifications/preferences/ritesh HTTP/1.1
X-HostCommonName:
ap1.salesforce.com
Authorization:
OAuth 00D90000000j9AW!AQkAQLba73cDzjXhQ4kkQ2PSru4XpFuJcwr5kg_W_MkZmQnm9vI653FBWeJaABwClQqtJZD_b6j7V0O_elkzvkh7IqRKSUop
X-PrettyPrint:
1
Host:ap1.salesforce.com
X-Target-URI:
https://ap1.salesforce.com
Content-Type: application/json
Connection:Keep-Alive
and the response i am getting is
HTTP/1.1 500 Internal Server Error
Date:
Fri, 25 Jan 2013 16:30:21 GMT
Content-Length:
203
Connection:
close
Content-Type:
application/json; charset=UTF-8
Server:
[
{
"message": "System.NullPointerException: Attempt to de-reference a null object\n\nClass.NotificationRestService.retrievingNotificationSettings: line 34, column 1",
"errorCode": "APEX_ERROR"
}
]
the line where i am getting error is
if(req.headers.get('Content-Type').equals('application/xml'))
i didn't understand why i am getting this error when in header i am passing value Content-Type as application/json ?? please tell me where i am making mistake
Can you System.debug(req.headers) to see if it was passed correctly?
To answer your null pointer question literally - use == instead of equals:
if(req.headers.get('Content-Type') == 'application/xml')
but that won't solve the real underlying problem.
You might want to try with different header (stupid, I know) or decide to add a GET parameter, something like &mode=json