Hi the following code works as expected.
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
val request = HttpRequest(uri = "http://www.google.com")
Http.get(system).singleRequest(request).map(_.entity.dataBytes.runWith(Sink.ignore))
.onComplete { _ =>
println("shutting down actor system...")
system.terminate()
}
However, if I change http://www.google.com to https://www.google.com like the following:
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
val request = HttpRequest(uri = "https://www.google.com")
Http.get(system).singleRequest(request).map(_.entity.dataBytes.runWith(Sink.ignore))
.onComplete { _ =>
println("shutting down actor system...")
system.terminate()
}
I get the following error message:
shutting down actor system...
[ERROR] [02/11/2017 13:13:08.929] [default-akka.actor.default-dispatcher-4] [akka.actor.ActorSystemImpl(default)] Outgoing request stream error (akka.stream.AbruptTerminationException)
Anyone knows why https produces above error and how can I fix it?
It is apparently a known issue, see the following tickets:
akka-http#497
akka#18747
The error seems harmless.
Related
I used the Taurus Gatling guide to create a simple performance test and uploaded the yaml and scala file to blazemeter. When i start the test in blazemeter there is no test result and the bzt.log contains a ClassNotFoundException.
The validator for the yaml says its fine and i can't find anything so I'm lost...
My blazemleter.yaml:
execution:
- executor: gatling
scenario: products
iterations: 15
concurrency: 3
ramp-up: 2
scenarios:
products:
script: productSimulation.scala
simulation: test.productSimulation
My productSimulation.scala is mostly copied from their documentation:
package test
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class productSimulation extends Simulation {
// parse load profile from Taurus
val t_iterations = Integer.getInteger("iterations", 100).toInt
val t_concurrency = Integer.getInteger("concurrency", 10).toInt
val t_rampUp = Integer.getInteger("ramp-up", 1).toInt
val t_holdFor = Integer.getInteger("hold-for", 60).toInt
val t_throughput = Integer.getInteger("throughput", 100).toInt
val httpConf = http.baseURL("https://mydomain/")
val header = Map(
"Content-Type" -> """application/x-www-form-urlencoded""")
val sessionHeaders = Map("Authorization" -> "Bearer ${access_token}",
"Content-Type" -> "application/json")
// 'forever' means each thread will execute scenario until
// duration limit is reached
val loopScenario = scenario("products").forever() {
// auth
exec(http("POST OAuth Req")
.post("https://oauth-provider")
.formParam("client_secret", "...")
.formParam("client_id", "...")
.formParam("grant_type", "client_credentials")
.headers(header)
.check(status.is(200))
.check(jsonPath("$.access_token").exists
.saveAs("access_token")))
// read products
.exec(http("products")
.get("/products")
.queryParam("limit", 200)
.headers(sessionHeaders))
}
val execution = loopScenario
.inject(rampUsers(concurrency) over rampUp) // during for gatling 3.x
.protocols(httpConf)
setUp(execution).maxDuration(rampUp + holdFor)
}
After learning that i can execute the scala file as a test directly if i click the file directly and not the yaml i got better exceptions.
Basicly i made two mistakes:
my variables are named t_concurrency, ... while the execution definition uses a different name. ups.
since gatling 3.x the keyword for the inject is during, so the correct code is: rampUsers(t_concurrency) during t_rampUp
Now everything works.
I hope someone can point me into the right direction!
I try to run one scenario which has several steps that have to be executed in order and each with the same user session to work properly. The below code works fine with one user but fails if I use 2 or more users...
What am I doing wrong?
val headers = Map(
Constants.TENANT_HEADER -> tenant
)
val httpConf = http
.baseURL(baseUrl)
.headers(headers)
val scen = scenario("Default Order Process Perf Test")
.exec(OAuth.getOAuthToken(clientId))
.exec(session => OAuth.createAuthHHeader(session, clientId))
.exec(RegisterCustomer.registerCustomer(customerMail, customerPassword,
tenant))
.exec(SSO.doLogin(clientId, customerMail, customerPassword, tenant))
.exec(session => OAuth.upDateAuthToken(session, clientId))
.exec(session =>
UpdateCustomerBillingAddr.prepareBillingAddrRequestBody(session))
.exec(UpdateCustomerBillingAddr.updateCustomerBillingAddr(tenant))
.exec(RegisterSepa.startRegisterProcess(tenant))
.exec(session => RegisterSepa.prepareRegisterRequestBody(session))
.exec(RegisterSepa.doRegisterSepa(tenant))
setUp(
scen
.inject(atOnceUsers(2))
.protocols(httpConf))
object OAuth {
private val OBJECT_MAPPER = new ObjectMapper()
def getOAuthToken(clientId: String) = {
val authCode = PropertyUtil.getAuthCode
val encryptedAuthCode = new
Crypto().rsaServerKeyEncrypt(authCode)
http("oauthTokenRequest")
.post("/oauth/token")
.formParam("refresh_token", "")
.formParam("code", encryptedAuthCode)
.formParam("grant_type", "authorization_code")
.formParam("client_id", clientId)
.check(jsonPath("$").saveAs("oauthToken"))
.check(status.is(200))
}
def createAuthHHeader(session: Session, clientId: String) = {
val tokenString = session.get("oauthToken").as[String]
val tokenDto = OBJECT_MAPPER.readValue(tokenString,
classOf[TokenDto])
val session2 = session.set(Constants.TOKEN_DTO_KEY, tokenDto)
val authHeader = AuthCommons.createAuthHeader(tokenDto,
clientId, new util.HashMap[String, String]())
session2.set(Constants.AUTH_HEADER_KEY, authHeader)
}
def upDateAuthToken(session: Session, clientId: String) = {
val ssoToken = session.get(Constants.SSO_TOKEN_KEY).as[String]
val oAuthDto = session.get(Constants.TOKEN_DTO_KEY).as[TokenDto]
val params = new util.HashMap[String, String]
params.put("sso_token", ssoToken)
val updatedAuthHeader = AuthCommons.createAuthHeader(oAuthDto,
clientId, params)
session.set(Constants.AUTH_HEADER_KEY, updatedAuthHeader)
}
}
def createAuthHHeader(session: Session, clientId: String) = {
val tokenString = session.get("oauthToken").as[String]
val tokenDto = OBJECT_MAPPER.readValue(tokenString,
classOf[TokenDto])
val session2 = session.set(Constants.TOKEN_DTO_KEY, tokenDto)
val authHeader = AuthCommons.createAuthHeader(tokenDto,
clientId, new util.HashMap[String, String]())
session2.set(Constants.AUTH_HEADER_KEY, authHeader)
}
So I did add the two methods that dont work along as expected. In the first part I try to fetch a token and store in the session via check(jsonPath("$").saveAs("oauthToken")) and in the second call I try to read that token with val tokenString = session.get("oauthToken").as[String] which fails with the exception saying that there is no entry for that key in the session...
I've copied it and removed/mocked any missing code references, switched to one of my apps auth url and it seems to work - at least 2 firsts steps.
One thing that seems weird is jsonPath("$").saveAs("oauthToken") which saves whole json (not single field) as attribute, is it really what you want to do? And are you sure that getOAuthToken is working properly?
You said that it works for 1 user but fails for 2. Aren't there any more errors? For debug I suggest changing logging level to TRACE or add exec(session => {println(session); session}) before second step to verify if token is properly saved to session. I think that something is wrong with authorization request (or building that request) and somehow it fails or throws some exception. I would comment out all steps except 1st and focus on checking if that first request is properly executed and if it adds proper attribute to session.
I think your brackets are not set correctly. Change them to this:
setUp(
scn.inject(atOnceUsers(2))
).protocols(httpConf)
I have an Akka stream source from a Message queue, for example RabbitMq. For each message I want to execute an http request, map the http request to an object and proceed downward.
Is this possible by using a flow from the akka http (Http().outgoingConnection) or should the request be executed inside a map operation?
This is exactly what Http().outgoingConnection is used for (as mentioned in the question):
type MQMessage = ???
val messageToRequest : (MQMessage) => HttpRequest = ???
type ObjectType = ???
val responseToObjectType = (HttpResponse) => ObjectType = ???
val httpHost : String = ???
val messageFlow : Flow[MQMessage, ObjectType, _] =
Flow.map(messageToRequest)
.via(Http().outgoingConnection(httpHost))
.map(responseToObjectType)
In my long, but simple awesome Gatling simulation, I have few responses that ended with error 500. Is it possible to tell gatling to collect these error responses messages in a file during the simulation?
No in production mode. You only have them when debug logging is enabled.
It is possible to collect what ever you want and save it into simulation.log file. Use extraInfoExtractor method when you define protocol:
val httpProtocol = http
.baseURL(url)
.check(status.is(successStatus))
.extraInfoExtractor { extraInfo => List(getExtraInfo(extraInfo)) }
Then define in your getExtraInfo(extraInfo: ExtraInfo) method whatever criteria you want. Example bellow outputs request and response in case of debug is enables via Java System Property OR response code is not 200 OR status of request is KO (it can be KO if you have setup some max time and this max time gets increased)
private val successStatus: Int = 200
private val isDebug = System.getProperty("debug").toBoolean
private def getExtraInfo(extraInfo: ExtraInfo): String = {
if (isDebug
|| extraInfo.response.statusCode.get != successStatus
|| extraInfo.status.eq(Status.valueOf("KO"))) {
",URL:" + extraInfo.request.getUrl +
" Request: " + extraInfo.request.getStringData +
" Response: " + extraInfo.response.body.string
} else {
""
}
}
I want to use data.photos.service.PhotosService to push and pull photos from Picasa. I got a service key file XXXXXXXX-privatekey.p12 from Google console and am now trying to authenticate using said key with google.
The documentation for OAUTH2 using appengine has led me to believe that using the following would be of use:
f = file(settings.SITE_ROOT + '/aurora/' + settings.PRIVATE_KEY, 'rb')
key = f.read()
f.close()
credentials = SignedJwtAssertionCredentials(settings.SERVICE_ACCOUNT_NAME, key, scope = 'http://picasaweb.google.com/data https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile')
http = httplib2.Http()
http = credentials.authorize(http)
service = build("oauth2", "v2", http=http)
user_info = None
try:
user_info = service.userinfo().get().execute()
# neither of these two methods work
#gd_client.SetOAuthInputParameters(signature_method = gdata.auth.OAuthSignatureMethod.RSA_SHA1, consumer_key = "asdfasdfasdf.apps.googleusercontent.com", rsa_key = key, two_legged_oauth = True, requestor_id = user_info.get('email'))
#gd_client.auth_token = gdata.gauth.TwoLeggedOAuthRsaToken(consumer_key = user_info.get('email'), rsa_private_key = key, requestor_id = user_info.get('email'))
except errors.HttpError, e:
logging.error('An error occurred: %s', e)
user_inf0 = {u'verified_email': True, u'id': u'1234', u'name': u'asdfasdfasdf#developer.gserviceaccount.com', u'email': u'asdfasdfasdf#developer.gserviceaccount.com'}
The issue is that either method 1 using SetOAuthInputParameters returns a invalid token, or method 2 returns a 403 restricted.
I am at my wits' end reading through mountains of code that all do regular 3 legged oauth when I really and truly do not want to do it that way. Any ideas/articles I haven't seen yet?
Use gdata.gauth.OAuth2TokenFromCredentials.
auth2token = gdata.gauth.OAuth2TokenFromCredentials(credentials)
gd_client = auth2token.authorize(gd_client)
OAuth2TokenFromCredentials is designed to help you use apiclient and gdata at the same time. Under the covers, it uses the credentials for making sure it has the auth information it needs to perform gdata calls.
Note, if you still get 403, it may be something else entirely. I was using a service account to access a user's data and was getting 403 because I hadn't spec'd the user properly in the SignedJwtAssertionCredentials call.
UPDATE: Here's the basic pattern I used:
from oauth2client.client import SignedJwtAssertionCredentials
credentials = SignedJwtAssertionCredentials(
"XXXXXXXXXXX#developer.gserviceaccount.com",
open("keyfile").read(),
scope=(
"https://www.googleapis.com/auth/drive",
"https://spreadsheets.google.com/feeds",
"https://docs.google.com/feeds"
), # For example.
sub="user#gmail.com"
)
http = httplib2.Http()
http = credentials.authorize(http) # Not needed? See comment below.
auth2token = gdata.gauth.OAuth2TokenFromCredentials(credentials)
gd_client = gdata.photos.service.PhotosService() # For example.
gd_client = auth2token.authorize(gd_client)
If you are using MFA on your google account, you need to use the consent screen authentication method. With Picassa API, it does not work as is, as the request API is slightly different.
import gdata.gauth
import os
import pickle
import gdata.photos.service
clientid='xxx' # https://console.developers.google.com/apis/credentials
clientsecret='xxx'
Scope='https://picasaweb.google.com/data/'
User_agent='myself'
def GetAuthToken():
if os.path.exists(".token"):
with open(".token") as f:
token = pickle.load(f)
else:
token = gdata.gauth.OAuth2Token(client_id=clientid,client_secret=clientsecret,scope=Scope,user_agent=User_agent)
print token.generate_authorize_url(redirect_uri='urn:ietf:wg:oauth:2.0:oob')
code = raw_input('What is the verification code? ').strip()
token.get_access_token(code)
with open(".token", 'w') as f:
pickle.dump(token, f)
return token
token = GetAuthToken()
gd_client = gdata.photos.service.PhotosService()
old_request = gd_client.request
def request(operation, url, data=None, headers=None):
headers = headers or {}
headers['Authorization'] = 'Bearer ' + token.access_token
return old_request(operation, url, data=data, headers=headers)
gd_client.request = request
photos = gd_client.GetUserFeed(kind='photo', limit='10')
for photo in photos.entry:
print 'Recently added photo title:', photo.title.text