SolrJ - Can't commit updates when authentication plugin is enabled - solr

With Solr 6.3.0, in cloud mode, and 3 external zookeepers as cluster, and use solrJ as client.
A: Without authentication
Before enabling authentication, I use following code to add/update document:
CloudSolrClient client = cloudClientBuilder.build();
UpdateResponse resp = client.add(doc, 5000);
client.commit();
client.close();
return resp;
It works well, the new document is in searching result immediately.
B: With authentication enabled
Then I enabled basic authentication plugin and rule-based authorization plugin (and SSL if that matters).
In order to set credential information, the code is refactored as following:
// create request,
UpdateRequest req = new UpdateRequest();
// add doc to request,
req.add(doc, 5000);
// set credential,
req.setBasicAuthCredentials(user, password);
// create client,
CloudSolrClient client = cloudClientBuilder.build();
client.setDefaultCollection(ConfigUtil.getProp(ConfigUtil.KEY_SOLR_CORE));
// do request & get response,
UpdateResponse resp = req.process(client);
client.commit();
client.close();
Then it will get error similar as this:
Error 401 require authentication, require authentication.
When debugging, the error occurs at line client.commit();.
Try with curl
I use curl to make an update:
curl -k --user solr:password "https://localhost:8983/solr/corexxx/update?wt=json&indent=true&commit=true" -d '[{"id":"20041", "name":"xxx 41", "location":"xxx", "description":"xxx"}]'
It committed successfully ! And, the updates are visible in searching result immediately.
My guess
Since curl works well, I guess solr cloud itself works fine.
Thus the issue is due to the code from B which is based on SolrJ.
Questions:
Why code B get HTTP 401 error? How can I fix it?
Could I use code from A, and still able to provide credential information,if yes, then how?
Thanks.

You should change client.commit() for req.commit(client, ConfigUtil.getProp(ConfigUtil.KEY_SOLR_CORE)), the credentials are setted in the UpdateRequest.

It worked like this:
SolrClient client = new HttpSolrClient.Builder(urlString).build();
UpdateRequest up = new UpdateRequest();
up.setBasicAuthCredentials(user, pass);
up.add(doc1);
up.process(client, core);
up.commit(client, core);
client.close();

Related

com.google.cloud.pubsub.spi.v1.Publisher.publish is not sending data to PubSub

The call to the newer version of com.google.cloud.pubsub.spi.v1.Publisher.publish(pubsubMessage).get() is hanging forever. I'm not sure what the problem is.
Code snippet:
com.google.cloud.pubsub.spi.v1.Publisher publisher = Publisher.defaultBuilder(TopicName.parse("projects/" + projectId + "/topics/" + topicName))
.setChannelProvider(TopicAdminSettings
.defaultChannelProviderBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(ServiceAccountCredentials.fromStream(new FileInputStream(keyFile))))
.build())
.build();
ApiFuture<String> messageIdFuture = publisher.publish(pubsubMessage);
messageIdFuture.get() // HANGS FOREVER!!
The older API works fine where we do:
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(new NetHttpTransport())
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccount)
.setServiceAccountScopes(Arrays.asList(PubsubScopes.PUBSUB))
.setServiceAccountPrivateKeyFromP12File(new File(keyFile))
.build();
Pubsub pusub = new Pubsub.Builder(transport, JSON_FACTORY, credential).setApplicationName("bigquery").build();
PubsubMessage pubsubMessage = new PubsubMessage();
pubsubMessage.encodeData(message.getBytes());
PublishRequest publishRequest = new PublishRequest();
publishRequest.setMessages(Arrays.asList(pubsubMessage));
pubsub.projects().topics().publish(outputTopic, publishRequest).execute();
Can somebody point out what am I missing?
This may be because you have not configured subscription for the topic or given proper permissions in the GCP console.
It is required to have a subscription attached to the topic. Also make sure you give the correct permissions in the console. Note that you give this
"client_email" : (an auto-generated email id)
auto-generated email id with admin subscriber permissions in the console.
You will get this field in your projectname.json credentials file while configuring.
Hope it helps.
I have no idea why but once i've add a compile dependency for guava no more hangs on a get call.
compile group: 'com.google.guava', name: 'guava', version: '23.0'
For us, when we removed guava dependency, it worked. We are using following version for publishing:
com.google.cloud:google-cloud-pubsub:1.33.0

Unity/Android ServerAuthCode has no idToken on Backend

I have an unity app and use the google-play-games plugin with google *.aar versions 9.4.0. I lately changed my Backend (Google App Engine) from php to java. My problem is the following: in php the serverauthcode is used to get the users data (in JWT format) - it was working fine. So I changed to a Java servlet and I am failing since 2 days to get a valid idtoken. I am able to recieve the server auth code from my app and a valid token response is made by GoogleAuthorizationCodeTokenRequest (see code snippet). Unfortunately it does not contain any idtoken content but a valid auth_token. So I can not get the user id to identifiy the user. When I call tokenResponse.parseIdToken(); it is failing with a NullPointerException.
servlet code (authCode is the serverAuthCode I send from the play-games-plugin inside Unity to my GAE):
// (Receive authCode via HTTPS POST)
// Set path to the Web application client_secret_*.json file you downloaded from the
// Google Developers Console: https://console.developers.google.com/apis/credentials?project=_
// You can also find your Web application client ID and client secret from the
// console and specify them directly when you create the GoogleAuthorizationCodeTokenRequest
// object.
String CLIENT_SECRET_FILE = "/mypath/client_secret.json";
// Exchange auth code for access token
GoogleClientSecrets clientSecrets =
GoogleClientSecrets.load(
JacksonFactory.getDefaultInstance(), new FileReader(CLIENT_SECRET_FILE));
GoogleTokenResponse tokenResponse =
new GoogleAuthorizationCodeTokenRequest(
new NetHttpTransport(),
JacksonFactory.getDefaultInstance(),
clientSecrets.getDetails().getTokenUri(),
clientSecrets.getDetails().getClientId(),
clientSecrets.getDetails().getClientSecret(),
authCode,
REDIRECT_URI) // Specify the same redirect URI that you use with your web
// app. If you don't have a web version of your app, you can
// specify an empty string.
.execute();
String accessToken = tokenResponse.getAccessToken();
// Get profile info from ID token -> HERE IT THROWS AN EXCEPTION.
GoogleIdToken idToken = tokenResponse.parseIdToken();
GoogleIdToken.Payload payload = idToken.getPayload();
String userId = payload.getSubject(); // Use this value as a key to identify a user.
String email = payload.getEmail();
boolean emailVerified = Boolean.valueOf(payload.getEmailVerified());
String name = (String) payload.get("name");
String pictureUrl = (String) payload.get("picture");
String locale = (String) payload.get("locale");
String familyName = (String) payload.get("family_name");
String givenName = (String) payload.get("given_name");
the token response looks like (its invalid now):
{
"access_token" : "ya29.CjA8A7O96w-vX4OCSPm-GMEPGVIEuRTeOxKy_75z6fbYVSXsdi9Ot3NmxlE-j_t-BI",
"expires_in" : 3596,
"token_type" : "Bearer"
}
In my PHP GAE I always had a idToken inside this constuct which contained my encrypted data. But it is missing now?! So I asssume I do somthing differently in Java or I made a mistake creating the new OAuth 2.0 Client on the google console.
I checked the accessToken manually via:
https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=ya29.CjA8A7O96w-vX4OCSPm-GMEPGVIEu-RTeOxKy_75z6fbYVSXsdi9Ot3NmxlE-j_t-BI
{
"issued_to": "48168146---------.apps.googleusercontent.com",
"audience": "48168146---------.apps.googleusercontent.com",
"scope": "https://www.googleapis.com/auth/games_lite",
"expires_in": 879,
"access_type": "offline"
}
Is there something I do not see? Help is very much appreciated...
I found a root cause discussion inside the unity plugin "play-games-services" on github:
https://github.com/playgameservices/play-games-plugin-for-unity/issues/1293
and
https://github.com/playgameservices/play-games-plugin-for-unity/issues/1309
It seems that google switching their authentication flow. In the given links they are talking about adding the email scope inside the plugin to get the idtoken again. I'll try that in the next days and share my experience.
Here is a good explaination about what happens:
http://android-developers.blogspot.de/2016/01/play-games-permissions-are-changing-in.html
If you do what paulsalameh said here (Link to Github) it will work again:
paulsalameh: Sure. After you import the unitypackage, download NativeClient.cs and
PlayGamesClientConfig.cs from my commits (#1295 & #1296), and replace
them in the correct locations.
Afte that "unity play-services-plugin" code update you will be able to add AddOauthScope("email") to PlayGamesClientConfiguration, which allows your server to get the idtoken with the serverAuthCode again...
Code snippet from Unity:
PlayGamesClientConfiguration config = new PlayGamesClientConfiguration.Builder()
.AddOauthScope("email")
.AddOauthScope("profile")
.Build();
Now I am back in business:
{
"access_token" : "ya29.Ci8..7kBR-eBdPw1-P7Pe8QUC7e_Zv7qxCHA",
"expires_in" : 3600,
"id_token" : "eyJhbGciOi......I1NiE0v6kqw",
"refresh_token" : "1/HlSZOo......dQV1y4E",
"token_type" : "Bearer"
}

Windows Azure Active Directory Application Setup

I have created WAAD application with several reply urls, e.g.
https://localhost:4444/Search
https://server/Search
https://stage.company.com/Search
https://production.company.com/Search
I am using WSFederationAuthencationModule.CreateSignInRequest method and passing in the reply URL based upon where the code is being executed
public ActionResult Federated()
{
var module = FederatedAuthentication.WSFederationAuthenticationModule;
var scheme = Request.Url.Scheme;
var replyUrl = Url.Action("Index", "Search", null, scheme);
var requestMessage = module.CreateSignInRequest(Guid.NewGuid().ToString(), replyUrl, true);
return new RedirectResult(requestMessage.RequestUrl);
}
It seems that the code always redirects to the last Reply URL that I modified in the UI. It seems when you modify the Reply Url it is placed first in the manifest only that Reply Url is used.
Is my understanding Reply URL flawed?
I used your code in my project but wasn't able to reproduce the issue. The method that you are using produces a WSFed SSO request that contains the replyUrl encoded in the wctx parameter (pass the below SSO request produced by your code via a URL decoder and you'll see the encoded ru).
https://login.windows.net/dushyantgill.com/wsfed?wa=wsignin1.0&wtrealm=https%3a%2f%2fdushyantgill.com%2fWSFedTest&wctx=rm%3d1%26id%3d01b22db4-bfdc-4efd-abb5-2909cf445a51%26ru%3dhttps%253a%252f%252flocalhost%253a44311%252fHome%252fAbout&wct=2014-05-14T05%3a37%3a01Z
The OnAuthenticateRequest handler of the authentication module after processing the response, extracts the replyUrl from the wctx and redirects the user's agent. See http://msdn.microsoft.com/en-us/library/system.identitymodel.services.wsfederationauthenticationmodule.onauthenticaterequest(v=vs.110).aspx
Can you confirm that you have a SessionAuthenticationModule in the pipeline too.
Finally, you can always construct your owner WSFed SSO request, with an explicit WReply parameter with one of the reply URLs that you have configured with your application in AAD.
Hope this helps.
What worked for me was setting the passiveRedirectEnabled to true in the config file. Then the returnUrl parameter in the FederatedAuthentication.WSFederationAuthenticationModule.CreateSignInRequest call worked!
<wsFederation passiveRedirectEnabled="true" ....>

Google Channel API sending message with token

In documents it says 'client_id' part can actually be the token, however it doesn't work. Anyone know why?
https://developers.google.com/appengine/docs/python/channel/functions
If the client_id parameter is actually a token returned by a create_channel call then send_message can be used for different versions of the app. For instance you could create the channel on the front end and then send messages from a backend of the app.
the reason i want to use this, is because i want to send messages to anonymous users as well, without requiring them to login. i don't know if it is possible to assign them a 'client_id' if token doesn't work.
this is how i am creating the token
user = users.get_current_user()
if user:
token = channel.create_channel(user.user_id())
else:
token = channel.create_channel(str(uuid.uuid4()))
then injecting into client
template_values = {
'token' : token,
}
on the client side open the channel
openChannel = function() {
var token = '{{ token }}';
var channel = new goog.appengine.Channel(token);
var handler = {
'onopen': onOpened,
'onmessage': onMessage,
'onerror': function() {},
'onclose': function() {}
};
var socket = channel.open(handler);
socket.onopen = onOpened;
socket.onmessage = onMessage;
}
now send a message
var xhr = new XMLHttpRequest();
xhr.open('POST', path, true);
xhr.send();
in the server,
when the message is received send back a message using the token
channel.send_message(token, someMessage)
back to client
onMessage = function(m) {
alert("you have some message");
}
this sequence works fine if client_id() is used instead of token when calling send_message
In response to btevfik's initial question: Allowing tokens or client_id in send_message is a feature released in 1.7.5 (very recently). Some people may not be familiar with it yet so therefore they suggest to use client_id. Both should work!
The only thing that I can see in your code is the fact that you should not rely on token variable to be correct in between two requests. They may not even land on the same instance of the app. If you share your code with more details I may be able to spot something. The proper way would be to either store the token in the datastore or pass it from the client as a parameter when you send the message that will trigger a message back.
The purpose of this feature was to allow people to send messages from backends (or other versions). Before was not possible whereas now you can do it if you use directly the tokens instead of the client_id.
Long time this post has been around, but just curious about your usage of the token global variable?
I don't see this code:
global token
before you set the token
user = users.get_current_user()
if user:
token = channel.create_channel(user.user_id())
else:
token = channel.create_channel(str(uuid.uuid4()))
If that code is missing, then token will be set in the local scope of the function above and not globally. So, the token value used later will be None (or to what ever the token was initialised with.)
Just a thought, if its still relevant.
I don't think you actually have a problem here.
You are able to send messages to users that are logged in or not.
The problem you are having I think is knowing that there are multiple ways to use the channel API re: tokens.
https://developers.google.com/appengine/docs/python/channel/overview#Life_of_a_Typical_Channel_Message
In this example, it shows the JavaScript client explicitly requests a token and sends its Client ID to the server. In contrast, you could choose to design your application to inject the token into the client before the page loads in the browser, or some other implementation if preferred.
This diagram shows the creation of a channel on the server. In this
example, it shows the JavaScript client explicitly requests a token
and sends its Client ID to the server. In contrast, you could choose
to design your application to inject the token into the client before
the page loads in the browser, or some other implementation if
preferred.
Here's my demo implementation, hope it helps somehow: https://github.com/Paul1234321/channelapidemo.git
Here's the code for creating the channel on GAE:
client_id = str(uuid.uuid4()).replace("-",'')
channel_token = channel.create_channel(client_id)
And in the JS:
channel = new goog.appengine.Channel('{{ token }}');
Have a look at it in action: http://pppredictor.appspot.com/
You shouldn't store request-specific values in global variables. Store them in a cookie or pass them as a request parameter instead.

How to authenticate programmatically to google app engine (with Java)?

I'm trying to authenticate to google app engine programmatically.
I've tried the code sample from the "gae-app-manager" project but it fails:
tmp>java -jar net.sf.gae-app-manager-0.0.1-jar-with-dependencies.jar myaccount#gmail.com mypassword appname
Exception in thread "main" java.lang.Exception: Did not find ACSID cookie
at net.sf.gaeappmanager.google.LogonHelper.loginToGoogleAppEngine(LogonHelper.java:85)
at net.sf.gaeappmanager.google.appengine.Manager.retrieveAppQuotaDetails(Manager.java:34)
at net.sf.gaeappmanager.google.appengine.Main.main(Main.java:55)
Any idea? I'm able to get the token, but there are no cookies.
The code (taken from the gae-app-manager project - http://gae-app-manager.git.sourceforge.net/git/gitweb.cgi?p=gae-app-manager/gae-app-manager;a=blob;f=src/main/java/net/sf/gaeappmanager/google/LogonHelper.java;h=8e09a6d7f864c29b10847ac7fd2eeab2d3e561e6;hb=HEAD):
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add(new BasicNameValuePair("accountType", "HOSTED_OR_GOOGLE"));
nvps.add(new BasicNameValuePair("Email", userid));
nvps.add(new BasicNameValuePair("Passwd", password));
nvps.add(new BasicNameValuePair("service", "ah"));
nvps.add(new BasicNameValuePair("source", source));
HttpPost post = new HttpPost(
"https://www.google.com/accounts/ClientLogin");
post.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
HttpResponse response = client.execute(post);
if (response.getStatusLine().getStatusCode() != 200) {
throw new Exception("Error obtaining ACSID");
}
String authToken = getAuthToken(response.getEntity().getContent());
post.abort();
HttpGet get = new HttpGet(
"https://appengine.google.com/_ah/login?auth=" + authToken);
response = client.execute(get);
for (Cookie cookie : client.getCookieStore().getCookies()) {
if (cookie.getName().startsWith("ACSID")) {
return cookie.getValue();
}
}
get.abort();
throw new Exception("Did not find ACSID cookie");
Thanks,
Li
Have you considered using the OAuth support instead of trying to log in as a web client would? Every App Engine app can act as an OAuth provider with very little work required on the server side to set it up.
To solve the problem use "SACSID" instead of "ACSID"

Resources