I am working on a GAE web app which shows movie related data.To get the movie data I am using API from OMDB (http://www.omdbapi.com/) .Below is the code snippet I use to connect to the API.
When i run it locally it works perfectly fine, but doesn't work when deployed on GAE. It throws connection timeout exception, i tried increasing connection timeout period but that didn't work.
String URLstr = "http://www.omdbapi.com/?t="+URLEncoder.encode(Request,"utf-8");
URL url=null;
URLConnection uc = null;
BufferedReader bf = null;
try {
url= new URL(URLstr);
uc = url.openConnection();
uc.setConnectTimeout(15* 1000);
bf = new BufferedReader(new InputStreamReader(uc.getInputStream()));
} catch (IOException e) {
throw new IllegalArgumentException(e.getMessage());
}
Is my code incorrect? are there some restrictions with GAE that i missed?
Your code looks correct. I am having the exact same issue with OMDB API and Google App Engine as of a few weeks ago. I reached out to Brian who runs OMDB API regarding this and I think it had to do with the App Engine IP range being blocked because of abuse a few weeks ago.
I created the following webapp to figure out what external IP address the url fetch from my app was showing up as to the OMDB servers. I deployed the following to GAE to get the public IP.
import webapp2
import logging
from google.appengine.api import urlfetch
class ifconfig(webapp2.RequestHandler):
def get(self):
url="http://ipecho.net/plain"
urlfetch.set_default_fetch_deadline(60)
result = urlfetch.fetch(url)
logging.debug("I think my external IP is %s " % result.content)
self.response.write(result.content)
app = webapp2.WSGIApplication([
('/ifconfig', ifconfig)
])
In Google App Engine, I went to the instances tab and shutdown the instance, and checked what external IP the new instance had. I did this several times, and in my case it seemed like the external IPs were all coming from 107.178.195.0/24, so I provided this information to OMDB API.
I guess this was in the banned IP block, and Brian was able to unblock that range. This fixed my issue and requests to the API started working again.
This possibly might have resolved the issue for you as well, but if it didn't, you might want to figure out what your public IP is and reach out to Brian to see if it's in an IP range that's being blocked
Related
Below are my requirements.
Develop a flask app.
Use collections in the firebase in the app.
Deploy this app on Google App Engine using a standard service account
What I have done.
Created a service account
Downloaded the corresponding credentials json; I am calling it as key.json
written a main.py
cred = credentials.Certificate('key.json')
default_app = initialize_app(cred)
db = firestore.client()
user_ref = db.collection_group('Users')
#app.route('/', methods=['GET'])
def home():
return "<h1>Welcome to my first app</h1>"
#app.route('/users', methods=['GET'])
def getUsers():
try:
result = [user.to_dict() for user in user_ref .stream()]
return jsonify(result), 200
except Exception as e:
result = { "message:"failed"}
return jsonify(result), 500
I have tested this locally and also on deployed on Google App Engine.
In both the cases, key.json was in the same directory as the code.
I have verified that if this key.json is modified to store wrong data, then /users endpoint won't work and gives me a 500 error.
So far so good. I want to know if this is even the right approach.
I want the key.json authentication to applied even for the root / endpoint.
i.e., if the user supplies a valid key.json, only then the Welcome to my first app should be displayed.
Else, Unauthorized user message needs to be displayed.
As mentioned by #Gaefan and #DishantMakwana, as well as in this documentation:
An API key only identifies the application and doesn't require user authentication. It is sufficient for accessing public data.
So in order to authenticate/authorize your users you should reconsider your strategy. I would recommend you to follow the instructions in the Authenticating as an end user Documentation.
I have found that we can use Google Cloud Endpoints for API management. Works as a charm.
I am trying to migrate from Channel API in GAE to firebase. To do this, first, I am trying to setup a local development environment. I cloned the sample app from GAE samples. (Link to sample)
When I ran this, I get the following error, when the web client is trying to authenticate with the firebase DB.The error is in the console.
Screenshot of the error
i.e token authentication failed.Clearly, this points to the fact that generated JWT is not correct.
To be sure, I have done the following:
Created a service account in Google cloud console.
Downloaded the JSON and pointed to this JSON in the environment variable "GOOGLE_APPLICATION_CREDENTIALS"
Put the code snipped from the firebase into WEB-INF/view/firebase_config.jspf file
The code to generate the token is as follows ( from FirebaseChannel.java )
public String createFirebaseToken(Game game, String userId) {
final AppIdentityService appIdentity = AppIdentityServiceFactory.getAppIdentityService();
final BaseEncoding base64 = BaseEncoding.base64();
String header = base64.encode("{\"typ\":\"JWT\",\"alg\":\"RS256\"}".getBytes());
// Construct the claim
String channelKey = game.getChannelKey(userId);
String clientEmail = appIdentity.getServiceAccountName();
System.out.println(clientEmail);
long epochTime = System.currentTimeMillis() / 1000;
long expire = epochTime + 60 * 60; // an hour from now
Map<String, Object> claims = new HashMap<String, Object>();
claims.put("iss", clientEmail);
claims.put("sub", clientEmail);
claims.put("aud", IDENTITY_ENDPOINT);
claims.put("uid", channelKey);
claims.put("iat", epochTime);
claims.put("exp", expire);
System.out.println(claims);
String payload = base64.encode(new Gson().toJson(claims).getBytes());
String toSign = String.format("%s.%s", header, payload);
AppIdentityService.SigningResult result =
appIdentity.signForApp(toSign.getBytes());
return String.format("%s.%s", toSign,
base64.encode(result.getSignature()));
}
Instead of Step #2, have also tried 'gcloud auth application-default login' and then running after unsetting the environment variable - resulting in the same issue
Appreciate any help in this regard.
After further research, I found out additional info which may help others facing the same issue. I did the following to be able to run the sample locally.
Its important to ensure that the service account in appengine has the permissions to access resources. Chose "Editor" as the role for permissions (other levels may also work, but I chose this) for the default appengine service account. This will ensure that you do not run into "Unauthorized" error.
My application was enabled for domain authentication and did not use Google authentication. The sample however has been created for Google authentication. I had to make changes to the sample code to send the userId as part of the URL and removed the code that referred to UserServiceFactory.
The console error did show up even now, but the application worked fine. This error probably can be ignored. In the deployed environment, however, this error does not show up.
I would appreciate if Google/Firebase engineers update this answer with official responses, or update the sample documentation appropriately as currently this information does not seem to be mentioned anywhere.
I have written a WebApp, which is deployed on Google AppEngine. I am trying to fetch a url in my app.. It shows the following error:
Firewall Notification - Your access has been blocked by firewall
policy 732. If you have any further concerns, please contact your
network administrator for more information.
This is how I am fetching the url.. Am I doing something wrong here?
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(REQUEST_TIME_OUT);
conn.setReadTimeout(REQUEST_TIME_OUT);
conn.setDoOutput(true);
int length = dataToBePost.length();
conn.setRequestProperty("Content-Length", (String.valueOf(length)));
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(dataToBePost);
wr.flush();
InputStream inStream = conn.getInputStream();
//using inStream now
NOTE: This is happening only when I deploy my app on Google AppEngine, and if I deploy my app on my local server, it works fine.
The url which I am trying to fetch is outside from Google Cloud/AppEngine network..
Is there any specific thing I have to do to make it work on AppEngine?
This is a Fortinet Firewall notification.
That mean that your administrator blocked you that website.
I don't think Google use Fortinet, they have their own technology. BTW that sound correct as you are able to connect to it when localhost, and your IT block certain connection, like this one.
Source: I have the same at work. Good luck!
I am trying to send an email from google app engine using the python 2.7 library but I keep getting Unauthorized sender in the logs. I have tried my gmail account I created the application with as the sender, I registered another gmail address as a developer and tried that but still get Unauthorized sender. I am not sure if it matters but I do have a domain name registered to this application.
Here is the code I am trying:
message = mail.EmailMessage()
message.sender = "ron.....#gmail.com"
message.subject = "Inquiry"
message.to = "ron.....#gmail.com"
message.body = "Please work"
message.send()
I have looked at other articles to no avail.
Google Appengine sending emails: [Error] unauthorized sender
InvalidSenderError: Unauthorized sender (Google App Engine)
from google.appengine.api import mail
mail.send_mail(sender="stackoverflow.com Hossam <Hossam#stackoverflow.com>",
to="rsnyder <rsnyder#stackoverflow.com>",
subject="How to send an e-mail using google app engine",
body="""
Dear rsnyder:
This example shows how to send an e-mail using google app engine
Please let me know if this is what you want.
Best regards,
""")
EDIT:
Note that sender must be an administrator of the application, so in case that you are not and administrator, follow these steps from the post google app engine: how to add adminstrator account
I found the issue. It was the wrong version of the code. I switched to a version 2 and didn't realize I had to activate it.
My goal is to test out google's orchestrator and the compute engine api by first retrieving a list of active instances. The orchestrator project including the servlet file is stored in a jar.
I'm trying to test out the java google compute engine client api. I have a cron job which calls on the orchestrator servlet. The target for the cron is a backend. From which I try to get the list of instances:
...
AppIdentityCredential credential = getCredential(computeScope);
String appName = ConfigProperties.getInstance().getGceConfigProperties().get("projectId");
try {
httpTransport = GoogleNetHttpTransport.newTrustedTransport();
final Compute compute = new Compute.Builder(
httpTransport, JSON_FACTORY, credential).setApplicationName(appName)
.build();
logger.info("================== Listing Compute Engine Instances ==================");
Compute.Instances.List instances = compute.instances().list(projectId, zone);
InstanceList list = instances.execute();
if (list.getItems() == null) {
logger.info("No instances found. Sign in to the Google APIs Console and create "
+ "an instance at: code.google.com/apis/console");
} else {
for (Instance instance : list.getItems()) {
logger.info(instance.toPrettyString());
}
}
...
There error response I get is(I omitted my project name from the response, I confirmed that I'm using the correct project id in my code):
com.google.cloud.solutions.sampleapps.orchestration.orchestrator.server.GceClientApiUtils
getInstances: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 OK
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "The resource 'projects/<project-name-here>' was not found",
"reason" : "notFound"
} ],
"message" : "The resource 'projects/<project-name_here>' was not found"
}
I've also attempted this by retrieving an access token and making a RESTful call to get the list of instances and i received the exact same response. I confirmed the Url constructed was correct by comparing it against a successful query of the instances using the api explorer.
EDIT: I determined the solution to the issue with help of another post:
I was finally able to find the solution in the post Compute Engine API call fails with http 404
I needed to add my app engine service account as a team member with edit capabilities, which it does not have by default. Once I did this, the code worked as expected. I had to do this through cloud.google.com/console, as if done through appengine.google.com, a pending status will be given to the service account and will not have access.
For me i had to make sure i had authorization. Try this in the terminal gcloud auth login
Make sure you are in the right project, you can run this command on your vm to see if you are in the right project:
gcloud config list
Take a look at this post in Google Groups
Do you have access to the developers console https://console.developers.google.com?
It seems that the user account #appspot.gserviceaccount.com has not access to compute engine. In my case I see #developer.gserviceaccount.com.
If you don't have one, visit https://developers.google.com/console/help/new/#generatingoauth2 to create a new Client ID