I’m using Google App Engine for a golang api with algolia and I’ve been working (indexing records) without any issue in localhost, today I deployed to test it live and I got this error for all my indexing operations
Cannot perform request [POST] /1/indexes/INDEXNAME/batch (APPID.algolianet.com): Post https://APPID.algolianet.com/1/indexes/INDEXNAME/batch: dial tcp: lookup APPID.algolianet.com on [::1]:53: dial udp [::1]:53: socket: operation not permitted
Any solution ideas?
I struggled with setting up algolia with app engine in production, you have to set up a different transport for the algolia client... here is it:
client := algoliasearch.NewClient(ALGOLIA_APP_ID, ALGOLIA_API_KEY)
transport := &http.Client{
Transport: &urlfetch.Transport{
AllowInvalidServerCertificate: true,
Context: appengine.NewContext(r), // r *http.Request
},
}
client.SetHTTPClient(transport)
client.initIndex(INDEX_NAME)
EDIT:
This is now fixed by golang 1.11 release, you can simply use algolia golang as is
Related
I deployed a meteor app on App Engine on one instance, it works well.
However, when I want to scale on two instances, I sometimes got 401 on HTTP GET requests. Every call through websocket is successful with two instances.
More details:
I use meteor-files to handle upload and download.
When I download a file, the client makes an HTTP request (GET) to download the file from the server. In a method, I check this.userId (from the meteor) to compare it with the owner of the file (on mongoDb)
with one instance
when the user is authenticated, it always works: this.userId is always set
with two instances
when the client is authenticated with instance 1 AND the request is directed to instance 1 => OK
when the client is authenticated with instance 1 AND the request is directed to instance 2 => this.userId is null.
What I tried
In app.yaml:
network:
session_affinity: true
However when I check the config in the google app engine service, I got:
network:{}
It seems to be related to this bug: https://issuetracker.google.com/issues/154647126
My questions
How do you handle mutli instances with Meteor ?
How do you handle multi instances with Meteor in Google App Engine?
Thanks,
I followed the following documentation by Google to create ML engine and I deployed my online predicator there:
https://cloud.google.com/ml-engine/docs/scikit/quickstart
Now my question is what is the simplest way I can connect my iOS application to the prepared ML engine?
You can request Online Predictions, the quickstart provides how to request online predictions by using either the gcloud tool or the python client library, but you can also use a HTTP Request to the JSON API to get the predictions by using the projects.predict method [1]:
POST https://ml.googleapis.com/v1/{name=projects/**}:predict
You just have to add your request body with your list of instances using this JSON format [2]:
{
"instances": [
<simple list>,
...
]
}
[1] https://cloud.google.com/ml-engine/reference/rest/v1/projects/predict
[2] https://cloud.google.com/ml-engine/docs/v1/predict-request#request-body
I'm trying to use Firestore in my AppEngine (standard environment) app written in Go. I've been following the "Getting Started with Cloud Firestore" guide and have been using the firestore package documentation to implement a simple example that works fine when running it on my local dev server.
However when I deploy the app and try the deployed version the call to DocumentRef.Set() fails with the error
rpc error: code = Unavailable desc = all SubConns are in TransientFailure
This is my code that reproduces the issue:
func init() {
http.HandleFunc("/test", testHandler)
}
type testData struct {
TestData string `firestore:"myKey,omitempty"`
}
func testHandler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
var firestoreClient *firestore.Client
var firebaseApp *firebase.App
var err error
conf := &firebase.Config{ProjectID: "my-project"}
firebaseApp, err = firebase.NewApp(ctx, conf)
if err != nil {
fmt.Fprintf(w, "Failed to create a new firestore app: %v", err)
return
}
firestoreClient, err = firebaseApp.Firestore(ctx)
if err != nil {
fmt.Fprintf(w, "Failed to create a new firestore client: %v", err)
return
}
data := testData{"my value"}
_, err = firestoreClient.Collection("testCollection").Doc("testDoc").Set(ctx, data)
if err != nil {
fmt.Fprintf(w, "Failed to create a firestore document: %v", err)
return
}
firestoreClient.Close()
fmt.Fprint(w, "Data stored in Firestore successfully")
}
As mentioned before, on the dev server this works fine. So there the returned page contains the text Data stored in Firestore successfully.
When running the deployed code I get Failed to create a firestore document: rpc error: code = Unavailable desc = all SubConns are in TransientFailure instead. Why do I get this error and how can I avoid it?
I've raised an issue about this in the Firestore client library issue tracker and it seems like the situation is a bit complex.
When using App Engine the Firestore client library's network connections goes trough the App Engine socket library. However sockets is only available for paid App Engine apps:
Sockets are only available for paid apps, and traffic from sockets is billed as outgoing bandwidth. Sockets are also limited by daily and per minute (burst) quotas.
So this is the reason why the Firestore client library fails. For small scale projects it's possible to enable billing of your App Engine app and still stay within the free range. If billing is enabled it should work when the app is deployed as well.
However if you are living within the European Union you are not allowed to have a paid App Engine app for non commercial purposes due to Google policies:
If you are located in the European Union and the sole purpose for which you want to use Google Cloud Platform services has no potential economic benefit you should not use the service. If you have already started using Google Cloud Platform, you should discontinue using the service. See Create, modify, or close your billing account to learn how to disable billing on your projects.
So if you are in Europe or for some other reason are unable to use have a paid App Engine app you will not be able to use the Firestore client library.
One alternative in this case is to use the Firestore REST API instead and manually make HTTP requests to Firestore. It's a bit more work, but for smaller scale projects it works.
On AppEngine you need to create a client that uses Http client provided by urlfetch service.
The firestore.NewClient() function accepts ClientOptions parameters that you can create using WithHTTPCLient() function.
Here is an article on issuing HTTP requests from AppEngine Go.
That should help.
I am trying to integrate google pubsub into my project.
So far I managed to get everyhting to work (create/delete topics/subscriptions publish and pull messages) except for push endpoints, I am unable to sent messages to them.
I am using app engine flex environment, and used this code as sample:
https://github.com/GoogleCloudPlatform/cloud-pubsub-samples-java/tree/master/appengine-push
I am not getting any useful logs except a 403 error similar to this "
POST /_ah/push-handlers/receive_message? HTTP/1.1" 403 156 - "-" "versionId-dot-projectId.com"
I followed the document in here which explains what to do in case of 403, but no luck.
https://cloud.google.com/pubsub/docs/troubleshooting
https://cloud.google.com/pubsub/docs/advanced#gae-endpoints
My questions:
Does pubsub need ssl certificates even on app engine.
If it does, what sort of self signed certificates are acceptable.
We're having trouble publishing messages to a Google Cloud PubSub topic on Google AppEngine. Using the Application Default credentials works perfect locally. But once it's deployed on Google AppEngine it gives the following error:
<HttpError 403 when requesting https://pubsub.googleapis.com/v1/projects/our-project-id/topics/our-topic:publish?alt=json returned "The request cannot be identified with a project. Please pass a valid API key with the request.">
I would assume that it's will use the service account of app engine to access the PubSub API. Here is the code we used to create the credentials.
credentials = GoogleCredentials.get_application_default()
if credentials.create_scoped_required():
credentials = credentials.create_scoped(['https://www.googleapis.com/auth/pubsub'])
http = httplib2.Http()
credentials.authorize(http)
pubsub_service = build('pubsub', 'v1', http=http)
The error is thrown when publishing the actual message to PubSub.
pubsub_service.projects().topics().publish(topic="projects/out-project-id/topics/out-topic", body = { 'messages' : [ { 'data': base64.b64encode(request.get_data()) }]}).execute()
Not that the same flow works doing API call's to "BigQuery", so it's not a general Google API problem. It seems to be specific to PubSub...
It's a rare case of the service account without project id embedded in it. We fixed your service account and you should be good to go now. Sorry for the trouble.