Internal error during creation of App Engine via deployment manager - google-app-engine

I am trying to create an App Engine service via Deployment Manager.
My deployment-manager.yaml file looks like below
imports:
- path: create-app-engine-std.py
resources:
- name: create-app-engine-std-app
type: create-app-engine-std.py
properties:
name: app-engine-std-app
appsId: projectId
zip:
sourceUrl: https://storage.googleapis.com/some-bucket/xyz.zip
filesCount: 2
version: v1
runtime: nodejs8
My create-app-engine-std.py look like below
def GenerateConfig(cxt):
deployment = {}
if cxt.properties['zip']:
deployment = {
'zip': {
'sourceUrl': cxt.properties['zip']['sourceUrl'],
'filesCount': cxt.properties['zip']['filesCount']
}
}
resources = [{
'type': 'gcp-types/appengine-v1:apps.services.versions',
'name': 'app-engine-std-app',
'properties': {
'servicesId': 'app-engine-std-test-app',
'appsId': cxt.properties['appsId'],
'deployment': deployment,
'runtime': cxt.properties['runtime'],
'threadsafe': True,
'id': cxt.properties['version']
}
}]
return {'resources': resources}
When I execute the deployment command I get following error
message: '{"ResourceType":"gcp-types/appengine-v1:apps.services.versions","ResourceErrorCode":"500","ResourceErrorMessage":"An
internal error occurred."}'
I tried using the apps.services.versions.create API but I always get following error although I am owner of the project
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
Can someone guide me why I am getting the internal error and why can't I use the API. Thanks...

Related

Auth0 returns a 401 on token request. Auth0 logs show login is successful

I'm integrating auth0 from the tutorial into my own application and have encountered a couple of problems with authentication reflected in the auth0 logs.
This occurs on hitting my react login button:
Login.js
import React from "react";
import { useAuth0 } from "#auth0/auth0-react";
import '../components/App.css'
const LoginButton = () => {
const { loginWithRedirect } = useAuth0();
return <button class="btn btn-primary" onClick={() => loginWithRedirect()}>Log In</button>;
};
export default LoginButton;
However on the Auth0 Application logs I see that I am successfully authenticated and I also get a Failed Exchange, Successful Login and a Warning During Login.
Fixed Log: Warning During Login
Here's the text of the log for Warning During Login:
You are using Auth0 development keys which are only intended for use
in development and testing. This connection (google-oauth2) should be
configured with your own Development Keys to enable the consent page
to show your logo instead of Auth0's and to enable SSO for this
connection. AUTH0 DEVELOPMENT KEYS ARE NOT RECOMMENDED FOR PRODUCTION
ENVIRONMENTS. To learn more about Development Keys please refer to
https://auth0.com/docs/connections/social/devkeys.
This was fixed by following these instructions on the Auth0 website. Essentially:
Creating a google project and OAuth credentials
Adding the credentials inside my Auth0 connected apps
Broken: Login Successful
The log shows that it was a successful login. However on my application, I click the Login button and the expected auth0 modal does not appear.
{
"date": "2020-10-14T09:14:06.549Z",
"type": "s",
"connection_id": "",
"client_id": "<MyClientId>",
"client_name": "<MyClientName>",
"ip": "<MyIP>",
"user_agent": "Safari 13.1.2 / Mac OS X 10.15.6",
"details": {
"prompts": [],
"completedAt": 1602666846548,
"elapsedTime": null,
"session_id": "m0AeJer-FhZ0rb9UFPWgvDkvN7MW36h_"
},
"hostname": "<MyHost>",
"user_id": "<MyUserID>",
"user_name": "<MyUserName>",
"auth0_client": {
"name": "auth0-react",
"version": "1.1.0"
},
"log_id": "90020201014091409270008789595401783120816526823843168290",
"_id": "90020201014091409270008789595401783120816526823843168290",
"isMobile": false,
"description": "Successful login"
}
And looking at the response headers in safari, the token request has 401'd
URL: https://<testdomain>.auth0.com/oauth/token
Status: 401
Source: Network
Address: <testaddress>
Initiator:
auth0-spa-js.production.esm.js:15
Fixed Log: Failed Exchange
After ensuring that I was connecting to goole properly I saw that the issue persisted. Looking at the log I get the following under the heading of a Failed Exchange.
{
"date": "2020-10-14T09:14:07.304Z",
"type": "feacft",
"description": "Unauthorized",
"connection_id": "",
"client_id": "<MyClientId>",
"client_name": null,
"ip": "<TheIP>",
"user_agent": "Safari 13.1.2 / Mac OS X 10.15.6",
"details": {
"code": "*************Rw7"
},
"hostname": "<MyHostName>",
"user_id": "",
"user_name": "",
"log_id": "90020201014091410270002070951766882711015226887425228818",
"_id": "90020201014091410270002070951766882711015226887425228818",
"isMobile": false
}
This question fixed the Failed Exchange issue for me. Change your Auth0 Application properties settings to:
Application Type: Regular Web Application
Token Endpoint Authentication Method: None
This however, unearthed a new issue...
Broken Log: Failed Silent Auth
There's a number of fixes I did here so I'll document them in the answer.
Warning During Login
This was fixed by ensuring the my credentials provider had been properly set up. In this case google. For instructions on how to add google as a credentials provider see here.
Failed Exchange
This was fixed by going to the auth0 dashboard application settings and modifying the setting Application Type to Regular Web Application and the setting Token Endpoint Authentication Method to None.
Login Successful (but not really)
This disappeared once I fixed the Failed Exchange above.
Failed Silent Auth
This was never "fixed" and the error still appears on the log. However the comment on this question prompted me to revisit my Allowed Web Origins and Allowed Origins (CORS) on my auth0 to the below:
https://<mydomain>.eu.auth0.com, http://localhost:3000
This was the last issue in the chain and I could now use login and logout as expected.

Unhandled Rejection (MissingRequiredParameter): Missing required key 'userId' in params

I'm working on integration of react app with aws Lex Chatbot,
I have followed the manual process specified in this doc's - https://aws-amplify.github.io/docs/js/interactions
And each and every-time, when i send a message, I'm getting triggered with following error message
ERROR: Unhandled Rejection (MissingRequiredParameter): Missing required key 'userId' in params
Can someone pls help me with this...
This is the way i'm sending my request
Amplify.configure({
Auth: {
identityPoolId: 'us-east-1:xxxx-xxxx',
region: 'US_EAST_1',
},
Interactions: {
bots: {
"Chatbot": {
"name": "Chatbot",
"alias": "Bot1",
"region": "US_EAST_1",
},
},
}
});
Component:
<ChatBot
title="My Bot"
theme={myTheme}
botName="Chatbot"
welcomeMessage="Welcome, how can I help you today?"
onComplete={this.handleComplete.bind(this)}
clearOnComplete={true}
conversationModeOn={true}
/>

Error "The caller does not have permission" with Google Cloud Endpoints and Guice

I'm setting up Cloud Endpoints Framework for java in my GCP App Engine project, following the official documentation:
https://cloud.google.com/endpoints/docs/frameworks/java/set-up-environment
https://cloud.google.com/endpoints/docs/frameworks/java/using-guice
https://cloud.google.com/endpoints/docs/frameworks/java/adding-api-management
The whole thing works, but when I look at the logs, I can see the error below.
It seems that Guice is triggering it but I've no idea what's happening. Am I missing something?
com.google.api.control.Client check: direct send of a check request service_name: "myproject.appspot.com" (Client.java:210)
operation {
operation_id: "9e037ff9-7822-41b0-87cf-16074a59a153"
operation_name: "1.myproject_appspot_com.GetUser"
consumer_id: "project:myproject"
start_time {
seconds: 1581554680
nanos: 956000000
}
end_time {
seconds: 1581554680
nanos: 956000000
}
labels {
key: "servicecontrol.googleapis.com/referer"
value: "https://core-dot-myproject.appspot.com/_ah/api/static/proxy.html?usegapi=1&jsh=m%3B%2F_%2Fscs%2Fapps-static%2F_%2Fjs%2Fk%3Doz.gapi.en.OfYsKuVZ3qI.O%2Fam%3DwQE%2Fd%3D1%2Fct%3Dzgms%2Frs%3DAGLTcCMVod3aO7ybjljp3cyn2IsEoP0pUg%2Fm%3D__features__"
}
labels {
key: "servicecontrol.googleapis.com/caller_ip"
value: "65.204.128.202"
}
labels {
key: "servicecontrol.googleapis.com/user_agent"
value: "ESP"
}
labels {
key: "servicecontrol.googleapis.com/service_agent"
value: "EF_JAVA/1.0.12"
}
}
failed
endpoints.repackaged.com.google.api.client.http.HttpResponseException: 403
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"errors": [
{
"message": "The caller does not have permission",
"domain": "global",
"reason": "forbidden"
}
],
"status": "PERMISSION_DENIED"
}
}
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.newExceptionOnError(AbstractGoogleClientRequest.java:456)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at endpoints.repackaged.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.api.control.Client.check(Client.java:205)
at com.google.api.control.ControlFilter.doFilter(ControlFilter.java:249)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.api.control.ConfigFilter.doFilter(ConfigFilter.java:127)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at com.google.apphosting.runtime.jetty9.ParseBlobUploadHandler.handle(ParseBlobUploadHandler.java:119)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1182)
at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doHandle(AppEngineWebAppContext.java:187)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:293)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:213)
at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81)
at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:757)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:720)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:690)
at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:882)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:270)
at java.lang.Thread.run(Thread.java:748)
I get the error both locally and when deployed in App Engine
I use the appengine-maven-plugin
mvn clean package
mvn appengine:deploy or locally: mvn appengine:run
For deploying cloud endpoints:
mvn endpoints-framework:openApiDocs
gcloud --project=myproject endpoints services deploy target/openapi-docs/openapi.json
I use the Google API Explorer to test my API: https://apis-explorer.appspot.com/apis-explorer/?base=http://localhost:8080/_ah/api#p/
EDIT:
I get the error as soon as I deploy the openapi.json file. Before deploying it, I don't get the error locally. After deploying it, I get the error both locally and in App Engine.
I have Google Cloud SDK 280.0.0 and app-engine-java 1.9.78

Setting Stackdriver Logging log level for Docker's gcplogs log driver

Docker supported the --log-driver=gcplogs option to log a Google Compute Engine instance to the Stackdriver Logging service (aka Google Cloud Logging). However, all log entries in the Logs Viewer page at https://console.cloud.google.com/logs have no severity level - the entire log line gets placed in a data field like this:
{
metadata: {
projectId: "my-project"
serviceName: "custom.googleapis.com"
labels: {
custom.googleapis.com/secondary_key: "secondary_key"
custom.googleapis.com/primary_key: "primary_key"
}
timestamp: "2016-05-10T00:07:28.700617425Z"
projectNumber: "12345"
}
insertId: "2016-05-09|17:07:28.706161-07|1.2.3.4|1234"
log: "gcplogs-docker-driver"
structPayload: {
container: {
imageName: "us.gcr.io/my-image/name"
created: "2016-05-09T15:09:15.15158629Z"
name: "/my-container"
id: "7b6f9fa1d7517e6a3004b8fa7e52asdfddd8e63d02e43afb3942681a56f4e"
imageId: "sha256:20605d3fdc700asdfa8b08a4aed1f745a0b5d9a8f08ae0db2f90af0bd093"
}
data: "2016-05-10 00:07:28,700 DEBUG [my-file:191][uWSGIWorker1Core0] Some log message here"
instance: {
zone: "us-east1-b"
name: "broker-1"
id: "12345"
}
}
}
I know that stdout and stderr of a Container get logged by Docker, but how can I configure things (Docker, Compute Engine, Stackdriver Logging) to parse my messages to have a log level?

Google Compute Engine: Can't authorize request to Task Queue API

everyone.
I'm having trouble trying to authorize my Compute Engine instance to lease tasks on a Task Queue queue.
I've included de necessary scopes (I think), in the instance creation config:
"metadata": {
"kind": "compute#metadata",
"items": [
{
"key": "startup-script-url",
"value": "[MY-STARTUP-SCRIPT]"
},
{
"key": "service_account_scopes",
"value": "https://www.googleapis.com/auth/cloud-platform"
}
]
},
"serviceAccounts": [
{
"email": "[MY-SERVICE-ACCOUNT]",
"scopes": [
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/taskqueue",
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/compute"
]
}
Also in my queue.yaml, I have added the same service account to the acl directive with the "user_email" attribute:
queue:
- name: [MY-QUEUELIST]
mode: pull
retry_parameters:
task_retry_limit: 5
acl:
- user_email: [MY-COMPUTE-ENGINE-SERVICE-ACCOUNT]
Finally, the script that I run on my instance uses the GoogleCredentials.get_application_default() function to obtain the credentials. This credentials are passed as argument to the build() method (as stated here: https://cloud.google.com/compute/docs/authentication):
The end result is that when I try to list the task of the given taskqueue I get this error:
googleapiclient.errors.HttpError: https://www.googleapis.com/tasks/v1/lists/documentation-compiler-queue/tasks?alt=json
returned "Insufficient Permission">
What am I missing?!
Thanks in advance.
I got my own mistake!
Just ignore this cuestion. I was using:
from googleapiclient.discovery import build
taskqueue_service = build('task', 'v1beta2', credentials=credentials)
instead of:
from googleapiclient.discovery import build
taskqueue_service = build('taskqueue', 'v1beta2', credentials=credentials)
Note the [API name] string in the build method

Resources