How to enable api-key auth for all version when deploying multiple versions to same configuration in Google Clould Endpoint - google-app-engine

I deployed 2 versions of openapi.yaml file to Google Cloud Endpoint using the Cloud Endpoint's versioning feature(i.e gcloud service-management deploy openapi_v1.yaml openapi_v2.yaml). Each version of the yaml file contains a version number and basepath different from the other, one endpoint that use api-key authentication, and definition for api-key authentication tag. After deployed to Endpoint, the configuration shows both yaml file, however deploying an api to GAE using this configuration will only have api-key authentication turned on for the newer version.
Does anyone know if this is a known bug, or there is something else I need to do to enable authentication for all versions?
The .yaml file looks like the following. The two versions I used to test on are identical except version and bathpath:
swagger: "2.0"
info:
description: "This API is used to connect 3rd-party ids to a common user identity"
version: "0.0.1"
title: "****"
host: "uie-dot-user-id-exchange.appspot.com"
basePath: "/v0"
...
- "https"
x-google-allow: all
paths:
...
/ids/search:
get:
operationId: "id_search"
produces:
- "application/json"
security:
- api_key: []
tags:
- "Ids"
summary: "Privileged endpoint. Provide any id (3rd party or otherwise) and get a hash of all ids associated with it."
parameters:
- in: "query"
name: "id_type"
description: "Type of id to search"
required: true
type: string
- in: "query"
name: "id_value"
description: "Value of id to search"
required: true
type: string
responses:
200:
description: "AssociatedIdsHash"
schema:
$ref: '#/definitions/AssociatedIdsHash'
400:
description: "Bad request. Requires both id_type and id_value query parameters."
401:
description: "Unauthorized. Please provide a valid api-key in the \"api-key\" header."
404:
description: "Not found - no entry found for key provided"
...
################ SECURITY DEFINITIONS ################
securityDefinitions:
# This section configures basic authentication with an API key.
api_key:
type: "apiKey"
name: "key"
in: "query"

I can replicate this issue and it appears to be a bug.
What does work is adding the API key restriction on the global level for both versions rather than at the per-path level. Perhaps this workaround will suffice for your use case.
...
security:
- api_key: []
path:
...

Related

ShinyProxy 2.6.1 access with Identity Server

I followed the instruction from this website to add the authentication with Identity Server. The configuration is quite simple
proxy:
title: Open Analytics Shiny Proxy
port: 8080
authentication: openid
openid:
auth-url: https://identityserverurl/connect/authorize
token-url: https://identityserverurl/connect/token
jwks-url: https://identityserverurl/.well-known/openid-configuration/jwks
logout-url: https://identityserverurl/Account/Logout?return=http://yourshinyproxy:8080/
client-id: ShinyProxy
client-secret: secret
scopes: [ "openid", "profile", "roles" ]
username-attribute: aud
roles-claim: role
And the authentication seems working. When I add the access-groups to display only the app for a particular role, it doesn't work
specs:
- id: 01_hello
display-name: Hello Application
description: Application which demonstrates the basics of a Shiny app
container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
container-image: openanalytics/shinyproxy-demo
container-network: sp-example-net
access-groups: 200122-user
The same code is working with the version 2.4.3 of ShinyProxy.
Is there anything I missed for this configuration in the ShinyProxy 2.6.1?

GCP Extensible Service Proxy encounters error when forwarding request

I have a the following setup:
1. Application (Java microservice) deployed on app engine.
2. Custom domain mapped to hit this service:.
myfavmicroservice.project-amazing.dev.corporation.com
3. This endpoint is secured to require authentication by enabling IAP.
4. Configured ESP to intercept, authenticate and fulfill request to all
backend microservices (like above) with a common gateway endpoint.
5. Microservice is deployed using app.yaml.
6. ESP endpoint is configured using api.yaml (OpenAPI API Surface document)
This is the tutorial I am following:
https://cloud.google.com/endpoints/docs/openapi/get-started-app-engine-standard
app.yaml to deploy the microservice:
runtime: java11
entrypoint: java -jar tar/worker.jar
instance_class: F2
service: myfavmicroservice
handlers:
- url: /.*
script: this field is required, but ignored
The ESP api.yaml for describing microservice api surface is like this
swagger: "2.0"
info:
title: "My fav micro Service"
description: "Serve my favorite microservice content"
version: "1.0.0"
# This field will be replaced by the deploy_api.sh script.
host: microservice-system-gateway-5c4s43dedq-ue.a.run.app
schemes:
- https
produces:
- application/json
paths:
/myfavmicroservice:
get:
summary: Greet the user
operationId: hello
description: "Get helloworld mainpage"
x-google-backend:
address: https://myfavmicroservice.project amazing.dev.corporation.com
jwt_audience: .....
responses:
'200':
description: "Success."
schema:
type: string
'400':
description: "The IATA code is invalid or missing."
schema:
type: string
But the problem is that whenever I make request to endpoint like this:
GET
https://microservice-system-gateway-5c4s43dedq-ue.a.run.app/myfavmicroservice
I always get gateway 500 error. Upon inspection of ESP logs I am finding primarily
1. SSL Handshake Error with Error no 40
2. upstream server temporarily disabled while SSL handshaking to upstream
3. request: "GET /metadatasvc-hello HTTP/1.1", upstream: "https://[3461:f4f0:5678:a13::63]:443/myfavmicroservice
So the ESP is intercepting my request correctly, perhaps forwarding the request in correct format as well as evidenced from #3. But I am getting SSL error.
Why am I getting this error?
Ok figured out the issue. For the benefit of stackoverflow community I am posting the solution here.
I figured that if you use custom domains that you map to app engine like this in the OpenAPI Configuration (That you deploy to ESP), SSL handshake fails:
x-google-backend:
address: https://my-microservice.my-custom-domain.company.com
However if you use the default URL that is assigned by APP Engine upon startup of the microservice like this, everything is fine:
x-google-backend:
address: https://my-microservice.appspot.com
So I am trying to figure out how to use custom domain mappings in ESP OpenAPI configuration. For now though, if I do that the SSL proxying is not working inside ESP.

App Engine deploy: Permissions error fetching application

We are using a service account to deploy our app to App Engine using Travis.
On every merged PR, Travis pulls the code from our GitHub repository, and pulls a Docker image which contains Google Cloud SDK and executes the gcloud app deploy command. We use a Service Account to perform the deployment with "Project Owner" role.
Everything used to work fine until I added a new service to the project which automates SSL certificate generation and renewal, along with a dispatch.yaml file to route traffic incoming from Let's Encrypt for domain verification. I needed to add more permissions to allow updating the SSL certificates we use for our custom domain. I removed the current service account, and created a new one with a new private key. I created a new role with the required permissions to update and view SSL certificates in addition to the previous permissions (all appengine.* permissions). I assigned the new role and the Project Owner role to the new account. After these changes, the deployment fails with the following error when executing the deploy command:
Permissions error fetching application [apps/hollowverse-c9cad]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
I used the same service account on my local machine with logging level set to debug. I got this error:
DEBUG: HttpError accessing <https://appengine.googleapis.com/v1/apps/hollowverse-c9cad?alt=json>: response: <{'status': '403', 'content-length': '335', 'x-xss-protection': '1; mode=block', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'vary': 'Origin, X-Origin, Referer', 'server': 'ESF', '-content-encoding': 'gzip', 'cache-control': 'private', 'date': 'Wed, 02 Aug 2017 14:33:50 GMT', 'x-frame-options': 'SAMEORIGIN', 'alt-svc': 'quic=":443"; ma=2592000; v="39,38,37,36,35"', 'content-type': 'application/json; charset=UTF-8'}>, content <{
"error": {
"code": 403,
"message": "Operation not allowed",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"resourceType": "gae.api",
"description": "The \"appengine.applications.get\" permission is required."
}
]
}
}
>
DEBUG: (gcloud.beta.app.deploy) Permissions error fetching application [apps/hollowverse-c9cad]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
The description says that appengine.applications.get is required to perform the deployment. Looking at the permissions granted to the role assigned to the Travis account we use to deploy, appengine.applications.get is clearly granted:
I assigned every possible App Engine and Project role to the account, but deployment still fails with the same error. However, using the default service account, which is automatically created for every new project on GCP, seems to be working.
I removed the current service account, and created a new one with a new private key.
This is where it went wrong. The new account had the same ID as the previous one. Although I could not find this behavior documented anywhere, it looks like that once an ID is used for a service account, it cannot be used again for a new account, even if the previous one is removed.
We created a new account with a new ID (travis2#hollowverse-c9cad.iam.gserviceaccount.com) instead of travis#hollowverse-c9cad.iam.gserviceaccount.com) and the issue is now resolved.

deploying to google app engine flexible env

i am following
https://cloud.google.com/endpoints/docs/quickstart-app-engine
but when i run
gcloud service-management deploy openapi.yaml
i am hitting:
ERROR: (gcloud.service-management.deploy) PERMISSION_DENIED: Not allowed to get project settings for project instasmarttagger-162719
i am not sure what i have to do to resolve it.
the openapi.yml looks like
VSKUMAR-mac:appengine vskumar$ vi openapi.yaml
- "application/json"
responses:
200:
description: "Authenication info."
schema:
$ref: "#/definitions/authInfoResponse"
x-security:
- google_id_token:
audiences:
# Your OAuth2 client's Client ID must be added here. You can add
# multiple client IDs to accept tokens from multiple clients.
- "YOUR-CLIENT-ID"
definitions:
echoMessage:
properties:
message:
type: "string"
authInfoResponse:
properties:
id:
type: "string"
email:
type: "string"
# This section requires all requests to any path to require an API key.
security:
- api_key: []
securityDefinitions:
# This section configures basic authentication with an API key.
api_key:
type: "apiKey"
name: "key"
in: "query"
# This section configures authentication using Google API Service Accounts
# to sign a json web token. This is mostly used for server-to-server
# communication.
google_jwt:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
# This must match the 'iss' field in the JWT.
x-google-issuer: "jwt-client.endpoints.sample.google.com"
# Update this with your service account's email address.
x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/jwk/YOUR-SERVICE-ACCOUNT-EMAIL"
# This section configures authentication using Google OAuth2 ID Tokens.
# ID Tokens can be obtained using OAuth2 clients, and can be used to access
# your API on behalf of a particular user.
google_id_token:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "accounts.google.com"
x-google-jwks_uri: "https://www.googleapis.com/oauth2/v1/certs"
Looks like i was signed into a different account and was trying to deploy to the app hosted on another account. doing a gcloud list projects helped me identify it

Unable to SSH to Google Cloud

I installed the Google Cloud SDK
Thought Web UI I created a new instance. I am not knowledgeable of SSH. I followed steps as described here: https://cloud.google.com/compute/docs/instances#sshkeys
I have Window 7 OS
I checked firewall rules as suggested here: https://cloud.google.com/compute/docs/troubleshooting#ssherrors
I checked these through Web UI and found rule
"default-allow-ssh 0.0.0.0/0 tcp:22 Apply to all targets"
STEPS I FOLLOWED:
1) > gcloud auth login
(default browser opens up and I authorize the Google Cloud SDK)
Google SDK Shell outputs:
"Saved Application Credentails. You are now logged as [someuser#gmail]
Your current project is [some-project-999].
2) > gcloud compute ssh my-instance --zone us-central1-a
Google SDK Shell outputs:
WARNING: You do not have an SSH key for Google Compute Engine.
WARNING: [C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\..\bin\sdk\ssh-keygen.EXE] will be executed to generate
a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
The key fingerprint is:
ssh-rsa 2048 06:73:ac:e8:f2:31:c8:df:d4:b0:a2:3b:a2:53:6c:09
Your private key has been saved in C:\Users\First Last\.ssh\google_compute_engine.
Your public key has been saved in C:\Users\First Last\.ssh\google_compute_engine.pub.
Your putty key has been saved in C:\Users\First Last\.ssh\google_compute_engine.ppk.
Updated [https://www.googleapis.com/compute/v1/projects/arctic-depth-863].
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
FATAL ERROR: Network error: Software caused connection abort
FATAL ERROR: Network error: Connection timed out
ERROR: (gcloud.compute.ssh) Could not SSH to the instance. It is possible that your SSH key has not propagated to the i
nstance yet. Try running this command again. If you still cannot connect, verify that the firewall and instance are set
to accept ssh traffic.
On the browser's Web UI, I open the Broser's SSH and I navigate to .ssh folder
someuser_gmail_com#my-instance:~$ cd .ssh
someuser_gmail_com#my-instance:~$ cat authorized_keys
# Added by Google
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4OxYxWvIlp...F7As google-ssh {"userName":"someuser#gmail.com","expireOn":"2015-02-21T23:29:06+0000"}
# Added by Google
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzd...KRqcUZmvWr= google-ssh {"userName":"someuser#gmail.com","expireOn":"2015-02-21T23:28:55+0000"}
on Web UI, I navigate to Project's > Compute > Compute Engine > Metadata > SSH KEYS and I see three records
USERNAME KEY
someuser_gmail_com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4...", "edpireOn":"2015-02-21T23:29:06+0000"}
someuser_gmail_com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTIt...", "edpireOn":"2015-02-21T23:29:06+0000"}
First Last ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAi...ZkpSpRt6RQ== First Last#MYPC
In my local computer, I navigate to Users/First Last/.ssh/google_compute_engine.pub and I see
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAit...mGhUKZRgFZkpSpRt6RQ== First Last#MYPC
QUESTIONS:
Does white space in the user's folder path causes problems (i.e. "First Last")?
When the key is created by the Google Cloud SDK, it sets the comment to First Last#MYPC. Is this the correct setting? (I have been reading and trying this and that and I suspect it should be something like someuser#my-instance-public-IP)
When I
Google Cloud SDK > gcloud compute instances describe my-instance --zone us-central1-a --format yaml
canIpForward: false
creationTimestamp: '2015-02-21T14:53:37.276-08:00'
disks:
- autoDelete: true
boot: true
deviceName: my-instance
index: 0
interface: SCSI
kind: compute#attachedDisk
licenses:
- https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/licenses/ubuntu-1204-precise
mode: READ_WRITE
source: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/disks/my-instance
type: PERSISTENT
id: '111812933445597333'
kind: compute#instance
machineType: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/machineTypes/g1-small
metadata:
fingerprint: w3steEkuQUS=
kind: compute#metadata
name: my-instance
networkInterfaces:
- accessConfigs:
- kind: compute#accessConfig
name: External NAT
natIP: 112.134.99.170
type: ONE_TO_ONE_NAT
name: nic0
network: https://www.googleapis.com/compute/v1/projects/some-project-999/global/networks/default
networkIP: 10.356.252.66
scheduling:
automaticRestart: true
onHostMaintenance: MIGRATE
selfLink: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/instances/my-instance
serviceAccounts:
- email: 78111222333-compute#developer.gserviceaccount.com
scopes:
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
status: RUNNING
tags:
fingerprint: DLYFgkKTlB3=
items:
- http-server
zone: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a
C:\Program Files\Google\Cloud SDK>
This is a known issue when using Cloud SDK from Windows.
Please download pageant.exe at [1] and use it to load your ppk key or use Putty (downloadable from the same link) to SSH to instance as documented at [2].
As a workaround you can even rename ssh.exe to ssh-bak.exe and ssh-term.exe to ssh.exe in C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\sdk\
Link:
[1] - http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
[2] - https://cloud.google.com/compute/docs/console#sshkeys

Resources