Using grpc with latest appengine sdk (golang 1.4.2) - google-app-engine

Using latest appengine SDK which is go version 1.4.2 under the hood:
go get google.golang.org/grpc fails due to dependency on http2 which evidently doesn't work with the current appengine sdk release. Seems http2 stuff is only available only from 1.6+:
from Go 1.6 and later
(ref: https://godoc.org/golang.org/x/net/http2)
calling go get golang.org/x/net/http2 did not fix the errors
While I originally caught this in working with google cloud storage, seems it's used throughout this library: https://github.com/GoogleCloudPlatform/gcloud-golang/search?utf8=%E2%9C%93&q=grpc

update the http2 package using
go get -u golang.org/x/net/http2
from the description of go commands (https://golang.org/cmd/go/):
The -u flag instructs get to use the network to update the named
packages and their dependencies. By default, get uses the network to
check out missing packages but does not use it to look for updates to
existing packages.

This turned out to be an issue with my local configuration somehow:
Just for the record...
davidk-osx:Desktop davidk$ more ~/.bash_profile
export GOROOT=/usr/local/go
export PATH=$PATH:$GOROOT/bin
export GOPATH=/Users/davidk/Documents/golang
export PATH=$GOPATH/bin:$PATH
davidk-osx:Desktop davidk$ go version
go version go1.6 darwin/amd64
davidk-osx:Desktop davidk$ /Applications/go_appengine/goapp version
go version go1.4.2 (appengine-1.9.33) darwin/amd64
davidk-osx:Desktop davidk$ mkdir ./TEMP_GOPATH
davidk-osx:Desktop davidk$ export GOPATH=/Users/davidk/Desktop/TEMP_GOPATH
davidk-osx:Desktop davidk$ go get google.golang.org/grpc
davidk-osx:Desktop davidk$ go get google.golang.org/cloud/storage
(i.e. everything is fine, but then...)
davidk-osx:Desktop davidk$ export GOPATH=/Users/davidk/Documents/golang
davidk-osx:Desktop davidk$ go get google.golang.org/grpc
# google.golang.org/grpc/transport
../Documents/golang/src/google.golang.org/grpc/transport/http2_client.go:678: undefined: http2.MetaHeadersFrame
../Documents/golang/src/google.golang.org/grpc/transport/http2_client.go:774: undefined: http2.MetaHeadersFrame
../Documents/golang/src/google.golang.org/grpc/transport/http2_server.go:140: undefined: http2.MetaHeadersFrame
../Documents/golang/src/google.golang.org/grpc/transport/http2_server.go:253: undefined: http2.MetaHeadersFrame
../Documents/golang/src/google.golang.org/grpc/transport/http_util.go:279: f.fr.ReadMetaHeaders undefined (type *http2.Framer has no field or method ReadMetaHeaders)
davidk-osx:Desktop davidk$

Related

Openstack keystone token issue. Failed to discover available identity versions

Im attempting to setup keystone identity service on virtualized Ubuntu 20.4 via Windows Hyper-V on Windows 10.
I followed the instructions on: https://docs.openstack.org/keystone/pike/install/index-ubuntu.html very carefully & attempted this 4 times from scratch with a new VM.
However, all calls to OpenStack web service API either via curl, or command line result in some type of 404 Error.
Here my detailed steps (not including VM setup):
Install & Setup MySQL for keystone:
apt install keystone :
configure keystone:
fernet setup:
Bootstrap keystone:
Apache2 setup:
In /etc/apache2/apache2.conf
Set ServerName controller
source confif file
check port 5000.
set local host name resolution:
Start apache. service apache2 restart
Here is the issue:
Attempt # 1 - Using curl to call keystone API (404):
Attempt # 2 - Using openstack command line tool, again (404):
Troubleshooting:
Keystone logs look good with no error:
Furthermore, apache2 is running as I can see default server page:
So why cant keystone API be reached?
Appreciated
export these variables and test again
export OS_AUTH_VERSION=3
export OS_ENDPOINT_TYPE=internalURL
export OS_INTERFACE=internalURL
export OS_AUTH_TYPE=password
export OS_TENANT_NAME=admin
export OS_NO_CACHE=1
check your wsgi and apache2 vhost config too, to make sure apache will redircet it correctly
would you please check if libapache2-mod-WSGI is installed as it is mentioned in the documentation
enter image description here

App Engine deploy failing with "fatal: unable to access 'https://gopkg.in/yaml.v3/': server certificate verification failed."

Anyone else getting this issue when deploying Go/Other projects to the App Engine?
Deployment was working fine on Tuesday, Sept 28. I tried to deploy on Friday, Oct 1 (yesterday) and today, but I still got the same error. It's not a code-related error as the code hasn't changed since the last deployment.
From the log:
Step #2 - "build": go:
gopkg.in/yaml.v3#v3.0.0-20200313102051-9f266ea9e77c: git fetch -f
origin refs/heads/:refs/heads/ refs/tags/:refs/tags/ in
/layers/google.go.gomod/gopath/pkg/mod/cache/vcs/5ea86ba1b933025fb7a7a539058d4acea777e0b3175c573a70130f7ea565323f:
exit status 128: Step #2 - "build": fatal: unable to access
'https://gopkg.in/yaml.v3/': server certificate verification failed.
CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
If I'm reading this right, is there a missing certificate in the App Engine? But then, I'm not sure what certificate it needs since I never had to provide one before. Maybe if I delete the certificate file, App Engine will recreate it?
Any ideas?
Experienced the certificate error too.
Assuming you are using the official golang image, to fix the certificate issue, you need to update ca-certificates and libgnutls30 packages.
RUN apt update && apt install ca-certificates libgnutls30 -y
I had the same issue is Travis CI build.
I changed the build image to ubuntu 20.04 (from 16.04)
and update package ca-certificates
sudo apt-get update
sudo apt-get install --reinstall ca-certificates
Solved ... mostly
This solution is for those running on App Engine Standard environment using Go 1.11
change your runtime in app.yaml to
runtime: go115
and change your go version in go.mod file:
module myapp
go 1.11
This will allow you to deploy your application. In my case, there were other changes I had to make to the app.yaml. For example, I had to add
app_engine_apis: true
I have my app deployed now but not completely running. I'll post relevant fixes here if I discover anything else. Hope this helps those that run into this issue.
To sum it all up -- it was an issue on the App Engine side. Other users experienced the same problem and the Google team resolved the issue.
For reference I've posted the issue here: https://issuetracker.google.com/issues/201753230
For my project, I reverted the "runtime: go115" changes to go111 (also removed app_engine_apis from app.yaml) and the application deployed as normal.

Go app engine dev_appserver.py doesn't support runtime go112

I am trying to migrate my Go app to runtime go112, but dev_appserver.py throw the following error:
RuntimeError: Unknown runtime 'go112'; supported runtimes are 'custom', 'go', 'go111', 'java', 'java7', 'java8', 'php55', 'php72', 'python', 'python-compat', 'python27', 'python37'.
Current Go version is 1.12.11
Edit: I used apt-get to install the SDK.
apt-get update && apt-get install google-cloud-sdk google-cloud-sdk-app-engine-python\
google-cloud-sdk-app-engine-python-extras\
google-cloud-sdk-app-engine-java\
google-cloud-sdk-app-engine-go\
google-cloud-sdk-datalab\
google-cloud-sdk-datastore-emulator\
google-cloud-sdk-pubsub-emulator\
google-cloud-sdk-cbt\
google-cloud-sdk-cloud-build-local\
google-cloud-sdk-bigtable-emulator\
kubectl -y
That way it seems I can't use gcloud components update and the packages are not up-to-date in the repos.
Now my questions is: Does google-cloud-sdk-app-engine-go support go runtime 1.12? If not, when will it support runtime 1.12?
Update: I installed cloud SDK and run gcloud components update successfully on another computer, dev_server.py still throw the same error. Does the local SDK actually support rumtime 1.12?
go112 runtime is not available on local dev_appserver.py. This has been discussed on the App Engine Go forum at
https://groups.google.com/forum/#!topic/google-appengine-go/kHxZ9zIb_QE
The expectation is that starting from go112 runtime, there is no "appengine" way of running the go server and works normally like any go server outside GAE. There are a few open issues like dispatching urls though.
It sounds like you gcloud SDK is an old version and not recognize "runtime: go112", please run this command in order to update your SDK.
gcloud components update
After the update you can be able to deploy an app engine service using GO 1.12 runtime, check the changes that you need to perform in you App.yaml in this link

Can't find dev_appserver.py with gcloud installation

I've installed gcloud by following the instructions on:
https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu
gcloud is in my path at /usr/bin/gcloud, but the package doesn't seem to have dev_appserver.py in my path. Is it installed? How do I run it?
Platform: Ubuntu 16.04
Edit: By running dpkg -L google-cloud-sdk I've found it at /usr/lib/google-cloud-sdk/bin/dev_appserver.py but when I try to run it I get:
This action requires the installation of components: [app-engine-
python]
You cannot perform this action because this Cloud SDK installation is
managed by an external package manager. If you would like to get the
Also not sure why it wasn't added to my path.
I know the original question concerns Ubuntu, but I just wanted to share some notes for macOS/OS X in case it's helpful for someone else.
I installed the google-cloud-sdk via Homebrew-Cask and overlooked the caveats note:
brew cask install google-cloud-sdk
After installing the SDK cask, I installed the Python App Engine component, as #Rodney Jonace mentioned:
gcloud components install -q app-engine-python
Going back to the caveats note mentioned above, I appended the following the my ~/.zshrc file:
source $(brew --prefix)/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/path.zsh.inc
source $(brew --prefix)/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/completion.zsh.inc
Opening a new terminal tab, I was able to call the extra Python App Engine scripts (e.g., dev_appserver.py) and use the Zsh completions. Hope that helps!
The following articles were also useful:
http://www.javatronic.fr/tips/2014/10/17/installing_google_cloud_sdk_on_ubuntu_with_oh-my-zsh.html
http://www.rainbowbreeze.it/how-to-setup-a-google-app-engine-python-environment-on-mac-osx-using-homebrew/
The google-cloud-sdk deb package comes with the built-in component manager disabled, which is preventing that copy of dev_appserver.py from working through gcloud. If you update your apt-cache, you can install the google-cloud-sdk-app-engine-python and/or google-cloud-sdk-app-engine-java packages that have just started to be published. Directions here:
https://cloud.google.com/sdk/downloads#apt-get

can't find import: "code.google.com/p/goauth2/oauth"

I am using the Google Glass Go quickstart for Goggle App Engine. And got stuck at this issues when starting the app:
can't find import: "code.google.com/p/goauth2/oauth"
I have a propper GOPATH and did get the outauth package with
go get code.google.com/p/goauth2/oauth
I confirmed that the package is available at GOPATH. I ran various other Go App Engine apps without problems. Has anyone seen the same issue?
> go version
go version go1.1.1 (appengine-1.8.2) darwin/amd64
> go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="<EXISTING_PATH>/gopath"
GORACE=""
GOROOT="<EXISTING_PATH>/goroot"
GOTOOLDIR="<EXISTING_PATH>/darwin_amd64"
CC="gcc"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread -fno-common"
CGO_ENABLED="1"
AppEngine for Go currently uses its own version of Go and also sets up a custom GOPATH. You need to put the sources somewhere in your yourapp/ directory (preferable under the very same name of the package, e.g. code.google.com/p/goauth2/oauth/ so that a) the files get transmitted to the server during deployment and b) so that you local GAE environment can find them.
I think the Go GAE team is already working to remove those restrictions, but they haven't announced anything yet.

Resources