New question: Everything I can find on Google suggests that the CloudSQL connector should be working, with the exception of the source code itself (and importing the SDK's cloudsql doesn't work). Is there some place I can find this updated library, if it exists?
---EDIT: I have my answer: it seems the problem is with CloudSQL itself rather than with the driver or something similar.---
I'm trying to access CloudSQL from a Google App Engine Go program.
I've tried both go-sql-driver/mysql and ziutek/mymysql, using Go 1.2.1 and Go 1.4.2. I've tried both the go get version of go-sql-driver and cloned it directly from Github. I've tried both the App Engine installer and the archive.
Every time I try to access a database with my application, the resulting webpage states:
cloudsql: not supported in dev yet
I've seen the other similar question here, tried everything noted there, none of it worked.
The code in question:
import (
_ "github.com/go-sql-driver/mysql"
_ "appengine/cloudsql"
"database/sql"
"net/http"
)
func adminLogin(w http.ResponseWriter, r *http.Request) {
username := formatter(r.FormValue("username"))
password := formatter(r.FormValue("password"))
db, err := sql.Open("mysql",
"root:password#cloudsql(ws-virtual-classroom:database)/logins") // And all the variations on that string I could think of...
defer db.Close()
if err != nil {
log.Print(err)
} else {
rows, err := db.Query("SELECT username FROM admin_logins WHERE username=? AND password=? LIMIT 1", username, password)
defer rows.Close()
if err != nil {
log.Print(err)
} else {
var user string
for rows.Next() {
err = rows.Scan(&user)
if err != nil {
log.Print(err)
} else {
makeCookie(w, r, user, true, true)
}
}
}
}
teachersHome(w, r)
}
The result (this is displayed in my browser after I submit the login form):
the runtime process gave a bad HTTP response: ''
2015/05/17 01:53:06 cloudsql: not supported in dev yet
2015/05/17 01:53:06 http: panic serving 127.0.0.1:56970: runtime error: invalid memory address or nil pointer dereference
goroutine 12 [running]:
net/http.func·011()
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1130 +0xbb
database/sql.(*Rows).Close(0x0, 0x0, 0x0)
/tmp/appengine/go_appengine/goroot/src/database/sql/sql.go:1659 +0x31
main57750.adminLogin(0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
main.go:208 +0x25a
net/http.HandlerFunc.ServeHTTP(0x927c78, 0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1265 +0x41
github.com/gorilla/mux.(*Router).ServeHTTP(0xc20800c730, 0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/home/daniel/go/src/github.com/gorilla/mux/mux.go:98 +0x297
net/http.(*ServeMux).ServeHTTP(0xc20803a6f0, 0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1541 +0x17d
appengine_internal.handleFilteredHTTP(0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/tmp/appengine/go_appengine/goroot/src/appengine_internal/api_dev.go:98 +0x413
net/http.HandlerFunc.ServeHTTP(0x927248, 0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1265 +0x41
net/http.serverHandler.ServeHTTP(0xc208042540, 0x7f76a72ef5d8, 0xc208045860, 0xc2080c4820)
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc2080457c0)
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
/tmp/appengine/go_appengine/goroot/src/net/http/server.go:1751 +0x35e
This is when running it with goapp serve. If I deploy it I seem to have the same problem--the resulting page is blank rather than showing that text, but the log error messages are the same.
The updated library is here: https://godoc.org/google.golang.org/appengine. It doesn't have CloudSQL though. Perhaps you should just use a regular mysql connection locally:
func dialSQL() (*sql.DB, error) {
if appengine.IsDevAppServer() {
// or sql.Open("mysql", "user-name:password#ip-address-of-google-cloud-sql-instance/dbname")
return sql.Open("mysql", "user:password#/dbname")
}
return sql.Open("mysql", "cloudsql:my-instance*dbname/user/passwd")
}
Using a local database is usually the best option because connecting to a live database can be super dangerous. For example it's all too easy to accidentally run your tests against your production database and drop everything.
Nevertheless, Google has instructions for connecting to CloudSQL here: https://cloud.google.com/sql/docs/introduction. The instructions for 3rd party tools will work for Go too.
I don't think cloudsql does anything? The source just looks like it returns that error no matter what lol
https://code.google.com/p/appengine-go/source/browse/appengine/cloudsql/cloudsql.go
// Dial connects to the named Cloud SQL instance.
func Dial(instance string) (net.Conn, error) {
return connect(instance)
}
https://code.google.com/p/appengine-go/source/browse/appengine/cloudsql/cloudsql_dev.go
func connect(instance string) (net.Conn, error) {
return nil, errors.New("cloudsql: not supported in dev yet")
}
Related
When i tried to implement push notification in golang App Engine using onesignal enviorment.But iam getting error "http.DefaultTransport and http.DefaultClient are not available in App Engine".This is my code,
func (c *PushNotificationController) CreateNotification() {
client := onesignal.NewClient(nil)
client.AppKey = "MyAppKey"
client.UserKey = "MyUserKey"
notifID := CreateNotifications(client)
log.Println(notifID)
}
func CreateNotifications(client *onesignal.Client) string {
playerID := "SamplePlayerId" // valid
notificationReq := &onesignal.NotificationRequest{
AppID: "MyAppKey",
Contents: map[string]string{"en": "English message"},
IsIOS: true,
IncludePlayerIDs: []string{playerID},
}
if createRes, res, err := client.Notifications.Create(notificationReq){
if err != nil {
log.Fatal(err)
}
return createRes.ID
}
...
}
Use http on appengine, you have to use urlfetch.
https://cloud.google.com/appengine/docs/standard/go/urlfetch/reference
i.e. the package you use doesn't support appengine.
A similar issue has been addressed by Robby Colvin in this blog
It explains how to make a third party package run in such conditions. Hoping this will help.
What is the GO appropriate way to create a FetchUrl/GetURL function that works from the command line and works from google app engine with its custom way to fetch a url.
I have basic code that fetches and processes some data on a URL. I want to be able to call it from code I use on my desktop, and code deployed to app engine.
Hopefully thats clear, if not please let me know and Ill clarify.
If you have some code which works both on local machine and on AppEngine environment, you have nothing to do.
If you need to do something which should or must be done differently on AppEngine, then you need to detect the environment and write different code for the different environments.
This detection and code selection is easiest done using build constraints. You can put a special comment line in the beginning of your .go file, and it may or may not be compiled and run depending on the environment.
Quoting from The Go Blog: The App Engine SDK and workspaces (GOPATH):
The App Engine SDK introduces a new build constraint term: "appengine". Files that specify
// +build appengine
will be built by the App Engine SDK and ignored by the go tool. Conversely, files that specify
// +build !appengine
are ignored by the App Engine SDK, while the go tool will happily build them.
So for example you can have 2 separate .go files, one for AppEngine and one for local (non-AppEngine) environment. Define the same function in both (with same parameter list), so no matter in which environment the code is built, the function will have one declaration. We will use this signature:
func GetURL(url string, r *http.Request) ([]byte, error)
Note that the 2nd parameter (*http.Request) is only required for the AppEngine (in order to be able to create a Context), so in the implementation for local env it is not used (can even be nil).
An elegant solution can take advantage of the http.Client type which is available in both the standard environment and in AppEngine, and which can be used to do an HTTP GET request. An http.Client value can be acquired differently on AppEngine, but once we have an http.Client value, we can proceed the same way. So we will have a common code that receives an http.Client and can do the rest.
Example implementation can look like this:
url_local.go:
// +build !appengine
package mypackage
import (
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Local GetURL implementation
return GetClient(url, &http.Client{})
}
url_gae.go:
// +build appengine
package mypackage
import (
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Appengine GetURL implementation
ctx := appengine.NewContext(r)
c := urlfetch.Client(ctx)
return GetClient(url, c)
}
url_common.go:
// No build constraint: this is common code
package mypackage
import (
"net/http"
)
func GetClient(url string, c *http.Client) ([]byte, error) {
// Implementation for both local and AppEngine
resp, err := c.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, nil
}
You could get some clues in the golang/appengine project itself.
For instance, its remote_api/client.go provides the client for connecting remotely to a user's production application.
Im trying to serve images from google storage via google images
In order to do that i need to create blob key.
i tried several ways togenerate the key but got errors
loc := fmt.Sprintf("/gs/%s/%s", BUCKET, s)
applog.Infof(appCtx, "%s", loc)
bkey, err := blobstore.BlobKeyForFile(appCtx, loc)
if err != nil {
gc.JSON(500, model.GenericResponse{500, err.Error()})
return
}
opt := &image.ServingURLOptions{}
u, err := image.ServingURL(appCtx, bkey, opt)
if err != nil {
gc.JSON(500, model.GenericResponse{500, err.Error()})
return
}
// i tried with file extension to
/gs/bucktname/CXlvJUKiTmo9joe6
OBJECT_NOT_FOUND
gs://bucktname/rUAJOYKQbORzOYvs
"description": "API error 6 (images: INVALID_BLOB_KEY)"
gs:/bucktname/MlY77iFNbBca2KCA
"description": "API error 6 (images: INVALID_BLOB_KEY)"
What is the correct path ?
Dose the images cached and behind cdn ?
I got this error as well (among others) and here are the fixes which worked for me:
When saving the file to Google Cloud Storage you need to make it public like so: writer.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
The correct path is fmt.Sprintf("/gs/%s/%s", bucketName, objectName) where objectName is the path to the file within the bucket.
You have to make the bucket public. How to do that is described in here.
In the Go Runtime i used the method c.Infof to log messages , but it fails to compile with the following error
c.Infof undefined (type context.Context has no field or method Infof) .
The Error clearly tells that the app engine context returned from c := appengine.NewContext(r) is of type context.Context and it doesnt have a method c.Infof on it. But contrary to this the documentation in https://godoc.org/google.golang.org/appengine/log suggests that this method exists . Another point to note , The method existed on the context returned by "appengine" (import "appengine" ) package , and this doesnt seem to exist on the context returned by the new package google.golang.org/appengine , what is c.Infof equivalent on the new Context of type context.Context returned by package "google.golang.org/appengine" ?
The example in the package documentation is not correct.
Use the log package functions to write to the App Engine log. Here's the corrected example:
c := appengine.NewContext(r)
query := &log.Query{
AppLogs: true,
Versions: []string{"1"},
}
for results := query.Run(c); ; {
record, err := results.Next()
if err == log.Done {
log.Infof(c, "Done processing results")
break
}
if err != nil {
log.Errorf(c, "Failed to retrieve next log: %v", err)
break
}
log.Infof(c, "Saw record %v", record)
}
The example in the package documentation was copied from the App Engine Classic package, but not updated to use the new functions. I suggest reporting this to the App Engine Team.
I built a simple web app using google app engine and golang. in code below, I use fmt.Println twice to print out somehting for debugging purpose. I have no problem running the app. everything works except nothing print out on the terminal.
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("Post").Ancestor(goblogKey(c)).Order("-CreatedOn").Limit(10)
//posts := make([]entity.Post, 0, 10)
var posts []entity.Post
if _, err := q.GetAll(c, &posts); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Println(string(len(posts)) + "...........")
postList := []dto.Post{}
for _, val := range posts {
newpost := dto.Post{
Post: val,
BodyHTML: template.HTML(val.Body),
}
fmt.Println(val.Title)
postList = append(postList, newpost)
}
page := dto.PageData{Title: "Home", Posts: postList}
templates.ExecuteTemplate(w, "index", page)
}
In the real appengine enviroment you can't se anything output to stdout.
Appengine context give you away to log (that you can check in you appengine admin's page and in console playground).
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
c.Debugf("The message: %s", "foo")
...
Read more: https://developers.google.com/appengine/docs/go/reference#Context
standard i/o Or Error is used communicate with app server used by the devleoper. In production system there's no meaning of using standard i/o. In production systems log is used to track the results. In app engine there's some limitations. like fmt, socket etc.
Its always better to use log when testing or running program in remote server.