In all tutorials on using datastore with appengine standard go environment suggest that I should obtain appengine context from the http request and use this context when manipulating the datastore. Golang appengine datastore introduction
This is unfortunate because it makes it hard for me to use dependency injection of data services into my controllers (handlers). I can't create one instance of aservice while initializing the controller, but I have to pass requests/contexts to every data related operation.
func createHandler(dbService db.DBService) http.HandleFunc
return func (req http.Request, resp http.Response) {
entity := bindEntity(req)
/*
I undertand that I can pass whole request to the dbService
to at least remove dependency on datastore from this handler
but it doesn't seem right to pass request everywhere
*/
ctx := appengine.NewContext(req)
dbService.StoreEntity(ctx, entity)
}
}
Is there a way of getting the appengine context from somewhere else except for request? Or is there some common known design pattern how to separate the responsibility of handling request and manipulating data in app engine?
dbService := db.CreateServcie(somehowObtainedContext)
func createHandler(dbService db.DBService) http.HandleFunc
return func (req http.Request, resp http.Response) {
entity := bindEntity(req)
dbService.StoreEntity(entity)
}
}
This would make the code clearer and testing easier.
Related
I am creating a custom web handler to handle my routes in an web application.
The handler is
type CustomHandler struct{
Db *gorm.DB
}
Then a receiver function:
func (h CustomHandler) Index() http.Handler {
return http.handlerFunc(w http.ResponseWriter,r *http.Request){
//Some code
//use h.Db.Find() ,etc.
})
I am passing this to my router as
//In package
customHandler := &CustomHandler{*gormInstance} //I've already got the instance
router.Handle("/someroute", customHandler.index() )
But a problem with this is gorm.DB instance, being a pointer will mutate once it pass through a route during one request.
Do I have to generate different copies of the initial gorm instance? If so, How can I pass them through each routes on each requests?
According to Gorm documentation :
All Chain Methods will clone and create a new DB object (shares one
connection pool), GORM is safe for concurrent use by multiple
goroutines.
Gorm creates clones when it needs to automatically, there is no need to handle this manually. For more information check this page in the documentation: http://gorm.io/docs/method_chaining.html
What is the GO appropriate way to create a FetchUrl/GetURL function that works from the command line and works from google app engine with its custom way to fetch a url.
I have basic code that fetches and processes some data on a URL. I want to be able to call it from code I use on my desktop, and code deployed to app engine.
Hopefully thats clear, if not please let me know and Ill clarify.
If you have some code which works both on local machine and on AppEngine environment, you have nothing to do.
If you need to do something which should or must be done differently on AppEngine, then you need to detect the environment and write different code for the different environments.
This detection and code selection is easiest done using build constraints. You can put a special comment line in the beginning of your .go file, and it may or may not be compiled and run depending on the environment.
Quoting from The Go Blog: The App Engine SDK and workspaces (GOPATH):
The App Engine SDK introduces a new build constraint term: "appengine". Files that specify
// +build appengine
will be built by the App Engine SDK and ignored by the go tool. Conversely, files that specify
// +build !appengine
are ignored by the App Engine SDK, while the go tool will happily build them.
So for example you can have 2 separate .go files, one for AppEngine and one for local (non-AppEngine) environment. Define the same function in both (with same parameter list), so no matter in which environment the code is built, the function will have one declaration. We will use this signature:
func GetURL(url string, r *http.Request) ([]byte, error)
Note that the 2nd parameter (*http.Request) is only required for the AppEngine (in order to be able to create a Context), so in the implementation for local env it is not used (can even be nil).
An elegant solution can take advantage of the http.Client type which is available in both the standard environment and in AppEngine, and which can be used to do an HTTP GET request. An http.Client value can be acquired differently on AppEngine, but once we have an http.Client value, we can proceed the same way. So we will have a common code that receives an http.Client and can do the rest.
Example implementation can look like this:
url_local.go:
// +build !appengine
package mypackage
import (
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Local GetURL implementation
return GetClient(url, &http.Client{})
}
url_gae.go:
// +build appengine
package mypackage
import (
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Appengine GetURL implementation
ctx := appengine.NewContext(r)
c := urlfetch.Client(ctx)
return GetClient(url, c)
}
url_common.go:
// No build constraint: this is common code
package mypackage
import (
"net/http"
)
func GetClient(url string, c *http.Client) ([]byte, error) {
// Implementation for both local and AppEngine
resp, err := c.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, nil
}
You could get some clues in the golang/appengine project itself.
For instance, its remote_api/client.go provides the client for connecting remotely to a user's production application.
I'm attempting to run a very simple query from Javascript against the Google Cloud Datastore, and no matter what I do, it always seems to just return empty results. My request is being made from a fully authenticated (OAuth2) GAPI client. Here's the relevant code:
var gqlQuery = 'SELECT * FROM swell';
var request = gapi.client.datastore.datasets.runQuery({
datasetId : datasetId,
gqlQuery: {queryString: gqlQuery}
});
request.execute(function(err, result) {
...... Process the results ......
}
I'm using a datasetId that I know is valid. I've tried switching err and result in the callback function, but it doesn't help. No matter what I do, my err and result callback parameters just seem to be blank object collections.
I have an app that tries to store small images (less than 50kB) in memcache, but each call to memcache.Set() results in an error memcache: server error.
I'm on the shared memcache class, so I understand there is no service guarantee, but currently I have no service at all.
Is it a temporary outage? Am I just unlucky?
Here is the piece of code creating an item and calling memcache. ctx is the appengine context for the request. memkey is my key (a string). img_data is a string with my data.
This code works well in the local dev environment.
cache_item = &memcache.Item{
Key: memkey,
Value: bytes.NewBufferString(img_data).Bytes(),
}
err = memcache.Set(ctx, cache_item)
if err != nil {
ctx.Infof("Could not store image in memcache: %s", err)
}
If it's still happening, file a bug, but I suspect it was just a transient problem.
Incidentally, your Value initialiser is unnecessarily complex. This would work the same:
cache_item = &memcache.Item{
Key: memkey,
Value: []byte(img_data),
}
We're building a web application using Django/TastyPie as the back-end REST service provider, and building an AngularJS based front end, using lots of $resource based services to CRUD objects on the server. Everything is working great so far!
But, we would like to reduce the amount of data that we're shipping around when we want to update only one or two changed fields on an object.
TastyPie supports this using the HTTP PATCH method. We have defined a .diff() method on our objects, so we can determine which fields we want to send when we do an update. I just can't find any documentation on how to define/implement the method on the instance object returned by $resource to do what we want.
What we want to do is add another method to the object instances, (as described in the Angular.js documentation here) like myobject.$partialupdate() which would:
Call our .diff() function to determine which fields to send, and then
Use an HTTP PATCH request to send only those fields to the server.
So far, I can't find any documentation (or other SO posts) describing how to do this, but would really appreciate any suggestions that anyone might have.
thank you.
I would suggest using
update: {
method: 'PATCH',
transformRequest: dropUnchangedFields
}
where
var dropUnchangedFields = function(data, headerGetter) {
/* compute from data using your .diff method by */
var unchangedFields = [ 'name', 'street' ];
/* delete unchanged fields from data using a for loop */
delete data['name'] ;
delete data['street'];
return data;
}
PS: not sure from memory, whether data is a reference to your resource of a copy of it, so you may need to create a copy of data, before deleting fields
Also, instead of return data, you may need return JSON.stringify(data).
Source (search for "transformRequest" on the documentation page)
We implemented $patchusing ngResource, but it's a bit involved (we use Django Rest Framework on the server-side). For your diff component, I'll leave to your own implementation. We use a pristine cache to track changes of resources, so I can poll a given object and see what (if any) has changed.
I leverage underscore's _.pick() method to pull the known fields to save off the existing instance, create a copy (along with the known primary key) and save that using $patch.
We also use some utility classes to extend the built-in resources.
app.factory 'PartUpdateMixin', ['$q', '_', ($q, _) ->
PartUpdateMixin = (klass) ->
partial_update: (keys...) ->
deferred = $q.defer()
params = _.pick(#, 'id', keys...)
o = new klass(params)
o.$patch(deferred.resolve, deferred.reject)
return deferred.promise
]
Here's the utility classes to enhance the Resources.
app.factory 'extend', ->
extend = (obj, mixins...) ->
for mixin in mixins
obj[name] = method for name, method of mixin
obj
app.factory 'include', ['extend', (extend) ->
include = (klass, mixins...) ->
extend klass.prototype, mixins...
return include
]
Finally, we can enhance our Resource
include TheResource, PartUpdateMixin(TheResource)
resourceInstance = TheResource.get(id: 1234)
# Later...
updatedFields = getChangedFields(resourceInstance)
resourceInstance.partial_update(updatedFields...)
I would suggest using Restangular over ngResource. The angular team keeps improving ngResource with every version, but Restangular still does a lot more, including allowing actions like PATCH that ngResource doesn't. Here'a a great SO question comparing the two What is the advantage of using Restangular over ngResource?