I am new to Go and AppEngine. I am trying to figure out how to create packages but I keep running into conflicts. My directory structure is below:
GOPATH
third-party-libs
app
app.yaml
controllers
default.go -- package controllers
models
models.go -- package models
templates
templates.go -- package templates
I am importing the templates package as follows import ("app/templates") inside default.go
When I do goapp serve I get this error:
Failed parsing input: app file templates.go conflicts with
same file imported from GOPATH
I have tried a bunch of things and nothing has worked so far. These are things I have tried:
Changed the templates directory to apptemplates and the corresponding file to apptemplates.go, changed package name to apptemplates. I imported it as app/apptemplates
I tried different combinations by changing the file name but not the package name, vice versa, etc. Either it does not find the file or has a conflict.
I am importing html/template in my templates.go file. So I commented out the entire file just keeping the package declaration but did not make the conflict go away
I thought may be another file is named templates.go but when I do this (at the GOPATH level) find . -name "*.go" | grep "templates.go" I only see the one file I have created.
I am confused as to how packages are created. I have changed the name to something generic so it does not look like a naming issue. Can someone please tell me how I can debug this error?
Thanks!
Rename the package to a non-conflicting name as in #1. Import the package using the path "apptemplates".
Packages inside of the application directory (the directory containing app.yaml) are imported with a path relative to the application directory. See Organizing Go Apps for the complete details.
Related
I'm confused how "control" and "control.in" files works and what exactly is the difference between them.
I looked into a postgresql extension's debian folder and both files contains the same code, but the packaging build fails if I remove control.in.
I looked into the documentation of control fields but didn't get what I was looking for.
The "control" file is used by Debian package managers to specify the metadata and dependencies of a package. The "control.in" file is a template file used to generate the "control" file. It may contain variables that get replaced by values during the package build process. The package build may fail if "control.in" is removed because it is used to generate the final "control" file.
So upon further review, I found, as answered by #fahad-zaheer ,
Control.in serves as a template to build control file and after specifying the variables to make a generic control.in file which can be dynamic in nature,
We can create the control file by :
pg_buildext updatecontrol
This will look for control.in in the same directory, and will replace all the variable like postgreSQL-$Version with the value of $version by fetching from environment variable or if initialized in the control.in itself.
The main difference between the two is that "control.in" is a template file that gets processed during the packaging process to generate the final "control" file.
The reason why the build fails if you remove the "control.in" file is that it is used as a template to generate the "control" file, which is required for the package to be built and installed properly. The "control.in" file contains variables and placeholders that are replaced with actual values during the package building process, so it's essential to have it in place.
This is an XY problem; answering any part would do.
X: I want to generate multiple bundles out of a single source tree. I'm aware of webpack supporting this, but I'm afraid, I can't use it. For example, my Router gets fed by one or more maps mapping a route (string) to a page (component). I'm storing these maps in separate files and comment importing and using them out as needed (I wrote a simple script doing this for me).
Is there a better solution?
Y: So when building the admin bundle, the user pages are not reachable from the index.tsx. Nonetheless, I'm getting typescript errors for them. The same happens even when I create a new unused file containing an error.
How can I avoid compiling unused files?
I'm not posting here all my config files as I hope, there's a simple setting for this somewhere. I'll do it when needed. Alone the list makes me depressed:
.babelrc
.env
.eslintrc
.gitignore
config-overrides.js
package.json
tsconfig.json
tslint-imports.json
This is a partial answer, but hopefully it will take you in the right direction. Typescript looks at tsconfig.json. Assuming your initiation is from an npm script, you can specify which tsconfig to use with the --project flag, like tsc --project tsconfig-1.json.
In your tsconfig.json file, only include the entry file, like
include: [
"src/app/index.ts"
]
If you have something like "src/**/*", then it will process all files. But it you specify only the entry file, then it will process only the files in that import tree. The tree is defined by the import/import() statements.
For reference: https://www.typescriptlang.org/docs/handbook/tsconfig-json.html
If you want to create multiple bundles, then you'll need to run multiple npm scripts and have multiple tsconfig files or else specify the files from the command line.
We have a program people can compile on their machines. It has an HTTP interface but can also be invoked by command line.
In order to provide some nice-looking error pages for HTTP clients, we want to provide error pages. We are using a very simple solution with go's html/template package.
So in order for the program to find the templates, we currently do:
func init() {
prefStr := "path/to/http/tmpl"
pathPrefix,err := filepath.Abs(prefStr)
if err != nil {
log.Warn("Template path %s is not available!", prefStr)
}
pathPrefix + "/err.html"
}
Now when debugging the app, this usually works well - we are in the package's root directory, so filepath.Abs() resolves correctly, like so:
$GOPATH/github.com/user/repo/path/to/http/tmpl (expanding $GOPATH correctly)
But when we invoke the app via executable from the command line, this doesn't work. The command line could of course be invoked from anywhere on the filesystem, for convenience for example to provide a file in the current directory as a parameter.
In short, running /some/other/path/on/fs/our-executable filename.txt results in the init() function above breaking due to wrong concatenation of the directory: it takes /some/other/path/on/fs/ to create the absolute path, which is wrong. Thus it crashes with
panic: open /some/other/path/on/fs/path/to/http/tmpl/err.html: no such file or directory
I've searched and so far only found this:
How can I open files using relative paths in Go?
But this exactly doesn't apply for us.
Another solution proposes to bundle compiled go resources, but this seems rather odd as the error pages are html text.
We've also tried
https://stackoverflow.com/a/31464648/169252
but it has the same effect.
How can we make sure that the paths are always resolved correctly? This seems rather something which shouldn't be too difficult to be done, but we haven't managed so far.
EDIT: This is not an exact duplicate of the question How can I open files using relative paths in Go?. As already mentioned in my question text, I already had looked it up myself. It suggests to use filepath.Abs(). But as laid out in my question, for us this doesn't work, as if our executable is called from different places, filepath.Abs() doesn't return the same value, and thus doesn't work for us.
I think your challenge here is that people can install the program anywhere on the disk and the program will have to be smart enough to know where it is later on.
One of the common approach that I have seen is that people typically use environment variables to anchor them to the application's installation path. I believe you may have seen environment variables with naming pattern of *_HOME like JAVA_HOME, MAVEN_HOME and their values are always filepath to the installation place.
I guess you can do the same here. Force your users to have MYAPP_HOME variable define and at the start of the application make sure that it is set or else throw an error saying MYAPP_HOME is not set.
Then all you need to do would be a simple lookup of the value for MYAPP_HOME + /http/tmpl to source for the template html files.
Example:
package main
import "os"
func main() {
// Assuming MYAPP_HOME has been verified that it is set
// Then:
tmlPath := os.Getenv("MYAPP_HOME") + "/http/tmpl/"
errTml := tmlPath + "err.html"
}
If you're not keen on using the current working directory, or passing the directory in, you can find the absolute executable path by calling os.Executable from the os package.
appPath, err := os.Executable()
The os package will generally contain os specific stuff like how to get the current working directory. It's worth looking through the pkg docs and list of packages at golang.org if your'e ever stuck, as they are pretty good typically you'll find an answer there.
https://golang.org/pkg/os
Another approach you can take here if users install with go get is to rely on the fact that your templates will be installed with the pkg under GOPATH, so you can always find them at $GOPATH/src/your/project/path/templates (or ~/go the default gopath now that it is not strictly required).
The safest way is probably to bundle them with the binary in a virtual file system as this means you depend on nothing external and don't care where your app is hosted or even if it has access to files at all.
I recommend using a relative path in this case.
According to your description, it seems like you are developing a web application. While it works fine on individual developer's machine, you need to be mindful that your application can be deployed under any directory on your production server. You cannot determine where you app is going to be deployed, but you can always determine where static files are relative to the root directory of your app.
When you invoke your app in the command line, you should have all the required static files copied to the same relative path exactly the same with your development environment. My typical structure is:
project/
|- config.json
|- main.go
|- package1/
|- package2/
|- static/
|- templates/
| |- index.html
| |- base.html
|- css/
|- javascript/
|- image/
When you are ready to run your application from command line, be sure to copy your config.json and the static/ directory to the same level as your executable binary. Then all you need to do is to use relative paths in your code without any nightmares.
For the records: as we just have two templates, we resorted to store them as strings in a go file so that they get compiled. Our html templates are very simple, so this is a reasonable way to do it.
So I have spent the better part of two days trying to figure this one out and no matter what I do I can't get things straightened out. Here is what is going on:
Using Go and Appengine. I am running into issues when trying to
get proper unit tests working.
I have tried lots of structures but here is a sample of where I am now: https://github.com/markhayden/SampleIssue
I am running into dependency issues in either goapp serve or goapp test -v ./src/lib1 depending on how I have my import paths set.
If I use "src/lib1" for my import path and then goapp serve. My app boots and runs fine, but when I run tests I get the following failure:
src/lib1/lib1.go:5:2: cannot find package "src/lib2" in any of:
/Users/USERNAME/go_appengine/goroot/src/pkg/src/lib2 (from $GOROOT)
/Users/markhayden/Projects/go/src/src/lib2 (from $GOPATH)
Likewise, if I use "dummy/src/lib1" as my path, my tests are happy and run fine but upon goapp serve ing the app I now get:
2014/11/06 20:33:34 go-app-builder: Failed parsing input: app file lib1.go conflicts with same file imported from GOPATH
Have fiddled with all sorts of different options and can't figure out how to handle dependencies and still have solid testing. Maybe its a appengine / golang bug? Or am I missing something?
Any help would be very much appreciated. Thanks in advance!
Updated everything based on first comment feedback. I can run tests (as I was able to do before) but I still can not serve the app. Here is what I get when running goapp serve
INFO 2014-11-07 17:24:48,727 devappserver2.py:745] Skipping SDK update check.
INFO 2014-11-07 17:24:48,748 api_server.py:172] Starting API server at: http://localhost:60732
INFO 2014-11-07 17:24:48,751 dispatcher.py:185] Starting module "default" running at: http://localhost:8080
INFO 2014-11-07 17:24:48,754 admin_server.py:118] Starting admin server at: http://localhost:8000
ERROR 2014-11-07 17:24:49,041 go_runtime.py:171] Failed to build Go application: (Executed command: /Users/markhayden/go_appengine/goroot/bin/go-app-builder -app_base /Users/markhayden/Projects/go/src/github.com/markhayden/SampleIssue -arch 6 -dynamic -goroot /Users/markhayden/go_appengine/goroot -nobuild_files ^^$ -unsafe -gopath /Users/markhayden/Projects/go -print_extras_hash lib1/lib1.go lib2/lib2_test.go main_test.go main.go lib1/lib1_test.go lib2/lib2.go)
2014/11/07 09:24:49 go-app-builder: Failed parsing input: app file lib2.go conflicts with same file imported from GOPATH
$GOPATH = /Users/markhayden/Projects/go
$GOROOT = not set (according to docs it doesnt need to be if you dont use a custom directory)
App Structure:
$GOPATH/src/github.com/markhayden/SampleIssue/
- app.yaml
- /lib1
- lib1_test.go
- lib1.go
- /lib2
- lib2_test.go
- lib2.go
- main_test.go
- main.go
In main.go:
import (
"fmt"
"github.com/markhayden/SampleIssue/lib1"
"net/http"
)
In lib1/lib1.go:
import (
"fmt"
"github.com/markhayden/SampleIssue/lib2"
)
Appengine "conflicts with same file imported from GOPATH" issue:
Appengine is importing things underneath the root directory (i.e. where the app.yaml is). This will cause two imports, one by appengine when it scans the directories, and a second by your source when it is explicitly imported.
You have two choices:
Don't use the full import path (for sub-folder packages) with appengine.
Remove the source repository part of import. So instead of
"github.com/blah/blah" it would be "blah/blah".
Note: This kinda sucks as it makes your build and software appengine specific. You could make this a little better -maybe- by using build constraints. e.g. +build !appengine or
+build !appengine to include/remove certain files from the build depending on if you are targeting appengine.
Move your modules/dependencies (sub-folders) to a separate and independent project to make it work with the full path import convention:
Get rid of all directories / dependencies in the main project (where
your app.yaml is), so that appengine can't scan and find them.
Move them to another independent project (I did SampleIssueDeps)
with no app.yaml that is not a sub-directory (e.g.
/MarkHayden/SampleIssueDeps).
Then pull those dependencies via
full path import. e.g. github.com/MarkHayden/SampleIssueDeps/lib1.
Summary: For sub-folder packages in an appengine project don't include the "source repository" part of the import path OR only use appengine to init() and move all of your other code to separate projects and use like external dependencies.
I came up with another option that isn't discussed here and in my opinion is much easier to deal with (and keep your app less appengine specific). Lets say you have the repo at github.com/blah/blah and right now the root folder of the repo defines your app engine server.
First, move the app.yaml and other app engine specific files (NOT .go files) into github.com/blah/blah/appengine/app.yaml.
Next, wherever you run your init function for app engine, rename it to something like func Run() { ... }, and then in github.com/blah/blah/whatever.go write something like this:
package appengine
import "github.com/blah/blah"
func init() {
blah.Run()
}
From my experience this has resolved the issue and made things much easier. I'll update this if I run into any major issues that make this a bad solution.
I had a lot of trouble following various answers and understanding how to solve the problem.
But after a lot of research, I believe I understand both cause and solution:
Google app-builder tooling does some path-munging and is causing this.
They are aware of the bug but no ETA to fix it.
Problem summary:
any .go files inside or below the directory holding main.go / app.yaml will be double imported…
In summary, just make sure that ALL of our files/packages are siblings and not decedents of the directory holding those two files...
I have a menu link with the alias "inside" on a joomla 3.1.5 site here: http://naiwellness.com
Every morning I check the site and a folder named "inside" has been created in the root dir which prevents the site from loading as it should, instead displaying a directory contents listing at http://naiwellness.com/inside
Anyone have any ideas how to prevent this please?
Thanks
This is just a theory, but check the xml files of your installed modules and plugins (most likely a plugin) for this line...
<folder>inside</folder>
and if it exists try deleting that line of code - or better still uninstall the extension - that should fix it. It's possible that a plugin requires that folder and has php code to create the folder if it doesn't exist - and so creates the folder each time it is run - i.e. when the site is loaded. The above line of xml code I have asked you to look for will go someway to helping determine if this is the case.