We recently converted our app engine project into modules as per the structure below. The problem with this new dev workflow is that we have to rebuild the EAR on every change and relaunch the app engine local dev server. This makes us loose 30s to a minute every time we make a change to the code and want to test it.
/commons
-pom.xml
/model
-pom.xml
/webapp //app engine module
-pom.xml
/apis //app engine module
-pom.xml
/ear
-pom.xml
pom.xml //main (parent) project pom
In our previous workflow, with the monolithic app, we could use app engine's hot reload functionality, where modifying code in an IDE (e.g. eclipse) would be picked up automatically.
What do you guys recommend as the best maven config and/or dev workflow in this case? Ideally, a change in any of the modules would not require a full rebuild of the project.
I am using a similar structure with a small difference. The top level directory has war and ear and then they contain their specific pom.xml. I use Eclipse for debugging, and I am able to hot deploy "most of the time" and I am not using Eclipse plugin, which (I understand) is what you want.
Directory Structure
.
|-- pom.xml
|-- README.md
|-- my-ear
| |-- devpid
| |-- pom.xml
| `-- src
| `-- main
| `-- application
| `-- META-INF
`-- my-war
|-- build
| `-- classes
| |-- main
| | |-- java
| | `-- webapp
| `-- test
| `-- java
|-- pom.xml
`-- src
|-- main
| |-- java
| | `-- com
| `-- webapp
| |-- css
| |-- favicon.ico
| |-- index.html
| |-- js
| |-- test.html
| `-- WEB-INF
`-- test
`-- java
Tools
Eclipse Luna without Google App Engine Plugin (or SDK)
Maven 3.2.1
Google App Engine SDK 1.9.6
Dev Workflow
If you already have source code, keep it somewhere else and generate a skeleton using mvn appengine command.
Run the first cut with a simple Hello World using only maven and terminal and mvn appengine:devserver command.
Once done, generate the eclipse project.
Import the eclipse project as a Maven project. It will see the jars via Maven. I won't have written this answer before Luna as it required too many tweaks. In Luna, this works automatically.
The step above will create three projects, top level, ear and war each with pom.xml - It's OK.
In eclipse, provide the output directory as war/target directory. This is the step which makes it possible to hot deploy.
In maven ear/pom.xml, add xArgs to appengine plugin for running in debug mode.
<plugin>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-maven-plugin</artifactId>
<version>${appengine.target.version}</version>
<configuration>
<jvmFlags>
<jvmFlag>-Xdebug</jvmFlag>
<jvmFlag>-Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=n</jvmFlag>
</jvmFlags>
<disableUpdateCheck>true</disableUpdateCheck>
</configuration>
</plugin>
Notice the suspend=n.
Run the app engine from outside eclipse using mvn appengine:devserver from the ear directory. I use this command:
mvn appengine:devserver > ~/.logs/.appengine.devserver.logs & echo $! > devpid
Let's call this Terminal 1.
An advantage of this method is that your console is not captured by Eclipse, so you are free to use a tool of your choice to view it, like multitail etc. I use this simple tail command:
tail -f ~/.logs/.appengine.devserver.logs | sed 's/INFO/^[[0;34m&^[[0m/g;s/ERROR/^[[0;31m&^[[0m/g;s/WARN\|WARNING/^[[0;35m&^[[0m/g;s/SEVERE\|FATAL/^[[0;31;47m&^[[0m/g'
The above is a difficult to type command. Every instance of ^[ is actually Ctrl+V Esc - it is worth the effort of typing it once. But this is of course subjective and up to you.
In Eclipse, create a Debug Profile for your project under Remote Java Application - select the war project and socket attach options. This step is available on the internet at many places, here is an image nevertheless
Open another terminal, Terminal 2 in the war directory and keep it open in order to run mvn compile install when you need to.
You are good to go. You should be able to integrate your source code by just pasting it at the right place. You should also be able to use standard debugging techniques. Eclipse will compile at the right location and devserver will detect it all right. If Eclipse throws a warning, ignore it.
This works most of the time. Sometimes, you save something that breaks compilation of the whole project, or change a function name being called from a pre compiled class or simply change web.xml which is loaded at start up. Of course then hot deploy will not work.
In such a case, stop your remove debug from within eclipse, complete your tasks, run mvn compile install from Terminal 2. Devserver will autodetect.
Mostly, I hardly need to touch the tail running in Terminal 1. Devserver does not tend to need restart.
Unless I am changing web.xml or refactoring, I do not need to run mvn compile install from outside.
My reason for giving list of windows (Eclipse, Terminal 1 and Terminal 2) is just to show that Alt+Tab is actually faster than Shift+F7 from within eclipse. It is subjective and of course up to you.
Related
I currently have a project structure as shown below:
Parent (Maven)
|-- SpringBoot (Maven + Kotlin)
|-- Shared module (Maven + Kotlin)
|-- React with KotlinJS (Preferably maven)
the last module is React with KotlinJS. As seen in the "new project" wizard of IntelliJ, it says it's only available with Gradle. Is there a possibility that I can also build the React KotlinJS app with Maven? Because I want to remain the current project structure in Maven for code sharing between modules.
Anyone has a clue? I tried adding Gradle build files to the other modules but that doesn't seem to work.
I have finished the tutorial for sphinx-versioning completely. After running the following command, I obtained a new index.html.
sphinx-versioning build -r feature_branch docs docs/_build/html
open docs/_build/html/index.html
However, I want to have version number such as 0.5.0 and 0.6.0 instead of the branches. How to stack the documentation with version numbers instead of the branches? I can't seem to find it in the official sphinx-versioning documentation.
In Pyramid, we create branches with the version number, e.g., 1.10-branch. Alternatively you can use git to tag a version number, then go into the RTD Admin for your project, and under "Versions" activate it for publication.
In the end I realized that sphinx-versioning does not recognize tag, instead it will only recognize release.
However, another problem with sphinx-versioning is that if you are using the autodoc extension, the documentation generated will be the same as your existing version across all branches and versions. The reason is that the autodoc will only generate the documentation with the package you are having on your computer now, it will not automatically download the old package and generate the older version of documentation for you. But there is a workaround for it.
Complete solution (hacks)
Say you have two releases v1.0 and v2.0 on GitHub.
Then you do a git checkout v1.0, and build the html with sphinx-versioning build <your source location> output1.
Similarly, do git checkout v2.0, and build the html with sphinx-versioning build <your source location> output2.
Then you will have two output folders like this:
output1
├── index.html
├── master
├── v1.0
└── v2.0
output2
├── index.html
├── master
├── v1.0
└── v2.0
I am omitting other unimportant files here.
Now, we just need to delete that v1.0 folder under output2 and move the v1.0 folder from output1 to output2. Then you will have a perfectly working autodoc generated documentation together with a working versioning.
Of course, the drawback for this is that the build time would increase exponentially as you have more versions and you need to manually build so many versions. But still works as a quick fix. Maybe we can write a script to do this for us so that we do not need to build them manually?
TL;DR
sphinx-versioning does not support versioning that well especially when you use autodoc. There are hacks to make it works, but it would be very slow and tedious.
If you want something convenient and don't mind having ads on your documentation, just follow Steve Piercy's suggestions and use the hosting service provided by RTD.
I need to some help to identify the correct solution that would help me create a seamless documentation for my friends.
We have several repositories in which a doc folder with several .MD files are going to be placed
Repo1
|- Readme.MD
|-docs
|- Installation.MD
|- Usage.MD
Repo2
|- Readme.MD
|-docs
|- Installation.MD
|- Usage.MD
Repo3
|- Readme.MD
|-docs
|- Installation.MD
|- Usage.MD
We would like to use something like vuepress to generate a static site.
If there is any tool/framework which can easily solve this issue. I would be grateful
Thanks a lot for any response,we will definitely put below what we have done
You can accomplish this using MkDocs as your static site generator and the multirepo plugin. Below are the steps to get it all setup. I assume you have Python installed and you created a Python venv.
python -m pip install git+https://github.com/jdoiro3/mkdocs-multirepo-plugin
mkdocs new my-project
cd my-project
Add the below to your newly created mkdocs.yml. This will configure the plugin.
plugins:
- multirepo:
repos:
- section: Repo1
import_url: {Repo1 url}
- section: Repo2
import_url: {Repo2 url}
- section: Repo3
import_url: {Repo3 url}
Now, you can run mkdocs serve or mkdocs build, which will build a static site with all the documentation in one site.
Here is an idea,
Initialize a repository, lets call it TheRepo
Add all depending repos (in our case Repo1, Repo2, Repo3) as git submodules to TheRepo
Now we have everything from Repo1, Repo2 and Repo3 into TheRepo, which I assume might be undesirable for a documentation website.
Create a bash script (using find, grep, rm or similar bash charm) to retain the desired *.md files and remove the undesirable source files. Here is a sample on how-to:
// remove everything else that is not markdown file
find . -type f ! -name '*.md' -delete
Initialize vuepress at TheRepo's root and let vuepress generate cosolidated documentation.
If you do a good job at step 3. you can have a seperate section/classification for each individual dependency Repo in the resulting documentation.
To refresh the contents, simple use git submodule update to update documentation and then chain it with script created in step3 as to automate the refresh process.
git submodule update && ./remove-undesirable-files.sh
I want to start a new app that will have both web and reactnative interfaces.
I decided to move all business -non enviroment dependent- code into a third package -aka sdk- that i can share between both react & react native .
So my project now has 4 modules
Web -- created with cra
Sdk -- mainly redux + redux saga + react containers + Hoc's
Mobile -react native
Server - nodejs express api.
All web, mobile and server will depend on Sdk module.
sdk module will depend on server module -mainly to impory mocks and data interfaces.
Is there any standard way to achieve such structure ?
Most probably i would love to
use yarn workspaces to hoist all node-modules into one folder to avoid reinstalling packages in every project
i will be working in all 4 projects at same time, so i need hotreload to be aware of this.
** challenges im facing **
Cra doesnot transpile code outside src folder so although web project does refresh qhen i make changes on sdk. It cannot understand es6 code.
Jest also doesnot understand es6 from node_modules
How can i avoid rebuilding step while working on both sdk and web modules simultaneous ?
Yarn workspace sounds like a good approach for the project structure you're thinking.
You can have a packages directory where you can add your projects:
/packages
- web
- sdk
- native
Now you can use babel to watch for code changes for each of your package using babel -w and yarn workspace will take care of linking them together.
If the babel watchers are running, any changes that you make to the sdk will be reflected to both web and native packages. You can also club all of these together using something like concurrently to fire up watchers using a single command.
I have co-authored an open-source library where we follow a similar structure which you may check here. The difference in this project is that our redux logic is in a separate repo.
In order for jest to work, you can add a test env into your .babelrc file which transpiles modules. So you can add two different environments like test which transpiles into commonjs modules and an es environment which keeps ES modules so your users can take advantage of tree-shaking. Example config
Hope this gives you a good starting point :)
You could try a Project structure like this:
| package.json
|- node_modules
|- Web
| package.json
|- SDK
| package.json
|- Mobile
| package.json
|- Server
| package.json
Then you could install everything within the root folder and set the NODE_PATH env variable :
export NODE_PATH='yourdir'/node_modules
So I'm trying to run my go app with google's app engine. When I run goapp server I get this error:
go-app-builder: Failed parsing input: app file model.go conflicts with same file imported from GOPATH
This is my project layout:
.
├── model
│ └── model.go
├── reqres
│ └── reqres.go
├── app.yaml
├── service.go
├── main.go
└── transport.go
If I run it without app engine I don't any get errors and the app runs fine.
According to my experience you get this error because your project folder is also under your GOPATH. "goapp" kind of clone your project folder and builds it against the go environment GOPATH and GOROOT... Doing so it finds duplicated symbols for all package that you have been declared under your project.
Here is the explanation in go appengine documentation
If you include your package sources in GOPATH, you must be careful not to place the source code at or below any directories in your App Engine project that contain app.yaml files. If that happens, a package could be loaded twice, once for the path relative to a module's directory, and once for the fully-qualified path. This can cause subtle problems, so the Go SDK scans your project and your GOPATH, detects this case, and reports it as an error.
Under the same link you will find some advises by google for your project structure and one of them is (your project break that guideline):
Do not include any subdirectories in a module's directory.
If you want a repository with your application definition and go packages I encourage you to adopt the folliwing structure:
projectRoot
|- modules
| |- myModule1
| | |- init.go // router pattern to handler
| | |- myModule1.yaml // configuration for the module
| |- myModule2
| |- init.go // router pattern to handler
| |- myModule2.yaml // configuration for the module
|
|- pkg
| |- myModule1
| | |- *.go // sources, subfolders(packages)
| | // with handlers and business code
| |- myModule2
| | |- *.go // sources, subfolders(packages)
// with handlers and business code
This structure is convinient and improves debugging experience as explained in the article debugging Go appengine module with visual studio code