Following this community tutorial for setting up ktor in GCP AppEngine, I additionally modified the 'webapp' folder to be a full react-based SPA. Running npm start in the webapp directory works fine, as does running ./gradlew appengineRun in the outer directory. But when trying to deploy, it complains that I have >10k files. Without details, it seems the node_modules are most likely to blame.
Looking online for several hours, the old solution was to add a 'skipFiles' section to app.yaml, and the newer solution is to create a .gcloudignore file. In both, listing node_modules should cause them to not be deployed. But, with the tutorial I'm following, I don't have an app.yaml file - it's apparently being constructed from the various .xml configs as part of the staging step of the appEngine gradle plugin. I can create the .gcloudignore file, but it doesn't get pushed to the deploy staging folder (called 'build/staged-app'). Even manually placing it there, it doesn't build properly because then there is a skipFiles and .gcloudignore, which can't coexist.
I feel there is some 1 line change to make this work, but I have no idea what it might be. I've tried creating app.yaml files in the project with overridden settings, but the deploy step doesn't pick them up - it just always generates one from scratch it seems.
Any idea how I can get this to work? Am I doing something fundamentally wrong perhaps?
You do not have an app.yaml file becuaseu you are using the App Engine Gradle Plugin.
EDIT
It seems you have to follow exclude syntax using <exclude> elements. In a pattern, * represents zero or more of any character in a file or dir name, and ** represents zero or more dirs in a path. Files matching this element will not be uploaded when you deploy your AppEngine app.
An <include> element overrides the default behavior of including all files. An <exclude> element applies after all patterns (as well as the default if no explicit <include> is provided).
The following example demonstrates how to designate all .png files as static files (except those in the data/ directory and all of its subdirectories):
<static-files>
<include path="/**.png" />
<exclude path="/data/**.png" />
</static-files>
You can use this with your path node_modules
Related
I've recently tried getting into the whole Node ecosystem and am trying to set up some continuous deployment for my app to AWS Amplify.
For background, my project structure looks like this:
project
public
index.html
src
App.tsx/App.js
package.json
As far as I know, this is basically what create-react-app gave me to start with, and I didn't change the file structure.
For most of my time working on the app, I've been able to go to the base project directory and use
npm start
to launch the app. This will bring me to the App.tsx/js homepage.
However, when I hosted this to AWS Amplify via GitHub, the default build settings actually point to the public directory, so the published site is actually point to index.html (which is basically just an empty placeholder).
While debugging, I ran
npm build
in my root project directory, which constructed a build folder, so now the overall project looks like this:
project
build
index.html
public
index.html
src
App.tsx/App.js
package.json
Now, running
npm start
will bring me to the index.html from the build directory, instead of App.js/tsx as it used to.
The AWS setup says that it will run
npm build
so I assume that what I've done on my local machine is mirroring what the AWS server is doing behind the scenes and explains why AWS is serving the empty index.html.
I've read a few articles and watched some videos about hosting a create-react-app on AWS, and in every version, it looks like AWS will serve the App.tsx/App.js right out of the box, rather than build/index.html, and I've not been able to find a good guide on how to configure this behavior. Quite frankly, there is an overwhelming number of similar-but-slightly-different answers for questions like this, which use different combinations of package managers, packages, hosting services, all on different release versions, with different setups, and it's very difficult for me to tell which ones apply to my scenario.
So I'm hoping someone can help straighten some of this out for me, or point me towards a good resource for learning more about this type of thing. Particularly interested in learning the right way to do these things, rather than a quick hack around whatever my particular issue is.
Some specific questions...
Is deploying things from a /build folder standard convention?
Why does create-react-app create a separate /src/app.tsx and /public/index.html that seem to be competing with one another as the app's "homepage"?
Why does the behavior of
npm start
change depending on whether
npm build
has been run?
Is the correct fix here to just insert my App.tsx component into the index.html? This doesn't seem hard, but doesn't seem right either
I have seen a lot of answers discussing tweaks to webpack.config.js to solve issues like this one. My project does have webpack installed, but as best I can tell, there is no webpack.config.js anywhere. Am I expected to create this file, or should it exist already? In either case, in which directory is it supposed to live? I've seen a couple answers saying it should be in /node_modules/webpack/, but also some saying it needs to live in the same directory as package.json
Things I've tried already: Spent a bunch of time reading through other StackOverflows and watching a few videos, but as outlined above, I'm finding it difficult to tell which could apply to my situation and which are unrelated, given the huge number of unique combinations of build/packages/platforms/versions. Also spent some time monkeying around with file structure/moving code around, but not very productively.
Eventually found my issue. In the production built version of my app (aka, /build), the bundled script created by webpack was failing in the browser because exports was undefined, so index.html was being served in its vanilla state, rather than with the TSX/JSX content. I changed the "module" property in tsconfig.json from commonjs to es6 and this fixed most of the problems.
Also of note is that the reason I couldn't find my webpack.config.js is that I had hidden ALL js files in my project, so VSCode wasn't finding it. I swapped to the suggestion from this blogpost to hide only js files with a matching TS file.
For general learning about how create-react-app works, I eventually found this page, which I found helpful:
https://blog.logrocket.com/getting-started-with-create-react-app-d93147444a27/
For the basic create-react-app
npm start
Is a short command for react-scripts start that sets up the development environment and starts your development server usually localhost:3000
npm build
After you are done developing, this command short for react-scripts build correctly bundles your app for production and optimizes the build for the best performance.
The files generated in the build folder are solely the files you serve to the public folder accessible by the public URL.
In short the files in the build folder should be copied to the public folder
AWS Amplify
Provides a CI/CD process where you don't have to set all this up by yourself, as long as you have a well-configured package.json file.
There are so many methods to deploy your react app to a production server but using AWS Amplify this link might help you out: https://youtu.be/kKwyKQ8Jxd8
More on create-react-app deployment: https://create-react-app.dev/docs/deployment/
I have two webapps - "manager" and "viewer" - coded in separate VSCode projects. These are deployed to a common Firebase project where they share a common database. The "manager" webapp is used to maintain the database and the "viewer" provides public read-only access.
To create the "page" structure I have added a robocopy to React's build script for each VSCode project to produce a structured "mybuild" folder with the page subfolder within it. Firebase.json's "public": setting is then used to deploy from "mybuild".
Individually the two pages work fine, but each deployment overrides the functionality of the other. So, following the deployment of "manager", webapp/viewer returns a 404 (not found) error and vice versa.
To cut a long story short, the only way I've found around this is to manually copy the results of a deployment for one project into the "mybuild" folder of the other and then deploy from this. But this is no way to proceed.
I think I've taken a wrong turn somewhere here. Can anyone suggest the correct "firebase solution" for this requirement? In the longer term I'd like the viewer webapp to be available at the root of some user-friendly "appurl" while the manager is accessed via "appurl/manager", but other arrangements would be acceptable. The main issue right now is finding a simple way of maintaining the arrangement.
I needed to fix this fast, so here's my own answer to my question.
It seems that when you deploy a project, firebase replaces the current public folder for your URL with the content of whatever folder is specified in your firebase.json. So I decided that I had to accept that whenever either of my projects was deployed it must deploy from a "composite" folder that contains the build files for the other project as well as its own build.
That being the case, it seemed I was on the right lines with my "manual copy" approach and that what I now needed to do was simply to automate the arrangement.
Each of my projects now contains a script file with the following pattern:
npm run build
ROBOCOPY build ./composite/x /E
ROBOCOPY ../y/build ./composite/y /E
firebase deploy --only hosting
In each script, x is the owner project and y is the other. Additionally, firebase.json in each project is edited to deploy from composite.
When I run the script for a project it first builds a composite folder combining the latest build state for both that project and its partner, and then deploys the pair.
The final twist is to tell the react build process that the result is to be deployed from a relative folder and so that the build therefore also needs to use relative references. I do this by adding a
"homepage": "https://mywebapp/x",
to the package.json for each project. Here, x is the name of the project.
I'm not able to convince myself that there's not something wrong with my whole approach to this issue, but at least I have a fix for the immediate problem.
i created a create-react-app and want to use filetypes like webp or mp3.
When i run my application on localhost via npm run start everything works fine, but after my deployment on my server (which uses npm run build and delivers the build folder) it doesn't load filetypes like mp3 or webp anymore. Why is this happening? i think its any simple configuration in react or anything like that, but i cant solve this problem by my own. Thanks for your help.
The issue may be with typescript (if that is what you're using). Typescript will convert .ts and .tsx files to .js, but not move most other files over to build. If they are in a separate assets directory, you have to ensure that gets deployed too. If this is the issue, you have a few choices.
You can manually move the files over to build as a 'post' deploy step (using say, a shell script).
You can use a bundler like webpack to help you maintain the references to those other assets and bundle them correctly.
I finally found the problem that caused this behaviour. Amazon AWS Amplify creates a rewrite rule for single page applications (SPA). You can find this setting under Rewrites and redirects in your Amplify application settings. There you will find a rewrite rule with following source address:
</^[^.]+$|\.(?!(css|gif|ico|jpg|js|png|txt|svg|woff|ttf|map|json)$)([^.]+$)/>
...change it to...
</^[^.]+$|\.(?!(css|gif|ico|jpg|js|png|txt|svg|woff|ttf|map|json|mp3)$)([^.]+$)/>
... for example, to allow mp3 files. This is also important to allow webp-Images or woff2-Fonts.
Is it possible to optionally override a static files directory in the Google App Engine app.yaml file if another directory exists? I have a source directory (unminified) and a build directory (minified and concatenated). I want Google App Engine to automatically use the build directory instead of the src directory, if it exists. That way I can dev using the src directory, then create a build and deploy it. Then, if I delete the build directory, GAE goes back to serving my static files from the src directory.
The reason I need to do this is because I am building an application with Backbone.js & Require.js as modules. I need to be able to optimize my code and deploy without changing my app.yaml file every time.
I'm pretty happy with my current system where my framework uses different paths in the templates to the source javascript files. Then at startup, through a combination of checking os.environ and get_application_id() I automatically detect whether I'm running locally on dev_appserver, or under my test appid or production appid on GAE.
And on to the next step, you most likely want to cache your minified JS aggressively, in which case you'd be unable to force clients to update a new version. The typical workaround is to append a hash or date string to the minified js filename whenever it's updated. This is something you'll also need to do in your framework/templating layer instead of app.yaml.
I would do this at the template layer - when you go to render the template that includes links to your assets, check to see if the minified version exists. If it does, link to that - otherwise, link to the unminified version.
This also helps if you accidentally deploy without creating a build - you'll just be serving unoptimized assets.
I'm trying to use a library project with my Google AppEngine project as a project on build path.
Is there a way to get that included in the AppEngine project without having to copy the entire source or make a jar ?
I only use a small portion of the library so it seems like overkill to copy the whole jar and a lot of work to find the dependencies within the source to get only the parts I'm using.
I added an ant project builder that copies the class files from the dependent project.
The ant file resides in the gae project and copies the class files from the referenced project.
Here's the ant:
<?xml version="1.0" encoding="UTF-8"?>
<project name="CommonsCopy" default="copy" basedir=".">
<target name="copy">
<copy todir="war/WEB-INF/classes">
<fileset dir="../Commons/bin" includes="**/*.class">
</fileset>
</copy>
</target>
</project>
I named it 'CommonsCopyBuilder.xml' as it copies code from a commons project.
This will copy the files to the appropriate location just before running the project.
You are not going to like this answer, but No.
In order to include the library as a dependency in the project once deployed the required library must be found in your WEB-INF/lib directory as a jar. This is going to require you to you to create a jar based on the library you want to use. The other option is to do just as you said pull the dependent source into your project and use it from there.
During development you can make the library project a dependency of your app engine project by doing the following:
Under Project->Properties->Java Build Path->Projects Tab
select the "Add.." button to add a subproject to your build.
Note: This will not address the the running in a production environment.
I managed to do this by creating the libary project is a "Web Fragment Project" and adding it to the "Deployment Assembly" in the project properties of the app engine project.
However, the app engine project does not seem to work with non-trivial dependency structures. In my case, I had two app engine modules within an EAR, both depending on the libary project. However, the Google plugin only bundled the library project with one of the modules, never with both at the same time despite identical configurations. Obviously a bug.
My work-around was to add a linked source folder to the app engine projects, pointing to the source folder of the library project. Ugly, but it does the job.