What is the point of implementing migrations if the down script is missing while checking out to previous state of repository?
Example:
I have implemented migrations succesfully. Repo version is 0.0.1.
// repo version 0.0.1
migrations/
└── 001.js
I have implemented a new feature that requires migrations in version 0.0.2:
// repo version 0.0.2
migrations/
├── 001.js
└── 002.js (new feature)
I checkout repository to previous 0.0.1 version, due to a bug. But I cant run the down script of migration 002.js because at this point it didn exist yet:
// repo version 0.0.1
migrations/
└── 001.js
So do I have to run the migration to the correct version manually before checking out to the previous state (npm run migrate:001)? This is a very easy and doable example. But it might get confusing over multiple branches.
Related
I am trying to share enums and dtos between my backend and frontend (NestJS at the back, React at the front. Both use typescript).
To do so, I tried to add my "shared" files to the client's tsconfig.json > compilerOptions > paths like so:
"paths": {
"#project/shared/*": [
"../server/src/common/*"
]
}
Unfortunately, this doesn't seem to work. When I run the react client, it throws "webpack Module not found."
This is my file structure:
App.tsx consumes the TestEnum from the server's common directory.
What you are trying to achieve is essentially a monorepo. The best way to solve the dependency issue is of monorepo is to refer each package as a separate package; it's clear in the comment that you do not want to do so but it's de facto standard already.
It's because importing what's outside the workspace(where the package.json lies) is really really bad idea. I've tried this but not recommend it; the package managers, bundlers and IDEs(let's say "the tools") will work strangely due to conflict issues:
the file's root has it's own config such as tsconfig.json, package.json which can conflict with project's setting. it is not clear which one they should follow.
the tools usually have their boundary due to security issues; if the tool can leave the project root, it can do something sneaky, such as compromising internal system files.
when deploying the apps in the clouds, they(client/server) have to stick together all the time. it becomes devOps issue.
Solution
the best way to deal with this, is to use workspace feature of package managers. here the term workspace means separate packages inside monorepo. npm's explanation of workspace clearly fits to your settings.
Workspaces is a generic term that refers to the set of features in the npm cli that provides support to managing multiple packages from your local file system from within a singular top-level, root package.
npm and yarn both support workspace features.
npm workspace
yarn workspace
from now on, the project structure should be perceived as this.
project root --> place workspace setting here
|-package root(client) --> workspace
|-package root(server) -> workspace too
most workspace features support 'hoist' feature. without publishing the actual packages, your packages in different folders can be imported and referred as if they are published, via softlink(symlink, something like shortcut in windows). it's same as typescript config's path alias but it actually creates a real file so that the tool won't have any side effects.
to do so, setup workspace config at your project root so that the tool can perceive this packages as workspaces.
# create shared package directory first
$(projectRoot) mkdir shared
// projectRoot/package.json
{
"name": "project",
"workspaces": [
"client",
"server",
"shared"
]
}
// projectRoot/client/package.json
{
"name": "#project/client",
"private": true, // set this to private so that this won't get published by mistake
"packages": {
// your default packages
"#project/shared": "1.0.0" // <-- version has to match
}
// ...
}
// projectRoot/server/package.json
{
"name": "#project/server",
"private": true
// ...
}
// projectRoot/shared/package.json
{
"name": "#project/shared",
"private": false, // make it public so that you can publish it later
"version": "1.0.0"
// ...
}
# remove all `node_modules` folder.
$(projectRoot) rm -rf client/node_modules
$(projectRoot) rm -rf server/node_modules
# reinstall node_modules so that the projects can be symlinked as packages.
$(projectRoot) npm i
then it'll work as workspace from now on and the packages will be linked. it's that simple!
Caveat
as you can see, cleaning up and bootstrapping the package is bit of a work. that's where monorepo tool(i.g. lerna) comes handy; try expand your monorepo knowledge from here.
if it's deployed to AWS, it'll probably work. however, it's best to publish the #project/shared package to npm in the end. many cloud instances don't support symlinks(vercel, netlify...) still, symlink is useful in local/dev environment.
Edit
here's working demo:https://github.com/rabelais88/monorepo-workspace-npm
check readme before proceed. after typing npm install, symlinked hollow modules are seen. as far as they are there, scripts will run fine
edit 2: after setting up the workspace, some dependencies or all the dependencies will be placed at project root instead of each workspace. it is intentional by design, to avoid having duplicated packages in each workspace. this might surprise some but actually very useful in maintaining bigger systems.
I want to share what I had to do in my projects and file structure to share models, types, enums, entities, etc., between the server (NestJS) and the client (React).
First of all, I started setting up npm workspaces to achieve a monorepo, as #sungryeol described in his detailed answer (thanks for it).
Unfortunately, it was only half of the way to making it work. So here is a solution I came up with (remarks are welcome) in addition to workspaces:
Client (react)
create react app is designed to compile code only from the src directory. Because I am using typescript in my shared code, I need to compile it to make it work. To make CRA compile my shared code, I used react-app-rewired to modify the webpack config with the following config-overrides.js file:
const path = require('path');
module.exports = function override(config, env) {
const babelLoaderModule = require.resolve('babel-loader');
const rule = config.module.rules[1].oneOf.find(rule => rule.loader === babelLoaderModule);
rule.include = [
rule.include,
path.resolve(__dirname, '../shared')
];
return config;
}
React finally started accepting my shared code without any issues.
Server (NestJS)
Since NestJS uses tsc as the compiler, I had to specify the directory of the shared code in the server's tsconfig.json as a path.
"paths": {
"#managed-businesses/shared/*": [
"../shared/*"
]
}
This made my shared code compile when I started the nest app, but it couldn't find the entry point, as it compiled the code in the following structure
dist
├── server
│ ├── src
│ ├── main.d.ts
│ └── main.js
├── shared
│ ├── dtos
│ └── enums
└── tsconfig.build.tsBuildInfo
while Nest lookup for the src/main.js file to be at the root of the dist directory.
To fix that, I had to tweak the nest-cli.json file to lookup the file in the dist/server/src directory. Here is my new config for nest-cli.
{
"collection": "#nestjs/schematics",
"sourceRoot": "server/src"
}
After all, the server finally works with the shared code, and I don't need to duplicate it between the projects. I am probably missing something or did something incorrectly, and I'll be happy to know if there is a better way, but at the bottom line, it works.
How to set the version of an NX App/Library build.
If i ran nx build or nx affected or nx build:MYAPP or Lib (if publishable) the version end allways as 0.0.1 and is not in sync with the version if the main package.json
which is even not clever if publishing apps + libs and some are not affected.
I tried nx Semver Plugin, which tags the build with a correct tag but the build (prod or not) ends as 0.0.1. ??
I found the azure/gitlab CI example there is nothing related to a package/version
But for CI we need allways a new package version!
As the docu lacks ...docu ...
HOW?
isn't possible.
Possible solutions are
create a package.json per App/Lib
Maybe not the best idea as it turns the idea of NX Monorepo upside down
Write a postTarget "Hook" to update the package.json after a successful version creation via a npm node script using git describe --tags to get latest version. and npm version ${version} to update package.json.
create-react-app version
react: v17.0.1
react-scripts: v4.0.1
storybook version
#storybook/react: v6.1.6
#storybook/addon-docs: v6.1.6
#storybook/core: v6.1.6
and i could run yarn start to run react app and could run start-storybook -p 9009 -s public to start storybook.
when it comes to build react app , it comes issue. look below.
when run "yarn run build", it actually run react-app-rewired start.
it shows below
There might be a problem with the project dependency tree.
It is likely not a bug in Create React App, but something you need to fix locally.
The react-scripts package provided by Create React App requires a dependency:
"babel-loader": "8.1.0"
Don't try to install it manually: your package manager does it automatically.
However, a different version of babel-loader was detected higher up in the tree:
/Users/yejinlei/Documents/playground/personal/react-temp/node_modules/babel-loader (version: 8.2.1)
Manually installing incompatible versions is known to cause hard-to-debug issues.
If you would prefer to ignore this check, add SKIP_PREFLIGHT_CHECK=true to an .env file in your project.
That will permanently disable this message but you might encounter other issues.
To fix the dependency tree, try following the steps below in the exact order:
1. Delete package-lock.json (not package.json!) and/or yarn.lock in your project folder.
2. Delete node_modules in your project folder.
3. Remove "babel-loader" from dependencies and/or devDependencies in the package.json file in your project folder.
4. Run npm install or yarn, depending on the package manager you use.
In most cases, this should be enough to fix the problem.
If this has not helped, there are a few other things you can try:
5. If you used npm, install yarn (http://yarnpkg.com/) and repeat the above steps with it instead.
This may help because npm has known issues with package hoisting which may get resolved in future versions.
6. Check if /Users/yejinlei/Documents/playground/personal/react-temp/node_modules/babel-loader is outside your project directory.
For example, you might have accidentally installed something in your home folder.
7. Try running npm ls babel-loader in your project folder.
This will tell you which other package (apart from the expected react-scripts) installed babel-loader.
If nothing else helps, add SKIP_PREFLIGHT_CHECK=true to an .env file in your project.
That would permanently disable this preflight check in case you want to proceed anyway.
P.S. We know this message is long but please read the steps above :-) We hope you find them helpful!
I run npm ls babel-loader and see below show
react-temp#0.1.0 /Users/yejinlei/Documents/playground/personal/react-temp
├─┬ #storybook/addon-docs#6.1.6
│ └─┬ #storybook/core#6.1.6
│ └── babel-loader#8.2.1 deduped
└── babel-loader#8.2.1
and i follow the instruction and run yarn add babel-loader#8.1.0. when it done i get below :
react-temp#0.1.0 /Users/yejinlei/Documents/playground/personal/react-temp
├─┬ #storybook/addon-docs#6.1.6
│ └─┬ #storybook/core#6.1.6
│ └── babel-loader#8.2.1
└── babel-loader#8.1.0
then run yarn run build, which actually run react-app-rewired build and get below error
playground/personal/react-temp/node_modules/react-scripts/scripts/build.js:19
throw err;
^
TypeError: aGeneratedCode.split is not a function
at Function.SourceNode_fromStringWithSourceMap [as fromStringWithSourceMap] (playground/personal/react-temp/node_modules/source-map/lib/source-node.js:64:41)
can not build
so then i target to the code node_modules/source-map/lib/source-node.js
and see the error code:
/node_modules/source-map/lib/source-node.js
64 var remainingLines = aGeneratedCode.split(REGEX_NEWLINE);
I run and analysis the code. the the aGeneratedCode will be "obejct", so i change the code as below.
var remainingLines = typeof(aGeneratedCode) === "string" ? aGeneratedCode.split(REGEX_NEWLINE) : [];
then i could run build / start of the cra app and storybook now.
but why and how to permantly fix it ?
I was able to work it around with "resolutions" property
(https://classic.yarnpkg.com/en/docs/package-json/#toc-resolutions)
"resolutions": {
"babel-loader": "8.1.0"
},
I am trying to introduce Go vendoring (storing dependencies in a folder called vendor) to an existing App Engine project. I have stored all dependencies in the vendor folder (using Godep as a helper) and it looks right, but running the application locally I get the following error:
go-app-builder: Failed parsing input: package "golang.org/x/net/context" is imported from multiple locations: "/Users/erik/go/src/github.com/xyz/abc/vendor/golang.org/x/net/context" and "/Users/erik/go/src/golang.org/x/net/context"
I believe the two locations should resolve to the same location, as Go applications should look in the vendor folder first. Is there a way to make Appengine understand that both dependencies are the same?
Your project directory (where app.yaml is) is probably in the GOPATH/src.
It shouldn't be.
The go-app-builder will take everything in the app.yaml folder (and below) and additionally merge your GOPATH into it, meaning now you have it twice.
The solution is to move app.yaml out of the GOPATH/src folder.
Additionally you'll find that goapp test works differently from goapp serve and goapp deploy when it comes to resolving dependencies.
So this is the solution I have been using (haven't used golang app engine in a while already) and it's the only setup I've found to work properly for all the goapp commands and for govendor to work properly (not sure about godep)
/GOPATH
├──/appengine
| ├── app.yaml
| └── aeloader.go
└──/src
└── /MYPROJECT
├── main.go
├── /handler
| └── handler.go
└── /vendor
details:
file: GOPATH/appengine/aeloader.go (NOTE the init function is necessary, probably a bug though)
package mypackage
import (
_ "MYPROJECT"
)
func init() {
}
now run goapp serve and goapp deploy from ../GOPATH/appengine/ and goapp test ./... from ../GOPATH/src/MYPROJECT
P.S. I find the global GOPATH thing silly and simply set my GOPATH to current project folder (in the example above /GOPATH) and check the whole thing into version control.
I use a Makefile to move the vendor directory to a temporary GOPATH:
TMPGOPATH := $(shell mktemp -d)
deploy:
mv vendor $(TMPGOPATH)/src
GOPATH=$(TMPGOPATH) gcloud app deploy
mv $(TMPGOPATH)/src vendor
I store this Makefile at the root of my service near the vendor directory and simply use make deploy to deploy manually or from the CI.
It works with Glide, Godeps or any tool that respects the Go vendor spec.
Please note, that you really need to move the vendor directory out of the build directory, otherwise the GoAppEngine compiler will try to build the vendor dependencies, potentially causing compile errors.
I just ran into this issue myself actually. The problem occurs when you're using the App Engine tools to build any package which imports something that is using vendoring, but the package you're trying to run doesn't have the import within it's vendor directory.
So, for example, if I'm trying to run package foo, which imports package bar, and both of which use the github.com/gorilla/mux library, if the bar repository has a vendor/ directory that contains gorilla/mux, but the foo package doesn't have gorilla mux in it's vendor/ directory, this error will occur.
The reason this happens is that the bar package will prioritize it's own vendor package over the one in the GOPATH, which is what foo will be using, causing a difference in the actual location of the imported paths.
The solution I found to this issue is to make sure that the foo directory is in the GOPATH and has the vendor directory properly installed. It's important to note that the vendor/ convention only works from within the GOPATH.
I managed to resolve this error using govendor instead of Godeps. The root cause appears to have been that vendored references with their own vendored references was not resolved correctly by Godeps.
The answer provided by Su-Au Hwang is also correct - you do have to keep app.yaml separate from your source.
Also got the same problem.
In the docs Google suggests the following:
For best results, we recommend the following:
Create a separate directory in your app's directory for each service.
Each service's directory should contain the service's app.yaml file and one or more .go files.
Do not include any subdirectories in a service's directory.
Your GOPATH should specify a directory that is outside your app's directory and contain all the dependencies that your app imports.
But this messes up my project structure, which looks like this:
GOPATH/
└── src
└── github.com
└── username
└── myproject
├── app.yaml
├── cmd
│ └── myproject
│ └── main.go
├── handlers
│ └── api.go
├── mw
│ ├── auth.go
│ └── logger.go
└── vendor
Where the myproject directory is a git project and the vendor folder contains all dependencies.
Running gcloud deploy from the myproject directory where app.yaml file lives doesn't work because first, main.go file is not in the same directory and second (from the same doc):
you must be careful not to place your source code at or below your app's directory where the app.yaml file is located
What I ended up doing is building my own custom runtime instead, which turned out to be a very clean solution.
Simply generate the Dockerfile with the following command:
gcloud beta app gen-config --custom
Modify it, then specify runtime: custom in your app.yaml and deploy normally.
The trick here is of course that you're in control what gets copied where.
Here is my Dockerfile:
# Dockerfile extending the generic Go image with application files for a
# single application.
FROM gcr.io/google-appengine/golang
ENV GOPATH /go
# The files which are copied are specified in the .dockerignore file
COPY . /go/src/github.com/username/myproject/
WORKDIR /go/src/github.com/username/myproject/
RUN go build -o dist/bin/myproject ./cmd/myproject
# All configuration parameters are passed through environment variables and specified in app.yaml
CMD ["/go/src/github.com/username/myproject/dist/bin/myproject"]
Don't forget that App Engine expects your application listening on port 8080. Check out Building Custom Runtimes doc for more details.
I'm starting to experiment with Polymer 1.0 and am having issues with the vulcanize process. I have included a separate html file for all of my component imports that is loaded via an import in index.html. This site is currently a single page with a paper-drawer-panel a couple of paper-header-panels (drawer and main) with paper-toolbars inside them. This all works perfectly before I vulcanize the site. Once I vulcanize the site the paper-drawer-panel renders the "main" section above the "drawer" section no matter the screen size.
Before Vulcanize (drawer opened)
After Vulcanize
I've done some investigation and this page states that vulcanize versions 1.0+ are for Polymer version 0.8+. The current version of vulcanize is 1.8.1 so I'm assuming that works for Polymer 1.0. Now, when I install grunt-vulcanize that includes vulcanize 0.7.9 which according to the documentation is not compatible with Polymer 1.0. Here is the output from npm list:
npm list -g vulcanize
/usr/local/lib
├─┬ grunt-vulcanize#0.6.4
│ └── vulcanize#0.7.9
└── vulcanize#1.8.1
So how do I get grunt-vulcanize to use vulcanize 1.8.1?
grunt-vulcanize is not compatible with vulcanize 1.8.1, because last update on github was on Feb 9 with version 0.6.4. It's out of date plugin for Polymer 0.8+. I've met same issue in my grunt project with Polymer 1.0 today and I didn't find how to vulcanize polymer elements in grunt project
grunt-vulcanize on github https://github.com/Polymer/grunt-vulcanize
after little search I found that issue on that was opened 5 days ago
https://github.com/Polymer/grunt-vulcanize/issues/21
one github user leave comment that he made temporary adaptation for grunt-vulcanize which works with vulcanize 1.8.1 and Polymer 1.0. I hope it will help
https://github.com/kgadzinowski/grunt-vulcanize/