I'm using Turborepo for my monorepo project, i have 2 react apps. How can i configure Turborepo and CircleCI (repos are on Github) so if i make changes to one project that pipeline is not going to run for second project?
I know turbo is using hash algo to check if there is any changes to a project and then rebuild it.
I have tried looking here https://turborepo.org/docs/ci/circleci but does not explain the behavior of this.
Steps would be:
Make code change to Project 1
Commit changes of monorepo to Github
Github detects a commit and triggers CircleCI to run CI/CD
So this part is what I'm not sure, if it triggers CI/CD it will trigger for the both projects right? And if so how can i prevent only for the one i have made changes?
I've been working on such a solution for days now. There are two core-concepts in turborepo to achieve this:
Filtering workspaces
Caching Buildoutputs and store the cache in the cloud (not what you're looking for)
So, you can filter your monorepo for a specific project, e.g:
pnpm turbo run build --filter='my-project...[HEAD^1]' --dry=json
-> This will look if the task build is needed to run for the project "my-project", comparing the current source with "HEAD^1". The option dry=json helps to just look if there would be a need to run "build" or not for "my-project".
You could filter a whole lot more, check the docs.
Now, what i have built on top of this:
A new job on the github workflow looks with the help of this filter command if a deployment of my graphql-server is needed and he will set the output of this decision as an artifact, to provide this information for later jobs (https://github.com/actions/upload-artifact)
My actual docker-build and deploy-to-fly-io jobs that run afterwards, will download this artifact and set a CONTINUE environment variable, depending if it should build + deploy or not.
Every job coming after that is having an if: ${{ env.CONTINUE == 'true' }} to skip them if no build/deploy is needed.
It could be much simpler if you can run your build/deploy job directly with the turbo cli, because then you can just combine your filter and the execution of the build - but that was not possible in my case.
If you need to "skip" jobs that are coming later in your workflow, it's harder thant it should, as github is not supporting "abortion" of jobs.
For all the other commands like lint, typecheck and test -> just add an appropriate filter option to them and you will achieve that they only run on your "affected" workspaces/projects in your PR.
Ressources:
https://dev.to/ngconf/deploying-nx-affected-apps-from-github-actions-54n4
How can I get the previous commit before a push or merge in GitHub Action workflow?
https://github.com/orgs/community/discussions/26313
Related
I created a Gitlab Ci-CD to remove react and react-dom from devDependencies in package.json when i push my code to master.
but as you can see it says that pipline is passed and job is succeeded but when i look at my master branch react and react-dom are not removed from devDependencies and they are still there.
what should i do with that?
A GitLab CI job doesn't change any of your repository files. It pulls a copy of the repository so that it can run a script, test, etc. against it. If the specified job runs successfully without throwing any errors (exit code), then the job succeeds/passes. You can find a more detailed explanation in the GitLab CI/CD documentation.
If you want to make changes to the repository, you need to have the script run to make the changes, then commit the changes, and push those changes to the repository using the standard git commands.
Just remember that you'll have to authenticate using HTTPS or using SSH.
You'll also likely find this StackOverflow answer on pushing to a GitLab repository through a CI job helpful as well.
We're using Jest to perform our React.js unit tests (on the frontend) of our Node.js app which runs in a docker container.
We have set up a Pipeline in Jenkins but I'm unsure of the best way (or best practice) to include the tests as part of the pipeline.
The steps we have are the following:
Check out the code from source control
NPM install and npm run build (front-end)
Docker build + publish
Deploy app
Bump version
Git push
Docker cleanup
I have 3 main queries:
A. I'm assuming it's best to include npm run test between Step 1 and Step 2 and if all tests pass successfully to move further?
B. But how are the snapshots handled? For example, if there's some change which occurred that generates a difference in a snapshot this will not be "checked" back into the source control.
C. I read that people use Cobertura, jest-junit, etc to have unit tests and coverage within Jenkins - what is the best?
Thanks in advance.
Good questions!
A. You can run tests after npm install. And if all tests pass you move further. Another common thing to do is to run linting or code style check.
B. A bad snapshot will fail tests. Which is why it's important to update snapshots before committing. If your jenkins is hooked up to a code review system, you may disable merges that fail builds, to make sure bad snapshots don't get on your master branch.
C. I have seen people use jest-junit, but that's only because there was a requirement to have the coverage report combined with a junit coverage report. If you don't have any particular requirements around the structure of the report, then the default report jest produces should be fine, and you don't need anything extra.
I am building a Google App Engine (GAE) project under Android Studio. I want to be able to have different "buildTypes" similar to how one can do for an android app project, however this is not supported by the appEngine Gradle plugin (see https://github.com/GoogleCloudPlatform/gradle-appengine-plugin/issues/177). As a workaround, I want to run some kind of script (e.g.: ant, bash, or similar), to do some arbitrary command prior to the build, so that I can copy or rename the right source files into place for the build. Unfortunately, being somewhat unfamiliar with Gradle, I am at a loss of how to do that. I've been reading the Gradle documentation but have been unable to discover how to add a hook before the :backend:compileJava task is executed. Can anyone explain how I might be able to do this?
If you haven't solved this yet
You can have any task depend on any other :
task myPrecompileTask {
doLast {
// execute some code here
}
}
compileJava.dependsOn myPrecompileTask
That will force any call to compileJava to call myPrecomileTask first.
I am building a sample react native application. Currently i am running it using the node server.Node server is serving the js file.
You can see this in following screenshot:
I want to shift to the option2, for this, if there is any change in the js file, i need to run the curl command manually.
Is there any alternative for this?
AFAIK there's nothing in place and this is work in progress. See:
https://github.com/facebook/react-native/issues/12
We plan on putting in some sort of build step that "compiles" the JS
source directly into a resource file in the app bundle. Obviously in
production you wouldn't have a server running nearby.
There's another bit of discussion here.
At the moment I think you're stuck with the curl option.
All this does is packing all your JavaScript together and writing it into a single file.
Option 1 has a small http server running, providing the latest packed file when you request it.
Option 2 takes the file from the local disk.
You can setup a tool that looks watches your project files and repacks everything if you make changes.
You can do this by yourself, using the packaging tool shipped with react-native (react-native bundle [--minify]) and re-run it everytime things changes using gulp (and gulp-watch).
Also you can use webpack as your packaging tool and use the --watch option. (see example)
I have been trying out Codeship and Heroku for continuous deployment of an AngularJS application I writing at the moment. The app was created using Yeoman and uses bower and grunt. Initially I thought this seemed like a really good setup as Codeship was free to use and I was quickly able to configure this to build my AngularJS project and it offered the ability to add a deployment step after the build. There were even many PaaS providers to choose from (Heroku, S3, Google App Engine etc). However I seem to have become a bit stuck with getting the app to run on Heroku.
The problem started from the fact that all the documentation suggested that I remove the /dist path from my .gitignore so that this directory is published to Heroku post build. This was mainly from documentation that talked about publishing to Heroku from a local machine, but I figure this is all Codeship is doing under the hood anyway. I didn't want to do this as I don't believe I should be checking build output into source control. The /dist folder was added to .gitignore for a good reason. Furthermore, this kind of defeats the point of having a CI server somewhat, as I might as well just push the latest build from my machine.
After some more digging around I found out that I could add a postinstall step to my packages.json file such as bower install && grunt build which would re-run the build on Heroku and hence repopulate all the bower dependencies (something else they wanted me to check in to source control!) and the dist directory.
After giving this a try it became apparent that I would need to add bower and grunt as dependencies in packages.json, which meant moving them from devDependencies which is where they should belong!
So I now seem to be stuck. All I want to do is publish my build artefacts (/dist) the dependencies (/bower_components) and the server.js file that will run the site. Does anyone know how to achieve this with Heroku and Codeship? Alternatively has anyone had any success with this using different tools. I am looking for something that is free and I am willing to accept that it will not be production stable (won't scale to multiple servers etc), but this is fine for now as all I want to do is continuously deploy the app for internal testing and to be able to share the output with non-technical members of my team so we can discuss features we'd like to prioritise etc.
Any advice would be greatly appreciated.
Thanks
Ahoy, Marko from the Codeship crew here. Did you already send us an in app message about this? I'm sure we can get your application building on Codeship and deploying to Heroku successfully.
As a very short answer, the easiest way to get this running would be to add both bower and grunt to your dependencies in the package.json. Another possibility would be to look for a custom buildpack with both tools already installed.
And finally you could also run the tools on Codeship, add the newly installed files to the repository, commit the changes and push this new commit to Heroku. If you want to use this, you'd very probably need to force push the changes though.
Feel free to reach out to me via the in app messenger (lower right corner of the site) and I'd be happy to help you get this working!
I found two ways to get this to work.
Heroku Node Custom Buildpack
Use the mbuchetics Heroku build pack. This works by basically re-building the app once it has been pushed to Heroku.
There were a few tricks I had to employ still to make this work. In Gruntfile.jstwo new tasks needed to be configured called heroku:production and heroku:development. This is what the buildpack executes to build the app. I initially just aliased the main build task, but found that the either the buildpack or Heroku had a problem with running jshint so in the end I copied the build task and took out the parts that I didn't need.
Also in packages.json I had to add this:
"scripts": {
"postinstall": "bower cache clean && bower install"
}
This made sure the bower_components were available in Heroku.
Pros
This allowed me to keep the .gitignore file in tact so that the 'binaries' in the dist directory and the dependencies in the bower_components directory were not committed into source control.
Cons
This is basically re-building the app once it is on Heroku and I generally prefer to use the same 'binaries' throughout the entire build and deployment pipeline. That way I know that the same code that was built, is the same code that was tested and is the same code that was deployed.
It also slows down the deployment as you have to wait for the app to build twice.
CodeShip Custom Script Deployment
Not being satisfied with the fact I was building my app twice, I tried using a Custom Script pipeline in CodeShip instead of the pre-existing Heroku one. The script basically modified the .gitignore file to allow the dist folder to be committed and then pushed to the Heroku remote (which leaves the code on the origin remote unaffected by the change).
I ended up with the following bash script:
#!/bin/bash
gitRemoteName="heroku_$APP_NAME"
gitRemoteUrl="git#heroku.com:$APP_NAME.git"
# Configure git remote
git config --global user.email "you-email#example.com"
git config --global user.name "Build"
git remote add $gitRemoteName $gitRemoteUrl
# Allow dist to be pushed to heroku remote repo
echo '!dist' >> .gitignore
# Also make sure any other exclusions dont apply to that directory
echo '!dist/*' >> .gitignore
# Commit build output
git add -A .
herokuCommitMessage="Build $CI_BUILD_NUMBER for branch $CI_BRANCH. Commited by $CI_COMMITTER_NAME. Commit hash $CI_COMMIT_ID"
echo $herokuCommitMessage
git commit -m "$herokuCommitMessage"
# Must merge the last build in Heroku remote, but always chose new files in merge
git fetch $gitRemoteName
git merge "$gitRemoteName/master" -X ours -m "Merge last build and overwrite with new build"
# Branch is in detached mode so must reference the commit hash to push
git push $gitRemoteName $(git rev-parse HEAD):refs/heads/master
Pros
This only require a single build of the app and deploys the same binaries that were tested during the test phase.
Cons
I've used this script quite a few times now and it seems relatively stable. However one issue I know of is that when a new pipeline is created there will be no code on the master branch so this script fails when it tries to do the merge from the heroku remote. At the moment I get around this by doing an initial push of the master branch to Heroku before kicking off a build, but I imagine there is probably a better Git command I could run along the lines of; 'only merge this branch if it already exists'.