a question about building flink-avro.
i downloaded the source, and i am able to build complete source.
but when i move to folder
flink/flink-staging/flink-avro
and run command mvn package as i want to compile and run the tests, it starts downloading latest flink SNAPSHOT jars.
whereas, i have built the same jars from source, everything is there in local repo.
why it downloads the same things which are built locally?
You need to use mvn install on the top level module first. This will copy the built jars into your local maven repository located at ~/.m2/respository/. mvn package only builds the jars and put them in folder target (no copying into local maven repo happens).
After that, you can step into flink-avro and build there (either with mvn package or mvn install -- as long as no other module depends on the built jars, mvn package is fine). If you change something in a module A and another module B depend on A, you need to do mvn install in module A. Otherwise, module B cannot access the newly built jars from A.
Maven always looks for dependent jars in your local m2-repository first. If it does not find the jars there, it starts downloading them from the Internet. If it finds "old" jars in your local repo, it might still use them (depending on your specified maven update interval). On the other hand, even if you built mvn install locally, and a newer version is available online, maven might download the newer version.
Last but not least, sometime maven does not resolve the timestamp dependencies correctly or does not update the local repo even if you do a mvn install. Clearing the local maven repository (just do a rm for the corresponding jars or the whole repo) can resolve this problem.
Related
Error executing command: Error resolving artifact
com.sample:features:xml:features:4.1.4-SNAPSHOT:
\[Could not find artifact com.sample:features:xml:features:4.1.4-SNAPSHOT in default
local (file:/C:{/Users}.m2/repository/),
Could not find artifact com.sample:features:xml:features:4.1.4-SNAPSHOT in apache
(https://repository.apache.org/content/groups/snapshots-group/),
Could not find artifact com.sample:features:xml:features:4.1.4-SNAPSHOT
in ops4j.sonatype.snapshots.deploy
(https://oss.sonatype.org/content/repositories/ops4j-snapshots/)\] :
com.sample:features:xml:features:4.1.4-SNAPSHOT.
I could not resolve this error after uncommenting local repository in org.ops4j.pax.url.mvn and add my local .m2 path
Thanks in advance for your suggestions.
You might have to reset your Apache karaf instance to fresh state.
Stop Karaf
Delete data folder from Karaf installation folder.
Start Karaf
Re-Install the features and bundles you need.
Alternatively you can try removing any features, feature-repositories and bundles that have been installed from there before removing the maven repository.
It's generally bad idea to remove/replace maven repositories from Apache Karaf from where user(s) might have already added new feature repositories, installed features and bundles.
Once tried to replace offline repository I had made with a new one which was missing some older artifacts. This lead to bunch of issues like inability to uninstall some of the older features.
I have a question, I want to use angular 2 in maven project, and as you now the modules should download with ( npm install or ng new .. (cli) ).
The problem is if I generate the war file with all this modules, it will be very large because of the presence of all the nodejs modules.
In one of Github example they install this modules with ( npm install ) and finally goes to the home directory and run spring-boot:run
I want directly deploy my war file, so my question is : i should generate the war file with the all the modules and dependencies of nodejs or there is another solution ?
Three ways:
add all node_modules dependencies in to your version control, so source is always there, or copy necessary js libraries manually in specific source folder, like angular.min.js and so on (if your node.js is not available on your server, by security reason)
create execution goal inside pom.xml, something like
How to deploy a node.js app with maven?
use https://github.com/eirslett/frontend-maven-plugin and check existing examples, I am sure your case is straight forward
At the moment I have Flink 0.8.1 installed on my machine. I installed it via Maven and would like to know how I can update it. Do I have to deinstall everything in order to update to the current version? Or can I do it via Maven?
If you have a Maven project which depends on Apache Flink you can simply update the version of the Flink dependencies in your project's pom.xml file. Just run mvn clean compile again and Maven will fetch all required dependencies.
The old dependencies will remain in your local Maven repository (~/.m2/repository/). You can "uninstall" the old Flink dependencies by deleting the corresponding jar files.
I have been trying out Codeship and Heroku for continuous deployment of an AngularJS application I writing at the moment. The app was created using Yeoman and uses bower and grunt. Initially I thought this seemed like a really good setup as Codeship was free to use and I was quickly able to configure this to build my AngularJS project and it offered the ability to add a deployment step after the build. There were even many PaaS providers to choose from (Heroku, S3, Google App Engine etc). However I seem to have become a bit stuck with getting the app to run on Heroku.
The problem started from the fact that all the documentation suggested that I remove the /dist path from my .gitignore so that this directory is published to Heroku post build. This was mainly from documentation that talked about publishing to Heroku from a local machine, but I figure this is all Codeship is doing under the hood anyway. I didn't want to do this as I don't believe I should be checking build output into source control. The /dist folder was added to .gitignore for a good reason. Furthermore, this kind of defeats the point of having a CI server somewhat, as I might as well just push the latest build from my machine.
After some more digging around I found out that I could add a postinstall step to my packages.json file such as bower install && grunt build which would re-run the build on Heroku and hence repopulate all the bower dependencies (something else they wanted me to check in to source control!) and the dist directory.
After giving this a try it became apparent that I would need to add bower and grunt as dependencies in packages.json, which meant moving them from devDependencies which is where they should belong!
So I now seem to be stuck. All I want to do is publish my build artefacts (/dist) the dependencies (/bower_components) and the server.js file that will run the site. Does anyone know how to achieve this with Heroku and Codeship? Alternatively has anyone had any success with this using different tools. I am looking for something that is free and I am willing to accept that it will not be production stable (won't scale to multiple servers etc), but this is fine for now as all I want to do is continuously deploy the app for internal testing and to be able to share the output with non-technical members of my team so we can discuss features we'd like to prioritise etc.
Any advice would be greatly appreciated.
Thanks
Ahoy, Marko from the Codeship crew here. Did you already send us an in app message about this? I'm sure we can get your application building on Codeship and deploying to Heroku successfully.
As a very short answer, the easiest way to get this running would be to add both bower and grunt to your dependencies in the package.json. Another possibility would be to look for a custom buildpack with both tools already installed.
And finally you could also run the tools on Codeship, add the newly installed files to the repository, commit the changes and push this new commit to Heroku. If you want to use this, you'd very probably need to force push the changes though.
Feel free to reach out to me via the in app messenger (lower right corner of the site) and I'd be happy to help you get this working!
I found two ways to get this to work.
Heroku Node Custom Buildpack
Use the mbuchetics Heroku build pack. This works by basically re-building the app once it has been pushed to Heroku.
There were a few tricks I had to employ still to make this work. In Gruntfile.jstwo new tasks needed to be configured called heroku:production and heroku:development. This is what the buildpack executes to build the app. I initially just aliased the main build task, but found that the either the buildpack or Heroku had a problem with running jshint so in the end I copied the build task and took out the parts that I didn't need.
Also in packages.json I had to add this:
"scripts": {
"postinstall": "bower cache clean && bower install"
}
This made sure the bower_components were available in Heroku.
Pros
This allowed me to keep the .gitignore file in tact so that the 'binaries' in the dist directory and the dependencies in the bower_components directory were not committed into source control.
Cons
This is basically re-building the app once it is on Heroku and I generally prefer to use the same 'binaries' throughout the entire build and deployment pipeline. That way I know that the same code that was built, is the same code that was tested and is the same code that was deployed.
It also slows down the deployment as you have to wait for the app to build twice.
CodeShip Custom Script Deployment
Not being satisfied with the fact I was building my app twice, I tried using a Custom Script pipeline in CodeShip instead of the pre-existing Heroku one. The script basically modified the .gitignore file to allow the dist folder to be committed and then pushed to the Heroku remote (which leaves the code on the origin remote unaffected by the change).
I ended up with the following bash script:
#!/bin/bash
gitRemoteName="heroku_$APP_NAME"
gitRemoteUrl="git#heroku.com:$APP_NAME.git"
# Configure git remote
git config --global user.email "you-email#example.com"
git config --global user.name "Build"
git remote add $gitRemoteName $gitRemoteUrl
# Allow dist to be pushed to heroku remote repo
echo '!dist' >> .gitignore
# Also make sure any other exclusions dont apply to that directory
echo '!dist/*' >> .gitignore
# Commit build output
git add -A .
herokuCommitMessage="Build $CI_BUILD_NUMBER for branch $CI_BRANCH. Commited by $CI_COMMITTER_NAME. Commit hash $CI_COMMIT_ID"
echo $herokuCommitMessage
git commit -m "$herokuCommitMessage"
# Must merge the last build in Heroku remote, but always chose new files in merge
git fetch $gitRemoteName
git merge "$gitRemoteName/master" -X ours -m "Merge last build and overwrite with new build"
# Branch is in detached mode so must reference the commit hash to push
git push $gitRemoteName $(git rev-parse HEAD):refs/heads/master
Pros
This only require a single build of the app and deploys the same binaries that were tested during the test phase.
Cons
I've used this script quite a few times now and it seems relatively stable. However one issue I know of is that when a new pipeline is created there will be no code on the master branch so this script fails when it tries to do the merge from the heroku remote. At the moment I get around this by doing an initial push of the master branch to Heroku before kicking off a build, but I imagine there is probably a better Git command I could run along the lines of; 'only merge this branch if it already exists'.
My application is having 70 jars with ant application. Now we are moving ant to maven, but i am unable not find 10 jars (Local jars) in maven repository.
Shell we import 10 jars manually.(I read somewhere that if you include more jars manullay don't choose maven, is it correct?)
Yes, to use Maven, jars must be fetched from $HOME/.m2 local repository. If those missing 10 jars are individual maven projects with a pom.xml & you had run maven install in each one of their project folder, i see no reason why those jars are not there in your local repo.