I am trying to download an artifact from a Jenkins project using a DOS batch script. The reason that this is more than trivial is that my artifact is a ZIP file which includes the Jenkins build number in its name, hence I don't know the exact file name.
My current plan of attack is to use wget pointing at: /lastSuccessfulBuild/artifact/
to do some sort of recursive/mirror download.
If I do the following:
wget -r -np -l 1 -A zip --auth-no-challenge --http-user=**** --http-password=**** http://*.*.*.*:8080/job/MyProject/lastSuccessfulBuild/artifact/
(*s are chars I've changed for posting to SO)
I never get a ZIP file. If I omit the -A zip option, I do get the index.html, so I think the authorisation is working, unless it's some sort of session caching issue?
With -A zip I get as part of the response:
Removing ...+8080/job/MyProject/lastSuccessfulBuild/artifact/index.html since it should be rejected.
So I'm not sure if maybe it's removing that file and so not following its links? But doing -A zip,html doesn't work either.
I've tried several wget options, and also curl, but I am getting nowhere.
I don't know if I have the wrong wget options or whether there is something special about Jenkins authentication.
You can add /*zip*/desired_archive_name.zip to any folder of the artifacts location.
If your ZIP file is the only artifact that the job archives, you can use:
http://*.*.*.*:8080/job/MyProject/lastSuccessfulBuild/artifact/*zip*/myfile.zip
where myfile.zip is just a name you assign to the downloadable archive, could be anything.
If you have multiple artifacts archived, you can either still get the ZIP file of all of them, and deal with individual ones on extraction. Or place the artifact that you want into a separate folder, and apply the /*zip*/ to that folder.
Related
I installed MongoDB and tried to run it on terminal. It just shows up 'mongo' is not recognized as an internal or external command, operable program or batch file.
I have set the path to bin folder inside Environment variables too. One thing I noticed is I might have a missing file inside bin folder and that is mongo. Because I have mongod and mongos file inside the bin folder. I tried to uninstall and reinstall the program and it was still not working.
I have no idea it's what that I'm missing. Please help out
Finally I have found the solution,
Mongo shell no longer ships with server binaries. We can download it from MongoDB Shell Download
Then we should extract the contents of the bin from the downloaded zip file to the bin file of the MongoDB folder and run mongosh instead of mongo on the terminal
I have to download all files from a ftp folder using Explicit FTP over SSL/TLS. I need that for a jenkins job, running on a windows machine and didnt find any plugins - so I am trying to use a batch script with curl and the following code lists the contents of the folder.
set "$FILEPATH=C:\temp"
set "$REMOTEPATH=/files/"
curl -u user:pass --ftp-ssl ftp://hostame.com:port%$REMOTEPATH% -o %$FILEPATH%
I figured out that with curl I have to download files one by one, but how can I achieve to go through all the files in a ftp directory and get them one by one?
Is there a better way to achieve that? I read about mget, but it doesnt seem to work with the explicit ftp over ssl.
Thanks
I couldnt bring it to work with batch directly in the script, so I wrote a python script instead and download it from git and execute it as a step in the pypeline. It has some nice libraries, so it works as a charm.
I am trying to download files with particular datestamp as an extension from a folder through FTP server. Since the folder contains all other files, I wanted to download only files with a particular datestamp.
I tried using wget files_datestamp*.extension, which didn't work.
I also tried using wget -i files_datestamp*.extension, which downloads all.
My question is: What option to use with wget to download only particular files that I am interested in?
wget http://collaboration.cmc.ec.gc.ca/cmc/CMOI/NetCDF/NMME/1p0deg/#%23%23/CanCM3_201904_r4i1p1_20190501*.nc4
The link you've shared is over HTTP and not FTP. As a result, it is not possible to glob over the filenames, that is feasible only over FTP.
With HTTP, it is imperative that you have access to a directory listing page which tells you which files are available. Then use -r --accept-regex=<regex here> to download your files
I have jenkin's job, which copy tar file from linux user folder and then copy binary file (compiled) from another job and make new tar file. Then jenkins user can copy that new tar file from jenkin's workspace.
It doesn't build anything or take files from SCM. Then after a while, suddenly tar file has been deleted from workspace, I have to run job again. How I can prevent that?
You really shouldn't rely on your workspace existing after a job has completed, as the workspace can be overwritten by another build starting, or when someone deletes a build, by a slave going offline, etc...
Since you want to save the file for later use, you should use the "Archive the artifacts" option in your job's post-build configuration. If you enter **/*.tar, for example, Jenkins would save all TAR files at the end of the build.
Then you can use Jenkins' permalinks to access the artifacts, e.g.:
http://JENKINS/job/JOB_NAME/lastSuccessfulBuild/artifact/bin/my-app.tar
As the URL suggests, this would give you the file from the last successful build.
As a sidenote, if you then want to copy archived files to another build, the best way to do this is with the Copy Artifact plugin.
That way Jenkins handles the file copying for you, even across multiple Jenkins slaves, and you don't have to do anything nasty like hard-coding paths to other Jenkins workspaces.
I am having a bit of trouble grabbing some files that have a strange file structure. What do I mean exactly? http://downloads.cloudmade.com/americas/northern_america/united_states/district_of_columbia#downloads_breadcrumbs
Look at that example. I want to start at the root of the site and recursively grab all the files that end with *.shapefile.zip. wget appears to treat this as two separate files ending in .shapefile and .zip. Anyone have some wget goodness to help me get started on this one?
You can recursively wget specific file types with:
wget -A 'shapefiles.zip' -r <url>
Although I don't think .shapefiles.zip is an extension of .zip but more that site's naming conventions