I'm using FossilSCM as my only solution for control version and tickets. So far, so good. Its self contained and minimalist approach suit my needs. But I would like to start to make some analysis on the projects history and development and a good soruce for that are the projects timelines. I could go with some html parsing trying to convert the Fossil timeline output to something else, but I would like if there is any option to export that info in other structured format (e.g JSON or similar). Web search has not produce any useful finding on that issue. Any pointers to a solution?
Thanks,
Offray
Have you tried fossil json timeline branch trunk?
fossil help json
Usage: fossil json SUBCOMMAND ?OPTIONS?
In CLI mode, the -R REPO common option is supported. Due to limitations
in the argument dispatching code, any -FLAGS must come after the final
sub- (or subsub-) command.
The commands include:
anonymousPassword
artifact
branch
cap
config
diff
dir
g
login
logout
query
rebuild
report
resultCodes
stat
tag
timeline
user
version (alias: HAI)
whoami
wiki
Run 'fossil json' without any subcommand to see the full list (but be
aware that some listed might not yet be fully implemented).
Compile json when you build from source:
./configure --json
The key for having this working is to enable json support in fossil by compiling it from sources. Current version have it disabled, so looking for any clue on it in command line help got me nothing originally. Thanks to user 2612611 for the initial clue about it. Here is the procedure I followed:
Go to https://www.fossil-scm.org/download.html and download the source tarball package.
Uncompress the previous package.
Go to the folder where you uncompressed the package (lets call it /uncompress-folder
Run ./configure --json
Run make.
Optional: Put your newly created fossil binary in your path or where the last one was installed (something like sudo mv /uncompress-folder/fossil /usr/bin/fossil.
Open the fossil repository that you want to export its history and launch the fossil web interface (fossil ui).
Go to http://localhost:8080/json/timeline/checkin?limit=0 ,where http://localhost:8080 is your local machine interface for fossil ui, and json/timeline/checkin?limit=0 is the json API call saying: json export of timeline (/json/timeline) chekins (/checkin) for all history (?limit=0). If instead of the 0 at the end of the url you put another integer you will have the last n checkins.
From command prompt you should be able to get the same result by running fossil json timeline checkin --limit=0 > timeline.json stored on the file timeline.json, instead of the web browser but in local test it didn't work.
API is still a moving target, but you can find documentation on this excellent project at [1] and a demo interface to test the parameters at [2]
[1] https://docs.google.com/document/d/1fXViveNhDbiXgCuE7QDXQOKeFzf2qNUkBEgiUvoqFN4/view?pli=1#
[2] http://fossil.wanderinghorse.net/repos/fossil-sgb/json/
Related
I have my qooxdoo project built and deployed by a CI server. Upon build, the server generates build info (version, VCS revision, CI build number, timestamp) that I would like to be passed to my qooxdoo app as qx.core.Environment keys.
At the moment, I have CI server generate a build.json file which is packaged together with the application, loaded at startup and converted to environment keys (by application code). This costs us an extra XHR.
On the other hand, I know that environment entries can be supplied during build, via config.json. Of course our build system can preprocess config.json to fill in environment entries, but I'm a bit skeptic of the idea of CI server fiddling with config.json. Is there any better solution? Is it possible to make generator script read environment entries from some auxiliary source?
I would write a #VERSION# tag into my script and at the end of the build process just search and replace this string in the compiled js file.
perl -i -p -e 's/#VERSION#/0.3.0/g' build/script/hello.js
I have the following files which I would like to upload to Artifactory as a 9.8.0 versioned artifact.
NOTE: The first two files DO NOT have an extension (they are executable files i.e. if you open them/cat on it, you'll see junk characters).
Folder/files of a given version 9.8.0 in CVS is like:
com.company.project/gigaproject/v9.8.0/linux/gigainstall
com.company.project/gigaproject/v9.8.0/solaris/gigainstall
com.company.project/gigaproject/v9.8.0/win32/gigainstall.exe
com.company.project/gigaproject/v9.8.0/gigafile.dtd
com.company.project/gigaproject/v9.8.0/gigaanotherfile.dtd
com.company.project/gigaproject/v9.8.0/giga.jar
com.company.project/gigaproject/v9.8.0/giga.war
Uploading the above files which have an extension is very easy... You log in to Artifactory as an administrator/user which has access to deploy artifacts, click on "Deploy" tab, browse for the Artifactory file and once you select the file, click on "Upload" button.
Next you'll see a screen (like shown above). You'll tweak what you want in the fields on this page and once you click on "Deploy Artifact", you are done. All you have to make sure is you select the correct file.extension file while uploading and make sure the file extension is shown in the "Target Path" box correctly (with the version -x.x.x, etc.).
My questions:
Question 1: How do I upload an artifact which doesn't have an extension? It seems like Artifactory by default takes an artifact as a .jar extension. How can I upload the "gigainstall" artifact as shown in the folder/file structure above for both Linux and Solaris? I see I can use the artifact name as gigainstall-linux and gigainstall-solaris and differentiate it, but I am not sure how to tell Artifactory that this artifact doesn't have any extension.
I don't think the development team will start generating this artifact with a proper extension (as this artifact may be hard coded everywhere in other projects where they are currently getting it from CVS/SVN source control somewhere - which is itself a bad practice to store an artifact in a source control version tool).
Question 2: How would I tell a build system (for example, Gradle) to consume a non-extensioned artifact during, let's say, 'compile' task. In build.gradle under section dependencies { .. }, I will add something like as shown below, but I am not sure for non-extensioned files (the first two in the folder/file structure I mentioned above).
dependencies {
//compile 'com.company.project:gigainstall-linux:9.8.0#'
//compile 'com.company.project:gigainstall-linux:9.8.0#??????'
//compile 'com.company.project:gigainstall-linux:9.8.0#""'
//compile 'com.company.project:gigainstall-linux:9.8.0#"none"'
//compile 'com.company.project:gigainstall-linux:9.8.0#"NULL_or_something"'
// The following will easily get giga.jar version giga-9.8.0.jar from Artifactory repository
compile 'com.company.project:giga:9.8.0'
// The following will easily get giga.war
compile 'com.company.project:giga:9.8.0#war'
// Similarly, other extension based artifacts can be fetched from Artifactory
compile 'com.company.project:gigafile:9.8.0#dtd'
compile 'com.company.project:gigaanotherfile:9.8.0#dtd'
}
Answer 1 (will cover 2 as well in a different sense): Using Artifactory "Artifact Bundle" feature section under "Deploy" tab can do the TRICK for AT LEAST uploading the artifacts in a way we want, by creating a zip file first (containing the structure and artifacts in it) --OR you can upload the artifacts using/calling Artifactory REST API way.
High level idea:
Create a zip file called gigaproject.zip OR anyname.zip/.tar/compressed file which Artifactory can read. Inside the zip, create the structure - how these artifacts will be loaded to Artifactory
i.e.
gigaproject.zip will contain the following folders/structure/files.
Case 1:
com/company/project/gigaproject/9.8.0/linux/gigainstall
com/company/project/gigaproject/9.8.0/solaris/gigainstall
com/company/project/gigaproject/9.8.0/win32/gigainstall.exe
com/company/project/gigaproject/9.8.0/gigafile.dtd
com/company/project/gigaproject/9.8.0/gigaanotherfile.dtd
com/company/project/gigaproject/9.8.0/giga.jar
com/company/project/gigaproject/9.8.0/giga.war
NOTE: In case 1 example, I didn't use any -x.x.x in the filename (i.e. I'm using plain and simple giga.jar instead of giga-9.8.0.jar).
The above Upload/Deploy will result the files (as shown in the following snapshot):
So, we have achieved what we wanted. Actually (visibly speaking yes), but not in a way Artifactory usually stores these artifacts (as they should -x.x.x version embedded in the file name and where artifact id should match the artifact filename). Now, if you want to consume the following in a Gradle build file, you CANNOT as first, you haven't uploaded the filename with -x.x.x version name in it, secondly, the artifact id in our case 1 tree was "gigaproject" (after com/company/project folder), so Gradle way of defining what artifact id and what artifact file name you want won't work.
compile 'com.company.project:gigaproject:CANNOTSAY_HOW_TO_GET_GIGA_JARorGIGAINSTALL_with_without_extension'
Conclusion: It's possible to upload any files (with/without extension in Artifactory) in any structure but it depends how your build system will consume it or will be able to consume it or not.
- I deleted the structure I just created with case 1 .zip file from Artifactory repository to try next case#2 and deleted the .zip file I created.
Case 2:
Let's create an individual versioned file name for each artifact and also create structure in the format - how Artifactory actually stores them (an artifact as seen in a repository in a tree view) and create a .zip file containing that structure. Let's use the same "Artifact Bundle" feature to upload this .zip file to upload individual artifacts that we need in Artifactory - where artifact-id (second value which we mention while trying to consume it) would match the artifactfile name in Artifactory.
Folder/file structure for the .zip file:
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.linux
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.solaris
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.exe
com/company/project/gigafile/9.8.0/gigafile-9.8.0.dtd
com/company/project/gigaanotherfile/9.8.0/gigaanotherfile-9.8.0.dtd
com/company/project/giga/9.8.0/giga-9.8.0.jar
com/company/project/giga/9.8.0/giga-9.8.0.war
NOTE: This time, we'll be using the same "Artifact Bundle" feature and for similar files (gigainstall under both Linux/Solaris folders), I took the approach of creating gigainstall folder (containing gigainstall-9.8.0.linux and gigainstall-9.8.0.solaris file names) i.e. when we'll consume these artifacts in Gradle under dependencies { ... } section for compile, we'll use x.x.x# way to fetch these artifacts from Artifactory.
OK, once "Artifact Bundle" Deploy/Upload was successfully complete, I got the following message.
Successfully deployed 7 artifacts from archive: gigaproject.zip (1 seconds).
Now, let's see how it looks like in Artifactory while searching for one of the artifact/in Tree view. You can see we have the files now in place, with filename-x.x.x.extension way so that I can consume them easily in Gradle.
In Gradle build file (build.gradle), I'll mention:
dependencies {
compile "com.company.project:gigainstall:9.8.0#linux"
compile "com.company.project:gigainstall:9.8.0#solaris"
compile "com.company.project:gigainstall:9.8.0#linux"
compile "com.company.project:giga:9.8.0
compile "com.company.project:giga:9.8.0#war
compile "com.company.project:gigafile:9.8.0#dtd
compile "com.company.project:gigaanotherfile:9.8.0#dtd
}
OH OH!! - That didn't work, see below for Gradle error. Why? - Artifactory Bundle upload/deploy feature uploads a zip file content what you have in the .zip but it DOES NOT create a .pom file per artifact it deploys. Thus, making the Gradle build to fail. May be in Ant this might succeed. This occurred for each individual .jar/.war/.dtd/etc file. I'm just showing one error example.
While doing gradle clean build
Could not resolve all dependencies for configuration ':compile'.
> Could not resolve com.company.project:gigafile:0.0.0.
Required by:
com.company.project:ABCProjectWhichConsumesGIGAProjectArtifacts:1.64.0
> Could not GET 'http://artifactoryserver:8081/artifactory/ext-snapshot-local/com/company/project/gigafile/0.0.0/gigafile-0.0.0.pom'. Received status code 409 from server: Conflict
Case 3: Let's take a simple approach (workaround but will save a lot of pain).
Create gigaproject.zip file with the following structure, this approach takes - No x.x.x version value embedded in the individual artifact/filename in the folder/file structure. We will use "Single Artifact" approach (which will create the .pom for gigaproject.zip file automatically during the upload/deploy process provided by Artifactory). You'll still be able to get gigainstall file without needing any extension to its name using this approach. During the upload/deploy step, as you already have seen, you upload gigaproject.zip and artifactory will upload it to a given Target Repository as "gigaproject-x.x.x.zip" where x.x.x is 9.8.0 in our case. See the image snapshot below.
gigaproject/linux/gigainstall
gigaproject/solaris/gigainstall
gigaproject/win32/gigainstall.exe
gigaproject/gigafile.dtd
gigaproject/gigaanotherfile.dtd
gigaproject/gigaproject.zip
gigaproject/giga.jar
gigaproject/giga.war
Now, upload it in Artifactory using "Single Artifact" feature. Click "Deploy Artifact" once you tweak the values for GroupId, ArtifactId, Version, etc.
Once this is uploaded. You'll see in the zip artifact in the target repository (I took a bad example, usually this would be libs-snapshot-local or libs-release-local instead of ext-...), you'll be able to consume the ZIP artifact directly in Graddle:
dependencies {
// This is the only line we need now.
compile "com.company.project:gigaproject:9.8.0#zip"
}
Once the .zip is available to Gradle build system, now you can tell Gradle to unpack this .zip file somewhere in your build/workspace area where you can feed the actual(unpacked) files (gigainstall, .dtd, .jar, .war, etc.) to the build process/steps.
PS: Case# 1 and 2 would have worked for Ant I guess.
Answer 2:
If you have uploaded a non-extensioned file in either way. Make sure you have manually created/uploaded its POM file as well (i.e. if I uploaded gigainstall-9.8.0 as an artifact under com/company/project/gigainstall/9.8.0/gigainstall-9.8.0, then at the same level, I have to/should create it's POM file (see a simple template .pom file for a custom jar artifact or while uploading an extensioned file via "Single Artifact" deploy, you'll see what POM Editor window shows you) and upload both so that Gradle won't error out saying no POM conflict/error. Ant might not need pom (I didn't check that).
Once it's there in Artifactory, the following line should work -- OR comment please if you find another way.
dependencies {
// See nothing mentioned after - x.x.x#
compile "com.company.package:gigainstall:9.8.0#"
}
I'm working on a Rails 3.2.9 app , on performing a certain action, the app doesnt go any further and when i chech the log file i get this line to be the last in the log
Connecting to database specified by database.yml
I have no idea what's causing this problem.. When i sign up or sign in also it needs to connect to db and it works fine then.. only when a function (called execute test case) is clicked, the app doesnt go further and freezes there itself..
Please help me if you have come across this ...or suggest what may be the cause!!
Check this answer. This may help you.
Rails Connecting to database specified by database.yml
I found the cause of this error.. Problem is when the gem ‘mysql2’ is installed for the app , it might not be the one compatible with the version of MySQL server that is installed in our machine. And also a corresponding libmysql.dll file to be copied to Ruby folder.
So install the gem by specifying the local directory of Mysql
1.In the cmd,
gem install mysql2 -- --with-mysql-dir=C:\Program Files\MySQL
As directed in the cmd, follow the link to download the dll. Extract the zip from that location and copy the file as instructed in cmd
If the zip is empty or the link shows file not exist.(which does happen for some versions!!)
--> Go to the link and follow the flow in the url.. Like the website.. http://dev.mysql.com ->downloads -> MySQL Connectors -> MySQL Connector/C -> the latest version zips are displayed.. Choose the one with the exact file name as in the empty zip/broken link. If not click on previous GA versions and find the according zip file. Download, extract and copy the libmysql.dll to Ruby’s bin folder
I have GWT app, which is deployed on GAE (Java). I'm trying to download data from App Engine datastore using appcfg.py . I did all the setup according to http://ikaisays.com/2010/06/10/using-the-bulkloader-with-java-app-engine/ .
GAE Python SDK version is 1.4.3
Python version is 2.5.4
appcfg.py is on my PATH. When I run appcfg.py on the command-line, I get the "help" message. But the problem is that no matter which command I use, it always returns the help message. I have not been able to run any command using appcfg.py.
It doesn't give any specific error message no matter what arguments I give. My app is using Google Accounts authentication, but I don't think it even gets to the point of authentication.
I'm able to use the Java appcfg (for other actions like rollback) without any problem. But the Python version simply refuses to work for all commands.
I've tried different formats like:
appcfg.py create_bulkloader_config --url=http://myappid.appspot.com/remote_api --application=myappid --filename=config.yml
appcfg.py create_bulkloader_config --filename=bulkloader.yaml --url=http://myappid.appspot.com/remote_api
appcfg.py --filename=bulkloader.yaml --url=http://myappid.appspot.com/remote_api create_bulkloader_config
All give me the same help message:
Usage: appcfg.py [options]
Action must be one of:
create_bulkloader_config: Create a bulkloader.yaml from a running application.
cron_info: Display information about cron jobs.
download_app: Download a previously-uploaded app.
download_data: Download entities from datastore.
help: Print help for a specific action.
request_logs: Write request logs in Apache common log format.
rollback: Rollback an in-progress update.
set_default_version: Set the default (serving) version.
update: Create or update an app version.
update_cron: Update application cron definitions.
update_dos: Update application dos definitions.
update_indexes: Update application indexes.
update_queues: Update application task queue definitions.
upload_data: Upload data records to datastore.
vacuum_indexes: Delete unused indexes from application.
Use 'help <action>' for a detailed description.
Options:
-h, --help Show the help message and exit.
-q, --quiet Print errors only.
-v, --verbose Print info level logs.
--noisy Print all logs.
-s SERVER, --server=SERVER
...
...
...
Even when I try "appcfg.py help create_bulkloader_config" for a detailed description, it still shows me the same standard help.
I have also tried on the local development server using the url http://127.0.0.1:8888/remote_api but it still gives the same help message.
I'm totally clueless as to what the problem is. I'm new to GWT and GAE, and any help will be appreciated.
Thanks.
The following fix worked for me. It looks like appcfg.py doesn't like PYTHON27 and ALWAYS returns the help menu. I fixed it by executing it with PYTHON25 and hard coded all my file locations:
C:\Python25-archive\python "C:\Program Files (x86)\Google\google_appengine\appcfg.py" rollback C:\scripts\myapp
The right way is to change the environment variables on Windows 7:
Go to System Properties
Go to Advance System Settings
Click on Environment Variables
Append to Path variable the values C:\Python27\
Click Ok and restart your computer. (Yes, it is needed.)
Another way is to:
Open command Prompt
Locate your python.exe file. For example:
C:\Python27>_
Then, run a python command that looks like this.
python <appcfg_directory> download_app -A <your_app_id> -V <your_app_version> <output-dir>
Where <appcfg_directory> is equal to C:\Program Files\Google\google_appengine\appcfg.py. (Depending on your file location)
Don't forget to put quotes before and after <appcfg_directory>
This seems like it should be very easy but I don't see a link to it anywhere.
How do I download the source code of a google app engine project?
Windows
appengine-java-sdk\bin\appcfg.cmd -A <your_app_id> -V <your_app_version> download_app <output-dir>
Linux
./appengine-java-sdk/bin/appcfg.sh -A <your_app_id> -V <your_app_version> download_app <output-dir>
For completeness, using the Python implementation:
appcfg.py download_app -A $appID -V $appVersionNumber $downloadDirectory --oauth2
--oauth2 is of course optional, you can omit it and provide your email + app-specific password (or your password, and then go implement two-factor authentication right after), but it's easier, and frankly there's no reason not to.
Documentation.
App Engine actually recently added the ability for the developer who uploaded a given app version to download its source code.
As of October 2019 you can simply go to --> App Engine --> Services and in the tool dropdown select 'source' and the source code is there
Posting this since none of the listed methods above didn't take me to the code (by June 2021)
You could try accessing it through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download it automatically but you can copy-paste the code.
Hope you will find this helpful.
IMHO, the best option today (Aug 2018) is:
Under the main menu, under Products, go to Tools -> Cloud Build -> Build history.
There, click the ID of the build you want (for me - the last one).
Then, in the opened window (Build details), click the "source" link, the download of your compressed code begins.
As simple as that.
HTH.
Working with App engine standard using Go, the debugger isn't available yet.
How I managed to download the source code for an existing service was to use the gcloud tool.
First: Get the version id of your service using the app engine console or running: gcloud app versions list
Second: use the version and service name and run: gcloud app versions describe <versionID> --service=<service name>
the describe parameter will give you the storage locations for your source files that looks like this:
cmd/main.go:
sha1Sum: e3fe5848c2640eca7ac3591490e1debc2d3a9b09
sourceUrl: https://storage.googleapis.com/<project>/<file id>
Third: you can then use the storage console, using the file id, to download the files you are interested in.
this process based on java sdk
Its works for me...
Download Google cloud SDK
gcloud init
enter image description here
Follow through process of logging in using your credentials
Enter following command from SDK
C:\Program Files (x86)\Google\appengine-java-sdk-1.9.49\bin
enter image description here
Enter Following command to download source code
appcfg.sh -A [YOUR_APP_ID] -V [YOUR_APP_VERSION] download_app [OUTPUT_DIR]
Eg: appcfg.sh -A my-project-name-1234 -V 2 download_app C:\Users\india\Desktop\my project
Note: this progress based on java-appengine sdk so we use appcfg.sh instead of appcfg.py
check if your app is uploaded with same email id that is in your app engine. if you are not sure then in app engine > control > Clear deployment credentials and then click on any project, deploy to sign in again then use this
appcfg.py download_app -A {app id from google app engine} -V {1} "{c:\path}" --oauth2_credential_file=C:\Users\{your account name}/.appcfg_oauth2_tokens
change all {} to your needs
Things have changed since this question was asked so I'm adding an updated answer. Note that this only applies to GAE Standard Environment
Google has deprecated appcfg.py and so the previous responses appcfg.py download_app no longer works.
gcloud which is the SDK in use (it replaced appcfg) does not have the functionality to download your source code.
When you deploy your app via gcloud app deploy, it copies your source code to a bucket. The default bucket is staging.<project_name>.appspot.com. Your files will stay in this bucket for a maximum of 15 days before they are deleted. You can modify the rule so that the files are retained for longer or less time.
The file names in the bucket are encoded so you can't figure out what each file is unless you open it (i.e. download it). Google has a mapping of the encoded names to the original file names. To get this mapping, you run the gcloud app versions describe command and it will list the file names and their encoded names. To download the files, you have to manually click each url one by one. So essentially, you have to download each file manually and then use the mapping to rename them (or open the file, check the content and then rename them). Also note that downloading the files manually will not maintain the folder structure in which they were uploaded.
If you do not wish to go through all of the above hassles (imagine having to manually open each url for each file if you have a small to mid-sized project which has hundreds of files), our App - https://nocommandline.com - now supports downloading source code from the default bucket - staging.<project_name>.appspot.com (so far as your files are still there which means any deployment i.e update not older than 15 days from your current date unless you previously increased the deletion age on your staging bucket's lifecycle page).
In simple terms, you enter your project name, the version number and our App will take care of retrieving the original file name to encoded name mapping, automatically downloading the files and renaming them to the original names, while maintaining the folder structure. For more information, refer to https://nocommandline.com/help/#faq_download_source_code_from_gae.
Log in to the console.developers.google.com
Select the project you want to download the code from (Google App Engine Standard Envoronment).
Go to the App Engine Dashboard. Under Summary is Debug and Source. Click on Source.
Select each file one at a time and copy it (highlight the code, copy and paste into your local editor.)
Select the next file....
You need to use svn to checkout the files.
If you are on Windows, you can use tortoise svn for your GUI end.
Here are tutorials on how to do it, here is the related question.