i'm new in azure devops pipeline.
I have three C project in Azure Repos Git and I've configured a linux self-hosted agents.
The C_project_3 depends from .h and .a files of C_project_2 which in turn depends on .h and .a files of C_project_1.
The C_project_1 needs to build a not versioned file stored on the agent.
Is it possible configure the YAML file, of each project, to start the building process on cascade resolving the dependencies of .h, .a and extern file?
i have found the solution for question "The C_project_1 needs to build a not versioned file stored on the agent"
The sources of project are loaded on the agent in folder _work/1/s.
The not versioned file must be stored there.
I found the answer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops
In detail:
Single repository: Your source code is checked out into a directory called s located as a subfolder of (Agent.BuildDirectory).
If (Agent.BuildDirectory) is C:\agent_work\1 then your code is
checked out to C:\agent_work\1\s.
Related
I want to install single folder which contains static data required in installer in STARTUP action how can I achieve this required behavior.In startup that folder should get installed in installation directory but single folder only not all of files.
When "Startup" actions are running, no files from the distribution tree are available.
If you need files at that point, you have to add them to
Installer->Custom Code & Resources
then they will be available in the ${installer:sys.resourceDir} folder.
If that is inconvenient, you have to move your actions after the "Install files" action, then the files from the distribution tree will be installed.
In a SQL Server Data Tools (SSDT) project in Visual Studio, we have a "core" set of SQL objects that are included in each SQL project we do - kind of like a class library. We keep these "core" SQL objects in a separate Git repo, and then include them in other projects as a Git submodule.
Once the "core" submodule is linked to the main project, we include the submodule files in our .SQLPROJ file like so:
<Content Include="..\CoreSubmodule\ProjectFolder\Scripts\**\*.*">
<Link>Scripts\%(RecursiveDir)%(FileName)%(Extension)</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
This works great for regular .sql files in project - they show up with a special icon in Visual Studio indicating that it's a referenced file, and the build engine is able to resolve references/dependencies just fine. Where we're running into a snag, though, is with our Pre- and Post-Deployment scripts.
We have a series of "core" Pre- and Post-Deployment master scripts that are common among projects, which we've just introduced into our "core" submodule. Here's how the directory structure looks at a high-level:
/Scripts/
/PostDeploy/
_PostDeployMaster.sql
/ReferenceData/
ReferenceDataScript1.sql
In the above structure:
_PostDeploymentMaster.sql is a local file in the project, and is set to Build Action = "PostDeploy". It has a reference to the *ReferenceDataScript1.sql file
ReferenceDataScript1.sql is a reference to a file that physically
exists in the submodule directory (one level up from our project),
and is set to Build Action = "None"
Note that Visual Studio displays it in the /ReferenceData/ folder as a linked file
The _PostDeploymentMaster script references other sub-scripts via a SQLCMD reference:
:r .\ReferenceData\ReferenceDataScript1.sql
go
Trying to build the project in this manner produces a SQL72001 error in Visual Studio ("The included file does not exist"). Obviously if we physically place the ReferenceDataScript1.sql file in the directory (without having a reference), it builds just fine.
Options we've explored include having a non-Build "buffer" script between the PostDeploy master and the core subscripts (same error), and having pre and post build actions set to physically copy files back and forth from the submodule to the project to satisfy the build engine (a little too hacky for our taste).
Has anyone run into this issue, or have a serviceable workaround?
We wound up working around this issue by using Peter Schott's suggested fix in the original question's comments - using relative paths back to the submodule on disk instead of the "virtual" link inside of the actual Visual Studio SQL project.
I was searching how can I organize SSDT project with Submodules and found your question.
I did some realizations, but I used "virtual" link inside of the actual Visual Studio SQL project. Here is my test projects.
https://github.com/gnevashev/Core
https://github.com/gnevashev/Installation01
In .sqlproj, I added:
<Build Include="Core\**\*.sql" Exclude="Core\**\*PostDepl*.sql" />
And in project PostDeploy Script, I added link on Core PostDeploy Script:
:r .\Core\Script.PostDeploymentPopulateData.sql
Here my package is net-snmp.
Here is the task : Addition of Net-SNMP Source directory at path personal/apps/snmp/
Requirement is that it should download from website if there is change in version name and it should patch , configure and build soruce.
Hence, for that we need to configure Config.in and netsnmp.mk files in build/package/netsnmp in order to fulfill this requirement.
By adding following configuration in netsnmp.mk
NETSNMP_VERSION = 5.7.2.1
NETSNMP_SOURCE = net-snmp-$(NETSNMP_VERSION).tar.gz
NETSNMP_SITE = http://www.sourceforge.net/projects/net-snmp/files/net-snmp/$(NETSNMP_VERSION)
We can download the latest net-snmp tar ball which will be downloaded in dl/ folder in buildroot.
From here it will untar in build/output/build and will patch configure and build. After that it will build the executable at appropriate location inside target folder.
Here the issue is I want netsnmp source code to be at personal/apps/snmp folder which seems to be problematic and can edit the source in path personal/apps/snmp/ and configure build from here.
Any help and suggestion will be appreciated.
There is already a netsmp package in Buildroot. Why do you want to create another one?
Also, the Buildroot community is going to be much more reactive if you ask questions on the project's mailing list.
I have the following files which I would like to upload to Artifactory as a 9.8.0 versioned artifact.
NOTE: The first two files DO NOT have an extension (they are executable files i.e. if you open them/cat on it, you'll see junk characters).
Folder/files of a given version 9.8.0 in CVS is like:
com.company.project/gigaproject/v9.8.0/linux/gigainstall
com.company.project/gigaproject/v9.8.0/solaris/gigainstall
com.company.project/gigaproject/v9.8.0/win32/gigainstall.exe
com.company.project/gigaproject/v9.8.0/gigafile.dtd
com.company.project/gigaproject/v9.8.0/gigaanotherfile.dtd
com.company.project/gigaproject/v9.8.0/giga.jar
com.company.project/gigaproject/v9.8.0/giga.war
Uploading the above files which have an extension is very easy... You log in to Artifactory as an administrator/user which has access to deploy artifacts, click on "Deploy" tab, browse for the Artifactory file and once you select the file, click on "Upload" button.
Next you'll see a screen (like shown above). You'll tweak what you want in the fields on this page and once you click on "Deploy Artifact", you are done. All you have to make sure is you select the correct file.extension file while uploading and make sure the file extension is shown in the "Target Path" box correctly (with the version -x.x.x, etc.).
My questions:
Question 1: How do I upload an artifact which doesn't have an extension? It seems like Artifactory by default takes an artifact as a .jar extension. How can I upload the "gigainstall" artifact as shown in the folder/file structure above for both Linux and Solaris? I see I can use the artifact name as gigainstall-linux and gigainstall-solaris and differentiate it, but I am not sure how to tell Artifactory that this artifact doesn't have any extension.
I don't think the development team will start generating this artifact with a proper extension (as this artifact may be hard coded everywhere in other projects where they are currently getting it from CVS/SVN source control somewhere - which is itself a bad practice to store an artifact in a source control version tool).
Question 2: How would I tell a build system (for example, Gradle) to consume a non-extensioned artifact during, let's say, 'compile' task. In build.gradle under section dependencies { .. }, I will add something like as shown below, but I am not sure for non-extensioned files (the first two in the folder/file structure I mentioned above).
dependencies {
//compile 'com.company.project:gigainstall-linux:9.8.0#'
//compile 'com.company.project:gigainstall-linux:9.8.0#??????'
//compile 'com.company.project:gigainstall-linux:9.8.0#""'
//compile 'com.company.project:gigainstall-linux:9.8.0#"none"'
//compile 'com.company.project:gigainstall-linux:9.8.0#"NULL_or_something"'
// The following will easily get giga.jar version giga-9.8.0.jar from Artifactory repository
compile 'com.company.project:giga:9.8.0'
// The following will easily get giga.war
compile 'com.company.project:giga:9.8.0#war'
// Similarly, other extension based artifacts can be fetched from Artifactory
compile 'com.company.project:gigafile:9.8.0#dtd'
compile 'com.company.project:gigaanotherfile:9.8.0#dtd'
}
Answer 1 (will cover 2 as well in a different sense): Using Artifactory "Artifact Bundle" feature section under "Deploy" tab can do the TRICK for AT LEAST uploading the artifacts in a way we want, by creating a zip file first (containing the structure and artifacts in it) --OR you can upload the artifacts using/calling Artifactory REST API way.
High level idea:
Create a zip file called gigaproject.zip OR anyname.zip/.tar/compressed file which Artifactory can read. Inside the zip, create the structure - how these artifacts will be loaded to Artifactory
i.e.
gigaproject.zip will contain the following folders/structure/files.
Case 1:
com/company/project/gigaproject/9.8.0/linux/gigainstall
com/company/project/gigaproject/9.8.0/solaris/gigainstall
com/company/project/gigaproject/9.8.0/win32/gigainstall.exe
com/company/project/gigaproject/9.8.0/gigafile.dtd
com/company/project/gigaproject/9.8.0/gigaanotherfile.dtd
com/company/project/gigaproject/9.8.0/giga.jar
com/company/project/gigaproject/9.8.0/giga.war
NOTE: In case 1 example, I didn't use any -x.x.x in the filename (i.e. I'm using plain and simple giga.jar instead of giga-9.8.0.jar).
The above Upload/Deploy will result the files (as shown in the following snapshot):
So, we have achieved what we wanted. Actually (visibly speaking yes), but not in a way Artifactory usually stores these artifacts (as they should -x.x.x version embedded in the file name and where artifact id should match the artifact filename). Now, if you want to consume the following in a Gradle build file, you CANNOT as first, you haven't uploaded the filename with -x.x.x version name in it, secondly, the artifact id in our case 1 tree was "gigaproject" (after com/company/project folder), so Gradle way of defining what artifact id and what artifact file name you want won't work.
compile 'com.company.project:gigaproject:CANNOTSAY_HOW_TO_GET_GIGA_JARorGIGAINSTALL_with_without_extension'
Conclusion: It's possible to upload any files (with/without extension in Artifactory) in any structure but it depends how your build system will consume it or will be able to consume it or not.
- I deleted the structure I just created with case 1 .zip file from Artifactory repository to try next case#2 and deleted the .zip file I created.
Case 2:
Let's create an individual versioned file name for each artifact and also create structure in the format - how Artifactory actually stores them (an artifact as seen in a repository in a tree view) and create a .zip file containing that structure. Let's use the same "Artifact Bundle" feature to upload this .zip file to upload individual artifacts that we need in Artifactory - where artifact-id (second value which we mention while trying to consume it) would match the artifactfile name in Artifactory.
Folder/file structure for the .zip file:
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.linux
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.solaris
com/company/project/gigainstall/9.8.0/gigainstall-9.8.0.exe
com/company/project/gigafile/9.8.0/gigafile-9.8.0.dtd
com/company/project/gigaanotherfile/9.8.0/gigaanotherfile-9.8.0.dtd
com/company/project/giga/9.8.0/giga-9.8.0.jar
com/company/project/giga/9.8.0/giga-9.8.0.war
NOTE: This time, we'll be using the same "Artifact Bundle" feature and for similar files (gigainstall under both Linux/Solaris folders), I took the approach of creating gigainstall folder (containing gigainstall-9.8.0.linux and gigainstall-9.8.0.solaris file names) i.e. when we'll consume these artifacts in Gradle under dependencies { ... } section for compile, we'll use x.x.x# way to fetch these artifacts from Artifactory.
OK, once "Artifact Bundle" Deploy/Upload was successfully complete, I got the following message.
Successfully deployed 7 artifacts from archive: gigaproject.zip (1 seconds).
Now, let's see how it looks like in Artifactory while searching for one of the artifact/in Tree view. You can see we have the files now in place, with filename-x.x.x.extension way so that I can consume them easily in Gradle.
In Gradle build file (build.gradle), I'll mention:
dependencies {
compile "com.company.project:gigainstall:9.8.0#linux"
compile "com.company.project:gigainstall:9.8.0#solaris"
compile "com.company.project:gigainstall:9.8.0#linux"
compile "com.company.project:giga:9.8.0
compile "com.company.project:giga:9.8.0#war
compile "com.company.project:gigafile:9.8.0#dtd
compile "com.company.project:gigaanotherfile:9.8.0#dtd
}
OH OH!! - That didn't work, see below for Gradle error. Why? - Artifactory Bundle upload/deploy feature uploads a zip file content what you have in the .zip but it DOES NOT create a .pom file per artifact it deploys. Thus, making the Gradle build to fail. May be in Ant this might succeed. This occurred for each individual .jar/.war/.dtd/etc file. I'm just showing one error example.
While doing gradle clean build
Could not resolve all dependencies for configuration ':compile'.
> Could not resolve com.company.project:gigafile:0.0.0.
Required by:
com.company.project:ABCProjectWhichConsumesGIGAProjectArtifacts:1.64.0
> Could not GET 'http://artifactoryserver:8081/artifactory/ext-snapshot-local/com/company/project/gigafile/0.0.0/gigafile-0.0.0.pom'. Received status code 409 from server: Conflict
Case 3: Let's take a simple approach (workaround but will save a lot of pain).
Create gigaproject.zip file with the following structure, this approach takes - No x.x.x version value embedded in the individual artifact/filename in the folder/file structure. We will use "Single Artifact" approach (which will create the .pom for gigaproject.zip file automatically during the upload/deploy process provided by Artifactory). You'll still be able to get gigainstall file without needing any extension to its name using this approach. During the upload/deploy step, as you already have seen, you upload gigaproject.zip and artifactory will upload it to a given Target Repository as "gigaproject-x.x.x.zip" where x.x.x is 9.8.0 in our case. See the image snapshot below.
gigaproject/linux/gigainstall
gigaproject/solaris/gigainstall
gigaproject/win32/gigainstall.exe
gigaproject/gigafile.dtd
gigaproject/gigaanotherfile.dtd
gigaproject/gigaproject.zip
gigaproject/giga.jar
gigaproject/giga.war
Now, upload it in Artifactory using "Single Artifact" feature. Click "Deploy Artifact" once you tweak the values for GroupId, ArtifactId, Version, etc.
Once this is uploaded. You'll see in the zip artifact in the target repository (I took a bad example, usually this would be libs-snapshot-local or libs-release-local instead of ext-...), you'll be able to consume the ZIP artifact directly in Graddle:
dependencies {
// This is the only line we need now.
compile "com.company.project:gigaproject:9.8.0#zip"
}
Once the .zip is available to Gradle build system, now you can tell Gradle to unpack this .zip file somewhere in your build/workspace area where you can feed the actual(unpacked) files (gigainstall, .dtd, .jar, .war, etc.) to the build process/steps.
PS: Case# 1 and 2 would have worked for Ant I guess.
Answer 2:
If you have uploaded a non-extensioned file in either way. Make sure you have manually created/uploaded its POM file as well (i.e. if I uploaded gigainstall-9.8.0 as an artifact under com/company/project/gigainstall/9.8.0/gigainstall-9.8.0, then at the same level, I have to/should create it's POM file (see a simple template .pom file for a custom jar artifact or while uploading an extensioned file via "Single Artifact" deploy, you'll see what POM Editor window shows you) and upload both so that Gradle won't error out saying no POM conflict/error. Ant might not need pom (I didn't check that).
Once it's there in Artifactory, the following line should work -- OR comment please if you find another way.
dependencies {
// See nothing mentioned after - x.x.x#
compile "com.company.package:gigainstall:9.8.0#"
}
I use JDIC in my executable Jar file, when it runs, it needs to have IeEmbed.exe and MozEmbed.exe in the same dir as the Jar file, I wonder if there is a way to package all 3 files into a single executable Jar file, so when I distribute the app, there is only one executable Jar file to worry about, besides, there are problems sending and downloading *.exe files, what's the solution ?
For instance, my executable Jar file is called Java_App.jar, how to get <1>IeEmbed.exe <2>MozEmbed.exe and <3>Java_App.jar into a Jar file called : My_App.jar, and when user double clicks on My_App.jar, it will run the Java_App.jar and find the needed IeEmbed.exe and MozEmbed.exe files from inside the My_App.jar package without unpacking and save them into a local dir.
There is no problem packaging them together per se. The jar file is finally a .zip file.
The problem will come when your app try to load it because it won't be able to find the .exe in the system path.
The solution is to make an installable file ( which can be a .zip file ) and expand and copy the .exe file for you for instance in a "bin" directory