I would like to add pre-built ipk package to the final image using Yocto at build time. What should be the correct workflow for this?
What I have tried:
1) As suggested here I used bin_package class and added ipk package to the SRC_URI
inherit bin_package
...
SRC_URI = "file://test.ipk;subdir=test-1.0"
This unpacks the package and repack it again. Unfortunately it omitted some control scripts, etc. when unpacking to work directory. Don't know why? With additional small modifications I got some usable solution, but I doubt that this is correct approach.
2) Use the package in original form and install it directly to rootfs
I tried something like this
inherit deploy
SRC_URI = "file://test.ipk;unpack=0"
do_deploy() {
install -D ${WORKDIR}/test.ipk ${DEPLOY_DIR_IPK}/test.ipk
}
addtask deploy after do_compile
I don't know if I can just copy package to e.g. "${DEPLOY_DIR_IPK}" and how to let Yocto know that the package is available and that I could use "IMAGE_INSTALL_append += "test"" to have it installed in the image.
Thanks for inputs!
To be honest I don't understand the first solution, secondly according to the documentation about do_deploy task : https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-tasks-deploy Why don't you use do_install ?
I'm not an expert but I think you should maybe try something like that:
SUMMARY = "sample recipe"
LICENSE = "CLOSED"
SRC_URI = "file://test.ipk;unpack=0"
do_install () {
install -d ${D}${foobar}
install -m 0755 ${WORKDIR}/test.ipk ${D}${foobar}
}
Related
I want to have additional files which can be imported in preprocess.py file
but i am not able to import these directly.
My directory looks like this:
Want to import from helper_functions directory into preprocess.
I tried to add this in setup.py file but it did not work.
package_data={"pipelines.ha_forecast.helper_functions": ["*.py"]},
One thing which kind of worked was to add this folder in input like this:
inputs = [
ProcessingInput(source=f'{project_name}/{module_name}/helper_functions',
destination="/opt/ml/processing/input/code/helper_functions"),
]
But this was putting the required files in some other directory which I was not able to import anymore.
What is standard way of doing this?
You have to specify the source_dir. Within your script then you can import the modules as you normally do.
source_dir (str or PipelineVariable) – Path (absolute, relative or an
S3 URI) to a directory with any other training source code
dependencies aside from the entry point file (default: None). If
source_dir is an S3 URI, it must point to a tar.gz file. Structure
within this directory are preserved when training on Amazon SageMaker.
Look at the documentation in general for Processing (you have to use FrameworkProcessor and not the specific ones like SKLearnProcessor).
P.S.: The answer is similar to that of the question "How to install additional packages in sagemaker pipeline".
Within the specified folder, there must be the script (in your case preprocess.py), any other files/modules that may be needed, and also eventually the requirements.txt file.
The structure of the folder then will be:
BASE_DIR/
|─ helper_functions/
| |─ your_utils.py
|─ requirements.txt
|─ preprocess.py
Within your preprocess.py, you will call the scripts in a simple way with:
from helper_functions.your_utils import your_class, your_func
So, your code becomes:
from sagemaker.processing import FrameworkProcessor
from sagemaker.sklearn import SKLearn
from sagemaker.workflow.steps import ProcessingStep
from sagemaker.processing import ProcessingInput, ProcessingOutput
BASE_DIR = your_script_dir_path
sklearn_processor = FrameworkProcessor(
estimator_cls=SKLearn,
framework_version=framework_version,
instance_type=processing_instance_type,
instance_count=processing_instance_count,
base_job_name=base_job_name,
sagemaker_session=pipeline_session,
role=role
)
step_args = sklearn_processor.run(
inputs=[your_inputs],
outputs=[your_outputs],
code="preprocess.py",
source_dir=BASE_DIR,
arguments=[your_arguments],
)
step_process = ProcessingStep(
name="ProcessingName",
step_args=step_args
)
It's a good practice to keep the folders for the various steps separate for each and don't create overlaps.
I am working with a very simple monorepo with the following structure:
.npmrc
package.json
pnpm-workspaces.yml
packages/
package-1/
package-2/
When I cd into package-1, I am trying to add a reference to package-2 by doing the following:
pnpm add #myorg/package-2
But I get the error #myorg/package-2 is not in the npm registry, or you have no permission to fetch it.. I expect that pnpm add would detect that I'm inside a workspace and automatically link workspace packages. Am I missing something here?
Here is the relevant information you should need:
My packages use the #myorg/[package-name] convention inside of package.json "name" field.
The pnpm-workspaces.yml file is as follows:
packages:
- 'packages/**'
My .npmrc file is as follows:
link-workspace-packages = true
prefer-workspace-packages = true
recursive-install = true
Ugh, I'm leaving this in case anybody else wants to feel embarrassed for themselves.
The pnpm-workspaces.yml should be named pnpm-workspace.yaml - I'm not sure where I got the other invalid name... but hopefully this helps you waste less time than I did.
I am trying to use react-native-linea in my react native app and during build I am getting React/RCTBridgeModule.h' file not found error.
Steps I tried -
1. react-native init ScannerApp
2. cd ScannerApp/
3. npm i react-native-linea --save
4. react-native link react-native-linea
5.Drag and drop the InfineaSDK Framework into the General > Embedded Binaries section of your Project. The framework will also display the Linked Frameworks and Libraries.
a. Verify that Copy Items if needed is checked.
6.Add the following to General > Linked Frameworks and Libraries:
• CoreLocation.framework
• ExternalAccessory.framework
• Foundation.framework
7.Add a new Run Script phase.
At the end of your project’s Build phase(s), add new running scripts to set up InfineaSDK.
FRAMEWORKS="${BUILT_PRODUCTS_DIR}/${FRAMEWORKS_FOLDER_PATH}" "${FRAMEWORKS}/InfineaSDK.framework/SDKSetup"
8.react-native run-ios
****Error*****
simulator/react-native-linea.build/Objects-normal/x86_64/RCTLinea.o
In file included from /Users/****/reactnative/ScannerApp/node_modules/react-native-linea/react-native-linea/RCTLinea.m:9:
/Users/****/reactnative/ScannerApp/node_modules/react-native-linea/react-native-linea/RCTLinea.h:9:9: fatal error: 'React/RCTBridgeModule.h' file not found
#import <React/RCTBridgeModule.h>
^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
** BUILD FAILED **
Solution 1: Adding podspec
Prerequisite: You will need to setup React as cocoapod dependency for this to work. Also, before you try this make sure you have package react-native-linea available in node_modules.
In this solution you need to create a podspec file for the linea. You can choose to keep it in your ScannerApp project or fork the original repo and add the podspec file to it. If you fork, please modify the git urls in the podspec to your repo url and add the react-native-linea package from your repo. Here's the podspec that worked for me,
require 'json'
package = JSON.parse(File.read(File.join(__dir__, '../node_modules/react-native-linea/package.json')))
Pod::Spec.new do |s|
s.name = 'LineaPro'
s.version = package['version']
s.summary = package['description']
s.license = package['license']
s.homepage = 'https://github.com/pablo-coco/react-native-linea'
s.authors = 'pablo-coco'
s.source = { :git => 'https://github.com/pablo-coco/react-native-linea.git', :tag => s.version }
s.source_files = '*.{h,m}','react-native-linea/*.{h,m}'
s.requires_arc = true
s.platforms = { :ios => "9.0" }
s.vendored_libraries = 'libdtdev.a'
s.frameworks = 'ExternalAccessory', 'CoreLocation'
s.dependency 'React'
end
Now you need to add this as cocoapod dependency in your ScannerApp podfile. If you add podspec file locally, make sure to specify its path as below,
pod 'LineaPro', :path => '../node_modules/react-native-linea', :podspec => '../ios/LineaPro.podspec'
If you created fork and added podspec to repo then skip the :podspec part.
Solution 2: Add source files directly
This is fairly simple solution and i'd recommend this. You can clone the react-native-linea repo locally on your machine.
Copy DTDevices.h, RCTLinea.h, RCTLinea.m source files to ios project
Copy LineaPro.js, NativeBridges.js to js project
Copy libdtdev.a static lib to project
Link ExternalAccessory, CoreLocation frameworks and libdtdev.a to your target
Compile and write js code to initialize LineaPro module.
I am wondering why the Biopython release in Macports does not contain the Phylo.TreeConstruction module as told here?
With iPython I tried :
In [1]: from Bio import Phylo
In [2]: Phylo. <-- hit tab here
Phylo.BaseTree Phylo.NexusIO Phylo.draw_graphviz
Phylo.CDAO Phylo.PhyloXML Phylo.parse
Phylo.NeXML Phylo.PhyloXMLIO Phylo.read
Phylo.NeXMLIO Phylo.convert Phylo.to_networkx
Phylo.Newick Phylo.draw Phylo.write
Phylo.NewickIO Phylo.draw_ascii
In [4]: ?Phylo
Type: module
String form: <module 'Bio.Phylo' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Bio/Phylo/__init__.pyc'>
File: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Bio/Phylo/__init__.py
Docstring:
Package for working with phylogenetic trees.
See also: http://biopython.org/wiki/Phylo
That's the Macports installation
I need this urgently to turn a distance matrix to a newick format for further analysis.
Thanks,
EL
Well, it's there, but for some reason it doesn't show up in dir(Bio.Phylo) and is thus not completed automatically by IPython.
Try:
/opt/local/bin/python2.7 -c 'from Bio.Phylo.TreeConstruction import _Matrix'
That's not something MacPorts did -- it seems this is either a deliberate or accidental decision made upstream. You should file a bug and request that these packages are properly exported.
I am using the solution to this question in order to apply configuration changes to App.config in a Winforms project. I also have an installer project for the project that creates an installable *.msi file. The problem is, the config file bundled in the installers is the original, un-transformed config file. So we're not getting the production connection strings in the production installer even though the config file for the built winforms project has all the correct transformations applied.
Is there any way to force the installer project to use the output of project build?
First of all: it is impossible to make the Setup Project point to another app.config file by using the Primary output option. So my solution is going to be a work around. I hope you find it useful in your situation.
Overview:
The basic idea is:
Remove the forced app.config from the Setup Project;
Add a file pointing to the app.config, manually;
Use MSBuild to get into the vdproj file, and change it to match the real output of the transformed app.config.
Some drawbacks are:
The setup project only gets updated, if the project it deploys build. ahhh... not a real drawback!
You need MSBuild 4.0... this can also be worked around!
Need a custom Task, called FileUpdate... it is open source and has installer.
Lets Work:
1) Go to your Setup Project, and select the Primary Output object, right click and go to Properties. There you will find the Exclude Filter... add a filter for *.config, so it will remove the hard-coded app.config.
2) Right click your Setup Project in the Solution Explorer -> Add -> File... select any file that ends with .config.
3) Download MSBuild Community Tasks Project, I recomend the msi installer.
4) Unload your project (the csproj) and replace the code from the other question with this one:
Code:
<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.Tasks.dll" />
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<Target Name="AfterCompile" Condition="exists('app.$(Configuration).config')">
<!-- Generate transformed app config in the intermediate directory -->
<TransformXml Source="app.config" Destination="$(IntermediateOutputPath)$(TargetFileName).config" Transform="app.$(Configuration).config" />
<!-- Force build process to use the transformed configuration file from now on. -->
<ItemGroup>
<AppConfigWithTargetPath Remove="app.config" />
<AppConfigWithTargetPath Include="$(IntermediateOutputPath)$(TargetFileName).config">
<TargetPath>$(TargetFileName).config</TargetPath>
</AppConfigWithTargetPath>
</ItemGroup>
<PropertyGroup>
<SetupProjectPath>$(MSBuildProjectDirectory)\$(IntermediateOutputPath)$(TargetFileName).config</SetupProjectPath>
</PropertyGroup>
<!-- Change the following so that this Task can find your vdproj file -->
<FileUpdate Files="$(MSBuildProjectDirectory)\..\Setup1\Setup1.vdproj"
Regex="(.SourcePath. = .8:).*\.config(.)"
ReplacementText="$1$(SetupProjectPath.Replace(`\`,`\\`))$2" />
<FileUpdate Files="$(MSBuildProjectDirectory)\..\Setup1\Setup1.vdproj"
Regex="(.TargetName. = .8:).*\.config(.)"
ReplacementText="$1$(TargetFileName).config$2" />
</Target>
5) The previous code must be changed, so that it can find your vdproj file. I have placed a comment in the code, indicating where you need to make the change.
Now, everytime you build your main project, the MSBuild will change the Setup project, so that it uses the correct app.config file. It may have drawbacks, but this solution can be polished and become better. If you need leave a comment, and I'll try to respond ASAP.
Resources I Used
MSBuild 4.0 is needed because I need to use String's Replace function, to replace single "\" to double "\" in the path. See
MSBuild Property Functions for details about using function in MSBuild.
I learned about the FileUpdate Task in this other question. The official project is MSBuild Community Tasks Project.
These two topics were important to my findings:
Trying to include configuration specific app.config files in a setup project
Problems with setup project - am I thick?
Another solution I've found is not to use the transformations but just have a separate config file, e.g. app.Release.config. Then add this line to your csproj file.
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x86' ">
<AppConfig>App.Release.config</AppConfig>
</PropertyGroup>
This will force the deployment project to use the correct config file when packaging.
I combined the best of the following answers to get a fully working solution without using any external tools at all:
1. Setup App.Config transformations
Source: https://stackoverflow.com/a/5109530
In short:
Manually add additional .config files for each build configuration and edit the raw project file to include them similar to this:
<Content Include="App.config" />
<Content Include="App.Debug.config" >
<DependentUpon>App.config</DependentUpon>
</Content>
<Content Include="App.Release.config" >
<DependentUpon>App.config</DependentUpon>
</Content>
Then include the following XML at the end of the project file, just before the closing </project> tag:
<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Web\Microsoft.Web.Publishing.Tasks.dll" />
<Target Name="AfterCompile" Condition="exists('app.$(Configuration).config')">
<TransformXml Source="app.config" Destination="$(IntermediateOutputPath)$(TargetFileName).config" Transform="app.$(Configuration).config" />
<ItemGroup>
<AppConfigWithTargetPath Remove="app.config" />
<AppConfigWithTargetPath Include="$(IntermediateOutputPath)$(TargetFileName).config">
<TargetPath>$(TargetFileName).config</TargetPath>
</AppConfigWithTargetPath>
</ItemGroup>
</Target>
Finally edit the additional .config files to include the respective transformations for each build configuration:
<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<!-- transformations here-->
</configuration>
2. Include the appropriate .config in the setup project
First, add a command in the postbuild event of your main project to move the appropriate transformed .config file to a neutral location (e.g. the main bin\ directory):
copy /y "$(TargetDir)$(TargetFileName).config" "$(ProjectDir)bin\$(TargetFileName).config"
(Source: https://stackoverflow.com/a/26521986)
Open the setup project and click the "Primary output..." node to display the properties window. There, add an ExludeFilter "*.config" to exclude the default (untransformed) .config file.
(Source: https://stackoverflow.com/a/6908477)
Finally add the transformed .config file (from the postbuild event) to the setup project (Add > File).
Done.
You can now freely add build configurations and corresponding config transforms and your setup project will always include the appropriate .config for the active configuration.
I accomplished this in a different manner with no external tools:
I added a post-build event that copied the target files to a 'neutral' directory (the root of the /bin folder in the project) and then added this file to the .vdproj. The deployment project now picks up whatever the latest built version is:
Post Build Command:
copy /y "$(TargetDir)$(TargetFileName).config" "$(ProjectDir)bin\$(TargetFileName).config"
This worked for what I needed without any external tools, and works nicely with SlowCheetah transformations.
Based off Alec's answer, here is a similar element that you can use along with the transformations and still get their full benefit:
<ItemGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<Content Include="$(OutputPath)$(AssemblyName).dll.config">
<InProject>false</InProject>
<Link>$(AssemblyName).dll.config</Link>
</Content>
</ItemGroup>
This way, you can use the SlowCheetah transforms or the built-in ones to transform your .config file, and then go into your Visual Studio Deployment Project (or other) and include the Content from the affected project in your Add -> Project Output... page easily, with minimal changes.
None of the above solutions or any articles worked for me in deployment/setup project. Spent many days to figure out the right solution. Finally this approach worked for me.
Pre requisites
I've used utility called cct.exe to transform file explicitly. You can download from here
http://ctt.codeplex.com/
I've used custom installer in setup project to capture installation events.
Follow these steps to achieve app config transformation
1) Add your desired config files to your project and modify your .csproj file like these
<Content Include="app.uat.config">
<DependentUpon>app.config</DependentUpon>
</Content>
<Content Include="app.training.config">
<DependentUpon>app.config</DependentUpon>
</Content>
<Content Include="app.live.config">
<DependentUpon>app.config</DependentUpon>
</Content>
I've added them as content so that they can be copied to output directory.
2) Add cct.exe to your project which you downloaded.
3) Add custom installer to your project which should look like this
[RunInstaller(true)]
public partial class CustomInstaller : System.Configuration.Install.Installer
{
string currentLocation = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
string[] transformationfiles = Directory.GetFiles(Path.GetDirectoryNam(Assembly.GetExecutingAssembly().Location), "app.*.config");
public CustomInstaller()
{
InitializeComponent();
// Attach the 'Committed' event.
this.Committed += new InstallEventHandler(MyInstaller_Committed);
this.AfterInstall += new InstallEventHandler(CustomInstaller_AfterInstall);
}
void CustomInstaller_AfterInstall(object sender, InstallEventArgs e)
{
try
{
Directory.SetCurrentDirectory(currentLocation);
var environment = Context.Parameters["Environment"];
var currentconfig = transformationfiles.Where(x => x.Contains(environment)).First();
if (currentconfig != null)
{
FileInfo finfo = new FileInfo(currentconfig);
if (finfo != null)
{
var commands = string.Format(#"/C ctt.exe s:yourexename.exe.config t:{0} d:yourexename.exe.config ", finfo.Name);
using (System.Diagnostics.Process execute = new System.Diagnostics.Process())
{
execute.StartInfo.FileName = "cmd.exe";
execute.StartInfo.RedirectStandardError = true;
execute.StartInfo.RedirectStandardInput = true;
execute.StartInfo.RedirectStandardOutput = true;
execute.StartInfo.UseShellExecute = false;
execute.StartInfo.CreateNoWindow = true;
execute.StartInfo.Arguments = commands;
execute.Start();
}
}
}
}
catch
{
// Do nothing...
}
}
// Event handler for 'Committed' event.
private void MyInstaller_Committed(object sender, InstallEventArgs e)
{
XmlDocument doc = new XmlDocument();
var execonfigPath = currentLocation + #"\yourexe.exe.config";
var file = File.OpenText(execonfigPath);
var xml = file.ReadToEnd();
file.Close();
doc.LoadXml(FormatXmlString(xml));
doc.Save(execonfigPath);
foreach (var filename in transformationfiles)
File.Delete(filename);
}
private static string FormatXmlString(string xmlString)
{
System.Xml.Linq.XElement element = System.Xml.Linq.XElement.Parse(xmlString);
return element.ToString();
}
}
Here I am using two event handlers CustomInstaller_AfterInstall in which I am loading correct config file and transforming .
In MyInstaller_Committed I am deleting transformation files which we don't need on client machine once we apply has been applied. I am also indenting transformed file because cct simply transforms elements were aligned ugly.
4) Open your setup project and add project output content file so that setup can copy config files like app.uat.config,app.live.config etc into client machine.
In previous step this snippet will load all available config files but we need supply right transform file
string[] transformationfiles = Directory.GetFiles(Path.GetDirectoryNam
(Assembly.GetExecutingAssembly().Location), "app.*.config");
For that I've added UI dialog on setup project to get the current config. The dialog gives options for user to select environment like "Live" "UAT" "Test" etc .
Now pass the selected environment to your custom installer and filter them.
It will become lengthy article if I explain on how to add dialog,how to set up params etc so please google them. But idea is to transform user selected environment.
The advantage of this approach is you can use same setup file for any environment.
Here is the summary:
Add config files
Add cct exe file
Add custom installer
Apply transformation on exe.config under after install event
Delete transformation files from client's machine
Modify setup project in such a way that
set up should copy all config files(project output content) and cct.exe into output directory
configure UI dialog with radio buttons (Test,Live,UAT..)
pass the selected value to custom installer
Solution might look lengthy but have no choice because MSI always copy app.config and doesn't care about project build events and transformations. slowcheetah works only with clickonce not setup project
The question is old, but the following could still help many folks out there.
I would simply use Wix WiFile.exe to replace the concerned file in the msi this way (for the sake of this example, we call your msi yourPackage.msi):
Step 1. From command prompt run: WiFile.exe "yourPackage.msi" /x "app.exe.config."
The above will extract the "wrong" app.exe.config file from the msi and place it the same directory as your msi;
Step 2. Place the new (prod) config file (must have the same name as the extracted file: app.exe.config) in same location as your msi;
This means that you are overwritting the app.exe.config that has just been extracted in step 1 above, with your new (production config file);
Step 3. From command prompt run: WiFile.exe "yourPackage.msi" /u "app.exe.config."
THAT'S ALL!
The above can be done in a few seconds. You could automate the task if you wanted, for instance, by running it as batch or else.
After running step 3 above, your msi will contain the new config file, which will now be installed at your clients' when they run the setup.