So I got to install a large number of requirements from pip for production servers. Most of those requirements require python-dev and build-essential to compile.
Now the way I look at it - python-dev and build-essential comes with some overhead, and one might argue, some security issues.
To save performance on production environment, my idea is to make binary packages for those requirements.
I have found that most pip installations come with installed-files.txt file, listing all installed files. So it is quite easy just to pack all of the listed files to some binary package.
Now my question - is it worth it ? and is there some easier way to go around it ?
Also, maybe someone has tried it and got some helper scripts ready on github.
What about creating some virtual env - installing all of the dependancies and making single large binary package for all of them ?
You can package all the Python packages as Debian Packages with py2dsc. See the Debian Python Wiki for more details. I have done that in the last hour and it works well.
Related
I'm new to c development, but I have some experience in other modern languages .so the first thing that I found hard is dependencies and deployment, while we got Gradle, maven, NuGet and pipy and... but in c I find it a bit difficult to manage this process.
for example, I have an app that should use mongo-c-library, log4c,libarchive so basically, in my development environment, I download and unzip all of the tar files of the above libraries and then followed their instruction(usually some make stuff) and installed them in order to include them in code make the code work.
I have studied a bit about CMake but I couldn't get a clear picture of how that could actually solve the problem.
at this moment my best solution is to create an install bash script and zip all dependencies unzipped folder with that install script and then send it to the production server to deploy it.
1.The first question is : is it possible to just copy and past all of .so .h and etc files in /path/of/installed/dependencies/include
and /path/of/installed/dependencies/lib in the destination server libary path.
2.if not what is the faster way?
while I was surfing the CMake source file I found that its developers just use this package source code directly.
cmxxx contains the xxx sources and headers files.
3.how can apt-get and Linux package manager help in the deployment process?
2 first question was more about dependencies. imagine we have a simple c app and we want to install(build and make a useable executable file) quickly. how it can be related to .deb packages.
1.The first question is : is it possible to just copy and past all of .so .h and etc files in /path/of/installed/dependencies/include and /path/of/installed/dependencies/lib in the destination server libary path.
Yes, technically it's possible. That's essentially what package managers do under the hood. However, doing that is a colossal mistake and screams bad practices. If that's what you want then in the very least you should look into package managers to build up your own installer, which handles this sort of stuff already for you.
2.if not what is the faster way?
You're actually asking an entirely different question, which is: how should I distribute my code, and how do I expect users to use/deploy it?
If you want users to access your source code and build it locally, as you've mentioned cmake then you just to set up your project right as cmake already supports that usecase.
If instead you just want to distribute binaries for a platform then you'll need to build and package that code. Again, cmake can also help you on that one, as cmake's cpack supports generating some types of packages like DEB packages used by Debian and Ubuntu, and which are handled by apt.
3.how can apt-get and Linux package manager help in the deployment process?
apt is designed to download and install packages from a repository.
Under the hood, apt uses DEB packages, which can be installed with dpkg.
If you're targeting a system that uses apt/deb, you can build DEB packages whenever you release a version to allow people to install their software.
You can also go a step beyond and release your DEB packages in a Personal Package Archive.
You would typically NOT download and install source packages. Instead you should generally rely on the libraries and development packages of the distribution. When building your own package you would typically just reference the packages or files that your package is dependent on. Then you build your own package and you're done. Upon installation of your package, all dependencies will automatically be resolved in an appropriate order.
What exactly needs to be done is dependent on the package management system, but generally the above statements apply. Be advised, package management apparently is pretty hard, because so many 3rd party developers screw it up.
I need to backup iphone with libimobiledevice, using ubuntu, the device is detected but going to launch the backup commands the following error is displayed:
Started "com.apple.mobilebackup2" service on port 49343.
Could not perform backup protocol version exchange, error code -1
What could it depend on?
Several Github issues have reported this problem, like this one.
Solution:
you need to use latest version of idevicebackup and libimobiledevice
Indeed, if you use Ubuntu 20.04 (for instance), the libimobiledevice package is outdated, as of now.
If that's your case, you'll have to either wait for the next Ubuntu release (22.04) or compile it from source, what may become necessary at some point after the release of Ubuntu 22.04 anyway.
Disclaimer: downside of compiling yourself is that your binaries are not managed by the package manager. You'll have to update yourself, git pulling or downloading the newest source code releases and re-compiling everything everytime. You might have to redo all of this after a distribution upgrade. Upside is that your binaries do work...
Note: compilation steps are described on the official site only for debian; I could perform them equally well on a Linux Mint 20.3 (based on Ubuntu, based on debian). OP does not mention the OS he or she uses, but debian based seem to be the only ones available for now, so what follows should work on debian based OSes.
Compilation from source, step by step:
uninstall the official package and its dependencies and:
install the build dependencies: sudo apt install build-essential checkinstall git autoconf automake libtool-bin libplist-dev libusbmuxd-dev libssl-dev usbmuxd (see "from source" here)
get libimobiledevice source code from its repo, using for instance git clone https://github.com/libimobiledevice/libimobiledevice.git. You might get to the releases page and use the latest tar.gz instead (1.3 at the moment).
also get source code of other libraries required by libimobiledevice: libplist, libimobiledevice-glue and libusbmuxd. (I also compiled usbmuxd instead of using the official package, but I am not sure it is necessary). For each one of them, you can git clone it or download and untar the latest source code release, if available.
choose a prefix directory, where libraries and binaries will go. Create it if necessary (official libimobiledevice site suggests /opt/local and I will use this too in the next steps; in order for the compilation to work, you'll have to sudo mkdir /opt/local and export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig before starting the first compilation)
to compile and install, cd to the root of each git-cloned (or source-downloaded) directory (in this order: lipblist, libimobiledevice-glue, libusbmuxd and libimobiledevice, because each one depends on the previous one) and execute, in each one of them: ./autogen.sh --prefix=/opt/local, then make and finally sudo make install. (Note, the autogen line for libimobiledevice may be ./autogen.sh --prefix=/opt/local --enable-debug, as suggested here).
Having done all of this, the iphone was not mounted automatically, I had to manually run idevicepair pair and then could mount it using ifuse ./iphone_mount_point/ (do sudo apt install ifuse if necessary) and perform a backup using idevicebackup2 backup --full iphone_backup/. Read the help of idevicebackup2 for more information.
I'm learning about Wireless Sensor Network in TinyOS. I tried to install this in Ubuntu 18.04 but it ain't worked and it return this kind of error (the picture included). Anyone could help me with this failure. Many thanks
https://i.stack.imgur.com/rZhtu.png
I came across the same problem. The reason is that TinyOS is now relatively old (in software terms) and the original development community have largely moved onto other things. This means that the packages are no longer maintained, and thus there are issues with unmet and broken dependencies. These issues will only get more and more problematic as Ubuntu evolves and leaves these packages far behind.
I can't remember exactly what the problem is as it was some time ago, but thankfully I wrote up a detailed explanation on how to install TinyOS on Vagrant here. It was tested on Ubuntu 16.04 but it should also work on 18.04.
The TL;DR version is to look carefully at the bootstrap script I use. For example, try this (the first line adds a signing key to your package manager - this came directly from Eric Decker's package repo):
wget -O - http://tinyprod.net/repos/debian/tinyprod.key | sudo apt-key add -
apt-get update
apt-get -y install nesc tinyos-tools avr-tinyos msp430-46 mspdebug
For further info see the official wiki and Eric Decker's repo, but these are also quite out of date now.
I've got a machine running an unsupported OS (rhel6.3). It no longer has access to current repos, and cannot be updated for budget reasons.
It needs to have tzdata updated, but the only version it has access to is from 2014.
tzdata is a noarch package - it's just a bunch of binary files - so it doesn't matter if I grab it from another repo, say, our local yum repo which has the latest tzdata packages for other OSs, like centos7.
I know I can simply download and put the latest tzdata rpm locally on the rhel machine, but I'd like this process to be automated - as in, Puppet checks that tzdata is the latest version - and not have to manually go put it in place whenever the tzdata is updated.
Is it possible for Puppet to specify, use this repo, but use the rpm specified for another OS?
tzdata is a noarch package - it's just a bunch of binary files - so it doesn't matter if I grab it from another repo, say, our local yum repo which has the latest tzdata packages for other OSs, like centos7.
Being a noarch package has no such implication. "Noarch" packages are independent of machine architecture, but not necessarily of OS version. For example, many Python packages are noarch, but the Python2 in EL6 is version 2.6, whereas the Python2 in EL7 is version 2.7. Noarch Python packages for EL7 do not work properly on EL6 if they rely on any of the features introduced in Python 2.7. If they are well packaged then such packages won't even install on EL6.
I suspect you're ok with tzdata in particular, but there is always a possibility that there has been a change or addition to the format of the packaged files that renders them unusable on older OS versions. You should be pretty safe, however, if you go with the CentOS 6.7 package for your EL6.3 system, rather than any EL7 package.
Is it possible for Puppet to specify, use this repo, but use the rpm specified for another OS?
Packages in the same Yum repo are not distinguished by OS or OS version. Where and to the extent that different OS flavors or versions must be served different package collections, it is done by giving them altogether different repos. So no, there is no way to instruct Puppet specifically as you describe, because there is no way to instruct yum that way.
On the other hand, if you have a local repo that is enabled on the target machine, then you can indeed drop a package built for some other RPM-based distro [version] into that repo. Clients will see it, and will (attempt to) use it where appropriate based on its name and epoch-version-release codes. You don't need to do anything special to make that happen.
On the third hand, you could also consider getting the source RPM for the version of tzdata you want, building the binary RPM from it on your EL6.3 machine, and dropping that in your local repo. You can get all manner of source (and binary) RPMs for Fedora and RHEL from Koji.
So I just installed macports (most recent version, on OS X 10.8.2). I now want to install ports. It seems that I need to search and install individual ports (though dependencies will be automatically installed for me), which are compiled instead of installed as a binary. Is this really the only way? Can I not ask to install a whole set/group of ports? (say everything related to latex)? Or perhaps there are some collections of useful ports which I can choose?
Macports contains a number of metaports, such as texlive, which are virtual packages which make it easier to install a group of ports.
E.g. installing the texlive package will set up everything you need for basic latex work. Some more advanced packages will still need to be installed by hand.
I do not know about a list of "ports that everybody should install".