How to list packages and dependencies that will be installed in Macports? - macports

Is there a way to just list all new packages and their dependencies that port will install for a given command?
For instance, consider installing the SciPy stack with the suggested:
sudo port install py27-numpy py27-scipy py27-matplotlib py27-ipython +notebook py27-pandas py27-sympy py27-nose
That installs a ton of packages and dependencies not listed in the above command.
Also, some of them I already have.
I'm already aware of the -y switch but that gives a verbose output of everything, including packages I had already installed.
I'm interested to have port tell me which new packages (be it dependencies or not) will be installed.
Is there a known way or do people just parse the -y output of the command, comparing each reported package against the existing installed packages?
Cheers
p.s. I'm fairly new to Macports and MacOSX (in Linux, apt-get always tells you which new packages will be installed)

You can use a port expression to print what will be installed:
port echo rdepof:$yourport and not installed
or for multiple ports
port echo \( rdepof:$yourport rdepof:$yourport2 ... \) and not installed
Due to the number of Portfiles involved in this and how the set operations are implemented, this will be rather slow. That being said, we're also working on improving this and providing feedback prior to installation like apt-get in a future MacPorts version.

neverpanic already gave a answer, but it seems unable to handle variants (like +notebook) and command-line options (like configure.compiler=macports-clang-3.7). I had a separate solution. The following Python script can display the new dependencies recursively:
#!/usr/bin/env python
#coding: utf-8
import re
import sys
import subprocess
# Gets command output as a list of lines
def popen_readlines(cmd):
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
p.wait()
if p.returncode != 0:
raise subprocess.CalledProcessError(p.returncode, cmd)
else:
return map(lambda line: line.rstrip('\n'), p.stdout.readlines())
# Gets the port name from a line like " gcc6 #6.1.0_0 (active)"
def get_port_name(port_line):
return re.sub(r'^ (\S+).*', r'\1', port_line)
# Gets installed ports as a set
def get_installed():
installed_ports_lines = popen_readlines(['port', 'installed'])[1:]
installed_ports = set(map(get_port_name, installed_ports_lines))
return installed_ports
# Gets port names from items that may contain version specifications,
# variants, or options
def get_ports(ports_and_specs):
requested_ports = set()
for item in ports_and_specs:
if not (re.search(r'^[-+#]', item) or re.search(r'=', item)):
requested_ports.add(item)
return requested_ports
# Gets dependencies for the given port list (which may contain options
# etc.), as a list of tuples (combining with level), excluding items in
# ignored_ports
def get_deps(ports, ignored_ports, level):
if ports == []:
return []
deps_raw = popen_readlines(['port', 'deps'] + ports)
uninstalled_ports = []
for line in deps_raw:
if re.search(r'Dependencies:', line):
deps = re.sub(r'.*Dependencies:\s*', '', line).split(', ')
uninstalled_ports += [x for x in deps if x not in ignored_ports]
ignored_ports |= set(deps)
port_level_pairs = []
for port in uninstalled_ports:
port_level_pairs += [(port, level)]
port_level_pairs += get_deps([port], ignored_ports, level + 1)
return port_level_pairs
def main():
if sys.argv[1:]:
ports_and_specs = sys.argv[1:]
ignored_ports = get_installed() | get_ports(ports_and_specs)
uninstalled_ports = get_deps(ports_and_specs, ignored_ports, 0)
for (port, level) in uninstalled_ports:
print ' ' * (level * 2) + port
if __name__ == '__main__':
main()
It can be invoked like port_rdeps.py libomp configure.compiler=macports-clang-3.7. As a bonus, it can show the uninstalled dependencies as a tree.

Related

How does one reinstall an opam package/proj using the absolute path to the project without relying on opam pin?

In the past I've used this command:
opam reinstall --yes --switch ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1 --keep-build-dir lf
I think this command only worked because I also pinned the pkg e.g. (ran it from within python from unimportant reasons):
command = (['opam', 'pin', '-y']
+ root_option()
+ ['--switch', switch]
+ [coq_package, coq_package_pin])
How do I reinstall it using the path to the proj instead i.e. using coq_package_pin?
I think something like this should work:
command: list = ['opam'] + ['reinstall'] + root_option() + ['--yes'] + ['--switch', switch] + \
['--keep-build-dir', coq_project_path] + \
[coq_project_path]
Note pining the package is giving me issues so I'd like to avoid it if possible.
related:
Is an opam pin project needed when one wants to install an opam project with opam reinstall or 'make'?
https://discuss.ocaml.org/t/how-does-one-reinstall-an-opam-package-proj-using-the-absolute-path-to-the-project-without-relying-on-opam-pin/10944
https://discuss.ocaml.org/t/what-is-the-difference-between-opam-pin-and-opam-install-when-to-use-one-vs-the-other/10942/5

Trying to create custom installation package through chocolatey

I am new to scripting. I was trying to create a chocolatey package that would automatically do a custom(not typical) install. For example with MariaDB installations, I would like to specify which parts of the server to install and the username and password for the database.
I was trying to practice on Libreoffice where the package chooses Custom install and intalls only libre Writer. But the following script does the default installations what am I missing here? thanks.
chocolateinstall.ps1
e$ErrorActionPreference = 'Stop'; # stop on all errors
$toolsDir = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
$fileLocation = ".\LibreOffice_7.2.7_Win_x64.msi"
$pp= Get-PackageParameters
if (!$pp['SetupType']){$pp['SetupType']='Custom'}
if (!$pp['InstallOption']){$pp['InstallOption']='LibreWriter'}
$packageArgs = #{
packageName = $env:ChocolateyPackageName
unzipLocation = $toolsDir
fileType = 'msi'
file = $fileLocation
softwareName = 'libre*'
checksum = '...'
checksumType = 'sha256'
silentArgs = "/qn /norestart /l*v `"$($env:TEMP)\$($packageName).$($env:chocolateyPackageVersion).MsiInstall.log`"" # ALLUSERS=1 DISABLEDESKTOPSHORTCUT=1 ADDDESKTOPICON=0 ADDSTARTMENU=0
validExitCodes= #(0, 3010, 1641)
}
Install-ChocolateyPackage #packageArgs
This is a pretty huge question, to one extent, and quite easy in another!
Short answer:
Your example above doesn't pass any of the package arguments you're crafting (e.g. $pp['InstallOption']) to the actual installer. They're being stored in the variable ($pp) and never used.
The values you want to use should be passed in to Install-ChocolateyPackage using the silentArgs parameter.
However, I don't think the arguments you have there are going to work, even if you pass them in (though I may be mistaken).
Longer answer:
MSIs don't just accept random arguments.
Accepted arguments vary hugely by installer, and by software, and there's no guarantee you can do what you want silently from the commandline.
You can use something like Orca to find out what arguments an MSI may support, or search for documentation (or other folk having done the work before), or create an MST file to apply.
You could also use the Chocolatey for Business Package Builder, which scans the file and tries to identify useful arguments you can pass - though this requires a paid Business license for Chocolatey.

Installing multiples packages with chef

When I try to install multiples packages with a wildcard naming I got the following error:
* yum_package[mysql-server] action install (up to date)
* yum_package[mysql*] action install
* No candidate version available for mysql*
============================================================================ ====
Error executing action `install` on resource 'yum_package[mysql*]'
============================================================================ ====
Recipe code is:
package 'mysql-server' do
action :install
end
package 'mysql*' do
action :install
end
You have to use the exact package name. The chef package resource does no magic to find matching packages.
The name of the resource (the part just after package) is used as the package name and given to the underlying system (yum on RH like systems, apt on debian like systems)
If you have multiples packages to install and a common configuration you can loop over them in your recipe instead:
['mysql-server','mysql-common','mysql-client'].each do |p|
package p do
action :install
end
end
The array creation could be simplified with some ruby syntax as the words builder %w:
%w(mysql-server mysql-common mysql-client).each [...]
Since chef 12.1 the package resource accept an array of packages directly like this:
package %w(mysql-server mysql-common mysql-client)
This can be resolved using chef cases. Please see below
add the following to your attributes file:
packages = []
case node[:platform_family]
when 'rhel' #based on your OS
packages = [
"package1",
"package2",
"package3",
"package4",
"package5",
"package6",
"package7" ## Last line without comma
]
end
default[:cookbookname][:packages] = packages
Then, add the following to your recipe file (recipes/default.rb):
node[:cookbookname][:packages].each do |pkg|
package pkg do
action :install
retries 3
retry_delay 5
end
end

VIM with youcomplete me

I've been coding for a while and moving from IDE to IDE is a real pain when what I need most is just ctag and code completion. I finally decided to go to vim, well neovim in this case but that's beside the point.
I have ctags working and was following this tutorial from the YCM site along with this one
but I can't seem to get YCM to work for me.
I followed these step:
cd ~/.nvim/bundle
git clone https://github.com/Valloric/YouCompleteMe.git
cd YouCompleteMe
git submodule update --init --recursive
./install.sh --clang-completer --system-libclang
this went through downloaded, compiled, installed some thing
Then I created the .ycm_extra_conf.py file at "~/.nvim/.ycm_extra_conf.py"
and added: let g:ycm_global_ycm_extra_conf = "~/.nvim/.ycm_extra_conf.py"
to the top of my ~/.nvimrc file
This is what I added to my .ycm_extra_conf.py file
# This file is NOT licensed under the GPLv3, which is the license for the rest
# of YouCompleteMe.
#
# Here's the license text for this file:
#
# This is free and unencumbered software released into the public domain.
#
# Anyone is free to copy, modify, publish, use, compile, sell, or
# distribute this software, either in source code form or as a compiled
# binary, for any purpose, commercial or non-commercial, and by any
# means.
#
# In jurisdictions that recognize copyright laws, the author or authors
# of this software dedicate any and all copyright interest in the
# software to the public domain. We make this dedication for the benefit
# of the public at large and to the detriment of our heirs and
# successors. We intend this dedication to be an overt act of
# relinquishment in perpetuity of all present and future rights to this
# software under copyright law.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# For more information, please refer to <http://unlicense.org/>
import os
import ycm_core
# These are the compilation flags that will be used in case there's no
# compilation database set (by default, one is not set).
# CHANGE THIS LIST OF FLAGS. YES, THIS IS THE DROID YOU HAVE BEEN LOOKING FOR.
flags = [
'-Wall',
'-Wextra',
'-Werror',
'-Wno-long-long',
'-Wno-variadic-macros',
'-fexceptions',
'-DNDEBUG',
'-ftrapv',
'-finstrument’-functions',
'-Wfloat’-equal',
'-Wundef',
'-Wshadow',
'-Wpointer-arith',
'-Wcast’-align',
'-Wstrict-prototypes',
'-Wstrict-overflow=5',
'-Wwrite-strings',
'-Waggregate-return',
'-Wcast-qual',
'-Wswitch-default',
'-Wswitch-enum',
'-Wconversion',
'-Wunreachable-code',
# You 100% do NOT need -DUSE_CLANG_COMPLETER in your flags; only the YCM
# source code needs it.
'-DUSE_CLANG_COMPLETER',
# THIS IS IMPORTANT! Without a "-std=<something>" flag, clang won't know which
# language to use when compiling headers. So it will guess. Badly. So C++
# headers will be compiled as C headers. You don\'t want that so ALWAYS specify
# a "-std=<something>".
# For a C project, you would set this to something like 'c99' instead of
# 'c99'.
'-std=c+99',
# ...and the same thing goes for the magic -x option which specifies the
# language that the files to be compiled are written in. This is mostly
# relevant for c headers.
'-x',
'c',
'-isystem',
'../BoostParts',
'-isystem',
# This path will only work on OS X, but extra paths that don't exist are not harmful
'/System/Library/Frameworks/Python.framework/Headers',
'-isystem',
'../llvm/include',
'-isystem',
'../llvm/tools/clang/include',
'-I',
'.',
'-I',
'./ClangCompleter',
'-isystem',
'./tests/gmock/gtest',
'-isystem',
'./tests/gmock/gtest/include',
'-isystem',
'./tests/gmock',
'-isystem',
'./tests/gmock/include',
]
# Set this to the absolute path to the folder (NOT the file!) containing the
# compile_commands.json file to use that instead of 'flags'. See here for
# more details: http://clang.llvm.org/docs/JSONCompilationDatabase.html
#
# You can get CMake to generate this file for you by adding:
# set( CMAKE_EXPORT_COMPILE_COMMANDS 1 )
# to your CMakeLists.txt file.
#
# Most projects will NOT need to set this to anything; you can just change the
# 'flags' list of compilation flags. Notice that YCM itself uses that approach.
compilation_database_folder = ''
if os.path.exists( compilation_database_folder ):
database = ycm_core.CompilationDatabase( compilation_database_folder )
else:
database = None
SOURCE_EXTENSIONS = [ '.cpp', '.cxx', '.cc', '.c', '.m', '.mm' ]
def DirectoryOfThisScript():
return os.path.dirname( os.path.abspath( __file__ ) )
def MakeRelativePathsInFlagsAbsolute( flags, working_directory ):
if not working_directory:
return list( flags )
new_flags = []
make_next_absolute = False
path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ]
for flag in flags:
new_flag = flag
if make_next_absolute:
make_next_absolute = False
if not flag.startswith( '/' ):
new_flag = os.path.join( working_directory, flag )
for path_flag in path_flags:
if flag == path_flag:
make_next_absolute = True
break
if flag.startswith( path_flag ):
path = flag[ len( path_flag ): ]
new_flag = path_flag + os.path.join( working_directory, path )
break
if new_flag:
new_flags.append( new_flag )
return new_flags
def IsHeaderFile( filename ):
extension = os.path.splitext( filename )[ 1 ]
return extension in [ '.h', '.hxx', '.hpp', '.hh' ]
def GetCompilationInfoForFile( filename ):
# The compilation_commands.json file generated by CMake does not have entries
# for header files. So we do our best by asking the db for flags for a
# corresponding source file, if any. If one exists, the flags for that file
# should be good enough.
if IsHeaderFile( filename ):
basename = os.path.splitext( filename )[ 0 ]
for extension in SOURCE_EXTENSIONS:
replacement_file = basename + extension
if os.path.exists( replacement_file ):
compilation_info = database.GetCompilationInfoForFile(
replacement_file )
if compilation_info.compiler_flags_:
return compilation_info
return None
return database.GetCompilationInfoForFile( filename )
def FlagsForFile( filename, **kwargs ):
if database:
# Bear in mind that compilation_info.compiler_flags_ does NOT return a
# python list, but a "list-like" StringVec object
compilation_info = GetCompilationInfoForFile( filename )
if not compilation_info:
return None
final_flags = MakeRelativePathsInFlagsAbsolute(
compilation_info.compiler_flags_,
compilation_info.compiler_working_dir_ )
else:
relative_to = DirectoryOfThisScript()
final_flags = MakeRelativePathsInFlagsAbsolute( flags, relative_to )
return {
'flags': final_flags,
'do_cache': True
}
I have a project where I created ctags and I'm able to get around the file in vim using it's ctags support but, code completion just isn't working.
I went through the steps, went inside my ~/.nvim/bundle
This error was caused by typos in my .ycm_extras file
I run :YcmDebugInfo
and I get this error that my server has crashed:
Printing YouCompleteMe debug information...
-- Server crashed, no debug info from server
-- Server running at: http://127.0.0.1:64594
-- Server process ID: 47040
-- Server logfiles:
-- /var/folders/64/d3t4_pcs06943d651dfgp3m00000gn/T/ycm_temp/server_64594_stdout.log
-- /var/folders/64/d3t4_pcs06943d651dfgp3m00000gn/T/ycm_temp/server_64594_stderr.log
reading the stderr log I saw that I had some of these "\xe2" characters in my .ycm_extra_conf.py file.
Running a python script from this answer, I found the culprits.
I first changed to the proper directory where the .ycm_extra file was located then ran python:
Brother:.vim blubee$ python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> with open(".ycm_extra_conf.py") as fp:
... for i, line in enumerate(fp):
... if "\xe2" in line:
... print i, repr(line)
...
45 "'-finstrument\xe2\x80\x99-functions',\n"
46 "'-Wfloat\xe2\x80\x99-equal',\n"
50 "'-Wcast\xe2\x80\x99-align',\n"
removing the quote marks fixed the problem, now YCM works just as expected.
you can see the file before:
'-finstrument’-functions',
'-Wfloat’-equal',
'-Wundef',
'-Wshadow',
'-Wpointer-arith',
'-Wcast’-align',
and after:
'-finstrument-functions', #removed quote after finstrument
'-Wfloat-equal', #removed quote after Wfloat
'-Wundef',
'-Wshadow',
'-Wpointer-arith',
'-Wcast-align', #removed quote after Wcast

fabric for offline package installation

The project I'm working in uses fabric for many build steps and requires a offline build as fallback.
I'm currently stuck at installing python packages provided in tarballs.
The thing is I have trouble getting into the newly extracted directory and running setup.py install in there.
#task
def deploy_artifacts():
"""Installs dependencies from local path, useful for offline builds"""
#TODO: Handle downloading files and do something like this bellow
tmpdir = tempfile.mkdtemp()
artifacts_path = ''
if not 'http' in env.artifacts_path:
artifacts_path = env.artifacts_path
with lcd(artifacts_path):
for f in os.listdir(artifacts_path):
if 'gz' in f:
put(f, tmpdir)
tar = os.path.join(tmpdir, f)
target_dir = os.path.join(tempfile.gettempdir(), normalize(f))
if not files.exists(target_dir):
run('mkdir %s' % target_dir)
else:
run('rm -rf %s' %target_dir)
run('mkdir %s' % target_dir)
run('tar xf %s -C %s' % (tar, target_dir))
run('rm %s' % tar)
with cd(target_dir):
sudo('python setup.py install')
I come from reading the tar man page for the bazillion time and I got nowhere near to getting what I want.
Did some of you face a situation like this? is there some other (read: better) approach to this scenario?
There's nothing wrong (in principle) with what you're trying do. Maybe just take smaller steps getting there. Rather than using temporary directories, it might make debugging easier if everything was put in a systematic location that has known permissions that nothing else writes to by convention. At least that would let you use some combination of fabric and manual intervention to check what is going wrong.
In the longer term, there are a few alternatives that I see. For simplicity you want the online and offline versions to work the same way, and that means fetching packages using easy_install / pip for both cases.
One way to do this is to build a mirror of PyPi. The right way to do this if you've got plenty of storage space (30Gb) is to use software that implements PEP381 (Mirroring Infrastructure for PyPI), there is already a client that does this (pep381client). A number of other projects are available that do similar things (basketweaver, djangopypi2, chishop).
An alternative is to consider a lighter weight proxying scheme. I've been looking a pip2pi and pipli. I'm unsure if they will work directly with easy_install, but it would be worth a try.
It's also worth noting that if you were using pip, you could have installed directly from the tarballs.

Resources