Use GitLab CI to deploy app with ftp - angularjs

I'm currently working on a little Angular Web project. And I found this great tool named Gitlab CI.
I read the docs and setup a node docker to build the webapp. Then I want to upload the builded app with ftp to my server. And this is where my trouble starts.
First here ist my gitlab-ci.yml
image: node:7.5.0
cache:
key: "$CI_BUILD_REF_NAME"
untracked: true
paths:
- node_modules/
- dist/
stages:
- build
# - test
- deploy
- cleanup
# - deployProd
runBuild:
before_script:
- npm install -g angular-cli
- npm install
stage: build
script:
- ng build --target=production --environment=test
except:
- tags
runProdBuild:
before_script:
- npm install -g angular-cli
- npm install
stage: build
script:
- ng build --target=production --environment=prod
only:
- tags
runDeployTest:
before_script:
- apt-get install ftp
variables:
DATABASE: ""
URL: "http://test.domain.de"
stage: deploy
environment:
name: Entwicklungssystem
url: https://test.domain.de
artifacts:
name: "$CI_BUILD_NAME/$CI_BUILD_REF_NAME"
paths:
- dist/
expire_in: 2d
except:
- tags
script:
- echo '<?php ini_set("max_execution_time", 300); function rrmdir($dir) { if (is_dir($dir)) { $objects = scandir($dir); foreach ($objects as $object) { if ($object != "." && $object != "..") { if (is_dir($dir."/".$object)) { rrmdir($dir."/".$object); } else { echo "unlink :".$dir."/".$object; unlink($dir."/".$object); } } } rmdir($dir); } } rrmdir(__DIR__."."); ?>' > delete.php
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . delete.php"
- wget "$URL/delete.php"
- cd ./dist
- zip -r install.zip .
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . install.zip"
- echo "<?php \$dateiname = __DIR__.'/install.zip'; \$ofolder = str_replace('/public','',__DIR__); exec('unzip '.\$dateiname.' -d '.\$ofolder.' 2>&1', \$out); print(implode('<br>', \$out)); unlink(\$dateiname); unlink('entpacker.php'); unlink(__DIR__.'/../delete.php'); unlink(__DIR__.'/../delete.php.1'); ?>" > entpacker.php
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . entpacker.php"
# Install
- wget $URL/entpacker.php
runDeployProd:
before_script:
- apt-get install ftp
variables:
DATABASE: ""
URL: "http://test.domain.de"
stage: deploy
environment:
name: Produktivsystem
url: https://prod.domain.de
artifacts:
name: "$CI_BUILD_NAME/$CI_BUILD_REF_NAME"
paths:
- dist/
expire_in: 2d
script:
- echo '<?php ini_set("max_execution_time", 300); function rrmdir($dir) { if (is_dir($dir)) { $objects = scandir($dir); foreach ($objects as $object) { if ($object != "." && $object != "..") { if (is_dir($dir."/".$object)) { rrmdir($dir."/".$object); } else { echo "unlink :".$dir."/".$object; unlink($dir."/".$object); } } } rmdir($dir); } } rrmdir(__DIR__."."); ?>' > delete.php
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . delete.php"
- wget "$URL/delete.php"
- cd ./dist
- zip -r install.zip .
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . install.zip"
- echo "<?php \$dateiname = __DIR__.'/install.zip'; \$ofolder = str_replace('/public','',__DIR__); exec('unzip '.\$dateiname.' -d '.\$ofolder.' 2>&1', \$out); print(implode('<br>', \$out)); unlink(\$dateiname); unlink('entpacker.php'); unlink(__DIR__.'/../delete.php'); unlink(__DIR__.'/../delete.php.1'); ?>" > entpacker.php
- lftp -d -c "set ftp:ssl-allow no; open -u $ftp_user,$ftp_password $ftp_server; cd $ftp_path; put -O . entpacker.php"
# Install
- wget $URL/entpacker.php
only:
- tags
cleanup:
stage: cleanup
script:
- rm -rf ./dist
- rm -rf ./node_modules
when: manual
So it works fine until I want to install ftp to the docker image.
My question is now: Is it possible to install ftp to the image?
Or is there a other way to handle things like this? I can't use ssh because there is no ssh access to the webspace.

I got a solution. As suggested I tried to create a own docker image. There I noticed that I can't install lftp too. So at creating an docker image you have to run apt-get update first.
So I tried this inside my script, and it worked.
So you need to run apt-get update first, then install any package you want.

Use lftp instead of ftp
runDeployProd:
before_script:
- apt-get install lftp
https://forum.gitlab.com/t/deploy-via-ftp-via-ci/2631/2

Related

Salesforce CLI destructive changes (sfdx sgd:source:delta) not showing difference in branches using GItHub Actions

I am using the Salesforce destructive changes mentioned here. However, sfdx sgd:source:delta --to "HEAD" --from "HEAD^" --output . --generate-delta command is not showing any destructive changes, it generates a destructiveChanges.xml without any deleted metadata.
--- destructiveChanges.xml generated with deleted metadata ---
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
<version>52.0</version>
</Package>
Here is the content of the yml file I am using:
name: sf-destructivechanges
on:
push:
branches:
- "test"
jobs:
sf-destructivechanges:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Install Salesforce CLI
run: |
wget https://developer.salesforce.com/media/salesforce-cli/sfdx-linux-amd64.tar.xz
mkdir sfdx-cli
tar xJf sfdx-linux-amd64.tar.xz -C sfdx-cli --strip-components 1
./sfdx-cli/install
sfdx update
- name: Install plugin
run: |
echo 'y' | sfdx plugins:install sfdx-git-delta
- name: Get delta files with SGD
run: sfdx sgd:source:delta --to "HEAD" --from "HEAD^" --output . --generate-delta
- name: Deploy changes
run: |
echo "--- package.xml generated with added and modified metadata ---"
cat package/package.xml
echo
echo "---- Deploying added and modified metadata ----"
sfdx force:source:deploy -x package/package.xml -u ${{ secrets.USERNAME}}
- name: Destructive Changes
run: |
echo "--- destructiveChanges.xml generated with deleted metadata ---"
cat destructiveChanges/destructiveChanges.xml
echo
echo "--- Deleting removed metadata ---"
sfdx force:mdapi:deploy -d destructiveChanges -u ${{ secrets.USERNAME}} -w -1

What correction can I make in DockerFile to pass my build successfully

My application deployment failing in building docker image while deploying the container with tomcat 9 and application war file inside it with below error :
ERROR: build step 1 "gcr.io/cloud-builders/docker" failed: exit status 2
I am beginner in docker containerization and took online guide in coding the deployment. Please let me know whats wrong in the below dockerFile & cloudbuild file
Dockerfile :-
FROM openjdk:8-jre
ENV CATALINA_HOME /usr/local/tomcat
ENV PATH $CATALINA_HOME/bin:$PATH
RUN mkdir -p "$CATALINA_HOME"
WORKDIR $CATALINA_HOME
# let "Tomcat Native" live somewhere isolated
ENV TOMCAT_NATIVE_LIBDIR $CATALINA_HOME/native-jni-lib
ENV LD_LIBRARY_PATH ${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$TOMCAT_NATIVE_LIBDIR
# runtime dependencies for Tomcat Native Libraries
# Tomcat Native 1.2+ requires a newer version of OpenSSL than debian:jessie has available
# > checking OpenSSL library version >= 1.0.2...
# > configure: error: Your version of OpenSSL is not compatible with this version of tcnative
# see http://tomcat.10.x6.nabble.com/VOTE-Release-Apache-Tomcat-8-0-32-tp5046007p5046024.html (and following discussion)
# and https://github.com/docker-library/tomcat/pull/31
ENV OPENSSL_VERSION 1.1.0j-1~deb9u1
RUN set -ex; \
currentVersion="$(dpkg-query --show --showformat '${Version}\n' openssl)"; \
if dpkg --compare-versions "$currentVersion" '<<' "$OPENSSL_VERSION"; then \
if ! grep -q stretch /etc/apt/sources.list; then \
# only add stretch if we're not already building from within stretch
{ \
echo 'deb http://deb.debian.org/debian stretch main'; \
echo 'deb http://security.debian.org stretch/updates main'; \
echo 'deb http://deb.debian.org/debian stretch-updates main'; \
} > /etc/apt/sources.list.d/stretch.list; \
{ \
# add a negative "Pin-Priority" so that we never ever get packages from stretch unless we explicitly request them
echo 'Package: *'; \
echo 'Pin: release n=stretch*'; \
echo 'Pin-Priority: -10'; \
echo; \
# ... except OpenSSL, which is the reason we're here
echo 'Package: openssl libssl*'; \
echo "Pin: version $OPENSSL_VERSION"; \
echo 'Pin-Priority: 990'; \
} > /etc/apt/preferences.d/stretch-openssl; \
fi; \
apt-get update; \
apt-get install -y --no-install-recommends openssl="$OPENSSL_VERSION"; \
rm -rf /var/lib/apt/lists/*; \
fi
RUN apt-get update && apt-get install -y --no-install-recommends \
libapr1 \
&& rm -rf /var/lib/apt/lists/*
# see https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/KEYS
# see also "update.sh" (https://github.com/docker-library/tomcat/blob/master/update.sh)
ENV GPG_KEYS 05AB33110949707C93A279E3D3EFE6B686867BA6 07E48665A34DCAFAE522E5E6266191C37C037D42 47309207D818FFD8DCD3F83F1931D684307A10A5 541FBE7D8F78B25E055DDEE13C370389288584E7 61B832AC2F1C5A90F0F9B00A1C506407564C17A3 79F7026C690BAA50B92CD8B66A3AD3F4F22C4FED 9BA44C2621385CB966EBA586F72C284D731FABEE A27677289986DB50844682F8ACB77FC2E86E29AC A9C5DF4D22E99998D9875A5110C01C5A2F6059E7 DCFD35E0BF8CA7344752DE8B6FB21E8933C60243 F3A04C595DB5B6A5F1ECA43E3B7BBB100D811BBE F7DA48BB64BCB84ECBA7EE6935CD23C10D498E23
ENV TOMCAT_MAJOR 9
ENV TOMCAT_VERSION 9.0.14
ENV TOMCAT_SHA512 6a06eabacb02eaa2c64772265f8ddb5ddbfbad72b4c5da82e68da739675c3ec7e4692ebb8aa80e1b5512006f60856c939c7c3a5fe4f46f30bd4b575ecd068d54
ENV TOMCAT_TGZ_URLS \
# https://issues.apache.org/jira/browse/INFRA-8753?focusedCommentId=14735394#comment-14735394
https://www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz \
# if the version is outdated, we might have to pull from the dist/archive :/
https://www-us.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz \
https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz \
https://archive.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz
ENV TOMCAT_ASC_URLS \
https://www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc \
# not all the mirrors actually carry the .asc files :'(
https://www-us.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc \
https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc \
https://archive.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc
RUN set -eux; \
\
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
\
apt-get install -y --no-install-recommends gnupg dirmngr; \
\
export GNUPGHOME="$(mktemp -d)"; \
for key in $GPG_KEYS; do \
gpg --batch --keyserver ipv4.pool.sks-keyservers.net --recv-keys "$key"; \
done; \
\
apt-get install -y --no-install-recommends wget ca-certificates; \
\
success=; \
for url in $TOMCAT_TGZ_URLS; do \
if wget -O tomcat.tar.gz "$url"; then \
success=1; \
break; \
fi; \
done; \
[ -n "$success" ]; \
\
echo "$TOMCAT_SHA512 *tomcat.tar.gz" | sha512sum -c -; \
\
success=; \
for url in $TOMCAT_ASC_URLS; do \
if wget -O tomcat.tar.gz.asc "$url"; then \
success=1; \
break; \
fi; \
done; \
[ -n "$success" ]; \
\
gpg --batch --verify tomcat.tar.gz.asc tomcat.tar.gz; \
tar -xvf tomcat.tar.gz --strip-components=1; \
rm bin/*.bat; \
rm tomcat.tar.gz*; \
command -v gpgconf && gpgconf --kill all || :; \
rm -rf "$GNUPGHOME"; \
\
nativeBuildDir="$(mktemp -d)"; \
tar -xvf bin/tomcat-native.tar.gz -C "$nativeBuildDir" --strip-components=1; \
apt-get install -y --no-install-recommends \
dpkg-dev \
gcc \
libapr1-dev \
libssl-dev \
make \
"openjdk-${JAVA_VERSION%%[.~bu-]*}-jdk=$JAVA_DEBIAN_VERSION" \
; \
( \
export CATALINA_HOME="$PWD"; \
cd "$nativeBuildDir/native"; \
gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
./configure \
--build="$gnuArch" \
--libdir="$TOMCAT_NATIVE_LIBDIR" \
--prefix="$CATALINA_HOME" \
--with-apr="$(which apr-1-config)" \
--with-java-home="$(docker-java-home)" \
--with-ssl=yes; \
make -j "$(nproc)"; \
make install; \
); \
rm -rf "$nativeBuildDir"; \
rm bin/tomcat-native.tar.gz; \
\
# reset apt-mark's "manual" list so that "purge --auto-remove" will remove all build dependencies
apt-mark auto '.*' > /dev/null; \
[ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
rm -rf /var/lib/apt/lists/*; \
\
# sh removes env vars it doesn't support (ones with periods)
# https://github.com/docker-library/tomcat/issues/77
find ./bin/ -name '*.sh' -exec sed -ri 's|^#!/bin/sh$|#!/usr/bin/env bash|' '{}' +; \
\
# fix permissions (especially for running as non-root)
# https://github.com/docker-library/tomcat/issues/35
chmod -R +rX .; \
chmod 777 logs work
# verify Tomcat Native is working properly
RUN set -e \
&& nativeLines="$(catalina.sh configtest 2>&1)" \
&& nativeLines="$(echo "$nativeLines" | grep 'Apache Tomcat Native')" \
&& nativeLines="$(echo "$nativeLines" | sort -u)" \
&& if ! echo "$nativeLines" | grep 'INFO: Loaded APR based Apache Tomcat Native library' >&2; then \
echo >&2 "$nativeLines"; \
exit 1; \
fi \
&& rm -rf ${CATALINA_HOME}/webapps/* || true
EXPOSE 8080
CMD ["catalina.sh", "run"]
COPY target/app-0.0.1.war ${CATALINA_HOME}/webapps/app.war
cloudbuild.yaml:-
steps:
- name: 'gcr.io/cloud-builders/mvn'
args: ['clean','install','-Dmaven.test.skip=true']
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '--tag=gcr.io/project/app:$TAG_NAME', '.']
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/project/app:$TAG_NAME']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '--image-url', 'gcr.io/project/app:$TAG_NAME', '--project', 'project', '--version', '$TAG_NAME', '--no-promote']
images: ['gcr.io/project/app:$TAG_NAME']
timeout: 2400s
options:
machineType: 'N1_HIGHCPU_32'
Here's the full log:
Step #1: Preparing to unpack .../libapr1_1.5.2-5_amd64.deb ...
Step #1: Unpacking libapr1:amd64 (1.5.2-5) ...
Step #1: Setting up libapr1:amd64 (1.5.2-5) ...
Step #1: Processing triggers for libc-bin (2.24-11+deb9u4) ...
Step #1: Removing intermediate container 1fc38e2094c7
Step #1: ---> 4176a4819577
Step #1: Step 11/21 : ENV GPG_KEYS 05AB33110949707C93A279E3D3EFE6B686867BA6 07E48665A34DCAFAE522E5E6266191C37C037D42 47309207D818FFD8DCD3F83F1931D684307A10A5 541FBE7D8F78B25E055DDEE13C370389288584E7 61B832AC2F1C5A90F0F9B00A1C506407564C17A3 79F7026C690BAA50B92CD8B66A3AD3F4F22C4FED 9BA44C2621385CB966EBA586F72C284D731FABEE A27677289986DB50844682F8ACB77FC2E86E29AC A9C5DF4D22E99998D9875A5110C01C5A2F6059E7 DCFD35E0BF8CA7344752DE8B6FB21E8933C60243 F3A04C595DB5B6A5F1ECA43E3B7BBB100D811BBE F7DA48BB64BCB84ECBA7EE6935CD23C10D498E23
Step #1: ---> Running in b9f5df938ebc
Step #1: Removing intermediate container b9f5df938ebc
Step #1: ---> e41d4c57146b
Step #1: Step 12/21 : ENV TOMCAT_MAJOR 9
Step #1: ---> Running in 93228b5fc301
Step #1: Removing intermediate container 93228b5fc301
Step #1: ---> 9493e206667a
Step #1: Step 13/21 : ENV TOMCAT_VERSION 9.0.14
Step #1: ---> Running in 2127ca636165
Step #1: Removing intermediate container 2127ca636165
Step #1: ---> 29cd9c11963b
Step #1: Step 14/21 : ENV TOMCAT_SHA512 6a06eabacb02eaa2c64772265f8ddb5ddbfbad72b4c5da82e68da739675c3ec7e4692ebb8aa80e1b5512006f60856c939c7c3a5fe4f46f30bd4b575ecd068d54
Step #1: ---> Running in 77ac801c3995
Step #1: Removing intermediate container 77ac801c3995
Step #1: ---> ab68501a7204
Step #1: Step 15/21 : ENV TOMCAT_TGZ_URLS https://www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz https://www-us.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz https://archive.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz
Step #1: ---> Running in 5475b4b3fdf0
Step #1: Removing intermediate container 5475b4b3fdf0
Step #1: ---> d35149a8d4e8
Step #1: Step 16/21 : ENV TOMCAT_ASC_URLS https://www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc https://www-us.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc https://archive.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz.asc
Step #1: ---> Running in 143651dc16c2
Step #1: Removing intermediate container 143651dc16c2
Step #1: ---> e8cd3a05124b
Step #1: Step 17/21 : RUN set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends gnupg dirmngr; export GNUPGHOME="$(mktemp -d)"; for key in $GPG_KEYS; do gpg --batch --keyserver ipv4.pool.sks-keyservers.net --recv-keys "$key"; done; apt-get install -y --no-install-recommends wget ca-certificates; success=; for url in $TOMCAT_TGZ_URLS; do if wget -O tomcat.tar.gz "$url"; then success=1; break; fi; done; [ -n "$success" ]; echo "$TOMCAT_SHA512 *tomcat.tar.gz" | sha512sum -c -; success=; for url in $TOMCAT_ASC_URLS; do if wget -O tomcat.tar.gz.asc "$url"; then success=1; break; fi; done; [ -n "$success" ]; gpg --batch --verify tomcat.tar.gz.asc tomcat.tar.gz; tar -xvf tomcat.tar.gz --strip-components=1; rm bin/*.bat; rm tomcat.tar.gz*; command -v gpgconf && gpgconf --kill all || :; rm -rf "$GNUPGHOME"; nativeBuildDir="$(mktemp -d)"; tar -xvf bin/tomcat-native.tar.gz -C "$nativeBuildDir" --strip-components=1; apt-get install -y --no-install-recommends dpkg-dev gcc libapr1-dev libssl-dev make "openjdk-${JAVA_VERSION%%[.~bu-]*}-jdk=$JAVA_DEBIAN_VERSION" ; ( export CATALINA_HOME="$PWD"; cd "$nativeBuildDir/native"; gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; ./configure --build="$gnuArch" --libdir="$TOMCAT_NATIVE_LIBDIR" --prefix="$CATALINA_HOME" --with-apr="$(which apr-1-config)" --with-java-home="$(docker-java-home)" --with-ssl=yes; make -j "$(nproc)"; make install; ); rm -rf "$nativeBuildDir"; rm bin/tomcat-native.tar.gz; apt-mark auto '.*' > /dev/null; [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; rm -rf /var/lib/apt/lists/*; find ./bin/ -name '*.sh' -exec sed -ri 's|^#!/bin/sh$|#!/usr/bin/env bash|' '{}' +; chmod -R +rX .; chmod 777 logs work
Step #1: ---> Running in f7a894cf216c
Step #1: [91m+ apt-mark showmanual
Step #1: [0m[91m+ savedAptMark=bzip2
Step #1: ca-certificates
Step #1: curl
Step #1: dirmngr
Step #1: fontconfig
Step #1: gnupg
Step #1: iproute2
Step #1: iputils-ping
Step #1: libapr1
Step #1: libfreetype6
Step #1: netbase
Step #1: unzip
Step #1: wget
Step #1: xz-utils
Step #1: + apt-get update
Step #1: [0mIgn:1 http://deb.debian.org/debian stretch InRelease
Step #1: Get:2 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Step #1: Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Step #1: Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Step #1: Get:5 http://deb.debian.org/debian stretch Release.gpg [2434 B]
Step #1: Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [492 kB]
Step #1: Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [27.2 kB]
Step #1: Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7082 kB]
Step #1: Fetched 7907 kB in 1s (4681 kB/s)
Step #1: Reading package lists...
Step #1: [91m+ apt-get install -y --no-install-recommends gnupg dirmngr
Step #1: [0mReading package lists...
Step #1: Building dependency tree...
Step #1: Reading state information...
Step #1: dirmngr is already the newest version (2.1.18-8~deb9u4).
Step #1: gnupg is already the newest version (2.1.18-8~deb9u4).
Step #1: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Step #1: [91m+ mktemp -d
Step #1: [0m[91m+ export GNUPGHOME=/tmp/tmp.8FhNStIe88
Step #1: + gpg --batch --keyserver ipv4.pool.sks-keyservers.net --recv-keys 05AB33110949707C93A279E3D3EFE6B686867BA6
Step #1: [0m[91mgpg: keybox '/tmp/tmp.8FhNStIe88/pubring.kbx' created
Step #1: [0m[91mgpg: keyserver receive failed: No data
Step #1: The command '/bin/sh -c set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends gnupg dirmngr; export GNUPGHOME="$(mktemp -d)"; for key in $GPG_KEYS; do gpg --batch --keyserver ipv4.pool.sks-keyservers.net --recv-keys "$key"; done; apt-get install -y --no-install-recommends wget ca-certificates; success=; for url in $TOMCAT_TGZ_URLS; do if wget -O tomcat.tar.gz "$url"; then success=1; break; fi; done; [ -n "$success" ]; echo "$TOMCAT_SHA512 *tomcat.tar.gz" | sha512sum -c -; success=; for url in $TOMCAT_ASC_URLS; do if wget -O tomcat.tar.gz.asc "$url"; then success=1; break; fi; done; [ -n "$success" ]; gpg --batch --verify tomcat.tar.gz.asc tomcat.tar.gz; tar -xvf tomcat.tar.gz --strip-components=1; rm bin/*.bat; rm tomcat.tar.gz*; command -v gpgconf && gpgconf --kill all || :; rm -rf "$GNUPGHOME"; nativeBuildDir="$(mktemp -d)"; tar -xvf bin/tomcat-native.tar.gz -C "$nativeBuildDir" --strip-components=1; apt-get install -y --no-install-recommends dpkg-dev gcc libapr1-dev libssl-dev make "openjdk-${JAVA_VERSION%%[.~bu-]*}-jdk=$JAVA_DEBIAN_VERSION" ; ( export CATALINA_HOME="$PWD"; cd "$nativeBuildDir/native"; gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; ./configure --build="$gnuArch" --libdir="$TOMCAT_NATIVE_LIBDIR" --prefix="$CATALINA_HOME" --with-apr="$(which apr-1-config)" --with-java-home="$(docker-java-home)" --with-ssl=yes; make -j "$(nproc)"; make install; ); rm -rf "$nativeBuildDir"; rm bin/tomcat-native.tar.gz; apt-mark auto '.*' > /dev/null; [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; rm -rf /var/lib/apt/lists/*; find ./bin/ -name '*.sh' -exec sed -ri 's|^#!/bin/sh$|#!/usr/bin/env bash|' '{}' +; chmod -R +rX .; chmod 777 logs work' returned a non-zero code: 2
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/cloud-builders/docker" failed: exit status 2
Step #1: [0m

Problem with executing only first element into array [duplicate]

This question already has answers here:
While loop stops reading after the first line in Bash
(5 answers)
Closed 1 year ago.
I thought that my problem is trivial, but I cannot figure out, why my scripts only performing once in array.
I have a jenkins job (bash script). This job gathering hostnames and sends ssh commands, through script, using gathered info:
rm /tmp/hosts
docker exec -t tmgnt_consul_1 consul members -status=alive | grep -v Node | awk '{print $1}' | cut -d : -f1 >> /tmp/hosts
sed -i '/someunnecessaryinfo/d' /tmp/hosts
echo >> /tmp/hosts
shopt -s lastpipe
while IFS= read -r line; do
echo "host is >>$line<<";
url="http://111.111.111.111:8500/v1/catalog/nodes"
term_IP=`curl -s $url | jq -r --arg Node "${line}" '.[] | select(.Node == "'${line}'" )|.Address' --raw-output`
echo $term_IP
sudo bash -x /home/rtm/t_mgnt/check_fw $term_IP
done < /tmp/hosts
Second script:
#!/bin/bash
term_IP=$1
sudo sshpass -p 'some.pass' ssh -o StrictHostKeyChecking=no user#$term_IP "sudo test -d /root/nv9"
if [ $? != 0 ]; then
sudo sshpass -p 'some.pass' \
scp -n -o StrictHostKeyChecking=no -r /home/rtm/t_mgnt/nv9 user#$term_IP:
sudo sshpass -p 'some.pass' \
ssh -n -o StrictHostKeyChecking=no user#$term_IP "sudo mv nv9 /root/"
sudo sshpass -p 'some.pass' \
ssh -n -o StrictHostKeyChecking=no user#$term_IP "sudo dpkg -i /root/nv9/libudev0_175-0ubuntu9_amd64.deb"
sudo sshpass -p 'some.pass' \
ssh -n -o StrictHostKeyChecking=no user#$term_IP "sudo /root/nv9/DetectValidator"
else
sudo sshpass -p 'some.pass' \
ssh -n -o StrictHostKeyChecking=no user#$term_IP "sudo /root/nv9/DetectValidator"
fi
The job is working fine, and returns correct values, but only for the first element of array.
PS - I already searched through this and other sites, and - following answer didn't help me - Shell script while read line loop stops after the first line (already "ssh -n -o").
Perhaps you can point me, what I missed.
Possibly this ssh call eats your input:
sudo sshpass -p 'some.pass' ssh -o StrictHostKeyChecking=no user#$term_IP "sudo test -d /root/nv9"
^^^
Try adding -n.

WildFly 10 rpmbuild

I am building a custom RPM for WildFly 10. I am stuck on deploying the systemd service. The spec file is able to deploy the code as well as create a user, however, no matter what avenue I try, I cannot get the RPM to create the service. I have tried install -m 644 but rpmbuild has tries finding the file, even if I specify full path:
e.g. install -m 644 %{buildroot}/opt/%{name}/docs/contrib/scripts/systemd/%{name}.service %{buildroot}/usr/lib/systemd/system/%{name}.service
I have also tried a series of systemd scriptlets as noted in https://fedoraproject.org/wiki/Packaging:Scriptlets, but that does nothing (the RPM will build with exit 0 status, but the service is never created). Any assistance would be appreciated.
$ cat SPECS/wildfly.spec
Name: wildfly
Version: 10.0.0.Final
Release: 1%{?dist}
Summary: WildFly (JBoss) Application Server
Group: System Environment/Daemons
License: LGPL 2.1
URL: http://wildfly.org
Source0: http://download.jboss.org/wildfly/%{version}/%{name}-%{version}.tar.gz
ExclusiveArch: x86_64 i686
ExclusiveOS: Linux
%{?systemd_requires}
Requires: systemd
Requires: shadow-utils
Requires: java >= 1.8.0
Requires: /etc/init.d/functions
Provides: %{name}
%description
WildFly Application Server packaged from the binary distribution.
%prep
%setup -q -n %{name}-%{version}
%install
mkdir -p %{buildroot}/opt/%{name}
mkdir -p %{buildroot}/var/log/%{name}
mkdir -p %{buildroot}/var/run/%{name}
cp -R . %{buildroot}/opt/%{name}
%pre
getent group %{name} >/dev/null || groupadd -r %{name}
getent passwd %{name} >/dev/null || \
useradd -r -g %{name} -d /opt/%{name} -s /sbin/nologin %{name}
%post
alternatives --install /etc/alternatives/%{name} %{name} /opt/%{name} 100
%systemd_post %{name}.service
%postun
alternatives --remove %{name} /opt/%{name}
%systemd_postun %{name}.service
userdel %{name}
%files
%defattr(-,root,root,0755)
%dir /opt/%{name}
/opt/%{name}/appclient
/opt/%{name}/bin
/opt/%{name}/docs
/opt/%{name}/domain
/opt/%{name}/jboss-modules.jar
/opt/%{name}/modules
%attr(-,%{name},%{name}) /opt/%{name}/standalone
/opt/%{name}/welcome-content
%dir /var/log/%{name}
%dir /var/run/%{name}
%doc /opt/%{name}/copyright.txt
%doc /opt/%{name}/LICENSE.txt
%doc /opt/%{name}/README.txt

How to replace MacPort's libiconv with Mac's default 64-bit version?

MacPorts installed "libiconv #1.14_0+universal" as a dependency on my system. This happens to be a 32-bit flavor and it started causing issue when I tried to compile a voice recognition software called Simon Listens.
While googling I found out that that Mac actually ships with a 64-bit flavor of libiconv by default and I was able to locate the said files on my system:
$ find /usr/lib -name libiconv*
/usr/lib/libiconv.2.4.0.dylib
/usr/lib/libiconv.2.dylib
/usr/lib/libiconv.dylib
In order to use the system library, the quickest way I could think of was to uninstall MacPort's version of libiconv so that the system's library would end up getting selected as a fallback as it has to present (my guess) somewhere downstairs on the PATH already.
But that failed due to dependecies:
$ sudo port uninstall libiconv #1.14_0+universal
Unable to uninstall libiconv #1.14_0+universal, the following ports depend on it:
...
So now my question is how can I tell MacPort to replace its dependency graph to point to and use the library already on my system?
Another approach to avoid MacPorts libiconv issues would be to build simon against a fresh MacPorts system plus the necessary packages such as cyrus-sasl2, zlib, portaudio and kdesdk4 in a custom location, e. g. /opt/macports-simon.
The following code worked for me on my machine running Mac OS X 10.6.8:
# compile simon on Mac OS X 10.6.8 using MacPorts for the installation of zlib, portaudio and kdesdk4
# http://www.simon-listens.org
# http://sourceforge.net/projects/speech2text/
# get a root shell
sudo -H -i
# prevent idle sleep
pmset -a force sleep 0 displaysleep 0 disksleep 0
mv -i /opt/local /opt/local-off
mv -i /usr/local /usr/local-off
cd /tmp
mkdir buildsimon || exit 1
cd buildsimon || exit 1
# create custom /opt/macports-simon to install zlib, portaudio and kdesdk4
# cf. http://guide.macports.org/#installing.macports.source.multiple
MP_PREFIX='/opt/macports-simon'
unset PATH
export PATH='/bin:/sbin:/usr/bin:/usr/sbin'
curl -L -O https://distfiles.macports.org/MacPorts/MacPorts-2.0.4.tar.bz2
tar -xjf MacPorts-2.0.4.tar.bz2
cd MacPorts-2.0.4 || exit 1
./configure --prefix="${MP_PREFIX}" --with-applications-dir="${MP_PREFIX}/Applications"
make
make install
cd /tmp/buildsimon
unset PATH
export PATH="${MP_PREFIX}/bin:/bin:/sbin:/usr/bin:/usr/sbin"
# get the Portfiles and update the system
port -v selfupdate
# install cyrus-sasl2
port -f uninstall cyrus-sasl2
port clean --all cyrus-sasl2
port extract cyrus-sasl2
cd "$(port dir cyrus-sasl2)"/work/cyrus-sasl-2.1.23
printf '%s\n' H '/\(darwin\[15\]\)/s//\1./g' wq | sudo ed -s config/ltconfig
printf '%s\n' H '/\(darwin\[15\]\)/s//\1./g' wq | sudo ed -s saslauthd/config/ltconfig
cd /tmp/buildsimon
port -f -s install cyrus-sasl2
otool -L /opt/macports-simon/lib/libsasl2.dylib
port -f install zlib
port -f install portaudio
port -f install kdesdk4
port installed zlib portaudio kdesdk4 cyrus-sasl2
# enable dbus with launchd
# http://www.freedesktop.org/wiki/Software/dbus
# open -e dbus-1.5.8/README.launchd
launchctl load -w /Library/LaunchDaemons/org.freedesktop.dbus-system.plist
launchctl load -w /Library/LaunchAgents/org.freedesktop.dbus-session.plist
sudo -u _mysql mysql_install_db5
sudo port load mysql5-server
# todo: how to configure simon to use /opt/macports-simon directly?
ln -isv "${MP_PREFIX}" /usr/local
cd /tmp/buildsimon
# http://sourceforge.net/projects/speech2text/
curl -L -O http://netcologne.dl.sourceforge.net/project/speech2text/simon/0.3.0/simon-0.3.0.tar.bz2
tar -xjf simon-0.3.0.tar.bz2
cd simon-0.3.0 || exit 1
# Note that /usr/local got symlinked to "${MP_PREFIX}" above!
unset PATH
export PATH='/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin'
# the following commands are taken from simon-0.3.0/build.sh
mkdir build 2> /dev/null
cd build || exit 1
cmake -DCMAKE_INSTALL_PREFIX=`kde4-config --prefix` ..
# append ${MP_PREFIX}/lib/libiconv.dylib to gcc command in link.txt file
printf '%s\n' H '/\/usr\/bin\/gcc/s|\(.*\)|\1 '"${MP_PREFIX}"'/lib/libiconv.dylib|' wq |
ed -s julius/julius/CMakeFiles/juliusexe.dir/link.txt
# replace gcc option ' -bundle ' with ' -dynamiclib '
egrep -Ilsr -Z -e ' -bundle ' . |
xargs -0 -n 1 /bin/sh -c 'printf "%s\n" H "g/ -bundle /s// -dynamiclib /g" wq | /bin/ed -s "${1}"' argv0
make
touch ./julius/gramtools/mkdfa/mkfa-1.44-flex/*
make
make install
# ldconfig # not needed on Mac OS X
kbuildsycoca4
echo -e "**** Build completed ****\n\nThe executable file \"simon\" is now ready and has been installed.\n\nIssue \"simon\" to start it."
unset PATH
export PATH="${MP_PREFIX}/bin:/bin:/sbin:/usr/bin:/usr/sbin"
otool -L "${MP_PREFIX}/bin/simon"
simon
mv -i /opt/local-off /opt/local
mv -i /usr/local-off /usr/local

Resources