I am executing the following command:
cap staging assets:precompile
How do I access the variable env in my Capistrano deploy.rb?
where env is equal to 'staging' in this case?
You can use fetch(:stage) or #{stage} in a string for example like this:
task :sunspot_stop do
run "cd #{current_path} && bundle exec rake sunspot:solr:stop RAILS_ENV=#{stage}"
end
Related
EDIT
I want to pass the last git short hash into my React app build with the following command:
git log -1 --pretty=%h
Thus in my Dockerfile I want something such as:
ARG REACT_APP_GIT_SHORTHASH
RUN git clone https://token#github.com/paulywill/repo.git && cd repo && export REACT_APP_GIT_SHORTHASH=`git log -1 --pretty=%h`
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
In my Github Actions build I'm getting the following:
Step 6/14 : ARG REACT_APP_GIT_SHORTHASH
---> Running in f45f530d5c76
Removing intermediate container f45f530d5c76
---> 87a91c010aaf
Step 7/14 : RUN git clone https://***#github.com/paulywill/repo.git && cd repo && export REACT_APP_GIT_SHORTHASH=$(git log -1 --pretty=%h)
---> Running in b8a8fa3cd703
Cloning into 'repo'...
Removing intermediate container b8a8fa3cd703
---> 5bbf3a76b928
Step 8/14 : ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
---> Running in f624f2e59dc6
Removing intermediate container f624f2e59dc6
---> d15c3c276062
Are these command even visible or able to pass values if they're different intermediate containers?
ROM ubuntu:20.04
ARG REACT_APP_GIT_SHORTHASH
RUN apt-get update -y
RUN apt-get install git -y
RUN git clone https://github.com/pooya-mohammadi/deep_utils.git
WORKDIR deep_utils
RUN REACT_APP_GIT_SHORTHASH=`git log -1 --pretty=%h`
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
Change deep_utils with your repo name. I found the cd <directory> to be problematic.
As per a previous answer and thanks in part for #david-maze for pointing me in the right direction, I can easily grab the git short hash before the docker build.
.github/deploy.yml
...
- name: Set outputs
id: vars
run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
- name: Check outputs
run: echo ${{ steps.vars.outputs.sha_short }}
...
docker build --build-arg REACT_APP_GIT_SHORTHASH=${{ steps.vars.outputs.sha_short }} -t $ECR_REPOSITORY_CLIENT .
Dockerfile
FROM node:16-alpine
ARG VERSION
ENV VERSION $VERSION
ARG REACT_APP_GIT_SHORTHASH
ENV REACT_APP_GIT_SHORTHASH $REACT_APP_GIT_SHORTHASH
...
I have a database for my app and I want to create it in run time docker
I have a file CreateDB.sh and it creates all the tables and stored procedure that I want.
I tried this :
FROM mcr.microsoft.com/mssql/server
ENV ACCEPT_EULA=Y \
SA_PASSWORD=qwe123QWE
USER root
RUN mkdir /home/db
COPY ./db /home/db
RUN chmod +x /home/db/DbScriptLinux.sh
WORKDIR /home/db/
CMD ["/bin/bash", "/home/db/DbScriptLinux.sh"]
but it returns an error :
LoginTimeout
is there any way to run my script after all services (sql-server) start?
You can use an if statement, for example, RUN if [[ -z "$arg" ]] ; then echo Argument not provided ; else echo Argument is $arg ; fi
Another way would be to use command1 && command2 so if the command 1 is successfull, then command 2 would run afterwards.
Your last line CMD ["/bin/bash", "/home/db/DbScriptLinux.sh"] if this is to start the database every time you start the container, as your default command to run, then should be alright, otherwise it would be better to use the RUN command.
I am trying to do the following tutorial:
https://itnext.io/docker-mongodb-authentication-kubernetes-node-js-75ff995151b6
However, in there, they use raw values for the mongo init.js file that is placed within docker-entrypoint-initdb.d folder.
I would like to use environment variables that come from my CI/CD system (Gitlab). Does anyone know how to pass environment variables to the init.js file? I have tried several things like for example use init.sh instead for the shell but without any success.
If I run manually the init shell version, I can have it working because I call mongo with --eval and pass the values, however, the docker-entrypoint-blabla is called automatically, so I do not have control of how this is called and I do not know what I could do for achieving what I want.
Thank you in advance and regards.
you can make use of a shell script to retrieve env variables and create the user.
initdb.d/init-mongo.sh
set -e
mongo <<EOF
use $MONGO_INITDB_DATABASE
db.createUser({
user: '$MONGO_INITDB_USER',
pwd: '$MONGO_INITDB_PWD',
roles: [{
role: 'readWrite',
db: '$MONGO_INITDB_DATABASE'
}]
})
EOF
docker-compose.yml
version: "3.7"
services:
mongodb:
container_name: "mongodb"
image: mongo:4.4
hostname: mongodb
restart: always
volumes:
- ./data/mongodb/mongod.conf:/etc/mongod.conf
- ./data/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
- ./data/mongodb/data/db/:/data/db/
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=root
- MONGO_INITDB_DATABASE=development
- MONGO_INITDB_USER=mongodb
- MONGO_INITDB_PWD=mongodb
ports:
- 27017:27017
command: [ "-f", "/etc/mongod.conf" ]
Now you can connect to development database using mongodb as user and password credentials.
Use shell script (e.g mongo-init.sh) to access variables. Can still run JavaScript code inside as below.
set -e
mongo <<EOF
use admin
db.createUser({
user: '$MONGO_ADMIN_USER',
pwd: '$MONGO_ADMIN_PASSWORD',
roles: [{
role: 'readWrite',
db: 'dummydb'
}]
})
EOF
Shebang line is not necessary at the beginning as this file will be sourced.
Until recently, I simply used a .sh shell script in the docker-entrypoint-initdb.d directory to access ENV variables, much like #Lazaro answer.
It is now possible to access environment variables from javascript files using process.env, provided the file is run with the newer mongosh instead of mongo, which is now deprecated.
However, according to the Docs (see 'Initializing a fresh instance'), mongosh is only used for .js files in docker-entrypoint-initdb.d if using version 6 or greater. I can confirm this is working using the mongo:6 image tag.
You can use envsubs.
If command not found : here. Install it on your runners host if you use shell runners, else, within the docker image used by the runner, or directly in your script.
(NB: Your link isn't free, so I can't adapt to your situation :p )
Example:
init.js.template:
console.log('$GREET $PEOPLE $PUNCTUATION')
console.log('Pipeline from $CI_COMMIT_BRANCH')
gitlab_ci.yml:
variables:
GREET: "hello"
PEOPLE: "world"
PUNCTUATION: "!"
# ...
script:
- (envsubst < path/to/init.js.template) > path/to/init.js
- cat path/to/init.js
Output:
$ (envsubst < init.js.template) > init.js
$ cat init.js
console.log('hello world !')
console.log('Pipeline from master')
At the end the answer is that you can use a .sh file instead of a .js file within the docker-entrypoint-initdb.d folder. Within the sh script, you can use directly environment variables. However, I could not do that at the beginning because I had a typo and environment variables were not created properly.
I prefer this method because it allows you to keep a normal .js file which you lint instead of embedding the .js file into a string.
Create a dockerfile like so:
FROM mongo:5.0.9
USER mongodb
WORKDIR /docker-entrypoint-initdb.d
COPY env_init_mongo.sh env_init_mongo.sh
WORKDIR /writing
COPY mongo_init.js mongo_init.js
WORKDIR /db/data
At the top of your mongo_init.js file, you can just define variables you need
db_name = DB_NAME
schema_version = SCHEMA_VERSION
and then in your env_init_mongo.sh file, you can just replace the strings you need with environment variables or add lines to the top of the file:
mongo_init="/writing/mongo_init.js"
sed "s/SCHEMA_VERSION/$SCHEMA_VERSION/g" -i $mongo_init
sed "s/DB_NAME/${MONGO_INITDB_DATABASE}/g" -i $mongo_init
sed "1s/^/use ${MONGO_INITDB_DATABASE}\n/" -i $mongo_init # add to top of file
mongo < $mongo_init
I'm trying to copy all my production database (that I have in Mongo) to my staging environment. So I'm trying to build a task. First I need to connect to production environment to be able to access to all my models in production (Model.all.each...) But I don't know how to reproduce the production environment. I know in console I can do 'export RAILS_ENV=heroku_production' but I don't know how to do it inside a Rake Task. This is what I'm trying for now but it does not work because I print Rails.env and it prints "development"...so I'm a bit lost
namespace :db do
namespace :sync_production_staging do
desc "Copy production database to staging"
task :staging => :environment do
system "export RAILS_ENV=heroku_production"
ap Rails.env
ap User.all
end
end
end
I have a script that copies my database from heroku to my local its a really strait forward process, I am sorry that this is PG and not mongo but I am sure that this should help
#lib/tasks/db.rake
namespace :db do
desc "Import most recent database dump"
task :import_from_prod => :environment do
puts 'heroku run pg:backups capture --app sushi-prod'
restore_backup 'sushi-prod'
end
def path_to_heroku
['/usr/local/heroku/bin/heroku', '/usr/local/bin/heroku'].detect {|path| File.exists?(path)}
end
def heroku(command, site)
`GEM_HOME='' BUNDLE_GEMFILE='' GEM_PATH='' RUBYOPT='' #{path_to_heroku} #{command} -a #{site}`
end
def restore_backup(site = 'sushi-prod')
dump_file = "#{Rails.root}/tmp/postgres.dump"
unless File.exists?(dump_file)
pgbackups_url = heroku('pg:backups public-url -q', site).chomp
puts "curl -o #{dump_file} #{pgbackups_url}"
system "curl -o #{dump_file} '#{pgbackups_url}'"
end
database_config = YAML.load(File.open("#{Rails.root}/config/database.yml")).with_indifferent_access
dev_db = database_config[Rails.env]
system "pg_restore -d #{dev_db[:database]} -c #{dump_file}".gsub(/\s+/,' ')
puts
puts "'rm #{dump_file}' to redownload postgres dump."
puts "Done!"
end
end
I'm trying to deploy code with Capistrano 3 to Ubuntu server from GIT repository, but I'm getting this error.
==========================================================================
Here is my Gemfile.
gem 'capistrano', '~> 3.1.0'
#//Use unicorn as the app server
gem 'unicorn'
#// Use Capistrano for deployment
group :development do
gem 'capistrano-rails'
gem 'capistrano-bundler'
gem 'capistrano-rbenv', "~> 2.0"
end
source 'https://rubygems.org'
==========================================================================
#deploy.rb
lock '3.1.0'
#// Define the name of the application
set :application, 'my_app'
#// Define where can Capistrano access the source repository
#// set :repo_url, 'https://github.com/[user name]/[application name].git'
set :scm, :git
set :repo_url, 'git#github.com:jaipratik/rw.git'
set :use_sudo, true
set :log_level, :debug
#// Define where to put your application code
set :deploy_to, "/var/www/my_app"
set :pty, true
set :format, :pretty
==========================================================================
#// production.rb
role :app, %w{ubuntu#{IP/Host}}
server '{IP/Host}', user: 'ubuntu', roles: %w{web app}
set :ssh_options, {
keys: %w(/Users/jay/.ssh/id_rsa),
forward_agent: false,
user: 'user'
# auth_methods: %w(password)
}
==========================================================================
result for $ bundle exec cap production deploy --trace
** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke deploy (first_time)
** Execute deploy
** Invoke deploy:starting (first_time)
** Execute deploy:starting
** Invoke deploy:check (first_time)
** Execute deploy:check
** Invoke git:check (first_time)
** Invoke git:wrapper (first_time)
** Execute git:wrapper
INFO[f8299d4f] Running /usr/bin/env mkdir -p /tmp/my_app/ on {IP/Host}
DEBUG[f8299d4f] Command: /usr/bin/env mkdir -p /tmp/my_app/
INFO[f8299d4f] Finished in 0.723 seconds with exit status 0 (successful).
DEBUGUploading /tmp/my_app/git-ssh.sh 0.0%
INFOUploading /tmp/my_app/git-ssh.sh 100.0%
INFO[b509dfb7] Running /usr/bin/env chmod +x /tmp/my_app/git-ssh.sh on {IP/Host}
DEBUG[b509dfb7] Command: /usr/bin/env chmod +x /tmp/my_app/git-ssh.sh
INFO[b509dfb7] Finished in 0.084 seconds with exit status 0 (successful).
** Execute git:check
DEBUG[9646aea0] Running /usr/bin/env git ls-remote git#github.com:jaipratik/rw.git on {IP/Host}
DEBUG[9646aea0] Command: ( GIT_ASKPASS=/bin/echo GIT_SSH=/tmp/my_app/git-ssh.sh /usr/bin/env git ls-remote git#github.com:jaipratik/rw.git )
DEBUG[9646aea0] c452c845bb80f72d3023557d2ea8f776950c659f
DEBUG[9646aea0]
DEBUG[9646aea0] HEAD
DEBUG[9646aea0]
DEBUG[9646aea0] c452c845bb80f72d3023557d2ea8f776950c659f
DEBUG[9646aea0]
DEBUG[9646aea0] refs/heads/master
DEBUG[9646aea0]
DEBUG[9646aea0] Finished in 0.940 seconds with exit status 0 (successful).
** Invoke deploy:check:directories (first_time)
** Execute deploy:check:directories
INFO[fa4b1f56] Running /usr/bin/env mkdir -pv /var/www/my_app/shared /var/www/my_app/releases on {IP/Host}
DEBUG[fa4b1f56] Command: /usr/bin/env mkdir -pv /var/www/my_app/shared /var/www/my_app/releases
INFO[fa4b1f56] Finished in 0.086 seconds with exit status 0 (successful).
** Invoke deploy:check:linked_dirs (first_time)
** Execute deploy:check:linked_dirs
** Invoke deploy:check:make_linked_dirs (first_time)
** Execute deploy:check:make_linked_dirs
** Invoke deploy:check:linked_files (first_time)
** Execute deploy:check:linked_files
** Invoke deploy:started (first_time)
** Execute deploy:started
** Invoke deploy:updating (first_time)
** Invoke deploy:new_release_path (first_time)
** Execute deploy:new_release_path
** Execute deploy:updating
** Invoke git:create_release (first_time)
** Invoke git:update (first_time)
** Invoke git:clone (first_time)
** Invoke git:wrapper
** Execute git:clone
DEBUG[fa77f295] Running /usr/bin/env [ -f /var/www/my_app/repo/HEAD ] on {IP/Host}
DEBUG[fa77f295] Command: [ -f /var/www/my_app/repo/HEAD ]
DEBUG[fa77f295] Finished in 0.081 seconds with exit status 1 (failed).
DEBUG[0cc407cc] Running /usr/bin/env if test ! -d /var/www/my_app; then echo "Directory does not exist '/var/www/my_app'" 1>&2; false; fi on {IP/Host}
DEBUG[0cc407cc] Command: if test ! -d /var/www/my_app; then echo "Directory does not exist '/var/www/my_app'" 1>&2; false; fi
DEBUG[0cc407cc] Finished in 0.075 seconds with exit status 0 (successful).
INFO[de52ca0e] Running /usr/bin/env git clone --mirror git#github.com:jaipratik/rw.git /var/www/my_app/repo on {IP/Host}`enter code here`
DEBUG[de52ca0e] Command: cd /var/www/my_app && ( GIT_ASKPASS=/bin/echo GIT_SSH=/tmp/my_app/git-ssh.sh /usr/bin/env git clone --mirror git#github.com:jaipratik/rw.git /var/www/my_app/repo )
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host {IP/Host}: git exit status: 1
git stdout: Nothing written
git stderr: Nothing written
/Users/jay/.rvm/gems/ruby-2.1.0/gems/sshkit-1.5.1/
Just write the command in console:
ssh-add
If that don't work, write below command in console:
ssh-add ~/.ssh/id_rsa
and then re-run the command (i.e. cap production deploy), It will work for sure.
Capistrano wasn't able to create the folder on ec2. Once I created the folder it worked like a charm.
So if you also have similar issues try creating the folder on ec2 and then execute the cap production deploy.
If all the above solutions did not work for you, try this. It works for me. Cheer.
ssh-add ~/.ssh/your_private_id_rsa
and run
eval `ssh-agent`
Ref: http://mjacobus.github.io/2015/08/20/solving-weird-capistrano-problems-with-ssh-authentication.html
So what are the permissions for /var/www/my_app in your remote machine ? Make sure they are owned by the same user that you specified in this config option:
set :user, "mydeployuser"
If not I believe Capistrano defaults to the user you are using to run the 'cap' command on the client machine. Make sure that it has permissions to modify/create /var/www/my_app
Since you have:
set :use_sudo, true
it's possible that sudo may not be setup as passwordless sudo like suggested in the Cap 3 docs
I had this issue when trying to deploy a Rails application to a server using Capistrano.
When I run the command cap deploy, I get the error:
SSHKit::runner::ExecuteError: Exception while executing as deploy
Here' how I fixed it:
The issue was that the server to which I was trying to deploy to did not have my SSH key authorized. I had to do the following:
Generate a new SSH key on my local machine:
ssh-keygen
Display the contents of the public file. In my case mine was id_rsa.pub:
cat ~/.ssh/your-public-file
Login to the server which you want to deploy the app to:
ssh your-username#server-ip-address
Paste the contents of the public file into the ~/.ssh/authorized_keys file on a new line.
That's all.
I hope this helps
I had the same problem. Turns out I forgot to add my Gitlab instance SSH key (the 'runner') to my server's ~/.ssh/authorized_keys file.
If you are setting up a new Ubuntu server, public key authentication may be switched off by default which you will have to switch on. See this answer