I've started to configure a CircleCI integration of my project. I've created the config file using the 2.0 standard. I've got it running smoothly, deploying and everything is working, except the phpunit tests that I've created. When I run the phpunit command, it returns an error (I believe he cannot connect to the database).
Unable to insert fixtures for "App\Test\TestCase\Model\Table\UsersTableTest" test case. SQLSTATE[HY000] [2002] No such file or directory in [***/vendor/cakephp/cakephp/src/TestSuite/Fixture/FixtureManager.php, line 356]
Here's my test database config:
'test' => [
'className' => 'Cake\Database\Connection',
'driver' => 'Cake\Database\Driver\Mysql',
'persistent' => false,
'host' => 'localhost',
// 'port' => '3306',
'database' => 'common_resources_test',
'username' => 'root',
'password' => 'root',
'encoding' => 'utf8',
'timezone' => 'UTC',
'cacheMetadata' => true,
'quoteIdentifiers' => false,
],
Here's my circleCI code:
version: 2
jobs:
build:
working_directory: *removed*
parallelism: 1
shell: /bin/bash --login
environment:
CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
DEBUG: true
docker:
- image: circleci/php:7.1-apache
steps:
- checkout
- run:
name: Install Maria DB
command: sudo apt-get install -y mariadb-server-10.1 mariadb-server-core-10.1 mariadb-client-10.1 mariadb-client-core-10.1
- run:
name: Initializing Mysql
working_directory: *removed*
command: 'sudo service mysql status || sudo service mysql restart; '
- run:
name: Set password
command: sudo mysql -u root -e "USE mysql; UPDATE user SET password=PASSWORD('root') WHERE User='root';FLUSH PRIVILEGES;"
- run:
name: Check settings
command: sudo mysql -u root -proot -e "USE mysql; SELECT host FROM user WHERE user = 'root';grant all privileges on *.* to root#'127.0.0.1' identified by 'root';FLUSH PRIVILEGES; SELECT host FROM user WHERE user = 'root';SELECT ##port;SELECT ##hostname;"
- run:
name: Restarting Mysql
working_directory: *removed*
command: 'sudo service mysql status || sudo service mysql restart; '
- run:
name: Create Mysql database and show current databases
command: sudo mysql -uroot -proot -e "create database common_resources_test;SHOW DATABASES"
- run:
name: Install required Libraries for PHP-GD
command: sudo apt-get update && sudo apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev
- run:
name: Install PHP Extensions (PHP-GD && PDO-MYSQL)
command: sudo docker-php-ext-install gd && sudo docker-php-ext-install pdo_mysql
- run:
name: Install Other required Libraries for PHP & Activating PHP-GD and PDO-MYSQL
command: sudo docker-php-ext-install -j$(nproc) iconv && sudo docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ && sudo docker-php-ext-install -j$(nproc) gd && sudo docker-php-ext-enable pdo_mysql
- run:
name: Create folder for Circle Artifacts and Circle Test Reports
working_directory: *removed*
command: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
- run:
name: Download Composer
command: curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin --filename=composer
- run:
name: Running Composer Install
working_directory: *removed*
command: composer install --no-interaction;
- run:
name: Download Yarn
working_directory: *removed*
command: curl -o- -L https://yarnpkg.com/install.sh | bash
- run:
name: Install Yarn
working_directory: *removed*
command: sudo yarn install
- restore_cache:
keys:
- v1-dep-{{ .Branch }}-
- v1-dep-pre-production-
- v1-dep-
- save_cache:
key: v1-dep-{{ .Branch }}-{{ epoch }}
paths:
- vendor/bundle
- ~/virtualenvs
- ~/.m2
- ~/.ivy2
- ~/.bundle
- ~/.go_workspace
- ~/.gradle
- ~/.cache/bower
- ~/.composer/cache
- ~/.yarn-cache
- run:
name: Creating folder for PHPUnit Tests
working_directory: *removed*
command: mkdir -p $CIRCLE_TEST_REPORTS/phpunit
- run:
name: Running PHPUnit Tests
working_directory: *removed*
command: ./vendor/bin/phpunit --configuration ./phpunit.xml.dist --log-junit $CIRCLE_TEST_REPORTS/phpunit/junit.xml
- store_test_results:
path: /tmp/circleci-test-results
- store_artifacts:
path: /tmp/circleci-artifacts
- store_artifacts:
path: /tmp/circleci-test-results
deployment:
docker:
- image: circleci/python:2.7-jessie
steps:
- run:
name: Install awsebcli
command: sudo pip install awsebcli
- run:
name: Deploy to S3
command: eb deploy --profile default
workflows:
version: 2
build_and_test:
jobs:
- build:
filters:
tags:
only: /.*/
- deployment:
requires:
- build
filters:
branches:
only: pre-production
Thanks for your time
I would handle the database the way it is in your config. You can use a second container for the database. Then set the DB creds with environment variables.
The config lines would look something like this:
- image: circleci/mariadb:10.1
environment:
MYSQL_DATABASE: "common_resources_test"
MYSQL_ROOT_PASSWORD: "root"
Founded a way to fix this problem. Basicly mysql root was not allowed to login using password. Had to set it up. Here's my app.php config:
'test_common_resources' => [
'className' => 'Cake\Database\Connection',
'driver' => 'Cake\Database\Driver\Mysql',
'persistent' => false,
'host' => '127.0.0.1',
'unix_socket' => '/var/run/mysqld/mysqld.sock',
'database' => 'common_resources_test',
'username' => 'root',
'password' => 'root',
'encoding' => 'utf8',
'timezone' => 'UTC',
'cacheMetadata' => true,
]
And here's my circleci config:
version: 2
jobs:
build:
working_directory: **removed**
parallelism: 1
shell: /bin/bash --login
environment:
CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
DEBUG: true
docker:
- image: circleci/php:7.1-node-browsers
steps:
- run:
name: Install Maria DB
command: sudo apt-get install -y mariadb-server-10.1 mariadb-server-core-10.1 mariadb-client-10.1 mariadb-client-core-10.1
- run:
name: Initializing Mysql
command: 'sudo service mysql status || sudo service mysql restart; '
- run:
name: Set password
command: sudo mysql -u root -e "USE mysql; UPDATE user SET password=PASSWORD('root') WHERE User='root';FLUSH PRIVILEGES;"
- run:
name: Check settings
command: sudo mysql -u root -proot -e "USE mysql; UPDATE user SET plugin='mysql_native_password' WHERE User='root'; SELECT User, Host, plugin FROM mysql.user WHERE user = 'root';grant all privileges on *.* to root#'127.0.0.1' identified by 'root';FLUSH PRIVILEGES; SELECT host FROM user WHERE user = 'root';SELECT ##port;SELECT ##hostname;"
- run:
name: Restarting Mysql
command: 'sudo service mysql status || sudo service mysql restart; '
- run:
name: Create Mysql database and show current databases
command: sudo mysql -uroot -proot -e "create database common_resources_test;SHOW DATABASES"
- run:
name: Install required Libraries for PHP-GD
command: sudo apt-get update && sudo apt-get install -y libfreetype6-dev libjpeg62-turbo-dev libpng-dev
- run:
name: Install PHP Extensions (PHP-GD && PDO-MYSQL)
command: sudo docker-php-ext-install gd && sudo docker-php-ext-install pdo_mysql
- run:
name: Install Other required Libraries for PHP & Activating PHP-GD and PDO-MYSQL
command: sudo docker-php-ext-install -j$(nproc) iconv && sudo docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ && sudo docker-php-ext-install -j$(nproc) gd && sudo docker-php-ext-enable pdo_mysql
- run:
name: Create folder for Circle Artifacts and Circle Test Reports
command: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
- run:
name: Download Composer
command: curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin --filename=composer
- run:
name: Download Yarn
command: curl -o- -L https://yarnpkg.com/install.sh | bash
- run:
name: Install Yarn
command: sudo yarn install
- restore_cache:
keys:
- v1-dep-{{ .Branch }}-
- v1-dep-pre-production-
- v1-dep-
- checkout
- run:
name: Running Composer Install
command: composer install --no-interaction;
- save_cache:
key: v1-dep-{{ .Branch }}-{{ epoch }}
paths:
- vendor/bundle
- ~/virtualenvs
- ~/.m2
- ~/.ivy2
- ~/.bundle
- ~/.go_workspace
- ~/.gradle
- ~/.cache/bower
- ~/.composer/cache
- ~/.yarn-cache
- run:
name: Creating folder for PHPUnit Tests
command: mkdir -p $CIRCLE_TEST_REPORTS/phpunit
- run:
name: Checking Mysql Status
command: 'sudo service mysql status'
- run:
name: Running PHPUnit Tests
command: ./vendor/bin/phpunit --configuration ./phpunit.xml.dist --log-junit $CIRCLE_TEST_REPORTS/phpunit/junit.xml
- store_test_results:
path: /tmp/circleci-test-results
- store_artifacts:
path: /tmp/circleci-artifacts
- store_artifacts:
path: /tmp/circleci-test-results
deployment:
docker:
- image: circleci/python:2.7-jessie
steps:
- run:
name: Install awsebcli
command: sudo pip install awsebcli
- run:
name: Deploy to S3
command: eb deploy --profile default
workflows:
version: 2
build_and_test:
jobs:
- build:
filters:
tags:
only: /.*/
- deployment:
requires:
- build
filters:
branches:
only: pre-production
Related
I have two containers that run via docker-compose, my docker-compose looks like this
version: "3.7"
services:
mssql:
build: ./Db
ports:
- 1433:1433
planning-poker:
build: .
restart: always
env_file:
- env.list
ports:
- 80:8080
depends_on:
- mssql
Dockerfile go-app:
FROM golang:latest
RUN apt-get -y update && \
apt-get install -y net-tools && \
apt-get install -y iputils-ping && \
apt-get install -y telnet
ADD . /go/src/planning-poker
WORKDIR /go/src/planning-poker
RUN go build -o main .
ENTRYPOINT ["./main"]
Dockerfile mssql:
FROM mcr.microsoft.com/mssql/server
ENV ACCEPT_EULA=Y
ENV MSSQL_SA_PASSWORD=Yukon_900
EXPOSE 1433
USER root
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN chmod +x ./run-initialization.sh
USER mssql
CMD /bin/bash ./entrypoint.sh
I am using database initialization scripts:
for i in {1..50};
do
/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P Yukon_900 -d master -i SQL_PlanningPoker.sql
if [ $? -eq 0 ]
then
echo "SQL_PlanningPoker.sql completed"
break
else
echo "not ready yet..."
sleep 1
fi
done
So is my entrypoint:
./run-initialization.sh & /opt/mssql/bin/sqlservr
The problem is that I can’t connect to mssql from the container with the golang application in any way, the connection passes from the host, I tried to connect via telnet to mssql from the go-app container on localhost 1433, 127.0.0.1 1433, 0.0.0.0 1433 but always I get an error that the connection is either reset or telnet cannot resolve the addresses.
MyProject: https://github.com/philomela/PlanningPoker/tree/master - master branch.
What am I doing wrong? thank you in advance!
try adding network and kepp all services run in same network
networks:
my-network:
services:
mssql:
build: ./Db
ports:
- 1433:1433
networks:
- my-network
planning-poker:
build: .
restart: always
env_file:
- env.list
ports:
- 80:8080
depends_on:
- mssql
networks:
- my-network
Also there is a possibility to check service is healthy with some heath check rather than just depends_on because docker may be up but SQL server would take some time to be up and running
https://docs.docker.com/engine/reference/builder/#healthcheck
The issue is resolved, I suffered for several days, but did not pay attention to port forwarding in my application, now everything is working stably!
I am new to docker and I am trying to deploy docker container on my local machine. However, Codeigniter3 cannot connect with the SQL server database.
Dockerfile:
FROM php:7.4.23-apache
WORKDIR /var/www/html
COPY src/ .
USER root
RUN chmod 777 ./system
RUN chown -R www-data:www-data /var/www/html
RUN a2enmod rewrite
ENV ACCEPT_EULA=Y
RUN apt-get update && apt-get install -y nano wget curl gnupg2
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list >
/etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN apt-get -y --no-install-recommends install msodbcsql17 unixodbc-dev
RUN pecl install sqlsrv
RUN pecl install pdo_sqlsrv
RUN docker-php-ext-enable sqlsrv pdo_sqlsrv
RUN echo "extension=pdo_sqlsrv.so" >> `php --ini | grep "Scan for additional .ini files" | sed -e "s|.*:\s*||"`/30-pdo_sqlsrv.ini
RUN echo "extension=sqlsrv.so" >> `php --ini | grep "Scan for additional .ini files" |
sed -e "s|.*:\s*||"`/30-sqlsrv.ini
RUN service apache2 restart
docker-compose.yaml
version: '3.9'
services:
app:
build: .
container_name: ci-mssql-db-app
image: ci-mssql-db-app:1.0.0
ports:
- 80:80
depends_on:
- db
links:
- db
db:
container_name: mssql
image: mcr.microsoft.com/mssql/server:2019-CU12-ubuntu-20.04
restart: always
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: "Testing123"
MSSQL_PID: "Developer"
Codeigniter3 database.php file configuration:
$db['default'] = array(
'dsn' => '',
'hostname' => 'db:1433',
'username' => 'admin',
'password' => 'admin',
'database' => 'ciapp',
'dbdriver' => 'sqlsrv',
'dbprefix' => '',
'pconnect' => FALSE,
'db_debug' => (ENVIRONMENT !== 'production'),
'cache_on' => FALSE,
'cachedir' => '',
'char_set' => 'utf8',
'dbcollat' => 'utf8_general_ci',
'swap_pre' => '',
'encrypt' => FALSE,
'compress' => FALSE,
'stricton' => FALSE,
'failover' => array(),
'save_queries' => TRUE
);
Here, I have set hostname = 'db:1433' as it points to the service name so that it will get the ip address of db service automatically. And, SQL server runs on 1433 port by default.
After running the docker command
docker-compose up --build
I am getting this error,
Unable to connect to your database server using the provided settings.
Error Image Link
I am struggling to solve it for almost two weeks. Please help me in this regard.
I am new to gitlab CI and I need help to configure gitlab-ci to run feature tests with Firefox (Ruby 2.7 / Rails 6 / Rspec / Capybara).
All is ok except feature tests.
I think I have to configure something with Firefox, or maybe install something.
Thanks for your help !
Error message when running tests :
Failure/Error: visit "/meth/methodologies/#{#meth.id}/edit"
Selenium::WebDriver::Error::WebDriverError:
Could not find Firefox binary (os=linux). Make sure Firefox is installed or set the path manually with Selenium::WebDriver::Firefox::Binary.path=
File .gitlab-ci.yml
stages:
- build
- test
- deploy
image: ruby:2.7.1
cache: &global_cache
key: ${CI_COMMIT_REF_SLUG}
paths:
- apt-cache/
- vendor/ruby
- node_modules
- .yarn-cache
policy: pull-push
.base:
cache:
# inherit all global cache settings
<<: *global_cache
before_script:
- gem install bundler --no-document
- bundle install --jobs $(nproc) "${FLAGS[#]}" --path=vendor
.base_db:
# extends: .base
services:
- name: mysql:8.0.21
command: ['--default-authentication-plugin=mysql_native_password']
- name: selenium/standalone-firefox
alias: selenium
variables:
MYSQL_ROOT_PASSWORD: xxxx
DB_USERNAME: xxxx
DB_PASSWORD: xxxx
DB_HOST: mysql
RAILS_ENV: test
DISABLE_SPRING: 1
BUNDLE_PATH: vendor/bundle
cache:
# inherit all global cache settings
<<: *global_cache
before_script:
# install yarn & dependencies
- export APT_CACHE_DIR=`pwd`/apt-cache && mkdir -pv $APT_CACHE_DIR
- wget -q -O - https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
- echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list
- apt-get update -qq && apt-get -o dir::cache::archives="$APT_CACHE_DIR" install -y yarn
- yarn config set cache-folder .yarn-cache
- yarn install
- gem install bundler --no-document
- bundle install --jobs $(nproc) "${FLAGS[#]}" --path=vendor/ruby
# Setup test database
- cp config/database.ci.yml config/database.yml
- RAILS_ENV=test bundle exec rails db:create db:migrate
rubocop:
extends: .base
stage: build
# cache:
# policy: pull-push
script:
- bundle exec rubocop app --fail-level W
rspec:
extends: .base_db
stage: test
script:
- bundle exec rspec -t ~type:feature
artifacts:
paths:
- coverage/
features:
extends: .base_db
stage: test
script:
- bundle exec rspec
# services:
# - name: selenium/standalone-firefox
# alias: selenium
# artifacts:
# paths:
# - coverage/
pages:
stage: deploy
dependencies:
- rspec
script:
- mv coverage/ public/
artifacts:
paths:
- public
expire_in: 30 days
# only:
# - master
EDIT :
I added the installation of firefox.
Tests that use js still don't work. The error is now as follows :
Failure/Error: visit "/meth/methodologies/#{#meth.id}/edit"
Selenium::WebDriver::Error::UnknownError:
Process unexpectedly closed with status 1
You have to use your browser with headless option
rails_helper.rb
Capybara.register_driver :headless_firefox do |app|
browser_options = Selenium::WebDriver::Firefox::Options.new()
browser_options.args << '--headless'
Capybara::Selenium::Driver.new(
app,
browser: :firefox,
options: browser_options
)
end
Capybara.javascript_driver = :headless_firefox
Capybara.configure do |config|
config.default_max_wait_time = 10 # seconds
config.default_driver = :headless_firefox
end
I met a problem when trying to apply CI/CD into our project using Github Action. The server has the firewall to enable access for a listed ip only.
I have found a method by using Github meta api https://api.github.com/meta but they denied to apply.
Is there any other way to apply this?
Our current ci.yml
name: remote ssh
on:
push:
branches: [ master ]
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: execute ssh command via using private key
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.REMOTE_HOST }}
username: ${{ secrets.REMOTE_USER }}
key: ${{ secrets.CICD_SSH_KEY }}
port: ${{ secrets.PORT }}
script:
pwd
In my case, I use an OpenVPN to access to the server.
About security. I think you should not load file VPN config to Git.
This is my config file.
name: remote ssh command to deploy
on:
push:
branches: [ master ]
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Install Open VPN
run: |
sudo apt-get install openvpn
echo "${{ secrets.VPN_FILE }}" > .github/vpn/config.ovpn
- name: Connect VPN
uses: golfzaptw/action-connect-ovpn#master
id: connect_vpn
with:
PING_URL: ${{ secrets.REMOTE_HOST }}
FILE_OVPN: '.github/vpn/config.ovpn'
env:
CA_CRT: ${{ secrets.CA_CRT}}
USER_CRT: ${{ secrets.USER_CRT }}
USER_KEY: ${{ secrets.USER_KEY }}
- name: Check Connect VPN
run: echo ${{ steps.connect_vpn.outputs.STATUS }}
- name: Execute ssh command via using private key
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.REMOTE_HOST }}
username: ${{ secrets.REMOTE_USER }}
key: ${{ secrets.CICD_SSH_KEY }}
port: ${{ secrets.PORT }}
script: |
pwd
cd ${{ secrets.REMOTE_TARGET }}
git pull
- name: kill vpn
if: always()
run: sudo killall openvpn
Follow https://github.com/marketplace/actions/connect-vpn#Example-prepare-file-.ovpn:
Copy data inside tag to encode base64 after that save to secret env github actions
Remove tag and replace to ca ca.crt cert user.crt key user.key
Aside OpenVPN, you can use Cloudflare WARP 1.1.1.1, its easy to use and no need for running any server or any kind of log in.
just make a job
name: remote ssh command to deploy
on:
push:
branches: [ master ]
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Check Connect VPN
run: |
curl https://pkg.cloudflareclient.com/pubkey.gpg | sudo gpg --yes --dearmor --output /usr/share/keyrings/cloudflare-warp-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/cloudflare-warp-archive-keyring.gpg] https://pkg.cloudflareclient.com/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/cloudflare-client.list
sudo apt update
sudo apt install cloudflare-warp
warp-cli --accept-tos register
warp-cli --accept-tos connect
put this there. Boom you're ready to go and surf anywhere.
Note:
the 1st line is to add the Cloudflare pkg host to apt host list because apt only use microsoft hosted pkg only, and it's not there. 2nd line for same reason.
5th line to register the service. --accept-tos part is for accepting TOS which needed to be done by human input if omitted
6th line Runs the service.
Full documentation here:
https://pkg.cloudflareclient.com/install
https://developers.cloudflare.com/warp-client/get-started/linux/
---
- name: install apache2, sqlite3, git pn remote server
hosts: host01
sudo: yes
tasks:
- name: Install list of packages
action: apt pkg={{item}} state=installed
with_items:
- apache2
- sqlite3
- git
INVENTORY FILE NAME: myhosts
$cat myhosts
[group1]
host01 ansible_ssh_user=ubuntu
COMMAND USED: ansible-playbook -i myhosts test.yml
ERROR is below one, I don't know what went wrong someone help me in this.
ERROR: Syntax Error while loading YAML script, test.yml
Note: The error may actually appear before this position: line 7, column 12
- name: Install list of packages
action: apt pkg={{item}} state=installed
^
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
Indendation seems wrong at it should be two spaces character by level so try with something like this regarding indentation issue.
---
- name: install apache2, sqlite3, git pn remote server
hosts: host01
sudo: yes
tasks:
- name: Install list of packages
action: apt pkg={{item}} state=installed
with_items:
- apache2
- sqlite3
- git
---
- hosts: all
become: yes
name: install apache2, sqlite3, git pn remote server
tasks:
- name: Install list of packages
action: apt pkg={{item}} state=installed
with_items:
- apache2
- sqlite3
- git
this works for me...
Given command as
---
- name: install apache2, sqlite3, git pn remote server
hosts: host01
become: yes
tasks:
- name: Install list of packages
action: apt pkg={{item}} state=installed
with_items:
- apache2
- sqlite3
- git
below error
ansible-playbook -i myhosts test.yml -b
PLAY [install apache2, sqlite3, git pn remote server] *************************
GATHERING FACTS ***************************************************************
fatal: [host01] => SSH Error: ssh: connect to host host01 port 22: Connection refused
while connecting to 172.17.3.177:22
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
TASK: [Install list of packages] **********************************************
FATAL: no hosts matched or all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit #/home/scrapbook/test.retry
host01 : ok=0 changed=0 unreachable=1 failed=0