Testing Linux Packages with Docker

During my apprenticeship at ownCloud  i had the opportunity to write a proof of concept for installation testing of ownCloud Linux packages for various distributions with the help of docker.

Requirements:

The goal was to create an testframework where each test could be ran individually and also locally on the developers machine. During my apprenticeship i used docker a lot and learned it is an awesome software  to do things quickly without the hassle of booting up VM’s. To get an ownCloud instance running all i needed was:

docker compose up

with the ownCloud Image from: https://github.com/owncloud-docker/server

So naturally  i went with docker for the testing framework i wanted to create.

Getting dirty:

I wanted to test the installation of ownCloud packages on four different Distros: ubuntu:16.04, CentOS7, Debian9 and openSuse Leap 42.3 seemed feasible.

You can find the projects Source Code right here:

https://github.com/Kawohl/abschlussprojekt

The general idea  was to fire up an docker container for each Distro and run the installation for the ownCloud Linux Packages in each container individually.

I wrote Scripts to do ownCloud Package installations for each distro. These Scripts are mounted into the corresponding containers. After the installation, the Scripts write an output to a log file on the host.

All in all, I had only one 35 hours to complete the scripts and write an elaborate documentation for school.  Unfortunately, i had to write in German because of school regulations.  Abschlussprojekt

[action-button color=”blue” title=”Download the Documentation here:” subtitle=”PDF” url=”https://jonaka.de/wp-content/uploads/2018/05/Abschlussprojekt_w2fjke.pdf”]

Why do installation Tests?

The ownCloud Server is a PHP application running typically on Linux with an Apache web server and a MYSQL/Mariadb database. In some cases, administrators prefer Postgres as a Database and for testing purposes with a minimal amount of Data they might want to use sqlite as a backend.  Different distros offer by default different versions of the Software dependencies, meaning different php Versions, different versions of databases, and even the default apache configuration may differ from distro to distro. For everything to work nicely together ownCloud needs a variety of additional php extensions :

PHP Extensions

Name Description
Ctype For character type checking
cURL Used for aspects of HTTP user authentication
DOM For operating on XML documents through the DOM API
GD For creating and manipulating image files in a variety of different image formats, including GIF, PNG, JPEG, WBMP, and XPM.
iconv For working with the iconv character set conversion facility.
intl Increases language translation performance and fixes sorting of non-ASCII characters
JSON For working with the JSON data-interchange format.
libxml This is required for the _DOM_, _libxml_, _SimpleXML_, and _XMLWriter_ extensions to work. It requires that libxml2, version 2.7.0 or higher, is installed.
Multibyte String For working with multibyte character encoding schemes.
OpenSSL For symmetric and asymmetric encryption and decryption, PBKDF2, PKCS7, PKCS12, X509 and other crypto operations.
PDO This is required for the pdo_msql function to work.
Phar For working with PHP Archives (.phar files).
POSIX For working with UNIX POSIX functionality.
SimpleXML For working with XML files as objects.
XMLWriter For generating streams or files of XML data.
Zip For reading and writing ZIP compressed archives and the files inside them.
Zlib For reading and writing gzip (.gz) compressed files.

And for the database integration:

Database Extensions

Name Description
pdo_mysql For working with MySQL & MariaDB.
pgsql For working with PostgreSQL. It requires PostgreSQL 9.0 or above.
sqlite For working with SQLite. It requires SQLite 3 or above. This is, usually, not recommended, for performance reasons.

 

That said: Whats the Problem?

Usually, sysadmins would just install the packages provided by the distro for all dependencies needed to get ownCloud running.

Lets have a closer look:

To get the ownCloud Server running on an ubuntu  16.04 we would need following packages:

Package Installation, PHP, and Apache configuration.

apt install -y apache2 mariadb-server libapache2-mod-php7.0 php7.0-gd php7.0-json php7.0-mysql php7.0-curl php7.0-intl php7.0-mcrypt php-imagick php7.0-zip php7.0-xml php7.0-mbstring

Then we start the Apache-Webserver:

service apache2 start

ownCloud installation and Apache Configuration

Then we get the ownCloud package:

    • Get the Release key of the ownCloud Release:
wget -nv https://download.owncloud.org/download/repositories/production/Ubuntu_16.04/Release.key -O Release.key< apt-key add - < Release.key
    • Add ownCloud to the repository list
echo 'deb http://download.owncloud.org/download/repositories/production/Ubuntu_16.04/ /' > /etc/apt/sources.list.d/owncloud.list
    • Update the repository list:
apt update
    • install the ownCloud package:
apt install -y owncloud-files
    • add the ownCloud vhost config to the apache webserver. So we write to the file:
      /etc/apache2/sites-available/owncloud.conf

<Directory /var/www/owncloud/>
Options +FollowSymlinks
AllowOverride All

<IfModule mod_dav.c>
Dav off
</IfModule>

SetEnv HOME /var/www/owncloud
SetEnv HTTP_HOME /var/www/owncloud

</Directory>

    • Create a Symbolic Link to enable the config:
ln -s /etc/apache2/sites-available/owncloud.conf /etc/apache2/sites-enabled/owncloud.conf
    • Enable phpmodules
a2enmod rewrite
    • Restart apache
service apache2 start

Database Configuration:

Create Database:

mysql -u root -e "create database owncloud"

Create ownCloud user:

mysql -u root -e "create user 'ownclouduser'@localhost identified by 'admin'"

grant the ownCloud user rights

mysql -u root -e "GRANT ALL PRIVILEGES ON owncloud. * TO 'ownclouduser'@'localhost'"

ownCloud Installation:

And finally we use the ownCloud occ command to finalize the installation

sudo -u www-data php /var/www/owncloud/occ maintenance:install --database "mysql" --database-name "owncloud" --database-user "ownclouduser" --database-pass "admin" --admin-user "admin" --admin-pass "admin"

Lets check if we succeeded:

First we need to install some additional packages:

apt install jq lynx

lynx is a textbased web browser that allows us to browse the ownCloud installation for the status.php within the docker container.  When ownCloud is correctly installed the status.php should show something like this:

{"installed":true,"maintenance":false,"needsDbUpgrade":false,"version":"10.0.6.1","versionstring":"10.0.6","edition":"Community","productname":"ownCloud"}

And then we run this command to check if ownCloud was installed successfully:

(echo "SUCCESS:$(lynx --dump localhost/owncloud/status.php| jq -r .versionstring ) installed! System $PRETTY_NAME" || echo "FAIL: Installation failed! System $PRETTY_NAME") >> /logs/server.install.log 2>&1

 

This results with a line in the server.install.log file : 

SUCCESS:10.0.4 installed! System Ubuntu 16.04.3 LTS

Summary package installation in ubuntu 16.04

The ownCloud package installation is pretty much forward:

Please Note: This is only for testing purposes and in no way an recommendation for deployments.

Docker:

To run the single installation process in a container we do:

docker run -ti --rm=true -v $(pwd)/server/ubuntu16.04/latestproduction/install.sh:/install.sh -v $(pwd)/logs/:/logs ubuntu:16.04 sh install.sh

In the root folder of the project

This starts the docker container and removes it when the script has run to save disk space. It mounts the install script for the latest ownCloud server for the ubuntu packages, runs it and outputs the results of the test to the server.install.log located in the root of the project.

The Folder structure looks like this in my Texteditor (Sublime)

jonaka.de-scripstructure

One Script to rule them all

In the root folder, there is a run.sh file that starts every container sequentially and starts the package installation for each and any distro.

Some remarks….

During writing these tests I ran into a couple of minor issues. Centos7 comes with php 5.4 by default. PHP 5.4 isn’t supported since 3rd of September 2015!

You can either compile the needed php or download it from some external repos:  Check out, for instance, Remi’s Repos:

https://blog.remirepo.net/post/2016/02/14/Install-PHP-7-on-CentOS-RHEL-Fedora

Dear openSuse!

I love your Distro and think you do awesome work. And I get it: different Distros have different package names: I can live with that: But:

apache2-mod_php7

Come on! Every other package name with php in it is with a hyphen and all of the sudden you add an underscore? That seems weird and adds IMHO unnecessary complexity.  I understand that this package is Apache related, but also a

zypper se php7

doesn’t give you the package name.  Searching for the package is an unnecessary p.i.t.a.. and has cost me some time.

tl;dr: I wrote a small little tool to test if packages install properly on different Linux Distros.

2 thoughts on “Testing Linux Packages with Docker”

  1. Pingback: HOW Dev ops brought our Projects up to speed - jonaka.de

  2. Pingback: Traefik as Reverse Proxy for your docker-compose setup! - jonaka.de

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.