Focker

Introduction

Focker is a FreeBSD image orchestration tool in the vein of Docker.

Table of Contents

• Introduction
• Table of Contents
• Installation
  • Installing the Python package from PyPi
  • Installing the Python package from GitHub
  • Setting up ZFS
  • Preparing base image
• Usage
  • focker command syntax
    • focker image|img|im|i
    • focker jail|j
    • focker volume|vol|v
    • focker compose|comp|c
  • Fockerfile syntax
  • focker-compose.yml syntax
    • Images
    • Jails
    • Volumes
    • Commands
• Further Reading
• Conclusion
• Links

Installation

In order to use Focker you need a ZFS pool available in your FreeBSD installation.

Installing the Python package from PyPi

Run:

pip install focker

Installing the Python package from GitHub

Run:

git clone https://github.com/sadaszewski/focker.git
cd focker/
python setup.py install

or (if you want an uninstaller):

git clone https://github.com/sadaszewski/focker.git
cd focker/
python setup.py sdist
pip install dist/focker-0.9.tgz

Setting up ZFS

Upon first execution of the focker command, Focker will automatically create the necessary directories and ZFS datasets. You just need to exclude the unlikely case that you are already using /focker in your filesystem hierarchy. The layout after initialization will look the following:

/focker
/focker/images
/focker/jails
/focker/volumes

images, jails, and volumes have corresponding ZFS datasets with canmount=off so that they serve as mountpoint anchors for child entries.

Preparing base image

To bootstrap the images system you need to install FreeBSD in jail mode to a ZFS dataset placed in /focker/images and provide two user-defined properties - focker:sha256 and focker:tags. One way to achieve this would be the following (using Bash shell):

TAGS="freebsd-latest freebsd-$(freebsd-version | cut -d'-' -f1)"
VERSION="FreeBSD $(freebsd-version)"
SHA256=$(echo -n ${VERSION} | sha256)
NAME=${SHA256:0:7}
zfs create -o focker:sha256=${SHA256} -o focker:tags="${TAGS}" zroot/focker/images/${NAME}
bsdinstall jail /focker/images/${NAME}
zfs set readonly=on zroot/focker/images/${NAME}
zfs snapshot zroot/focker/images/${NAME}@1

Usage

At this point, Focker is ready to use.

focker command syntax

The focker command is the single entrypoint to all of the Focker's functionality. The overview of its syntax is presented below as a tree where the focker command is the root, the first level of descendants represents the choice of Level 1 mode (image, jail, volume or compose), the second level - the Level 2 mode (dependent on L1 mode) and the final third level lists required and optional arguments specific to the given combination of L1/L2 modes.

focker
|- image|img|im|i
|  |- build|b
|  |  |- FOCKER_DIR
|  |  `- --tags|-t TAG [...TAG]
|  |- tag|t
|  |  |- REFERENCE
|  |  `- TAG [...TAG]
|  |- untag|u
|  |  `- TAG [...TAG]
|  |- list|ls|l
|  |  `- --full-sha256|-f
|  |- prune|p
|  `- remove|r
|     |- REFERENCE
|     `- --remove-dependents|-R
|- jail|j
|  |- create|c
|  |  |- IMAGE
|  |  |- --command|-c COMMAND (default: /bin/sh)
|  |  |- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
|  |  |- --mounts|-m FROM1:ON1 [...FROMN:ONN]
|  |  `- --hostname|-n HOSTNAME
|  |- start|s
|  |  `- REFERENCE
|  |- stop|S
|  |  `- REFERENCE
|  |- remove|r
|  |  `- REFERENCE
|  |- exec|e
|  |  |- REFERENCE
|  |  `- [...COMMAND]
|  |- oneshot|o
|  |  `- IMAGE
|  |  `- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
|  |  `- --mounts|-m FROM1:ON1 [...FROMN:ONN]
|  |  `- [...COMMAND]
|  |- list|ls|l
|  |  `- --full-sha256|-f
|  |- tag|t
|  |  |- REFERENCE
|  |  `- TAG [...TAG]
|  |- untag|u
|  |  `- TAG [...TAG]
|  `- prune|p
|     `- --force|-f
|- volume|vol|v
|  |- create
|  |  `- --tags|-t TAG [...TAG]
|  |- prune
|  |- list
|  |  `- --full-sha256|-f
|  |- tag
|  |  |- REFERENCE
|  |  `- TAG [...TAG]
|  `- untag
|     `- TAG [...TAG]
`- compose|comp|c
   |- build
   |  `- FILENAME
   `- run
      |- FILENAME
      `- COMMAND

Individual combinations are briefly described below:

focker image|img|im|i

The focker image mode groups commands related to Focker images.

build|b FOCKER_DIR [--tags TAG [...TAG]]

Build a Focker image according to the specification in a Fockerfile present in the specified FOCKER_DIR. Fockerfile syntax is very straightforward and explained below.

tag|t REFERENCE TAG [...TAG]

Applies one or more tags to the given image. REFERENCE can be the SHA256 of an image or one of its existing tags. It can be just a few first characters as long as they are unambiguous.

untag|u TAG [...TAG]

Removes one or more image tags.

list|ls|l [--full-sha256|-f]

Lists existing Focker images, optionally with full SHA256 checksums (instead of the default 7 first characters).

prune|p

Greedily removes existing Focker images without tags and without dependents.

remove|r REFERENCE

Removes the specified image.

focker jail|j

The focker jail mode groups commands related to Focker-managed jails.

create|c IMAGE [--command|-c COMMAND] [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [--hostname|-n HOSTNAME]

Creates a new Focker-managed jail. A jail consists of a clone of the given IMAGE and an entry in /etc/jail.conf. The configuration entry uses exec.prestart and exec.start to specify how the runtime environment (mounts and environmental variables) should be set up. It also calls COMMAND as last in exec.start. If not specified COMMAND defaults to /bin/sh. The hostname can be specified using the HOSTNAME parameter. Mounts and environment variables are provided as tuples separated by a colon (:). The environmental variable specification consists of variable name followed by variable value. The mount specification consists of the “from path”, followed by the “on path”. “From path” can be a local system path or a volume name.

start|s REFERENCE

Starts the given jail specified by REFERENCE. REFERENCE can be the SHA256 of an existing jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous. This command is equivalent of calling jail -c.

stop|S REFERENCE

Stops the given jail specified by REFERENCE. This command is equivalent to calling jail -r.

remove|r REFERENCE

Removes the given jail specified by REFERENCE. The jail is stopped if running, any filesystems mounted under its root directory are unmounted, its ZFS dataset and entry in /etc/jail.conf are removed.

exec|e REFERENCE [...COMMAND]

Executes given COMMAND (or /bin/sh if not specified) in the given running jail specified by REFERENCE. This command is the equivalent of calling jexec.

oneshot|o IMAGE [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [...COMMAND]

Create a new one-time Focker-managed jail. The syntax and logic is identical to focker jail create, the difference being that the hostname cannot be specified and that the jail will be automatically removed when the COMMAND exits.

Example: focker jail oneshot freebsd-latest -e FOO:bar -- ls -al

list|ls|l

Lists Focker-managed jails. For running jails their JIDs will be displayed.

tag REFERENCE TAG [...TAG]

Applies one or more tags to the given jail. REFERENCE can be the SHA256 of a jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous.

untag TAG [...TAG]

Removes one or more jail tags.

prune

Removes existing Focker jails without tags.

focker volume|vol|v

The focker volume mode groups commands related to Focker volumes.

create [--tags|-t TAG [...TAG]]

Create a new Focker volume optionally tagged with the given TAGs.

prune

Removes existing Focker volumes without tags.

list [--full-sha256|-f]

Lists existing Focker volumes. Full SHA256 is displayed if the -f switch is used, otherwise only the first 7 characters will be shown.

tag REFERENCE TAG [...TAG]

Applies one or more tags to the given volume. REFERENCE can be the SHA256 of a volume or one of its existing tags. It can be just a few first characters as long as they are unambiguous.

untag TAG [...TAG]

Removes one or more volume tags.

focker compose|comp|c

The focker compose mode groups commands related to Focker composition files - focker-compose.yml.

build FILENAME

Builds images, volumes and jails according to the specification provided in the file pointed to by FILENAME.

run FILENAME COMMAND

Runs one of the commands (specified by COMMAND) from the composition file pointed to by FILENAME.

Fockerfile syntax

A sample Fockerfile is pasted below.

base: freebsd-latest

steps:
  - copy:
    - [ '/tmp/x', '/etc/x' ]
    - [ 'files/y', '/etc/y' ]
  - copy: [ files/z, /etc/z ]
  - run: |
      pkg install -y python3
  - run:
      - pkg install -y py37-pip
      - pkg install -y py37-yaml
      - pkg install -y py37-certbot
  - run: |
      mkdir -p /persist/etc/ssh && \
      sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/rc.d/sshd && \
      sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/ssh/sshd_config && \
      sed -i '' -e 's/#HostKey/HostKey/g' /etc/ssh/sshd_config

Fockerfile currently supports only two entries - base and steps. base specifies the parent image on top of which the operations described by steps are executed. Each entry in steps results in creation of a new image. Focker determines a checksum for each step and if the corresponding image already exists the step is skipped and work continues on top of the existing image. This is a powerful paradigm for image building experimentation where we can split the task into multiple steps and resume work from the last successful step in case of problems. It is a big time saver. steps is a list that can contain copy and run entries. The copy entry specifies a single one or a list of copy operations from local files to the image in form of the [FROM, TO] tuples. The run entry specifies a chain of commands to be executed within the image. It can be a list of string or a single string.

focker-compose.yml syntax

A sample composition file illustrating all of the principles is pasted below.

images:
  wordpress-5: /path/to/wordpress_5_focker_dir

jails:
  wordpress:
    image: wordpress-5
    env:
      SITE_NAME: Test site
    mounts:
      wp-volume2: /mnt/volume2
      wp-volume1: /mnt/volume1
    ip4.addr: 127.0.1.1
    interface: lo1

volumes:
  wp-volume1: {}
  wp-volume2: {}
  wp-backup: {}

commands:
  backup:
    jail: wordpress
    command: |
      mysqldump >/mnt/volume2/backup.sql
    mounts:
      wp-backup: /mnt/backup

  restore:
    jail: wordpress
    command: |
      mysql </mnt/volume2/backup.sql
    mounts:
      wp-backup: /mnt/backup

Images

The images entry in Focker composition file specifies a dictionary from image tags to Focker directories (directories containing the Fockerfile and any supplementary files needed to build an image). Upon running focker compose build Focker will run focker image build for all of the specified directories and tag the results with the corresponding tags. This process can be repeated without significant performance penalty since the images will not need to be rebuilt unless their Fockerfiles or contexts change.

Jails

The jails entry in the Focker composition file specifies a dictionary from jail tags to jail specifications. A jail specification is a dictionary that can contain the following fields: image, env, mounts, exec.start, exec.stop, hostname, ip4.addr, interface. image, env and mounts have the same semantics as in the focker jail create command. The syntax for env and mounts is in the form of dictionaries. exec.start, exec.stop, hostname, ip4.addr and interface have the same semantics as the corresponding entries in /etc/jail.conf. The jails will be recreated each time focker compose build is executed. Hence, any persistent data should be stored in volumes.

Volumes

The volumes entry in the Focker composition file specifies a dictionary from volume tags to volume specifications. Currently a volume specification must be an empty dictionary. Specified volumes will be created by focker compose build and tagged with corresponding tags unless volumes with given tags already exist. Volumes are meant to persist data beyond the jail lifecycle.

Commands

The commands entry in the Focker composition file specifies a dictionary from command names to command specifications. A command specification can contain the following fields: jail, mounts and command. The jail field specifies in which jail the given command should be executed (the jail must be already running). The mounts entry specifies additional mounts that will be used only during the execution of the command. Finally the command entry specifies the command itself using the same syntax as the run step in a Fockerfile.

Further Reading

The best way to learn is by practice. Take a look at the example and start writing your own Focker specifications.

Conclusion

Focker provides powerful containerization primitives (images, volumes and containers) first introduced by the Docker platform without taking up the significantly more challenging task of achieving Docker compatibility. This has never been and never will be the goal of Focker which allows it to remain a lightweight tool with minimal dependencies and highly maintainable codebase. At the same time, the image building paradigm based on checksummed steps/layers and flexible composition builder offer significant time savings to pragmatic sysadmins.

Links

• PyPi entry for Focker
• Focker Announcement on ADARED