IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an email to s dot adaszewski at gmail dot com. User accounts are meant only to report issues and/or generate pull requests. This is a purpose-specific Git hosting for ADARED projects. Thank you for your understanding!
Du kannst nicht mehr als 25 Themen auswählen Themen müssen entweder mit einem Buchstaben oder einer Ziffer beginnen. Sie können Bindestriche („-“) enthalten und bis zu 35 Zeichen lang sein.

vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
vor 4 Jahren
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375
  1. # Focker
  2. ## Introduction
  3. Focker is a FreeBSD image orchestration tool in the vein of Docker.
  4. ## Table of Contents
  5. <!-- TOC depthFrom:2 depthTo:4 withLinks:1 updateOnSave:1 orderedList:0 -->
  6. - [Introduction](#introduction)
  7. - [Table of Contents](#table-of-contents)
  8. - [Installation](#installation)
  9. - [Installing the Python package from PyPi](#installing-the-python-package-from-pypi)
  10. - [Installing the Python package from GitHub](#installing-the-python-package-from-github)
  11. - [Setting up ZFS](#setting-up-zfs)
  12. - [Preparing base image](#preparing-base-image)
  13. - [Usage](#usage)
  14. - [`focker` command syntax](#focker-command-syntax)
  15. - [focker image|img|im|i](#focker-imageimgimi)
  16. - [focker jail|j](#focker-jailj)
  17. - [focker volume|vol|v](#focker-volumevolv)
  18. - [focker compose|comp|c](#focker-composecompc)
  19. - [`Fockerfile` syntax](#fockerfile-syntax)
  20. - [`focker-compose.yml` syntax](#focker-composeyml-syntax)
  21. - [Images](#images)
  22. - [Jails](#jails)
  23. - [Volumes](#volumes)
  24. - [Commands](#commands)
  25. - [Further Reading](#further-reading)
  26. - [Conclusion](#conclusion)
  27. - [Links](#links)
  28. <!-- /TOC -->
  29. ## Installation
  30. In order to use Focker you need a ZFS pool available in your FreeBSD installation.
  31. ### Installing the Python package from PyPi
  32. Run:
  33. ```bash
  34. pip install focker
  35. ```
  36. ### Installing the Python package from GitHub
  37. Run:
  38. ```bash
  39. git clone https://github.com/sadaszewski/focker.git
  40. cd focker/
  41. python setup.py install
  42. ```
  43. or (if you want an uninstaller):
  44. ```bash
  45. git clone https://github.com/sadaszewski/focker.git
  46. cd focker/
  47. python setup.py sdist
  48. pip install dist/focker-0.9.tgz
  49. ```
  50. ### Setting up ZFS
  51. Upon first execution of the `focker` command, Focker will automatically create the necessary directories and ZFS datasets. You just need to exclude the unlikely case that you are already using `/focker` in your filesystem hierarchy. The layout after initialization will look the following:
  52. ```
  53. /focker
  54. /focker/images
  55. /focker/jails
  56. /focker/volumes
  57. ```
  58. `images`, `jails`, and `volumes` have corresponding ZFS datasets with `canmount=off` so that they serve as mountpoint anchors for child entries.
  59. ### Preparing base image
  60. To bootstrap the images system you need to install FreeBSD in jail mode to a ZFS dataset placed in `/focker/images` and provide two user-defined properties - `focker:sha256` and `focker:tags`. One way to achieve this would be the following (using Bash shell):
  61. ```bash
  62. TAGS="freebsd-latest freebsd-$(freebsd-version | cut -d'-' -f1)"
  63. VERSION="FreeBSD $(freebsd-version)"
  64. SHA256=$(echo -n ${VERSION} | sha256)
  65. NAME=${SHA256:0:7}
  66. zfs create -o focker:sha256=${SHA256} -o focker:tags="${TAGS}" zroot/focker/images/${NAME}
  67. bsdinstall jail /focker/images/${NAME}
  68. zfs set readonly=on zroot/focker/images/${NAME}
  69. zfs snapshot zroot/focker/images/${NAME}@1
  70. ```
  71. ## Usage
  72. At this point, Focker is ready to use.
  73. ### `focker` command syntax
  74. The `focker` command is the single entrypoint to all of the Focker's functionality. The overview of its syntax is presented below as a tree where the `focker` command is the root, the first level of descendants represents the choice of Level 1 mode (`image`, `jail`, `volume` or `compose`), the second level - the Level 2 mode (dependent on L1 mode) and the final third level lists required and optional arguments specific to the given combination of L1/L2 modes.
  75. ```
  76. focker
  77. |- image|img|im|i
  78. | |- build|b
  79. | | |- FOCKER_DIR
  80. | | `- --tags|-t TAG [...TAG]
  81. | |- tag|t
  82. | | |- REFERENCE
  83. | | `- TAG [...TAG]
  84. | |- untag|u
  85. | | `- TAG [...TAG]
  86. | |- list|ls|l
  87. | | `- --full-sha256|-f
  88. | |- prune|p
  89. | `- remove|r
  90. | |- REFERENCE
  91. | `- --remove-dependents|-R
  92. |- jail|j
  93. | |- create|c
  94. | | |- IMAGE
  95. | | |- --command|-c COMMAND (default: /bin/sh)
  96. | | |- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
  97. | | |- --mounts|-m FROM1:ON1 [...FROMN:ONN]
  98. | | `- --hostname|-n HOSTNAME
  99. | |- start|s
  100. | | `- REFERENCE
  101. | |- stop|S
  102. | | `- REFERENCE
  103. | |- remove|r
  104. | | `- REFERENCE
  105. | |- exec|e
  106. | | |- REFERENCE
  107. | | `- [...COMMAND]
  108. | |- oneshot|o
  109. | | `- IMAGE
  110. | | `- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
  111. | | `- --mounts|-m FROM1:ON1 [...FROMN:ONN]
  112. | | `- [...COMMAND]
  113. | |- list|ls|l
  114. | | `- --full-sha256|-f
  115. | |- tag|t
  116. | | |- REFERENCE
  117. | | `- TAG [...TAG]
  118. | |- untag|u
  119. | | `- TAG [...TAG]
  120. | `- prune|p
  121. | `- --force|-f
  122. |- volume|vol|v
  123. | |- create
  124. | | `- --tags|-t TAG [...TAG]
  125. | |- prune
  126. | |- list
  127. | | `- --full-sha256|-f
  128. | |- tag
  129. | | |- REFERENCE
  130. | | `- TAG [...TAG]
  131. | `- untag
  132. | `- TAG [...TAG]
  133. `- compose|comp|c
  134. |- build
  135. | `- FILENAME
  136. `- run
  137. |- FILENAME
  138. `- COMMAND
  139. ```
  140. Individual combinations are briefly described below:
  141. #### focker image|img|im|i
  142. The `focker image` mode groups commands related to Focker images.
  143. ##### build|b FOCKER_DIR [--tags TAG [...TAG]]
  144. Build a Focker image according to the specification in a Fockerfile present in the specified FOCKER_DIR. Fockerfile syntax is very straightforward and explained below.
  145. ##### tag|t REFERENCE TAG [...TAG]
  146. Applies one or more tags to the given image. REFERENCE can be the SHA256 of an image or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  147. ##### untag|u TAG [...TAG]
  148. Removes one or more image tags.
  149. ##### list|ls|l [--full-sha256|-f]
  150. Lists existing Focker images, optionally with full SHA256 checksums (instead of the default 7 first characters).
  151. ##### prune|p
  152. Greedily removes existing Focker images without tags and without dependents.
  153. ##### remove|r REFERENCE
  154. Removes the specified image.
  155. #### focker jail|j
  156. The `focker jail` mode groups commands related to Focker-managed jails.
  157. ##### create|c IMAGE [--command|-c COMMAND] [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [--hostname|-n HOSTNAME]
  158. Creates a new Focker-managed jail. A jail consists of a clone of the given `IMAGE` and an entry in `/etc/jail.conf`. The configuration entry uses `exec.prestart` and `exec.start` to specify how the runtime environment (mounts and environmental variables) should be set up. It also calls `COMMAND` as last in `exec.start`. If not specified `COMMAND` defaults to `/bin/sh`. The hostname can be specified using the `HOSTNAME` parameter. Mounts and environment variables are provided as tuples separated by a colon (:). The environmental variable specification consists of variable name followed by variable value. The mount specification consists of the "from path", followed by the "on path". "From path" can be a local system path or a volume name.
  159. ##### start|s REFERENCE
  160. Starts the given jail specified by `REFERENCE`. `REFERENCE` can be the SHA256 of an existing jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous. This command is equivalent of calling `jail -c`.
  161. ##### stop|S REFERENCE
  162. Stops the given jail specified by `REFERENCE`. This command is equivalent to calling `jail -r`.
  163. ##### remove|r REFERENCE
  164. Removes the given jail specified by `REFERENCE`. The jail is stopped if running, any filesystems mounted under its root directory are unmounted, its ZFS dataset and entry in `/etc/jail.conf` are removed.
  165. ##### exec|e REFERENCE [...COMMAND]
  166. Executes given `COMMAND` (or `/bin/sh` if not specified) in the given running jail specified by `REFERENCE`. This command is the equivalent of calling `jexec`.
  167. ##### oneshot|o IMAGE [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [...COMMAND]
  168. Create a new one-time Focker-managed jail. The syntax and logic is identical to `focker jail create`, the difference being that the hostname cannot be specified and that the jail will be automatically removed when the `COMMAND` exits.
  169. Example: `focker jail oneshot freebsd-latest -e FOO:bar -- ls -al`
  170. ##### list|ls|l
  171. Lists Focker-managed jails. For running jails their JIDs will be displayed.
  172. ##### tag REFERENCE TAG [...TAG]
  173. Applies one or more tags to the given jail. REFERENCE can be the SHA256 of a jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  174. ##### untag TAG [...TAG]
  175. Removes one or more jail tags.
  176. ##### prune
  177. Removes existing Focker jails without tags.
  178. #### focker volume|vol|v
  179. The `focker volume` mode groups commands related to Focker volumes.
  180. ##### create [--tags|-t TAG [...TAG]]
  181. Create a new Focker volume optionally tagged with the given `TAG`s.
  182. ##### prune
  183. Removes existing Focker volumes without tags.
  184. ##### list [--full-sha256|-f]
  185. Lists existing Focker volumes. Full SHA256 is displayed if the `-f` switch is used, otherwise only the first 7 characters will be shown.
  186. ##### tag REFERENCE TAG [...TAG]
  187. Applies one or more tags to the given volume. REFERENCE can be the SHA256 of a volume or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  188. ##### untag TAG [...TAG]
  189. Removes one or more volume tags.
  190. #### focker compose|comp|c
  191. The `focker compose` mode groups commands related to Focker composition files - `focker-compose.yml`.
  192. ##### build FILENAME
  193. Builds images, volumes and jails according to the specification provided in the file pointed to by `FILENAME`.
  194. ##### run FILENAME COMMAND
  195. Runs one of the commands (specified by `COMMAND`) from the composition file pointed to by `FILENAME`.
  196. ### `Fockerfile` syntax
  197. A sample `Fockerfile` is pasted below.
  198. ```yaml
  199. base: freebsd-latest
  200. steps:
  201. - copy:
  202. - [ '/tmp/x', '/etc/x' ]
  203. - [ 'files/y', '/etc/y' ]
  204. - copy: [ files/z, /etc/z ]
  205. - run: |
  206. pkg install -y python3
  207. - run:
  208. - pkg install -y py37-pip
  209. - pkg install -y py37-yaml
  210. - pkg install -y py37-certbot
  211. - run: |
  212. mkdir -p /persist/etc/ssh && \
  213. sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/rc.d/sshd && \
  214. sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/ssh/sshd_config && \
  215. sed -i '' -e 's/#HostKey/HostKey/g' /etc/ssh/sshd_config
  216. ```
  217. `Fockerfile` currently supports only two entries - `base` and `steps`. `base` specifies the parent image on top of which the operations described by `steps` are executed. Each entry in `steps` results in creation of a new image. Focker determines a checksum for each step and if the corresponding image already exists the step is skipped and work continues on top of the existing image. This is a powerful paradigm for image building experimentation where we can split the task into multiple steps and resume work from the last successful step in case of problems. It is a big time saver. `steps` is a list that can contain `copy` and `run` entries. The `copy` entry specifies a single one or a list of copy operations from local files to the image in form of the `[FROM, TO]` tuples. The `run` entry specifies a chain of commands to be executed within the image. It can be a list of string or a single string.
  218. ### `focker-compose.yml` syntax
  219. A sample composition file illustrating all of the principles is pasted below.
  220. ```yaml
  221. images:
  222. wordpress-5: /path/to/wordpress_5_focker_dir
  223. jails:
  224. wordpress:
  225. image: wordpress-5
  226. env:
  227. SITE_NAME: Test site
  228. mounts:
  229. wp-volume2: /mnt/volume2
  230. wp-volume1: /mnt/volume1
  231. ip4.addr: 127.0.1.1
  232. interface: lo1
  233. volumes:
  234. wp-volume1: {}
  235. wp-volume2: {}
  236. wp-backup: {}
  237. commands:
  238. backup:
  239. jail: wordpress
  240. command: |
  241. mysqldump >/mnt/volume2/backup.sql
  242. mounts:
  243. wp-backup: /mnt/backup
  244. restore:
  245. jail: wordpress
  246. command: |
  247. mysql </mnt/volume2/backup.sql
  248. mounts:
  249. wp-backup: /mnt/backup
  250. ```
  251. #### Images
  252. The `images` entry in Focker composition file specifies a dictionary from image tags to Focker directories (directories containing the `Fockerfile` and any supplementary files needed to build an image). Upon running `focker compose build` Focker will run `focker image build` for all of the specified directories and tag the results with the corresponding tags. This process can be repeated without significant performance penalty since the images will not need to be rebuilt unless their `Fockerfile`s or contexts change.
  253. #### Jails
  254. The `jails` entry in the Focker composition file specifies a dictionary from jail tags to jail specifications. A jail specification is a dictionary that can contain the following fields: `image`, `env`, `mounts`, `exec.start`, `exec.stop`, `hostname`, `ip4.addr`, `interface`. `image`, `env` and `mounts` have the same semantics as in the `focker jail create` command. The syntax for `env` and `mounts` is in the form of dictionaries. `exec.start`, `exec.stop`, `hostname`, `ip4.addr` and `interface` have the same semantics as the corresponding entries in `/etc/jail.conf`. The jails will be recreated each time `focker compose build` is executed. Hence, any persistent data should be stored in volumes.
  255. #### Volumes
  256. The `volumes` entry in the Focker composition file specifies a dictionary from volume tags to volume specifications. Currently a volume specification must be an empty dictionary. Specified volumes will be created by `focker compose build` and tagged with corresponding tags unless volumes with given tags already exist. Volumes are meant to persist data beyond the jail lifecycle.
  257. #### Commands
  258. The `commands` entry in the Focker composition file specifies a dictionary from command names to command specifications. A command specification can contain the following fields: `jail`, `mounts` and `command`. The `jail` field specifies in which jail the given command should be executed (the jail must be already running). The `mounts` entry specifies additional mounts that will be used only during the execution of the command. Finally the `command` entry specifies the command itself using the same syntax as the `run` step in a `Fockerfile`.
  259. ## Further Reading
  260. The best way to learn is by practice. Take a look at the [example](example/) and start writing your own Focker specifications.
  261. ## Conclusion
  262. Focker provides powerful containerization primitives (images, volumes and containers) first introduced by the Docker platform without taking up the significantly more challenging task of achieving Docker compatibility. This has never been and never will be the goal of Focker which allows it to remain a lightweight tool with minimal dependencies and highly maintainable codebase. At the same time, the image building paradigm based on checksummed steps/layers and flexible composition builder offer significant time savings to pragmatic sysadmins.
  263. ## Links
  264. - [PyPi entry for Focker](https://pypi.org/project/focker/)
  265. - [Focker Announcement on ADARED](https://adared.ch/focker)