IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an email to s dot adaszewski at gmail dot com. User accounts are meant only to report issues and/or generate pull requests. This is a purpose-specific Git hosting for ADARED projects. Thank you for your understanding!
選択できるのは25トピックまでです。 トピックは、先頭が英数字で、英数字とダッシュ('-')を使用した35文字以内のものにしてください。

README.md 15KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378
  1. # Focker
  2. ## Introduction
  3. Focker is a FreeBSD image orchestration tool in the vein of Docker. This
  4. page contains a detailed reference of Focker's functionality. If it is your
  5. first time using Focker, please refer to the
  6. [Basic Usage Guide](docs/Basic_Usage_Guide.md) first.
  7. ## Table of Contents
  8. <!-- TOC depthFrom:2 depthTo:4 withLinks:1 updateOnSave:1 orderedList:0 -->
  9. - [Introduction](#introduction)
  10. - [Table of Contents](#table-of-contents)
  11. - [Installation](#installation)
  12. - [Installing the Python package from PyPi](#installing-the-python-package-from-pypi)
  13. - [Installing the Python package from GitHub](#installing-the-python-package-from-github)
  14. - [Setting up ZFS](#setting-up-zfs)
  15. - [Preparing base image](#preparing-base-image)
  16. - [Usage](#usage)
  17. - [`focker` command syntax](#focker-command-syntax)
  18. - [focker image|img|im|i](#focker-imageimgimi)
  19. - [focker jail|j](#focker-jailj)
  20. - [focker volume|vol|v](#focker-volumevolv)
  21. - [focker compose|comp|c](#focker-composecompc)
  22. - [`Fockerfile` syntax](#fockerfile-syntax)
  23. - [`focker-compose.yml` syntax](#focker-composeyml-syntax)
  24. - [Images](#images)
  25. - [Jails](#jails)
  26. - [Volumes](#volumes)
  27. - [Commands](#commands)
  28. - [Further Reading](#further-reading)
  29. - [Conclusion](#conclusion)
  30. - [Links](#links)
  31. <!-- /TOC -->
  32. ## Installation
  33. In order to use Focker you need a ZFS pool available in your FreeBSD installation.
  34. ### Installing the Python package from PyPi
  35. Run:
  36. ```bash
  37. pip install focker
  38. ```
  39. ### Installing the Python package from GitHub
  40. Run:
  41. ```bash
  42. git clone https://github.com/sadaszewski/focker.git
  43. cd focker/
  44. python setup.py install
  45. ```
  46. or (if you want an uninstaller):
  47. ```bash
  48. git clone https://github.com/sadaszewski/focker.git
  49. cd focker/
  50. python setup.py sdist
  51. pip install dist/focker-0.9.tgz
  52. ```
  53. ### Setting up ZFS
  54. Upon first execution of the `focker` command, Focker will automatically create the necessary directories and ZFS datasets. You just need to exclude the unlikely case that you are already using `/focker` in your filesystem hierarchy. The layout after initialization will look the following:
  55. ```
  56. /focker
  57. /focker/images
  58. /focker/jails
  59. /focker/volumes
  60. ```
  61. `images`, `jails`, and `volumes` have corresponding ZFS datasets with `canmount=off` so that they serve as mountpoint anchors for child entries.
  62. ### Preparing base image
  63. To bootstrap the images system you need to install FreeBSD in jail mode to a ZFS dataset placed in `/focker/images` and provide two user-defined properties - `focker:sha256` and `focker:tags`. One way to achieve this would be the following (using Bash shell):
  64. ```bash
  65. TAGS="freebsd-latest freebsd-$(freebsd-version | cut -d'-' -f1)"
  66. VERSION="FreeBSD $(freebsd-version)"
  67. SHA256=$(echo -n ${VERSION} | sha256)
  68. NAME=${SHA256:0:7}
  69. zfs create -o focker:sha256=${SHA256} -o focker:tags="${TAGS}" zroot/focker/images/${NAME}
  70. bsdinstall jail /focker/images/${NAME}
  71. zfs set readonly=on zroot/focker/images/${NAME}
  72. zfs snapshot zroot/focker/images/${NAME}@1
  73. ```
  74. ## Usage
  75. At this point, Focker is ready to use.
  76. ### `focker` command syntax
  77. The `focker` command is the single entrypoint to all of the Focker's functionality. The overview of its syntax is presented below as a tree where the `focker` command is the root, the first level of descendants represents the choice of Level 1 mode (`image`, `jail`, `volume` or `compose`), the second level - the Level 2 mode (dependent on L1 mode) and the final third level lists required and optional arguments specific to the given combination of L1/L2 modes.
  78. ```
  79. focker
  80. |- image|img|im|i
  81. | |- build|b
  82. | | |- FOCKER_DIR
  83. | | `- --tags|-t TAG [...TAG]
  84. | |- tag|t
  85. | | |- REFERENCE
  86. | | `- TAG [...TAG]
  87. | |- untag|u
  88. | | `- TAG [...TAG]
  89. | |- list|ls|l
  90. | | `- --full-sha256|-f
  91. | |- prune|p
  92. | `- remove|r
  93. | |- REFERENCE
  94. | `- --remove-dependents|-R
  95. |- jail|j
  96. | |- create|c
  97. | | |- IMAGE
  98. | | |- --command|-c COMMAND (default: /bin/sh)
  99. | | |- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
  100. | | |- --mounts|-m FROM1:ON1 [...FROMN:ONN]
  101. | | `- --hostname|-n HOSTNAME
  102. | |- start|s
  103. | | `- REFERENCE
  104. | |- stop|S
  105. | | `- REFERENCE
  106. | |- remove|r
  107. | | `- REFERENCE
  108. | |- exec|e
  109. | | |- REFERENCE
  110. | | `- [...COMMAND]
  111. | |- oneshot|o
  112. | | `- IMAGE
  113. | | `- --env|-e VAR1:VALUE1 [...VARN:VALUEN]
  114. | | `- --mounts|-m FROM1:ON1 [...FROMN:ONN]
  115. | | `- [...COMMAND]
  116. | |- list|ls|l
  117. | | `- --full-sha256|-f
  118. | |- tag|t
  119. | | |- REFERENCE
  120. | | `- TAG [...TAG]
  121. | |- untag|u
  122. | | `- TAG [...TAG]
  123. | `- prune|p
  124. | `- --force|-f
  125. |- volume|vol|v
  126. | |- create
  127. | | `- --tags|-t TAG [...TAG]
  128. | |- prune
  129. | |- list
  130. | | `- --full-sha256|-f
  131. | |- tag
  132. | | |- REFERENCE
  133. | | `- TAG [...TAG]
  134. | `- untag
  135. | `- TAG [...TAG]
  136. `- compose|comp|c
  137. |- build
  138. | `- FILENAME
  139. `- run
  140. |- FILENAME
  141. `- COMMAND
  142. ```
  143. Individual combinations are briefly described below:
  144. #### focker image|img|im|i
  145. The `focker image` mode groups commands related to Focker images.
  146. ##### build|b FOCKER_DIR [--tags TAG [...TAG]]
  147. Build a Focker image according to the specification in a Fockerfile present in the specified FOCKER_DIR. Fockerfile syntax is very straightforward and explained below.
  148. ##### tag|t REFERENCE TAG [...TAG]
  149. Applies one or more tags to the given image. REFERENCE can be the SHA256 of an image or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  150. ##### untag|u TAG [...TAG]
  151. Removes one or more image tags.
  152. ##### list|ls|l [--full-sha256|-f]
  153. Lists existing Focker images, optionally with full SHA256 checksums (instead of the default 7 first characters).
  154. ##### prune|p
  155. Greedily removes existing Focker images without tags and without dependents.
  156. ##### remove|r REFERENCE
  157. Removes the specified image.
  158. #### focker jail|j
  159. The `focker jail` mode groups commands related to Focker-managed jails.
  160. ##### create|c IMAGE [--command|-c COMMAND] [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [--hostname|-n HOSTNAME]
  161. Creates a new Focker-managed jail. A jail consists of a clone of the given `IMAGE` and an entry in `/etc/jail.conf`. The configuration entry uses `exec.prestart` and `exec.start` to specify how the runtime environment (mounts and environmental variables) should be set up. It also calls `COMMAND` as last in `exec.start`. If not specified `COMMAND` defaults to `/bin/sh`. The hostname can be specified using the `HOSTNAME` parameter. Mounts and environment variables are provided as tuples separated by a colon (:). The environmental variable specification consists of variable name followed by variable value. The mount specification consists of the "from path", followed by the "on path". "From path" can be a local system path or a volume name.
  162. ##### start|s REFERENCE
  163. Starts the given jail specified by `REFERENCE`. `REFERENCE` can be the SHA256 of an existing jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous. This command is equivalent of calling `jail -c`.
  164. ##### stop|S REFERENCE
  165. Stops the given jail specified by `REFERENCE`. This command is equivalent to calling `jail -r`.
  166. ##### remove|r REFERENCE
  167. Removes the given jail specified by `REFERENCE`. The jail is stopped if running, any filesystems mounted under its root directory are unmounted, its ZFS dataset and entry in `/etc/jail.conf` are removed.
  168. ##### exec|e REFERENCE [...COMMAND]
  169. Executes given `COMMAND` (or `/bin/sh` if not specified) in the given running jail specified by `REFERENCE`. This command is the equivalent of calling `jexec`.
  170. ##### oneshot|o IMAGE [--env|-e VAR1:VALUE1 [...VARN:VALUEN]] [--mounts|-m FROM1:ON1 [...FROMN:ONN]] [...COMMAND]
  171. Create a new one-time Focker-managed jail. The syntax and logic is identical to `focker jail create`, the difference being that the hostname cannot be specified and that the jail will be automatically removed when the `COMMAND` exits.
  172. Example: `focker jail oneshot freebsd-latest -e FOO:bar -- ls -al`
  173. ##### list|ls|l
  174. Lists Focker-managed jails. For running jails their JIDs will be displayed.
  175. ##### tag REFERENCE TAG [...TAG]
  176. Applies one or more tags to the given jail. REFERENCE can be the SHA256 of a jail or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  177. ##### untag TAG [...TAG]
  178. Removes one or more jail tags.
  179. ##### prune
  180. Removes existing Focker jails without tags.
  181. #### focker volume|vol|v
  182. The `focker volume` mode groups commands related to Focker volumes.
  183. ##### create [--tags|-t TAG [...TAG]]
  184. Create a new Focker volume optionally tagged with the given `TAG`s.
  185. ##### prune
  186. Removes existing Focker volumes without tags.
  187. ##### list [--full-sha256|-f]
  188. Lists existing Focker volumes. Full SHA256 is displayed if the `-f` switch is used, otherwise only the first 7 characters will be shown.
  189. ##### tag REFERENCE TAG [...TAG]
  190. Applies one or more tags to the given volume. REFERENCE can be the SHA256 of a volume or one of its existing tags. It can be just a few first characters as long as they are unambiguous.
  191. ##### untag TAG [...TAG]
  192. Removes one or more volume tags.
  193. #### focker compose|comp|c
  194. The `focker compose` mode groups commands related to Focker composition files - `focker-compose.yml`.
  195. ##### build FILENAME
  196. Builds images, volumes and jails according to the specification provided in the file pointed to by `FILENAME`.
  197. ##### run FILENAME COMMAND
  198. Runs one of the commands (specified by `COMMAND`) from the composition file pointed to by `FILENAME`.
  199. ### `Fockerfile` syntax
  200. A sample `Fockerfile` is pasted below.
  201. ```yaml
  202. base: freebsd-latest
  203. steps:
  204. - copy:
  205. - [ '/tmp/x', '/etc/x' ]
  206. - [ 'files/y', '/etc/y' ]
  207. - copy: [ files/z, /etc/z ]
  208. - run: |
  209. pkg install -y python3
  210. - run:
  211. - pkg install -y py37-pip
  212. - pkg install -y py37-yaml
  213. - pkg install -y py37-certbot
  214. - run: |
  215. mkdir -p /persist/etc/ssh && \
  216. sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/rc.d/sshd && \
  217. sed -i '' -e 's/\/etc\/ssh\/ssh_host_/\/persist\/etc\/ssh\/ssh_host_/g' /etc/ssh/sshd_config && \
  218. sed -i '' -e 's/#HostKey/HostKey/g' /etc/ssh/sshd_config
  219. ```
  220. `Fockerfile` currently supports only two entries - `base` and `steps`. `base` specifies the parent image on top of which the operations described by `steps` are executed. Each entry in `steps` results in creation of a new image. Focker determines a checksum for each step and if the corresponding image already exists the step is skipped and work continues on top of the existing image. This is a powerful paradigm for image building experimentation where we can split the task into multiple steps and resume work from the last successful step in case of problems. It is a big time saver. `steps` is a list that can contain `copy` and `run` entries. The `copy` entry specifies a single one or a list of copy operations from local files to the image in form of the `[FROM, TO]` tuples. The `run` entry specifies a chain of commands to be executed within the image. It can be a list of string or a single string.
  221. ### `focker-compose.yml` syntax
  222. A sample composition file illustrating all of the principles is pasted below.
  223. ```yaml
  224. images:
  225. wordpress-5: /path/to/wordpress_5_focker_dir
  226. jails:
  227. wordpress:
  228. image: wordpress-5
  229. env:
  230. SITE_NAME: Test site
  231. mounts:
  232. wp-volume2: /mnt/volume2
  233. wp-volume1: /mnt/volume1
  234. ip4.addr: 127.0.1.1
  235. interface: lo1
  236. volumes:
  237. wp-volume1: {}
  238. wp-volume2: {}
  239. wp-backup: {}
  240. commands:
  241. backup:
  242. jail: wordpress
  243. command: |
  244. mysqldump >/mnt/volume2/backup.sql
  245. mounts:
  246. wp-backup: /mnt/backup
  247. restore:
  248. jail: wordpress
  249. command: |
  250. mysql </mnt/volume2/backup.sql
  251. mounts:
  252. wp-backup: /mnt/backup
  253. ```
  254. #### Images
  255. The `images` entry in Focker composition file specifies a dictionary from image tags to Focker directories (directories containing the `Fockerfile` and any supplementary files needed to build an image). Upon running `focker compose build` Focker will run `focker image build` for all of the specified directories and tag the results with the corresponding tags. This process can be repeated without significant performance penalty since the images will not need to be rebuilt unless their `Fockerfile`s or contexts change.
  256. #### Jails
  257. The `jails` entry in the Focker composition file specifies a dictionary from jail tags to jail specifications. A jail specification is a dictionary that can contain the following fields: `image`, `env`, `mounts`, `exec.start`, `exec.stop`, `hostname`, `ip4.addr`, `interface`. `image`, `env` and `mounts` have the same semantics as in the `focker jail create` command. The syntax for `env` and `mounts` is in the form of dictionaries. `exec.start`, `exec.stop`, `hostname`, `ip4.addr` and `interface` have the same semantics as the corresponding entries in `/etc/jail.conf`. The jails will be recreated each time `focker compose build` is executed. Hence, any persistent data should be stored in volumes.
  258. #### Volumes
  259. The `volumes` entry in the Focker composition file specifies a dictionary from volume tags to volume specifications. Currently a volume specification must be an empty dictionary. Specified volumes will be created by `focker compose build` and tagged with corresponding tags unless volumes with given tags already exist. Volumes are meant to persist data beyond the jail lifecycle.
  260. #### Commands
  261. The `commands` entry in the Focker composition file specifies a dictionary from command names to command specifications. A command specification can contain the following fields: `jail`, `mounts` and `command`. The `jail` field specifies in which jail the given command should be executed (the jail must be already running). The `mounts` entry specifies additional mounts that will be used only during the execution of the command. Finally the `command` entry specifies the command itself using the same syntax as the `run` step in a `Fockerfile`.
  262. ## Further Reading
  263. The best way to learn is by practice. Take a look at the [example](example/) and start writing your own Focker specifications.
  264. ## Conclusion
  265. Focker provides powerful containerization primitives (images, volumes and containers) first introduced by the Docker platform without taking up the significantly more challenging task of achieving Docker compatibility. This has never been and never will be the goal of Focker which allows it to remain a lightweight tool with minimal dependencies and highly maintainable codebase. At the same time, the image building paradigm based on checksummed steps/layers and flexible composition builder offer significant time savings to pragmatic sysadmins.
  266. ## Links
  267. - [PyPi entry for Focker](https://pypi.org/project/focker/)
  268. - [Focker Announcement on ADARED](https://adared.ch/focker)