Zk | 2017-01-05-part-3

type: post date: 2017-01-05 slug: part-3 title: How Charming - Part 3

In the previous entry, we started the process of pulling together our charm, including a lot of configuration values. Some of those were standard charm fare, metadata and configuration values, but some were configuration for the charm layers. We also investigated what layers were and how they're different from hooks. If you haven't read the first two entries, please make sure that you do so, or this one won't make much sense!

How was your Christmas? How was your New Years Eve? How was your...uh...Tibb's Eve?!

Mine were good. I got struck by a hypomanic episode and started about eighteen projects. Very productive. Felt like garbage. Unfortunately, none of those projects involved Honeycomb. The end result is that I got a bit of a vacation and am now catching up on Work™.

Now we're back to it, though. I'm starting to scrape together the resources and requirements that will need to go into the charm. My task right now is to deploy Honeycomb to a test server (I picked up a droplet and named it after the next fox in line¹) and ensure that everything works when plugged in together.

And writing the charm.

Writing layered charms is weird as heck, for me. This project is as much me learning how to write them as it is explaining how to do so. We relied on writing just hook-based charms for so long that it's difficult to get up to speed with this new style and leave behind so much of what I learned. The docs and tooling are still being written, too, so I find myself stumbling along blind alleys quite often.

Well, come stumble with me as we work through the next part of the charm: fetching our source and setting some configuration values.

Lets begin with the process of fetching our project's source, using the Git layer and the charm configuration values. We'll be using the git-deploy layer which will help us in grabbing our code from GitHub/GitLab/BitBucket and deploy it to our machine.

Last time, we added the apt layer as one to compose into our final charm, so let's do the same thing with the git layer, in our layers.yaml:

includes:
  - layer:basic
  - layer:apt
  - layer:git-deploy
repo: https://github.com/makyo/wsgi-app.git
options:
  apt:
    packages:
      - python-pip

Easy peasy, lemon squeezy. Now, when we build our charm, we should see 'git-deploy' show up in the layers being processed:

makyo@corrin:~/work/charms/layers/wsgi-app$ charm -v build
build: Composing into /home/makyo/work/charms
build: Destination charm directory: /home/makyo/work/charms/builds/wsgi-app
build: Processing layer: layer:basic
build: Processing layer: layer:apt
build: Processing layer: layer:git-deploy
build: Processing layer: wsgi-app

Sweet, looking good.

When a layer is added to a layered charm like this, all of its configuration options get merged in with the configuration options that we specify in our layer. For example, lets go take a look at the config.yaml file in the built charm in ../../builds/wsgi-app:

"options":
  "extra_packages":
    "description": "Space separated list of extra deb packages to install.\n"
    "type": "string"
    "default": ""
  "package_status":
    "default": "install"
    "type": "string"
    "description": "The status of service-affecting packages will be set to this value\
      \ in the dpkg database. Valid values are \"install\" and \"hold\".\n"
  "install_sources":
    "description": "List of extra apt sources, per charm-helpers standard format (a\
      \ yaml list of strings encoded as a string). Each source may be either a line\
      \ that can be added directly to sources.list(5), or in the form ppa:<user>/<ppa-name>\
      \ for adding Personal Package Archives, or a distribution component to enable.\n"
    "type": "string"
    "default": ""
  "install_keys":
    "description": "List of signing keys for install_sources package sources, per\
      \ charmhelpers standard format (a yaml list of strings encoded as a string).\
      \ The keys should be the full ASCII armoured GPG public keys. While GPG key\
      \ ids are also supported and looked up on a keyserver, operators should be aware\
      \ that this mechanism is insecure. null can be used if a standard package signing\
      \ key is used that will already be installed on the machine, and for PPA sources\
      \ where the package signing key is securely retrieved from Launchpad.\n"
    "type": "string"
    "default": ""
  "commit-or-branch":
    "type": "string"
    "default": !!null ""
    "description": "Commit or branch to update to"
  "repo":
    "type": "string"
    "default": !!null ""
    "description": "The repository to clone from, this is required"
  "deploy-key":
    "type": "string"
    "default": ""
    "description": |
      A deploy key is an SSH key that is stored on the server and
      grants access to a repository.
  "key-required":
    "type": "boolean"
    "default": !!bool "false"
    "description": |
      This should be set to true to ensure that a deploy key is
      deployed if necessary

If you pick through all this yaml, what you'll see is that it's broken down into two sections: the configuration from the apt layer and the configuration from the git layer, four of each. From the git layer, which we're focusing on, you can see that we can specify a commit/branch to deploy and a repo from which to deploy, as well as some settings for fetching code from private repositories.

Config files, in charms, are used for specifying default values and any descriptions used on the charmstore. When the user deploys our charm, they will rely on defaults or pass in configurations when deploying through their own file. For instance, when deploying Honeycomb using this charm as it stands, my configuration file might be something like this:

wsgi-app:
  extra_packages: pandoc
  commit-or-branch: master  # Once we tag a release, we can use that branch
  repo: https://github.com/OpenFurry/honeycomb.git

When we deploy our charm, we'll deploy it with the config, using juju deploy --config honeycomb-config.yaml wsgi-app. When the charm is fetched, it'll grab pandoc (used to convert uploaded files to markdown), clone the honeycomb repository on the master branch, and then...and then magic. We'll get to how to serve our wsgi app in a future post.

For now, let's see what else we can do with our configuration while we're in there.

Hmm, well, Honeycomb requires quite a bit installed, actually. It needs Django, of course, and markdown, and, uh...oh gosh, quite a bit. Enough that it needs a pip requirements file. Actually, those are good practice to have, anyway. If they're that common, we should probably make running pip install a part of our charm installation. We can't just call pip install -r requirements.txt from our project's directory, though. What if you named your file differently? Or, say your project is part of a suite, and you're only going to be running one part of the suite, making your requirement in a subdirectory?

The same goes for our WSGI file, too --- in modern Django apps, that's in <project_name>/wsgi.py, but you had to write your own in older versions of Django; in Flask apps. it's usually the main python file, but might not be --- so we should be able to specify that as well.

So let's head back to our wsgi-app layer and dig into config.yaml. We'll need to add two configuration values --- pip-requirements and wsgi-file --- both of which will be strings:

options:
  pip-requirements:
    default: requirements.txt
    type: 'string'
    description: |
      The location within your project's directory of any requirements file
      needed to install Python dependencies. An empty string will mean that
      pip will not be run.
  wsgi-file:
    default: ''
    type: 'string'
    description: |
      REQUIRED
      The location of the file which exposes `application` to the WSGI server.

While we're at it, I must admit to a goof: in the previous installment, I mentioned that we had to add some config values to our layer's config.yaml file in order to make one of the layers happy. I was wrong, thank goodness. The configuration values fall all the way through to the built charm. So I deleted the ones from the last article, and what you see above is the entire config.yaml file now.

We've gone over quite a bit today around configuration values, but we've been tooling around in the edges of charm land. Next time, lets delve into writing some actual charm code! We'll see if we can play around with the states provided by the apt and git layers! Maybe even dive into some interfaces.

Source code for wsgi-app layer as it was at the point reached at the end of the article.