2020-04-25 04:55:06 +00:00
|
|
|
<!doctype html>
|
|
|
|
|
<html>
|
|
|
|
|
<head>
|
|
|
|
|
<title>Zk | 2015-04-06-dogfooding-1</title>
|
|
|
|
|
<link rel="stylesheet" type="text/css" href="/style.css" />
|
|
|
|
|
<meta name="viewport" content="width=device-width" />
|
|
|
|
|
<meta charset="utf-8" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<main>
|
|
|
|
|
<header>
|
|
|
|
|
<h1>Zk | 2015-04-06-dogfooding-1</h1>
|
|
|
|
|
</header>
|
|
|
|
|
<article class="content">
|
|
|
|
|
<hr />
|
|
|
|
|
<p>type: post
|
|
|
|
|
date: 2015-04-06
|
|
|
|
|
slug: dogfooding-1
|
|
|
|
|
title: Dogfooding - Pt. 1</p>
|
|
|
|
|
<hr />
|
|
|
|
|
<p>I work for <a href="http://canonical.com">Canonical, Ltd</a>, a company focused primarily
|
|
|
|
|
on open source software, best known for their operating system,
|
|
|
|
|
<a href="http://ubuntu.com">Ubuntu</a>, a distribution of Linux focused on both the ease of
|
|
|
|
|
desktop use, as well as a seamless experience across desktop, server, mobile,
|
2020-06-24 07:10:07 +00:00
|
|
|
and TV. That’s not all that we make, though, and our umbrella of products and
|
2020-04-25 04:55:06 +00:00
|
|
|
services stretches across several sectors of the software market.</p>
|
|
|
|
|
<p>I am on the Juju UI team. <a href="http://jujucharms.com">Juju</a> is a dev-ops solution
|
|
|
|
|
that allows packaging server software into Juju Charms, which act as biased
|
|
|
|
|
installations of software that are built by people who know how that software
|
|
|
|
|
can best be deployed. For instance, the <a href="https://jujucharms.com/wordpress/">Wordpress
|
|
|
|
|
charm</a> encapsulates an installation of
|
|
|
|
|
Wordpress, just as the <a href="https://jujucharms.com/mysql/">MySQL charm</a> packages an
|
|
|
|
|
installation of MySQL, such that, if you have a Juju environment up and running,
|
|
|
|
|
you can simply run:</p>
|
2020-04-25 06:55:07 +00:00
|
|
|
<p _="%" endhighlight="endhighlight">{% highlight bash %}
|
2020-04-25 04:55:06 +00:00
|
|
|
juju deploy wordpress
|
|
|
|
|
juju deploy mysql
|
|
|
|
|
juju add-relation wordpress mysql
|
2020-04-25 06:55:07 +00:00
|
|
|
juju expose wordpress</p>
|
2020-04-25 04:55:06 +00:00
|
|
|
<p>and wind up with a proper installation of Wordpress talking to a proper
|
|
|
|
|
installation of MySQL in the cloud of your choice.</p>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>I work on more than just the projects I have at Canonical, however. I’ve bought
|
2020-04-25 04:55:06 +00:00
|
|
|
wholeheartedly into both the ethos and implementation of open source software,
|
|
|
|
|
and so I work on releasing as much of my own free-time projects as open source
|
|
|
|
|
as possible. Lately, the project that has been taking up much of my weekends
|
|
|
|
|
and evenings has been an exploration of <a href="http://golang.org">Go</a>, a programming
|
|
|
|
|
language by Google with a focus on simplicity and concurrency. I chose this
|
|
|
|
|
language for this task because Juju itself, along with many of its tools, are
|
|
|
|
|
written in Go.</p>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>The project I’ve been working on,
|
2020-04-25 04:55:06 +00:00
|
|
|
<a href="https://github.com/warren-community/warren">Warren</a>, is a pretty simple bit of
|
|
|
|
|
web software that allows publishing of entities of various types - from blog
|
|
|
|
|
posts to pictures and so on - and linking of those entities into a web of
|
2020-06-24 07:10:07 +00:00
|
|
|
content. Or, rather, those are my hopes for it; it’s still in its early stages.
|
|
|
|
|
I’ve learned a lot about Go, many Go web utilities such as
|
2020-04-25 04:55:06 +00:00
|
|
|
<a href="http://martini.codegangsta.io/">Martini</a>,
|
|
|
|
|
<a href="https://github.com/codegangsta/gin">Gin</a>, and
|
|
|
|
|
<a href="http://www.gorillatoolkit.org/">Gorilla</a>, testing in Go using
|
|
|
|
|
<a href="http://goconvey.co/">GoConvey</a>, as well as the services that back Warren up
|
|
|
|
|
such as Mongo and ElasticSearch.</p>
|
|
|
|
|
<p>Writing a web app such as this, it was a natural solution to charm Warren so
|
|
|
|
|
that I could manage it, along with its backing services, in a scalable fashion
|
|
|
|
|
on any of the clouds that Juju supports.</p>
|
|
|
|
|
<h2 id="what-i-did">What I Did</h2>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>The work that I’ve done to date consists of two parts beyond Warren itself: the
|
2020-04-25 04:55:06 +00:00
|
|
|
<a href="https://github.com/warren-community/warren-charm">Warren charm</a> and the <a href="https://github.com/warren-community/warren-bundle">Warren
|
|
|
|
|
bundle</a>. Whereas charms
|
|
|
|
|
describe the way in which a singal service is deployed, bundles are groups of
|
|
|
|
|
charms that work together as a group of services providing a particular
|
|
|
|
|
functionality. For instance, the Warren bundle includes Warren, Mongo,
|
|
|
|
|
ElasticSearch, and haproxy for load balancing.</p>
|
|
|
|
|
<h3 id="the-warren-charm">The Warren charm</h3>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>I went with the idea of a “thin” charm. This means that the charm itself does
|
2020-04-25 04:55:06 +00:00
|
|
|
not contain any of the source code or binaries required for building Warren
|
2020-06-24 07:10:07 +00:00
|
|
|
itself. This is as opposed to “thick” (sometimes referred to as “fat”) charms,
|
2020-04-25 04:55:06 +00:00
|
|
|
which contain a binary, a zip of source code, or some other means of deploying
|
|
|
|
|
the service without reaching out to some external location. There are a few
|
|
|
|
|
advantages and disadvantages to each:</p>
|
|
|
|
|
<h4 id="thin-charms">Thin charms</h4>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>Advantages:<ul>
|
|
|
|
|
<li>Change to the code of the services does not mean change to the code of the
|
|
|
|
|
charm.</li>
|
|
|
|
|
<li>Charms are much smaller, and thus are uploaded to the environment fairly
|
|
|
|
|
quickly.</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</li>
|
|
|
|
|
<li>Disadvantages:<ul>
|
|
|
|
|
<li>Services will require access to wherever the source or binary to be
|
2020-06-24 07:10:07 +00:00
|
|
|
deployed. In Warren, that’s the GitHub repository, but it may be a PPA or
|
2020-04-25 04:55:06 +00:00
|
|
|
other source of packages. In some clouds, access such as this may be
|
|
|
|
|
heavily restricted.</li>
|
|
|
|
|
<li>Since the code must be fetched, potentially built, and then installed,
|
|
|
|
|
some of the speed gains for not having to upload the charm are lost.</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<h4 id="thick-charms">Thick charms</h4>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>Advantages:<ul>
|
|
|
|
|
<li>You can be guaranteed that the code that is in the charm is the only code
|
|
|
|
|
that will be deployed in the service without having to use any
|
|
|
|
|
work-arounds for pinning fetched source at a revision or tag.</li>
|
2020-06-24 07:10:07 +00:00
|
|
|
<li>Deployment is usually fairly easy and fast, since you don’t have to worry
|
2020-04-25 04:55:06 +00:00
|
|
|
about fetching source and building, and since no external sources are
|
|
|
|
|
required, you can safely deploy behind a strict firewall.</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</li>
|
|
|
|
|
<li>Disadvantages:<ul>
|
|
|
|
|
<li>Thick charms are rather large and may take a bit of time to actually
|
|
|
|
|
deploy, since the charm will need to be uploaded to the environment either
|
|
|
|
|
from your local machine or from the charmstore.</li>
|
|
|
|
|
<li>Changes in your codebase will not show up when a new charm is deployed,
|
|
|
|
|
since the code exists at one particular revision - this means that
|
|
|
|
|
emergency bug fixes in your code mean an emergency charm release
|
|
|
|
|
containing that fix, and a charm upgrade.</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>For Warren, I went with a thin charm for a few reasons:</p>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>I live in the mountains, where Internet connectivity is limited to microwave
|
2020-06-24 07:10:07 +00:00
|
|
|
line-of-sight of okay-but-not-great speeds. I didn’t want to have to
|
2020-04-25 04:55:06 +00:00
|
|
|
re-upload the charm to EC2 every time I made a fix to it (charm developing
|
|
|
|
|
being a particularly iterative process). Once the charm winds up in the
|
|
|
|
|
store, should it wind up there, this will not be as much of a concern.</li>
|
|
|
|
|
<li>I want to iterate fairly quickly on Warren, if it winds up going somewhere. I
|
|
|
|
|
want to be able to iterate on Warren much more than I want to be able to
|
2020-06-24 07:10:07 +00:00
|
|
|
iterate on the Warren charm, however, so I’d rather spend an initial
|
2020-04-25 04:55:06 +00:00
|
|
|
investment allowing a means of upgrading the Warren source in a deployed
|
|
|
|
|
instance of Warren through a configuration change (more on this later) than
|
|
|
|
|
have a repeated time sink involved in building a new version of the charm.</li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>In the end, the wins that thick charms get me are outweighed by the wins
|
|
|
|
|
involved in thin charms.</p>
|
|
|
|
|
<h3 id="the-warren-bundle">The Warren bundle</h3>
|
|
|
|
|
<p><img alt="Warren in the Juju GUI" src="/assets/tech/dogfooding-1.png" /></p>
|
|
|
|
|
<p>The Juju GUI is the project that I have spent the most time with during my time
|
|
|
|
|
at Canonical. I started out on the GUI, and continue to contribute to it even
|
|
|
|
|
as focus has shifted to other aspects of the various user interfaces to Juju.
|
2020-06-24 07:10:07 +00:00
|
|
|
One of the things that I like about it is that it’s really easy to see just how
|
2020-04-25 04:55:06 +00:00
|
|
|
your entire environment works and fits together. In the instance above, you can
|
|
|
|
|
see haproxy exposed, which leads to Warren, which depends on Mongo and
|
2020-06-24 07:10:07 +00:00
|
|
|
ElasticSearch. There’s a flow of traffic indicated in the diagram - a request
|
2020-04-25 04:55:06 +00:00
|
|
|
to the exposed bit, which is proxied to Warren, which may make a request of
|
|
|
|
|
either Mongo, ElasticSearch, or both.</p>
|
|
|
|
|
<p>Another benefit of the Juju GUI is that it allows you to export the current
|
|
|
|
|
environment in the format of a <a href="https://jujucharms.com/docs/1.20/charms-bundles">Juju
|
|
|
|
|
bundle</a>, which is a repeatable
|
|
|
|
|
representation of an environment. These bundles can then be reimported with the
|
|
|
|
|
GUI or with another of our tools, Juju Quickstart, which allows building and
|
|
|
|
|
managing Juju environments from the command line.</p>
|
|
|
|
|
<p>In this case, I created and exported my Warren bundle, but am not yet able to
|
|
|
|
|
utilize it. This is because the Warren charm does not yet live in the Juju
|
2020-06-24 07:10:07 +00:00
|
|
|
charmstore - it’s only deployed locally - and bundles do not support local
|
|
|
|
|
charms yet. For now, though, I’m happy to have the bundle here as a reference
|
2020-04-25 04:55:06 +00:00
|
|
|
to use when deploying Warren to a cloud.</p>
|
|
|
|
|
<h2 id="the-good">The Good</h2>
|
|
|
|
|
<p>My list of good things from this post will probably just read about the
|
|
|
|
|
advantages of using Juju. Once I had the charm up and running, it was
|
|
|
|
|
astoundingly easy to repeat the process: simply <code>juju deploy
|
2020-06-24 07:10:07 +00:00
|
|
|
local:trusty/warren-charm</code> from my charm repository. I have to say, there’s no
|
|
|
|
|
beating that. In the past, I’ve deployed and managed just about every one of my
|
2020-04-25 04:55:06 +00:00
|
|
|
projects by hand, and the lack of repeatability led to a ton of external cruft.
|
|
|
|
|
Deployment checklists, extended QA times, slower iterations, and so on. I think
|
|
|
|
|
that Juju helps alleviate all of these to some extent.</p>
|
|
|
|
|
<p>Additionally, in writing and testing the charm, I found all of the weak spots in
|
2020-06-24 07:10:07 +00:00
|
|
|
my code that weren’t evident from a development environment. For instance, I
|
2020-04-25 04:55:06 +00:00
|
|
|
had relied on several hacks and shortcuts supplied by the libraries I was using
|
|
|
|
|
which, while they looked elegant, were not good solutions for a repeatable
|
|
|
|
|
testing environment or a production environment. For instance, I relied on
|
|
|
|
|
defaults for hosting static files and templates. While this made my code look
|
|
|
|
|
clean and simple, it was not repeatable because there were no guarantees where
|
|
|
|
|
those assets would be stored. Once I charmed Warren, that became evident, and I
|
|
|
|
|
found ways to create a robust means of dealing with these problems that was both
|
|
|
|
|
elegant, and non-magical.</p>
|
|
|
|
|
<h2 id="the-bad">The Bad</h2>
|
|
|
|
|
<p>Docs.</p>
|
|
|
|
|
<p>I found the documentation around the python Charm Helpers, a package used for
|
|
|
|
|
building and supporting charms, to be quite lacking. It feels as though new
|
|
|
|
|
features were implemented in code while the documentation had not yet been fully
|
|
|
|
|
written; what docs are there read as though the user should already understand
|
|
|
|
|
how to use the package.</p>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>As a developer, I’m well aware of this pattern, and definitely guilty of it
|
2020-04-25 04:55:06 +00:00
|
|
|
myself. Even so, I was left spinning my wheels for days as I tried to charm up
|
2020-06-24 07:10:07 +00:00
|
|
|
Warren. I’ve helped with the charming of several different packages that we use
|
2020-04-25 04:55:06 +00:00
|
|
|
within the company, but none of them use the new, undocumented features of charm
|
|
|
|
|
helpers.</p>
|
|
|
|
|
<p>In the end, I wound up working off the old style of creating a python based
|
|
|
|
|
charm - that is, rather than utilizing the <a href="http://pythonhosted.org/charmhelpers/examples/services.html">service
|
|
|
|
|
framework</a> included
|
|
|
|
|
with charm helpers, I created methods that were decorated as hooks, which were
|
2020-06-24 07:10:07 +00:00
|
|
|
called simply whenever one of those hooks was fired. This is the style I’ve
|
2020-04-25 04:55:06 +00:00
|
|
|
most commonly seen in the charms hosted in the charmstore, and while it does
|
|
|
|
|
work, I get the impression that this newer services framework is much more
|
|
|
|
|
elegant and easy to use. Easy to use once, that is, you understand <em>how</em>.</p>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>It just so happens, though, that the task that I’ve given myself over the next
|
2020-04-25 04:55:06 +00:00
|
|
|
few months of free-project time at work is to help improve the documentation of
|
2020-06-24 07:10:07 +00:00
|
|
|
the products and services that we have, whether that’s code-side in the form of
|
2020-04-25 04:55:06 +00:00
|
|
|
docstrings, wider documentation included with the projects, tutorials, or
|
2020-06-24 07:10:07 +00:00
|
|
|
videos. Indeed, that’s much of the purpose behind these dogfooding posts! So
|
|
|
|
|
I’ll be working just has hard as the other teams, I hope, to help document Juju
|
2020-04-25 04:55:06 +00:00
|
|
|
and how it can be used for managing cloud deployments.</p>
|
2020-06-24 07:10:07 +00:00
|
|
|
<h2 id="whats-next">What’s next</h2>
|
|
|
|
|
<p>Next, I’d like to talk about more specifics about how the Warren charm works and
|
|
|
|
|
the services with which it’s deployed. I hope to make this at least a short
|
2020-04-25 04:55:06 +00:00
|
|
|
series, if not an account of my ongoing adventures with dogfooding Juju for my
|
|
|
|
|
own purposes.</p>
|
|
|
|
|
</article>
|
|
|
|
|
<footer>
|
2020-06-24 07:10:07 +00:00
|
|
|
<p>Page generated on 2020-06-24</p>
|
2020-04-25 04:55:06 +00:00
|
|
|
</footer>
|
|
|
|
|
</main>
|
2020-06-24 07:10:07 +00:00
|
|
|
<script type="text/javascript">
|
|
|
|
|
document.querySelectorAll('.tag').forEach(tag => {
|
|
|
|
|
let text = tag.innerText;
|
|
|
|
|
tag.innerText = '';
|
|
|
|
|
tag.innerHTML = `<a href="/tags.html#${text}">${text}</a>`;
|
|
|
|
|
});
|
|
|
|
|
</script>
|
2020-04-25 04:55:06 +00:00
|
|
|
</body>
|
|
|
|
|
</html>
|
