makyo
/
zk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
zk/work/gitea-4.md

16 KiB
Raw Blame History

How To Set Up CI/CD with Drone CI and Gitea

In the process of developing and releasing software, one of the most useful tools at a developer's disposal is a solution that performs continuous integration and continuous deployment, or CI/CD.

Continuous integration (CI) refers to the practice of multiple developers merging their changes to a codebase back into the main branch often, perhaps daily or even more frequently, relying on smaller chunks of work that are more easily reviewed to and tested during the process of development. In order to ensure that these changes don't leave the main branch in a broken state, a tool that helps with continuous integration will often include functionality that runs test suites within the software package. When a developer proposes a merge of their work back into the main branch, the tests will run automatically and report back whether or not they pass, which may mean that the code is not ready to land.

Continuous deployment or continuous delivery (CD) refers to the idea that changes to code --- whether they're security patches, bug fixes, or new features --- be released quickly and seamlessly to the end users of the application. In the case of applications that the user runs, this will mean a frequent release schedule, offering incremental improvements to the software they are using. In the case of a service such as a website, this will mean frequent deployments of the new features in the code to the site so that the user's experience is continuously improving.

There are several tools that can help with CI/CD. These run as their own web services, which work with many source code management (SCM) systems to perform these tasks automatically. Drone is one such solution, that provides a way of executing pipelines --- sets of steps that the service will run through such as building the software and running tests --- that are described in the code itself. Drone is a flexible CI/CD tool that can perform this on many different types of runners, containers or machines where the project can be repeatably built.

This tutorial aims to show how Drone can integrate with the source code management tool Gitea in order to offer fully self-hosted solutions for SCM and CI/CD. You will be installing Drone, connecting it with Gitea, and creating a sample project for Drone to run tests on in order to see how the two services work together.

Prerequisites

In order to complete this tutorial, you will need the following:

Step 1 — Creating an OAuth Application in Gitea

The first step to integrating Drone with Gitea --- before even installing Drone itself --- is to create an OAuth2 application in Gitea. OAuth2 is a way for one service provider to delegate access to another. For instance, you may want to be able to share information on one site without creating an entirely new account there when you already have an account elsewhere with that information. If both sites work with OAuth2, then you can authorize the first site to have access to that information on the second site. In this case, Drone, as an OAuth2 client of Gitea, will be granted access to information such as repositories and pull requests that it will need to run its CI/CD tasks.

OAuth2 works by creating a client ID by which a client (Drone, in this case) identifies itself and a client secret by which it authenticates itself. Gitea will generate these values for you to provide to Drone when starting it.

To generate the ID and secret, log in to your Gitea instance and click on your icon in the upper right corner to select Settings from the drop down menu. In that page, you will see a row of tabs along the top. Click Applications, and you'll be presented with a screen allowing you to create OAuth2 Applications.

Manage OAuth2 Applications

Enter Drone CI or similar as your application name. For the redirect URI, enter the domain you have chosen for your Drone instance. This should take the form of https://<^>your_domain<^>/login --- it's important that the protocol (HTTPS, in this case) and domain name match exactly, and that you include the /login path at the end of the URL.

When you click Create Application, you will be presented with a screen showing the information that you just entered along with the OAuth2 client ID and client secret. Copy these both into a temporary document now, as they'll be hidden as soon as you navigate away from the page. If you do lose them, note the Regenerate Secret link, which will allow you to create a new secret that you can use for your Drone installation.

Step 2 — Installing Drone

Now that you have your OAuth2 application created in Gitea, you can begin installing Drone. For this section, you will need the client ID and secret created in Step 1, the domain names for your Gitea instance and Drone instance, and an RPC secret. For this example, we will be using sammy-says.

On the server you created in the prerequisites, log in as your user and create a new directory named drone and move into it:

mkdir drone
cd drone

Now, create a new file named docker-compose.yml using your preferred text editor. The following example uses nano. This file will contain the descriptions of the containers that will run as part of your Drone installation:

nano docker-compose.yml

Add the following into this new file, changing the highlighted values as required:

version: "3"

services:
  server:
    image: drone/drone:2
    container_name: server
    environment:
      - DRONE_GITEA_SERVER=https://mscottclary-test.do-community.com
      - DRONE_GITEA_CLIENT_ID=06e6fcce-4b10-40f3-9672-8190849e6e02
      - DRONE_GITEA_CLIENT_SECRET=hAihmhcdag9kXvQQdGh2XfIZHzKXIBJAxRJirvAXUIKu
      - DRONE_RPC_SECRET=sammy-says
      - DRONE_SERVER_HOST=mscottclary-drone.do-community.com
      - DRONE_SERVER_PROTO=https
      - DRONE_TLS_AUTOCERT=true
      - DRONE_AGENTS_ENABLED=true
    volumes:
      - ./drone:/data
    ports:
      - "80:80"
      - "443:443"
  runner:
    image: drone/drone-runner-docker:1
    container_name: runner
    environment:
      - DRONE_RPC_PROTO=http
      - DRONE_RPC_HOST=server
      - DRONE_RPC_SECRET=sammy-says
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NAME=gitea-runner
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - "3000:3000"

Let's walk through what this file does:

  • version: "3": this lets Docker Compose know what version of configuration file this is.
  • services: for each of the containers that we have, we set up environment variables available to the service, assign volumes for data that's stored on the host, and expose various ports. Here are our services:
    • server: this service uses the drone/drone:2 image, meaning that it runs the drone server itself. The environment variables passed in provide the service with information such as how to connect to and authorize with Gitea, a secret for the remote procedure calls (RPC), and with DRONE_TLS_AUTOCERT, we let the drone server provision a SSL certificate using Let's Encrypt. The container exposes port 80 for HTTP and port 443 for HTTPS. The local directory drone will be mapped to the container's /data directory, allowing all of Drone's information to be stored locally, meaning that it can be backed up and persist across containers.
    • runner: this service uses the drone/drone-runner-docker:1 image, meaning that it will be in control of provisioning Docker containers for running tests on your code. This runner is what connects to the server container using RPC over port 3000 to start runs and report on the results. Because of this, /var/run/docker.sock is mapped as a volume. This special file is the means by which the runner process can communicate with Docker to start these containers.

<$>[note] Note: There are several different types of runners for Drone, each of which provides different benefits. Docker runners are good for ephemeral actions such as running tests, as they are cleaned up after the run completes and they do not persist any data. If you need the ability to persist data --- deploy your service, for instance --- you will need to use a different runner such as the Exec Runner or DigitalOcean runner. For more information on the available runners and the reasons for using them, the Drone Runner Documentation has instructions for each. <$>

Now that your Docker Compose file is complete, save and close it. If you used nano to edit the file, you can do so by pressing CTRL + X, Y, and ENTER.

With this file in place you can then bring the containers up using Docker Compose:

docker-compose up

This command will pull down the images, start the server and runner containers, and will return output like this:

[+] Running 5/5
 ⠿ server Pulled
   ⠿ 79e9f2f55bf5 Pull complete
   ⠿ 3534e21ebea8 Pull complete
   ⠿ 2f27386bf47c Pull complete
   ⠿ 631cac189eb7 Pull complete
[+] Running 2/2
 ⠿ Network drone_drone  Created
 ⠿ Container server     Created
Attaching to server
server  | {"acme":true,"host":"mscottclary-drone.do-community.com","level":"info","msg":"starting the http server","port":":443","proto":"https","time":"2022-06-29T21:01:12Z","url":"https://mscottclary-drone.do-community.com"}
server  | {"interval":"30m0s","level":"info","msg":"starting the cron scheduler","time":"2022-06-29T21:01:12Z"}
server  | {"interval":"24h0m0s","level":"info","msg":"starting the zombie build reaper","time":"2022-06-29T21:01:12Z"}

Give this a few minutes to finish running the Let's Encrypt certificate provisioning.

This will leave the container running in the foreground, however, and it will stop as soon as you exit the process with Ctrl + C or by losing your connection. In order to have the container run in the background as a separate process, you can append the -d flag to the Compose command:

docker-compose up -d

You will be notified when the container starts and then returned to your shell.

Step 3 — Connecting Drone to Gitea

Now that Drone is up and running, you can connect it to Gitea to authorize runs via OAuth2. Visit your Drone URL in your browser, where you will see a message that says "You will be redirected to your source control management system to authenticate" above a Continue button. If all of the information that has been entered to now has been valid, you will find yourself on a Gitea page asking for your authorization to give Drone permissions.

Authorize page

<$>[note] Note: If you run into an error saying that a client ID or redirect URL was not recognized, check the values entered in your docker-compose.yml to ensure they match your domain names and Gitea OAuth2 information exactly. When you are sure, running docker-compose restart will bring the containers back up with the correct information. <$>

Once you grant permission, you will be returned to your Drone dashboard, where you will see a list of your repositories.

Step 4 — Setting Up a Project for CI/CD

For this tutorial, we will be testing Drone's continuous integration capabilities for automatically running tests. In order to do so, create a sample repository in Gitea. Clone that repository to your development machine and move into the project directory:

git clone git@<^>your_gitea_domain<^>:sammy/drone-test.git
cd drone-test

Create a new file named drone_test.py and enter the following into it:

assert 1 == 1

Save and exit the file, then add it to your repository and push your changes:

git commit -am "Added test file"
git push origin main

Now, return to your Drone dashboard and click on the repository.

<$>[note] Note: If your repository does not appear in the list, you may need to click the Sync button in the upper right corner of the dashboard. <$>

In the settings tab, click + Activate Repository. This will tell Drone that it should be watching for changes from that repository.

Now that Drone is active for your test project, you will need to create a pipeline. Pipelines describe the steps that Drone should take in order to test your software. These are stored in a file named .drone.yml.

In your dev checkout of the project, create a new file named .drone.yml and add the following to it:

---
kind: pipeline
type: docker
name: default

steps:
- name: test
  image: python
  commands:
  - python drone_test.py

Let's take a look at what this file does. After the header of ---, we see that the type of configuration we are working with is a pipeline that runs on Docker containers. It has one step named test using the official python image, and when that container runs and checks out your repository, it will run the command python drone_test.py. Since this file contains a simple assertion that 1 is, indeed, equal to 1, the tests should pass.

Save and exit this file, then add it to your repository and push your changes as you did before.

Visit your Drone dashboard in the browser once more and click on your active repository. You should see that it has started running your pipeline. If it has not started yet, feel free to click the + New Build button in the upper right corner. When you click on the build, you will be presented with a few of the logs from that build updated in real time as your project is cloned and then your test step run. Since it turns out that 1 does, in fact, equal 1, your tests should pass and you will receive a green check mark.

What about 2, though? Check if 1 is equal to 2 by editing drone_test.py and updating the assertion to look like this:

assert 1 == 2

Commit and push your work as you did before, and then return to your Drone dashboard. Thankfully, 1 does not equal 2, and so our test fails as expected.

These results are also visible in Gitea, where you can see whether a particular commit broke the build. Visit your repository in Gitea and click on the Commits link above the file listing. You should see all of your commits that went into creating this test project. Before the commit where you checked whether 1 was equal to 2, you should see a red × indicating that CI has failed, whereas you should see a green ✓ indicating that the build passed. Prior to that, before the repository was hooked up to Drone, you should see no indicators.

Conclusion

In this tutorial, you set up the self-hosted CI/CD service Drone to work with your self-hosted Gitea service. CI/CD services can help keep your project in a clean state and your builds fresh and fast. Drone allows you to automatically run tests, checks, and deploys on your projects. Here, we are simply running a check every time a commit is pushed to Gitea, but the configuration files for Drone are flexible, allowing you to run checks on merges using triggers, run only on certain events using conditionals, and so on. For more information, the Drone Pipeline Docs describe many common scenarios.