As a former Lead Tools & Build Engineer at a small games studio, I often wondered, can we roll out updates to Jenkins, configs and jobs better…

The stability of a Jenkins controller is crucial, especially when it has become so crucial to everyone’s daily workflow, our controller, for instance, was responsible for the following:

• Regular builds to ensure stability and for publishing to QA
• PR Checks to do script-only builds and prevent compilation errors entering the mainline
• On-demand builds (either directly or via PR Comments)

This meant that any downtime on the Controller resulted in delays to pull requests (PRs) being approved for merging, a lack of on-demand build support, or, in worse cases, no builds for testing or shipping. Therefore, it was crucial to minimize downtime as much as possible.

On my own time, I opted to explore ways to avoid some of the key risks in updating a Controller such as:

• Updating a Job Config
• Updating Plugins
• Applying a Jenkins Update
• Modifying Settings

Each of the above comes with its risks, and with limited hardware available, setting up a staging environment was neither possible nor practical since we needed a reliable way to fully mirror each job's settings or configs.

Some of the key requirements for a replacement I had were

• Source-controlled
• Locally testable
• All configs, pipelines, settings etc must be defined by code
• Automated deployments

We already used pipelines for our jobs, and each project had its own pipeline file in its repository. However, all our settings and Jenkins configs had only ever been set up via the UI, so changing these settings happened live. If we made a mistake, the build farm could go down or pipelines become broken, causing delays for the team. To solve this problem, we needed a way to replicate these settings so that we could test changes before making them live.

Enter Docker…

Around this time I discovered that we no longer had to install the controller as a WAR file and instead we could deploy a docker image, and this is where the cogs began to turn…

With the docker deployment method, came the ability to customise that image, and at this point, the Configuration as Code Plugin was becoming stable and meant that we would be able to define the configuration of our controller in a code file. Adding this file to the docker image began to open a world of possibilities…

This initial version solved one key issue, we could now replicate our exact production config and run it locally or even on staging machines in the future.

Replicating the config was one thing, but much of the important information about how a job ran was in the job config, something Configuration as Code didn’t solve. Our options here were seemingly limited to the Job DSL plugin, however, this didn’t fully cover the options our jobs used and was laborious to write. This is when I stumbled across a far better option, XML. Internally Jenkins stores job configs in XML, we didn’t need to rewrite job configs into some other format to rebuild them in the new image, because we already had an easy-to-read config that could be source-controlled. Better yet, we could configure the job in Jenkin’s UI, save it and simply copy the XML file into our Jenkins Controller Repro to be included in the docker file — so long as we copied this to the correct folder in the image Jenkins would accept this job already existed.

At this point, we’d now achieved most of our goals; The config was defined by code, Job Configs were included in the docker image using XML files, and all of this was in source control. All of that meant I could run the image on my machine and have a replica of our production environment for me to make changes and test the results.

The last remaining goal, automating the deployment pipeline.

One important thing to note here, I didn’t want Jenkins involved in any part of this pipeline, if I accidentally pushed a change that broke it, I needed to be able to use the same CI/CD as normal to recover the system. This meant looking to Github actions to support deploying Jenkins.

Building and distributing the image was pretty straightforward using existing actions, but getting the Jenkins Controller machine to grab the latest image automatically meant some extra configuration was needed. This came in the form of WatchTower, a fantastic docker image that watches for updates to my other images and then pulls them. Using this we could schedule the new image to be pulled regularly at a time we knew it would be quiet.

In summary, this new approach offers several advantages over the previous setup. Unlike before, where all changes were made live and couldn't be tested or easily reverted, the new approach stores everything in source control. This allows for a thorough review before pushing to the mainline for automatic deployment at a safe time, typically overnight. In the event of issues, we have the capability to roll back to the previous image. Additionally, access to the live controller is heavily restricted as there should be no need for direct changes.

I’ve skipped over a lot of the detail here and will write a follow-up post delving into that detail, so follow along, leave a comment or clap to show your support.

Previously we briefly covered what a repository is and how to get one, now let’s talk about commits.

Commits are the lifeblood of Git, they describe every single change from the start of your project to its current state. Each commit describes a single set of changes in your project, whether that was adding lines or files, removing them or just editing those lines. All that information is stored in a commit. Whilst Git will automatically track what you’re changing, the creation of a commit is entirely driven by the user, you choose what to commit, and when.

A commit is made up of a few important elements:

• The change diff — This is in the form of a text snippet that describes the type of change and the new data in a line-by-line format.
• A commit message — This is a message written by the author to describe the change that was made. An author — The person who created the commit
• A unique identifier — Also known as the commit sha

To create a commit you’ll first need to “stage” some files in your chosen git gui, this is essentially a process of ticking which files should be in your commit — very useful if you’ve worked on multiple things at the same time but want to create multiple commits to describe the work you did. You’ll then need to write a message to describe what you’ve done, and then just commit and you’re done.

So, why should you create a commit, well it has a few purposes really:

1. It’s a useful "checkpoint" of a time when something functioned in a particular way, for example, maybe you made a change to the characters movement speed and thought it was good, a commit allows you to record that and continue making changes, safe in the knowledge you can get back to where you were before.
2. Commits are the backbone of Git and without them, you don’t have any way back up your changes
3. They provide a record that helps remind you of what you’ve done.

What about when to create a commit… whilst there’s no strict rule on when to commit, and it’s not an automatic process these are some good guiding principles for your commits:

• Each commit should be done at a time when your changes are functional, that is to say, the project could be built and run.
• A commit should describe a singular change, this is to make it easier to undo a specific change if needed and to maintain a cleaner version control history
• You should commit little and often, meaning to make small meaningful commits regularly.

When writing a commit message it’s important to ensure the message is clear but concise, the Conventional Commits standard is generally a good guide to follow when writing your commit messages, it enables you to quickly filter through commits to find the different types of commits you’ve made (bug fixes, features etc) and better yet you can ultimately use it to auto-generate a change list. Here’s an example of a conventional commit:

feat(player): added a new character

Conventional Commits

So in conclusion, create regular focused commits and write yourself and your colleagues helpful structured commit messages.

If you’ve found this helpful please drop me a follow or a like to help others find this content and to get updates when I post new articles

I see plenty of resources for teaching Git to programmers, this is quite natural of course since programmers spend a lot of time using git. But in the games industry especially, plenty of other disciplines utilise git a lot in their everyday work.

This series aims to describe Git, and how to best use it, in a much less technical manner.

Before we start I highly recommend using a program to interact with Git, typically referred to as a Git GUI (Simply meaning Git graphic user interface). My personal favourite is GitKraken but GitHub Desktop offers a very nice easy workflow as well. Many people will insist you learn git via the command line however, whilst useful this is overkill for most early users.

Before we begin… let’s quickly define what Git is:

Quite simply Git is software that allows multiple users to store and track the history of various files. Think of it as a clever way to back up your files at regular intervals, while simultaneously being able to share those files with others. Typically Git is used with code or text files but can be extended to other data types, including binary files (like images and models) with a little extra setup.

Unlike a regular back up however Git doesn’t store the entire file, what Git stores instead is information about the change you made. It calls this a commit, which is essentially an advanced save with some additional data.

A Git project is called a repository and there are many hosting services for Git including GitHub, GitLab and Bitbucket. A repository in its simplest form is a list of changes that describe the project from its creation to its current state.

So, how do you get a repository? Well, the first step is to pick a hosting service, GitHub is one of the more popular options at the moment and is great for new starters. Once you’ve created an account you’ll be guided through the process of creating your first repository by GitHub, at the end of which you’ll be able to perform an action known as cloning this is essentially just copying the repository to your computer. Cloning in a git GUI is as easy as logging in and picking the repository you want to clone, and where to. This will then copy all of the changes to your machine and allow you to start making your own changes.

One last important note, a repository clone is exactly that, you have an exact copy of the remote, hosted, version of the repository. This means that any changes you make are local to you until those are pushed back to the remote, we’ll cover that in more detail later though.

In conclusion, a Git repository is made up of a series of commits, each of which describes a change that someone made to the project. You clone these changes to your machine to get a local version of them and work within the project.

In a future article, we’ll start looking at what a commit is in more detail.

If you’ve found this helpful please drop me a follow or a like to help others find this content and to get updates when I post new articles

Technical skills are only part of what recruiters and hiring managers in the Games Industry seek. The other, often overlooked, skills are equally as important in this industry that demands a high level of collaboration. Let’s look at the soft skills that will help you join or progress within the games industry.

Here are a few of the things I am personally always on the lookout for in a candidate:

Communication

A programmer doesn’t just know how to write good code all day long, though that’s a key part of their role, it’s equally important to be a great communicator. Programmers spend a large portion of their time communicating with their peers, designers, producers and many others across the project, knowing how to communicate effectively massively increases your value as a programmer. It enables you to work closely with design to implement their vision, keep your producer up to date, and work with other programmers to help build your understanding of the systems you’re working in/with and resolve any technical challenges you face.

Willingness and Ability to learn

This one is a hugely important attribute I look for, often I’d rather take on a less technically competent candidate if they show a willingness and ability to learn over a candidate who is technically brilliant but has no aptitude towards learning. Let me explain why, the candidate who is willing to learn will do significantly better in the long term as the games industry is constantly evolving with new techniques, new technology, new hardware etc. More importantly than that the game itself is always changing and adapting, the underlying systems are ever-expanding and refactoring. Someone less willing or able to learn will ultimately have a harder time keeping up and will be a much less effective programmer in the long run.

Interest in games

This one is fairly obvious I would hope, but if you want to work in the games industry it helps to have an interest in games, not just playing them, but understanding how they work, what makes them tick, why it’s fun, why that system is implemented the way it is and the impact that has on the player. All this helps you anticipate problems in the systems you build and design your solution around this. On a more day-to-day note, you’re going to play your game A LOT, so having an interest in playing and analysing games helps.

Ability to take initiative

Being able to take initiative is a great attribute to have, in programmers, this often manifests in the form of cleaning up nearby code while working in that area, or fixing up another bug you spot whilst working on a feature. This goes a long way to improving the overall code in the game and helps you passively tackle tech debt, which all too often goes without dedicated time to address it.

Humility

For me, this translates to a very key skill, knowing when you need help. It is always okay and often preferred, to ask for help. All too often I see a programmer spending days giving vague updates on their work before ultimately admitting they’re stuck and need some help. Usually, this comes from a point of pride or not wanting to seem incompetent. But game development is hard, and often someone else on the team may have experienced a similar issue to you before and have suggestions that can save you days. Knowing when to reach out for help is crucial, don’t reach out too fast, but equally leaving it a full day isn’t good either. A good rule of thumb is if, after a couple of hours, you’ve made no real progress and have no solid leads, reach out to your colleagues, your producer, or your lead. They can all help point you in the right direction.

If you enjoyed this, clap or subscribe to support the content!

In Part 2, we learnt how to set up an automated build process for Unity to generate regular builds with automatically generated release notes.

Setting up a CI/CD pipeline for Unity Part 2

In this next part, we’ll look at some improvements and optimizations we can make to the pipeline. We’ll look at the following options:

1. Improving the stability of main by enforcing PRs are up to date before merging.
2. Caching to speed up job runs
3. Separation and Parallelisation of Jobs
4. Ensuring your main branch only contains conventional commit messages
5. Self-Hosted Runners and Cache Accelerators
6. Only testing code changes

1. Improving the stability of main by enforcing PRs are up to date before merging.

When setting up our build pipeline in the previous article, we noted that we still had to test our main branch because we couldn’t easily enforce PRs being up to date before we merge. Well, GitHub has some tools that will help with this! Firstly we should tick the “Require branches to be up to date before merging” branch protection rule in the Branches part of the repository settings.

This change alone however will cause us lots of problems in a particularly busy repository. Our very tiny project with just a single script, a few Unit tests and no playmode tests already takes over 5 minutes to run! On bigger projects, this will grow by quite a bit. Because this rule ensures that you have the latest changes before you can merge, as soon as someone else merges, you have to update your PR, GitHub can automate that for you with a handy button on the PR page, but it adds a lot of time and manual back and forth from developers.

Introducing the Merge Queue!

This handy feature allows you to queue up Pull Requests, and automatically merge the latest changes from the target branch, main in our case, into all pull requests in the queue. But the clever part, is it also merges all the changes from PRs ahead of it in the queue. This means that the 3rd PR in the queue, for example, would contain the latest changes from main, the changes from the first PR in the queue and the second PR. Each PR in the queue is then tested again to ensure that they still work with what main would contain once merged.

Because these can now be tested in parallel the rate at which PRs get merged back into main is much improved. It also has the added advantage that you can now guarantee main will always be in a state where it compiles and the unit tests pass. One downside worth mentioning is that this will eat your free (or paid) minutes faster as more checks are being run, 1 to validate your PR works on its own before it’s added to the queue, and 1 again once it’s in the queue with all the changes ahead of it. If you’re using self-hosted runners you’ll need to support the increased concurrency as well to get the benefits of this.

Merging a pull request with a merge queue - GitHub Docs

2. Use caching to speed up job runs

As you may already know, when you open Unity it will attempt to import any new files and process them to apply any rules or settings you’ve applied to them. This can be a pretty long process after a big PR with many files has been merged or when you open a project for the first time.

Currently, our pipeline, unfortunately, falls into the latter bracket, and every single time we run the pipeline it’s re-importing all of the project files from scratch. So, let’s see how we can fix that. GitHub actions have a very handy caching feature we can use here, it allows us to capture all of the import results generated by the build. We can then restore this cache at the start of the next build, import anything that has changed or is new and those will then be re-cached for the next build. This will massively speed up both of the CI/CD processes that we’ve made.

      # Cache
      - uses: actions/cache@v3
        with:
          path: Unity CI CD/Library
          key: Library-PR
          restore-keys: |
            Library-

The caching process isn’t fast for our use case however, so we’ll further optimise by using a different cache, we can do this by changing the cache key, for the build and test processes. The unit tests won’t need the full set of library imports as it’s a script-only process and won’t import other files like art assets which are much larger and will take a longer time to download.

3. Separation and Parallelisation of Jobs

If we don’t want or can’t use the PR queue feature then there’s another option we have. GitHub workflows are broken up into jobs. Currently, our workflows all use a single job. If we broke these up into 3 distinct jobs: Test, Build and Release, we could parallelise the Test and Build jobs since they’re not dependent on each other.

We can break the pipeline up something like below:

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest
    steps:
      #Checkout
      #Test Code
  
  build: 
    name: Build
    runs-on: ubuntu-latest
    outputs:
      releaseNotes: #release notes step output
      tag: #version step output
    steps:
      #Checkout
      #Build Code

  Release:
    name: Release
    runs-on: ubuntu-latest
    needs: [test, build]
    steps:
      #Release Code

(Full code in the repository at the end of the article)

Here’s how our pipeline looks like after the optimisation:

A few things to remember when breaking up your job

1. Each job that needs the code will need to checkout its copy of the code.
2. The way you access the outputs of previous jobs is a bit different to the outputs of previous steps and requires outputs to be defined at the top of the job.
3. Use needsto set up prerequisites if you want a job to wait for other jobs before they run.

4. Ensuring your main branch only contains conventional commit messages

We mentioned a previous article about the power of conventional commits in generating automated changelogs. But these aren’t always the most friendly format for less tech-orientated disciplines in their commit messages. We still want to harness the power of these in our pipeline though, so instead of enforcing these at a commit level, we’ll enforce them at the PR level.

First, let’s set up our PR merge rules.

To best support the workflows we want to encourage on our repository we disabled merge commits.

Looking at the remaining strategies we have Squash and Rebase. Our preferred option is Rebase as this provides a very clean history and preserves each commit from the PR, it also allows multiple changelog updates from the conventional commits. We also provide a Squash option for those who aren’t using conventional commits, and instead, we’ll enforce the conventional commit style on the title.

We’ll do that with another GitHub action and make this a required check in our branch protection setting.

name: Check PR title
on:
  pull_request:
    types:
      - opened
      - reopened
      - edited
      - synchronize

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: aslafy-z/conventional-pr-title-action@v3
        id: pr_title_check
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: PR Comment
        if: ${{ failure() }}
        uses: thollander/actions-comment-pull-request@v1
        with:
          message: |
            Add a prefix like "fix: ", "feat: " or "feat!: " to indicate the change type this pull request corresponds to. The title should match the commit mesage format as specified by https://www.conventionalcommits.org/.

This check should ensure all titles are matching the conventional commit spec. It’s not perfect however and in the future, we could use some advanced automation to auto-merge PRs in the correct manner based on their contents.

5. Self Hosted Runners and Cache Accelerators

GitHub’s self-hosted runners can only take you so far before you run out of free credits for the month, or need a more powerful machine for its faster speeds or bigger hard disk space. At this point, you’re faced with 2 options

1. Join GitHub’s larger runners beta program and pay for the more powerful machines.
2. Set up your own infrastructure.

Option one is certainly our easiest option if you want to stay hands-off from the infrastructure side of things.

However, option two comes with its own set of benefits (and problems) that we should consider. Firstly if we have our own infrastructure either cloud or locally based we will have greater control over the setup of those machines, ensuring we have the exact hardware we desire. But we could also set up a Cache Accelerator alongside our infrastructure. This would replace the caching changes we made above since Unity, with some extra arguments, can be configured to ask the cache accelerator for any library artifacts it needs and then download them instead of restoring a huge amount of data from the cache on the assumption we need it.

6. Only testing code changes

Imagine, every time you add art assets to your game you have to wait for the Unit tests to pass… This doesn’t make any sense, we don’t test the art with Unit tests and they should have no influence on our tests.

So let’s remove this case from our pipeline:

#Run on all pull requests that contain script, json or yaml changes
on: 
  pull_request:
    paths:
      - '**.cs'
      - '**.yml'
      - '**.json'

By adding this to the top of our PR Check job we ensure we only run the check if the check can influence the result of the pull request.

We also need to add another job, art_pr_check.yml to our project

name: Test project

#Run on anything that doesn't have .cs or .yml files
on:
    pull_request:
      paths-ignore:
        - '**.cs'
        - '**.yml'
        - '**.json'

jobs:
    build:
      name: PR Check
      runs-on: ubuntu-latest
      steps:
        - run: 'echo "No build required"'

This check uses the same name as our PR Check deliberately because it’s a required check, a check with this name MUST pass. So with the above two changes in place, we can ensure that our art PRs aren’t slowed down with Unit tests that they can’t break and everything can still be merged.

Conclusion

That about wraps it up for our optimisations and improvements for now, but that work is never complete so there’s always plenty more to do.

I’ll leave this on one last note, don’t just apply all these optimisations, take the time to understand your particular pipeline, understand where the bottlenecks are, where the improvements are needed, and strategically deploy improvements and optimisations where they are needed most.

As always, the code can be found on GitHub:

Release Release v0.0.2 · RunningMattress/UnityCI_CD_Pipeline

If you enjoyed this, clap or subscribe to support the content!

Setting up a CI/CD pipeline for Unity Part 1

In this second article, we’ll cover the CD side of things by way of setting up a pipeline for automating the build and release of our project. We’ll reuse a lot of the same concepts from our previous article and stick with GameCI for building.

Let’s define what our CD process needs to cover:

1. Something will need to trigger the pipeline
2. We need to checkout the project
3. Ideally, we should test it before we build it.
4. Next, we’ll do the actual build of the project
5. Finally, we need to release to a destination of our choice (In this article we’ll use GitHub Releases)

For our example project, we’re gonna imagine we’re ultimately aiming to ship a mobile project, and for simplicity’s sake, we’ll pick Android for our first pipeline. However with some very minor and easy adjustments everything we’ll talk about here can easily be adjusted for other platforms.

How often to trigger the build

We have a few different options to trigger our build. In a true CI/CD setup you want to deploy as often as possible, but realistically you should scale this to suit your team’s needs and capabilities.

Here’s some options to consider:

1. Every commit
2. Every day
3. Every week

The less frequent the builds the more changes are likely to be included and therefore the more possibilities for bugs to be introduced. So pick your cadence wisely but be cautious about creating more builds than you can test. With that in mind, we’ll create a weekly build trigger for now. As before we’ll make a new workflow file at .github/workflow/build.yml

name: Weekly Build
on:
  workflow_dispatch:
  schedule: 
    - cron: "0 0 * * 1"

Notice we’ve also added the workflow dispatch trigger as well, this leaves us the freedom to manually start our builds if we need to. Scheduling a GitHub action uses the cron syntax, if you’re not familiar with it I recommend using crontab guru as a great resource for building and understanding expressions.

Crontab.guru - The cron schedule expression editor

Checkout and Test

As before we’ll checkout and test in the same way. Unlike before we’re not operating on a PR but the GitHub checkout action will still help us out of the box by checking out the latest code on the branch it’s running on, in our case: main.

We’ll run our tests again now, this may seem counterintuitive since we ran tests before we merged. However, games can take a very long time to build and test, especially if you start adding playmode tests that need to run in real-time or near real-time. With multiple developers all raising changes all day long, we can’t easily enforce a rule to ensure PRs are fully up to date before merging. We do have a few other options however and we’ll talk about those in a future article, for the purpose of this article however we’ll continue to assume we can’t rely on the tests always being run on up-to-date code.

Build

Let’s take our new trigger code and combine it with the previous PR check code

name: Weekly Build
on:
  workflow_dispatch:
  schedule: 
    - cron: "0 0 * * 1"

jobs:
  testAllModes:
    name: Build
    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v2
        with:
          lfs: true

      - uses: game-ci/unity-test-runner@v2
        env:
          UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
        with:
          projectPath: path/to/your/project
          githubToken: ${{ secrets.GITHUB_TOKEN }}
          testMode: EditMode

      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: Test results
          path: artifacts

The build process isn’t too different from the test process, we just use the builder action instead and pass in the platform we wish to build.

- uses: game-ci/unity-builder@v2
  env:
    UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
    UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
    UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
  with:
    targetPlatform: Android
    projectPath: "path/to/your/project"
    buildsPath: "build"

As with the Test action, GameCI does all the hard work of figuring out which Unity version we’re using and installs it for us as well as any dependencies like the Android SDKs.

We can use the upload artifact action to store the build result.

- uses: actions/upload-artifact@v2
  with:
    name: Build
    path: build

This will start to consume a fair amount of space over time but we’ll look at resolving that in the next step.

Release

So now everything is building let’s look at creating a release, we will later look at more complex release pipelines and target actual storefronts, but for now, we’ll make simple GitHub releases instead.

We’ll first need to tag the release so GitHub knows what to call the release and what we’re actually releasing. We’ll put this step just above the Unity build step as we’ll need to use some of the outputs in our build.

# Tag
- name: Bump version and push tag
  id: tag_version
  uses: mathieudutour/github-tag-action@v6.1
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    release_branches: main

The above action will use our commit messages to decide on the appropriate version number, it does this based on semantic release conventions, I’m a big fan of the Conventional Commit format so this works out greatly in our favour. For example, any commit I prepend with fix: will result in a patch version bump where as feat: creates a minor version bump, and finally BREAKING CHANGE: will result in a major release.

Conventional Commits

This gives us a great deal of versatility if the conventional commit spec is followed, and in a future article, we’ll talk about how we can enforce following the spec in a manner that isn’t too controlling.

We’ll quickly amend our Unity build step now to grab the version out of our tag step. Below we’ve set the versioning type to Custom and the version we pass in is the result of our tag step ${{steps.tag_version.outputs.new_tag}}

That should enable our build version to always match the latest tag.

      # Build
      - uses: game-ci/unity-builder@v2
        env:
            UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
            UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
            UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
        with:
            projectPath: "path/to/your/project"
            buildsPath: "build"
            versioning: Custom
            version: ${{ steps.tag_version.outputs.new_tag }}
            targetPlatform: Android

On to the actual release side of things, we’ll again use another GitHub action to help us out here:

# Create Release
- uses: ncipollo/release-action@v1
  with:
    body: ${{ steps.tag_version.outputs.changelog }}
    token: ${{ secrets.GITHUB_TOKEN }}
    generateReleaseNotes: true
    tag: ${{ steps.tag_version.outputs.new_tag }}
    name: Release ${{ steps.tag_version.outputs.new_tag }}

This step reuses some of the information we previously generated when tagging the release, because as well as tagging our repository with the appropriate version it also created a changelog for us using the same conventional commit spec. This changelog uses the additional information provided in the spec to group changes by feature and change type providing a log that gives your quality engineers and users a very clear and easy-to-read log of exactly what’s new.

As mentioned above, always uploading the build to the workflows artifact storage will start consuming a lot of space over time. Since we need to upload the build to the release as well this is also wasting time uploading twice! So instead we’ll remove the artifact upload above, and add it to our release instead by adding the below line to the release step, and remove the build artifact upload (make sure to leave the test results upload!).

artifacts: "build"

Altogether your script should look something like this:

name: Weekly Build
on:
  workflow_dispatch:
  schedule: 
    - cron: "0 0 * * 1"  

jobs:
  testAllModes:
    name: Build
    runs-on: ubuntu-latest
    steps:

      #Checkout
      - uses: actions/checkout@v2
        with:
          lfs: true

      # Test
      - uses: game-ci/unity-test-runner@v2
        env:
          UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
        with:
          projectPath: "path/to/your/project"
          githubToken: ${{ secrets.GITHUB_TOKEN }}
          testMode: EditMode

      # Upload Test Results
      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: Test results
          path: artifacts

      # Tag
      - name: Bump version and push tag
        id: tag_version
        uses: mathieudutour/github-tag-action@v6.1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          release_branches: main

      # Build
      - uses: game-ci/unity-builder@v2
        env:
            UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
            UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
            UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
        with:
            projectPath: "Unity CI CD"
            buildsPath: "build"
            versioning: Custom
            version: ${{ steps.tag_version.outputs.new_tag }}
            targetPlatform: Android

      # Create Release
      - uses: ncipollo/release-action@v1
        with:
          body: ${{ steps.tag_version.outputs.changelog }}
          token: ${{ secrets.GITHUB_TOKEN }}
          generateReleaseNotes: true
          tag: ${{ steps.tag_version.outputs.new_tag }}
          name: Release ${{ steps.tag_version.outputs.new_tag }}
          artifacts: "build/Android/*.apk"

Once again raise a pull request and merge this to your main branch once the Unit tests have finished.

You’ve now got a complete CI/CD pipeline that can run as frequently/infrequently as you need!

As before the code is all available to look at on GitHub:

Release Release v0.0.1 · RunningMattress/UnityCI_CD_Pipeline

Follow for the rest of this series and other articles. Next, we’ll be looking at some improvements and optimisation we can do to this pipeline.

Throughout this series, we’ll discuss what value it can bring to your project, how to set one up, and how to scale this for larger projects.

For the first article in this series, we’ll use GitHub actions and GameCI to automate our Pull Requests back into our main branch.

Firstly let’s define a few key concepts for this series.

What is a CI/CD pipeline?

A CI/CD pipeline or Continuous Integration and Continuous Delivery is the automation of integration and delivery of your project to ensure consistency and quality. In terms of a Unity project, and in particular, what we’ll showcase in this article, this means automating the building and testing of your project before we merge it back into our mainline branch. In future parts of this series, we’ll look at automating the building and release of the project to various deployment targets.

What are GitHub Actions

GitHub actions are a yaml-based syntax that allows us to define a series of steps to automate a task. These automated tasks are run on servers hosted by GitHub with a variety of operating systems and configurations available. Often these are composed of several smaller actions such as GitHub’s clone action to checkout your repository. There’s a huge wealth of these available on the GitHub actions Marketplace allowing developers to piece together several of these to create incredibly powerful and robust pipelines.

Setting up the project

First, let’s take a simple Unity Project as an example and add some Unit tests to the codebase. This gives us something to test in the next step and starting a project with a test pipeline in place helps to further that practice as you develop more of the game.

Below you can see the very trivial feature and tests we created to prove our pipeline.

public static class SampleFeature
{
    public static List<string> UniqueStrings; 
    
    public static bool TryAddUniqueValue(string newValue)
    {
        //Init the list if null
        UniqueStrings ??= new List<string>();
        
        //Early exit if already added
        foreach (string item in UniqueStrings)
        {
            if (item == newValue)
            {
                return false;
            }
        }
        
        //Add the value
        UniqueStrings.Add(newValue);
        return true; 
    }
}

public class SampleFeatureTests
{
    [SetUp]
    public void SetUp()
    {
        SampleFeature.UniqueStrings?.Clear();
    }
    
    
    // Test that we can add a single value
    [Test]
    public void CanAddAValue()
    {
        SampleFeature.TryAddUniqueValue("test");
        
        Assert.AreEqual(1, SampleFeature.UniqueStrings.Count);
    }
    
    // Test that we can add many values
    [Test]
    public void CanAddManyValues()
    {
        SampleFeature.TryAddUniqueValue("test");
        SampleFeature.TryAddUniqueValue("test2");
        
        Assert.AreEqual(2, SampleFeature.UniqueStrings.Count);
    }

    // Test that we cannot add duplicates
    [Test]
    public void CannotAddTheSameValue()
    {
        SampleFeature.TryAddUniqueValue("test");
        SampleFeature.TryAddUniqueValue("test");
        
        Assert.AreEqual(1, SampleFeature.UniqueStrings.Count);
    }
}

Creating our simple CI/CD pipeline

Moving on to the GitHub actions side of things, as mentioned earlier GameCI will be doing the bulk of the work for our actions.

Here we want to create an action that will be run against every single pull request we raise. This will validate that the project compiles and all Unit tests pass, finally then providing some simple feedback to the person raising the pull request to let them know their PR is good to be merged.

Breaking this down into some more manageable steps that we can begin writing a script for we have:

1. Define what triggers the action (what causes it to run)
2. Checkout the project
3. Use GameCI to run the tests (this will trigger a compilation to achieve this)

Looking at GameCI’s documentation we can see we’ll need to do some work upfront to generate an appropriate serial key for the project to use so we’ll start with this.

We’ll run the provided workflow from GameCI to do this. We can then use the serial key as a GitHub Actions secret to enable our workflow to use the license.

Make a file at .github/workflow/activation.yml

name: Acquire activation file
on:
  workflow_dispatch: {}
jobs:
  activation:
    name: Request manual activation file 🔑
    runs-on: ubuntu-latest
    steps:
      # Request manual activation file
      - name: Request manual activation file
        id: getManualLicenseFile
        uses: game-ci/unity-request-activation-file@v2
      # Upload artifact (Unity_v20XX.X.XXXX.alf)
      - name: Expose as artifact
        uses: actions/upload-artifact@v2
        with:
          name: ${{ steps.getManualLicenseFile.outputs.filePath }}
          path: ${{ steps.getManualLicenseFile.outputs.filePath }}

Push this to your main branch and then follow these steps to get a licence ulf file ​

1. Follow these (one-time) steps for simple activation.
2. Manually run the above workflow.
3. Download the manual activation file that now appeared as an artifact and extract the Unity_v20XX.X.XXXX.alf file from the zip.
4. Visit license.unity3d.com and upload the Unity_v20XX.X.XXXX.alf file.
5. You should now receive your license file (Unity_v20XX.x.ulf) as a download. It's ok if the numbers don't match your Unity version exactly.
6. Open Github > <Your repository> > Settings > Secrets.
7. Create the following secrets;

UNITY_LICENSE - (Copy the contents of your license file here)

UNITY_EMAIL - (Add the email address that you use to log into Unity)

UNITY_PASSWORD - (Add the password that you use to log into Unity)

So now our license is all set up we’re good to create our pipeline. Let’s create another pipeline file in the .github/workflow folder, we’ll call it pr_check.yml.

We’ll start with the trigger conditions

name: Test project

on: [pull_request]

jobs:
  testAllModes:
    name: Run Tests
    runs-on: ubuntu-latest
    steps:

Let’s add GitHub’s checkout script.

- uses: actions/checkout@v2
  with:
    lfs: true

The default setup of this will checkout the head of your PR branch, or in simpler terms the latest commit on your branch. This makes it incredibly straightforward to use.

Next, we’ll add the GameCI step to run our tests.

- uses: game-ci/unity-test-runner@v2
  env:
    UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
  with:
    projectPath: path/to/your/project
    githubToken: ${{ secrets.GITHUB_TOKEN }}
    testMode: EditMode

And that’s it. That’s all we need to run our playmode and editmode tests.

To make it easy to view the results we’ll also upload them as artifacts.

- uses: actions/upload-artifact@v2
  if: always()
  with:
    name: Test results
    path: artifacts

Bringing that all together should give you something like this

name: Test project

on: [pull_request]

jobs:
  testAllModes:
    name: Run Tests
    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v2
        with:
          lfs: true

      - uses: game-ci/unity-test-runner@v2
        env:
          UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
        with:
          projectPath: path/to/your/project
          githubToken: ${{ secrets.GITHUB_TOKEN }}
          testMode: EditMode
      - uses: actions/upload-artifact@v2
        if: always()
        with:
          name: Test results
          path: artifacts

You can now commit that to a branch, raise a pull request and sit back and watch as the action springs to life to start validating your changes. Once this has been completed successfully, go ahead and merge it so all future branches can use this.

Set up GitHub rules

The pipeline we’ve just created is all well and good but without rules in place anyone can circumvent the check and merge bad code to the main branch causing instability, future test run breaks and ultimately someone having to spend time fixing it all.

So let’s head over to the repository settings and into the branch protections tab, from here we can create a new rule for our main branch and then add some restrictions. For now, we’ll use the “Require a pull request before merging” and the “Require status checks to pass before merging" rules and then add our check to this list “Run Tests”.

With these rules in place, it ensures that the pipeline must succeed before the PR gets merged. Of course, any admin can still override these in the event they need to. I would, however, always advise against this as when the inevitable emergency crops that makes you think you need to skip the checks are exactly the time you want something else double checking everything your most likely very stressed self has just done in an attempt to rectify the situation. Trust me, I speak from experience here. Shipping a fix that breaks more things than were previously broken is not fun.

And with that, you’re done, you have a simple pipeline that ensures your Unit tests pass on every single PR. Just remember that Unit tests are not a replacement for QA and only serve to prove that individual code units/modules are working the way you expect, there’s no guarantee how all that code will behave when you bring it all together, but you can help reduce a lot of risk with Unit testing. You can further reduce that risk by adding playmode tests which we’ll talk about a bit more in a future part of this series.

Check out the source code here:

Release Part 1 of a series of articles on creating a Unity CI/CD Pipeline · RunningMattress/UnityCI_CD_Pipeline

Follow for the rest of this series and other articles. Next, we’ll be looking at automating the Continuous Delivery side of CI/CD and creating a pipeline to create regular automated releases on GitHub. Check out part 2 here:

Setting up a CI/CD pipeline for Unity Part 2

What is Jenkins?

At a high level, Jenkins is a highly configurable free open-source build system with great community support through various plugins developed over the years. It’s very scalable and through creating your custom Groovy scripts it’s incredibly extendable as well. Here I want to share my top tips from my experience using and maintaining multiple Jenkins instances across several projects.

1. No Freestyle jobs

As far as I’m concerned Jenkins Freestyle jobs don’t and shouldn’t exist. They’re a hugely outdated way of writing your build pipeline and custom or complex functionality relies on scripts being deployed manually to the Jenkins Controller node. For many reasons, this is a terrible way to do things:

1. Bad iteration flows, you’re either editing scripts directly on the controller while it’s live, or you’re committing them to source control and then pulling them on the controller. It’s really slow and very risky.
2. There is no safe way to test this in the production environment and no branching.
3. You have to account for every other project on the Jenkins instance, this can result in some very messy code comparing project names before applying project-specific settings or processes.

2. Use shared libraries

The alternative to Freestyle jobs is scripted and declarative pipelines, both of which are capable of using Jenkins-shared libraries. These are custom libraries that Jenkins can check out alongside your project code to provide additional capabilities to your build pipeline. This can simplify complex custom tasks that need sharing across multiple projects. This does naturally add risk to those projects however as any change in the shared pipeline code will impact all projects using that code so make changes here very cautiously. If possible have a staging environment for testing these changes.

3. Blue Ocean

Available as a plugin for Jenkins, the Blue Ocean interface is a much cleaner UI for Jenkins, whilst it doesn’t offer admins everything they need it is perfect for the majority of end users. The visual representation of the build pipeline, dependent stages and parallel stages make it much easier to see where and why the pipeline failed. It’s trivial to swap between interfaces as needed and links can be generated to direct users to the desired interface.

4. Small Modular Jobs

Instead of having large encompassing jobs that contain all the steps and various configurations your project needs or uses break them down into smaller pipelines that can be reused or pieced together to create a complete pipeline.

Some examples of this could be

• Deploy to play store — a job to deploy an apk to the play store, this can be used by any Android project you have by setting the parameters to accept app id’s, APKs, credentials etc
• Build My Android Project — Project-specific pipeline for your Android build, this might also employ some shared libraries to execute more generic steps
• Build project — This pipeline could be seen as a conductor or orchestrator of other pipelines, it exists for a few purposes, kicking a matrix of builds off across all platforms on a timer, providing an easy way for an end user to start builds across platforms, etc.

With the above example pipelines you can build a great deal of power into your pipeline and reduce the number of times you re-write a simple deploy stage for example.

5. Deploy with Docker

Jenkins can be deployed in many ways, but my favourite is Docker, it’s a much more robust way of doing so, and you gain all the benefits of Docker such as an auto restart. Another huge advantage of deploying with Docker is that you can also create custom images based on Jenkins, adding in your own jobs configs and using Config As Code to have your own custom Jenkins instance that’s fully ready to go, more on this in a future article.

6. Avoid build work on the Jenkins Controller

Simply put the more time the master is doing work for your build pipelines, the less time it has to provide content (such as web pages and api responses to your users) and manage your build agents. Aim to run every part of your build pipeline on Jenkins Agents and you’ll have a much more responsive instance.

7. Provision agents as part of the pipeline

Building on the previous point, when using agents, don’t rely on software and tools being manually installed on the machine. Install the tools and software you need as part of your build process, the best case is it’s already set up and the install is skipped, worst case is some extra time spent provisioning resources. In either case, however, you’re able to rapidly expand your build farm due to low manual setup overheads and it opens up the possibility of connecting to cloud farms like Amazon’s EC2 to auto-scale the farm as needed.

Check out part one below

How to create private packages for your Unity project

So you’re up and running sharing code across your projects now, but you want to improve your CI/CD pipeline and provide better information for your packages’ users.

In this part two we’ll explore how we can improve the CI/CD pipeline to help us write higher-quality code and automatically generate release notes.

Add auto changelogs

To start with, we’re gonna ensure that we enforce conventional commits on this repository, this is a crucial first step in generating your release notes.

There are a few ways we can do this:

• Pre-commit checks
  These are great and if you’re the only one working on your project this is a nice straightforward option. However, it can get complicated if you’re working with others and don’t have the tooling set up to install pre-commit hooks for everyone. They’re also not so great for less tech focussed colleagues as the error messages are often hard to read.
• GitHub action to validate all commits
  This is also quite restrictive and checks too late, in my opinion (no one wants to rewrite all their commit messages after they’ve made the PR), as well as suffering some of the above issues.
• GitHub action to validate the PR title and enforced merge conventions
  This is how we’ll do this, it offers the most freedom whilst still ensuring our main branch only contains conventional commits. However, if the other options work for you then they’ll also work with the rest of the pipeline we’ll talk about today.

Let’s add the following GitHub action to our project

name: Check PR title
on:
  pull_request:
    types:
      - opened
      - reopened
      - edited
      - synchronize

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: aslafy-z/conventional-pr-title-action@v3
        id: pr_title_check
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: PR Comment
        if: ${{ failure() }}
        uses: thollander/actions-comment-pull-request@v1
        with:
          message: |
            Add a prefix like "fix: ", "feat: " or "feat!: " to indicate what kind of release this pull request corresponds to. The title should match the commit message format as specified by https://www.conventionalcommits.org/.

This checks we have formatted the title of the PR according to the conventional commits spec.

Next, we’ll change the way PRs are merged into the project to ensure the title is taken as the merge commit message.

In the settings page for our repository, we’ll scroll down to the Pull Requests section in the general tab, and set it up like so.

This means that when a PR is merged it’ll squash all commits on that branch into 1 and take the PR title as the commit message.

So that’s all the setup taken care of, let’s look at how to turn that into a changelog.

We’ll update our package publishing workflow for this. Add the following snippet just after the version bump and before we set up node.

    - name: 'Get Previous tag'
      id: previoustag
      uses: "WyriHaximus/github-action-get-previous-tag@v1"     
    
    - name: Update CHANGELOG
      id: changelog
      uses: requarks/changelog-action@v1
      with:
        token: ${{ secrets.GITHUB_TOKEN }}
        tag: ${{ steps.previoustag.outputs.tag }}

    - name: Commit CHANGELOG.md
      uses: stefanzweifel/git-auto-commit-action@v4
      with:
        branch: main
        commit_message: 'docs: update CHANGELOG.md for ${{ steps.previoustag.outputs.tag }} [skip ci]'
        file_pattern: CHANGELOG.md

This will grab the newly created tag, create/update the CHANGELOG.md file and push it to our main branch as well.

And there we are, automatically generated change logs.

Create a GitHub release

Whilst we’re improving the publish GitHub action, let’s go a step further and create a GitHub release for this as well. We’ll use the changelog we made previously to add some content to the release.

Add the below snippet at the end of the publish

    - name: Create Release
      uses: ncipollo/release-action@v1
      with:
        allowUpdates: true
        draft: false
        name: ${{ steps.previoustag.outputs.tag }}
        body: ${{ steps.changelog.outputs.changes }}
        token: ${{ secrets.GITHUB_TOKEN }}
        tag: ${{ steps.previoustag.outputs.tag }}

Now when we merge a PR, not only will a new package be published, but a GitHub release will be created so we can share the news and show what’s been changed.

Add some automated review help

Finally, to wrap up part two, let’s help ourselves out by adding an automated reviewer. We all make mistakes now and then and a little help goes a long way. We’re gonna use MegaLinter for this as they’ve already pulled together all the linters we want to use.

Open up a command prompt or terminal in your repository folder and run

npx mega-linter-runner --install

Follow the instructions to set it up according to your needs and desire.

You’ll then want to make a quick change to the mega-linter.yml file to prevent duplicate runs on pull requests, change the on: block to the below code instead

on: 
  pull_request:
    branches: [main]

The config file sometimes also includes the wrong linters, set your’s up to look like this instead or follow Megalinter’s guides to configure it how you’d like.

# Configuration file for MegaLinter
# See all available variables at https://megalinter.io/configuration/ and in linters documentation

APPLY_FIXES: all # all, none, or list of linter keys
# ENABLE: # If you use ENABLE variable, all other languages/formats/tooling-formats will be disabled by default
# ENABLE_LINTERS: # If you use ENABLE_LINTERS variable, all other linters will be disabled by default
# DISABLE:
  # - COPYPASTE # Uncomment to disable checks of excessive copy-pastes
  # - SPELL # Uncomment to disable checks of spelling mistakes
SHOW_ELAPSED_TIME: true
FILEIO_REPORTER: true
# DISABLE_ERRORS: true # Uncomment if you want MegaLinter to detect errors but not block CI to pass

ENABLE_LINTERS:
    # Looks for excessive uses of copying and pasting your code around your project.
    - COPYPASTE_JSCPD 
    # Formats your CSharp code according to CSharpier standards
    - CSHARP_CSHARPIER
    # Format CSS code (handy for uss files as well)
    - CSS_STYLELINT 
    # Formating for Json files
    - JSON_PRETTIER
    # Spell checker
    - CSPELL
    # Watches out for any missed merge conflict markers
    - REPOSITORY_GIT_DIFF

Now you can commit your files to a branch, create a pull request, sit back and relax as the robots check your code for you.

This was part two of a series about making Unity packages. Follow me for part three, coming soon.

All the code above is available in a public template repository: https://github.com/RunningMattress/upm-test-package