Gitlab CI - Commit & Push in a Job

Recently I needed to make a GitLab CI job where when a merge to master occurs, it runs a bash script, commits the files & pushes them back to master. I needed this to build a change log out of separate files, and now I have it working; it feels like a helpful thing to share how to do.

Get SSH fingerprint

You’re going to need the SSH fingerprint of your GitLab instance. If you don’t know this, you can get it by running the following command:

ssh-keyscan yourinstance.gitlab.com

# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9
# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSMqzJeV9rUzU4kWitGjeR4PWSa29SPqJ1fVkhtj3Hw9xjLVXVYrU9QlYWrOLXBpQ6KWjbjTDTdDkoohFzgbEY=
# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAfuCHKVTjquxvt6CM6tdG4SLp1Btn/nOeHHE5UOzRdf

You only want the line that contains ssh-rsa, so for instance from above:

gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9

Setup

  • Generate a SSH key, doesn’t matter where, as you just want the contents of it.
  • Add the public part of the key as a Project Deploy Key and grant it write access.
  • Create a project CI/CD variable called SSH_PUSH_KEY and add the private part of your key as the value.
  • Create a project CI/CD variable called CI_KNOWN_HOSTS and add the SSH fingerprint of your GitLab instance as the value.

GitLab CI config

You’ll then need to add a job to your .gitlab-ci.yml file like so:

build_and_commit_files:
stage: build
rules:
- if: $CI_COMMIT_BRANCH == "master" && $CI_PIPELINE_SOURCE == "push"
    when: always"
- when: never
before_script:
- mkdir ~/.ssh/
- echo "${CI_KNOWN_HOSTS}" > ~/.ssh/known_hosts
- echo "${SSH_PUSH_KEY}" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- git config user.email "gitlab@yourinstance.gitlab.com"
- git config user.name "GitLab"
- git remote remove ssh_origin || true  # May not have origin yet
- git remote add ssh_origin "git@$CI_SERVER_HOST:$CI_PROJECT_PATH.git"
script:
- touch example.txt # Or any command to change files
- git add -A && git commit -m 'Commit message' || true # May not have files to commit
- git push ssh_origin HEAD:master || true # May not have any commits to push

The above will commit & push to master, if you want to push to the current branch you should use $CI_BUILD_REF_NAME


Posted on July 26, 2022

Reflections on 2021

With the first month of 2022 ending, I thought it would make sense to reflect on what I’ve picked up in 2021 and my goals for 2022. With the last of the COVID-19 restrictions are being removed in England, I am cautiously optimistic about what the future has in store.

Flutter / App Development

In June, I decided to start learning how to create Android/iOS applications, fueled by a personal want to utilise the Health metrics from my iPhone. That was an unexpected venture for me, especially after finding out how easily PWA’s work.

I sized up two frameworks, React Native & Flutter. I decided to go with Flutter, mainly because I didn’t know the underlying programming language Dart, and I’m always keen to learn something new.

Coming from a PHP/Python/JS background Dart was initially confusing. I found myself comparing it most to ReactJS. Null-safety isn’t something I’ve worked with previously, but it wasn’t too difficult to figure out.

Flutter itself is a lot of fun. There is something very satisfying about changing a line of code and seeing it update an emulated device’s application. I’ll likely do a blog post just on my development setup for Flutter, but in case I don’t:

  • Vojtech Pavlovsky has a great article on setting up the architecture for Bottom Navigation in Flutter here, he also has an updated version here which utilises auto_route & flutter_bloc. I found this a great starting point for creating apps with bottom navigation & a simple state.
  • Flutter Platform Widgets is something else I’ve been utilising. This handy library allows you to create an application that will use either Cupertino (iOS) or Material (Android) styling depending on what type of device loads it.

Crypto

Nothing in this post should be considered financial advice. For full disclosure, I currently hold BTC, ETH, MATIC, FTM & KLIMA.

This year has certainly been interesting for cryptocurrencies. I’ve held off on mentioning cryptocurrencies at all on here, as there does seem to be a rather heated debate between “has technical value” and “another pyramid scheme” within the tech community. It is easy to look at NFT’s and think “this is a joke”, but I think the blockchain has much more potential than selling people links to images for absurd amounts of money.

I’m not going to discuss ETH or BTC, as I’m just holding those as something that may provide a return on investment over the next few years. I am interested to see how ETH switching to proof-of-stake affects its gas costs. However, I’m not particularly optimistic that it’s going reduce gas costs enough that ETH becomes reasonable to frequently trade/transfer over.

MATIC & FTM are much more interesting to me, and their networks seem to be where most of the more interesting Defi projects live. There also seems to be more developers working on creating apps on these networks.

Polygon

The Polygon Network (MATIC) aims to scale existing cryptocurrencies by providing faster staking, without sacrificing decentralization. Polygon is trying to interconnect the “layer 2” ecosystem on Ethereum. You can see some of the apps on Polygon here.

Fantom

The Fantom Foundation (FTM) is built on DAG technology, allowing faster transactions and lower fees. Fantom is a general-purpose “Layer 1” solution. Fantom doesn’t have a nice list of apps on it like Polygon does, but two I use are: Spirit Swap & Spooky Swap

Klima

Of those two networks projects, Klima (on Polygon) is one project that caught my eye this year, as it aims to drive carbon action, become a carbon-based reserve currency & facilitate the climate markets.

Their documentation lays out how they aim to achieve this much better than I can explain here, and this medium article has a great breakdown of it.

I want to get more involved in contributing to Klima DAO over 2022, as I believe climate change is the ultimate issue we face, and Klima (to me) looks to be one of the most promising projects to make positive change in that regard.

Solidity

With my goal to get more involved in contributing to Klima 2022, I’ve been learning Solidity. I found Crypto Zombies the best place to start, it’s a great resource to easily learn all about smart contracts and how development on the blockchain actually works. Ethernaut seems to be the next logical place to go, learning how to find issues with smart contracts.


Posted on January 31, 2022

Open source software I use for development

I love open source software, and I try to use it whenever possible. The ability to be able to fork the codebase and add (or fix) functionality you need means you’re not reliant on a third party for a bug fix.

In an effort to share more open source software, here are a couple of repositories for tools that I find useful day-to-day for development:

  • Super Productivity - An open source time tracking / task organization tool that integrates with Jira, GitHub and GitLab.

  • Pandoc - This is a great tool that enables easy conversion between markup formats. I mainly use it to write documentation in a markdown editor and then convert it to MediaWiki format.

  • ScreenToGif - ScreenToGif allows you to record your screen and save it as, you guessed it, a GIF. I find this incredibly useful for sharing quick demos / new functionality in video format.

  • Draw.io - It’s worth noting this project is open source, but closed to contributors. I use it to make UML diagrams and flowcharts to describe functionality and system architecture changes.

  • Git Recall - This a really useful tool that allows you to quickly browse through commit diffs in the terminal.


Posted on August 28, 2021

A demi-decade at Kobas

As of last Sunday, I’ve been working at Kobas for five years (a demi-decade 😅) , reading my post from a month in reminds me how much has changed and prompted me to provide an update.

Job Role

My job role has changed significantly throughout my time at Kobas. Initially, I was feature-focused and spent all my time programming. As time passed, I became more involved with bug fixes and technical product management: planning features, evaluating APIs and implementing technologies. We’ve also expanded our team along the way, so I’ve experienced our onboarding process from both sides. 😆

Issue Management

Previously we were using Pivotal Tracker for our issue management, while it did work great for planning features, it was lacking in terms of bug management. I worked on moving us over to YouTrack, which at the time required utilising both the Pivotal Tracker export API and the YouTrack API (a fun task 😁). YouTrack has worked great for us so far, we use their Agile Boards for sprints, project planning and issue triage.

Working Location

COVID-19 isn’t something I’ve posted about on here yet, which is strange considering that’s been the main subject of the past year. Before COVID-19, Kobas was hybrid-remote, having staff members working from home a certain % of the week, but still maintaining an office that could host all our staff members.

We’ve now transitioned to being completely remote, which I prefer. I’ve found GitLab guide on remote work fantastic for resources surrounding working remotely. Overall, I think our transition to remote work has gone smoothly as we already had a lot of technology in place to support remote work (like a VPN).

Features

We’ve made some impressive stuff for our clients since my last post, too much to really talk about without this becoming a changelog. The most recent thing that comes to mind is the Customer Interaction Centre, providing our clients with a way to support online ordering.

Other projects that come to mind when I think of my time here are: integrating PDQ terminals using socket connections (much fun), rewriting our stock system to support FIFO (not fun), and moving our API authentication over to OAuth 2.0 (easier to support third-party integrations).

Upgrades

We like to keep all our technology up to date in Kobas, and the last five years has seen a lot of upgrades.

Our back-end now runs on PHP 7.4 (currently working on upgrading to PHP 8) with CentOS 8 Stream as the OS. Our main framework is Symfony 5, after a lot of work migrating from Silex. 😬

On the front-end, we’ve added ReactJS into the mix, and new projects are now done in that. Lastly, on the automated testing front, we are still using Codeception but are moving our acceptance tests to Cypress.

Future

As evidenced by this year, it’s impossible to know what the future holds. But I hope to be able to post more about the work I’m doing at Kobas on here, as I’m aiming to add more open source projects. Stay tuned. 😉


Posted on June 08, 2021

Website revitalised using Gatsby!

This year, one of my personal goals was to revitalise this website and get the source code onto Github.

Historically it has been a blog, and not that anyone’s really noticed, but I haven’t done any blogging lately. While I do want to maintain my blog, and I intend on continuing to post to it, I don’t think it should be the focus of this website anymore.

Moving forward, I want this website to be somewhere:

  • I store technical information that I might need to refer to later.
  • I can highlight projects I’m working on & try out new ideas.
  • I can blog about different topics easily.

This presented a few issues for me with the websites existing setup:

  • It was built on WordPress, which I didn’t find very motivating to develop on anymore.
  • It was self-hosted on a server that included websites for family & friends. This made me hesitant to install new tools or languages on the server.
  • The existing UI/UX of the website didn’t cater to most of what I had in mind.

Naturally, I concluded the best course of action was to start from scratch—the sort of decision you can easily take on a project that’s entirely your own.

I had recently started to use ReactJS on a few projects at Kobas and was enjoying using it, so I decided I would use that for the frontend. I also knew I wanted to utilise some form of auto-deployment for the project, as that makes development much more comfortable.

After several iterations of trying different JAMstack frameworks, I landed on Gatsby hosted on AWS Amplify.

I started the project using the ”Gatsby WordPress starter”, immediately giving me a ReactJS frontend with the data sourced from my existing WordPress instance.

This allowed me to quickly get to a point where I could work on the design using real data and recognise the functionality I needed to code myself. While I did have data sourced from WordPress, I didn’t have a comment system, contact page, search, or sidebar widgets for things like tags/categories.

I needed to decide what I didn’t immediately require, as I wanted to get the new version out as soon as possible. A design I was happy with was the first thing to get added to my MVP list. The sidebar widgets I considered design-related, the website looked bare without them, they went into my MVP list. The contact page also went onto the MVP list, mainly as it was trivial to add utilising getform.io.

I decided that I could live without a comment system, it had never gotten much engagement anyway. I also thought that if I wanted one later, I could use something like Disqus. Adding search functionality seemed the most complex out of the features I was missing, so I didn’t add it to the MVP list.

Over the next few weeks, I worked on the above MVP list. Doing my best to avoid adding more functionality along the way.

Once I was done with the MVP list, I started looking at deployment options. I wanted something I wouldn’t need to spend much time configuring. AWS Amplify fit that requirement. First, I moved my domain over to Route53. Then I pointed Amplify to my Github repository, which automatically picked up the build command in my package.json. So simple!

I’m pretty happy where I’ve got to at this point, any future development I want to do here is much more streamlined for me. More fun stuff to come I hope. 😀


Posted on October 29, 2020