Gitlab CI - Commit & Push in a Job

Recently I needed to make a GitLab CI job where when a merge to master occurs, it runs a bash script, commits the files & pushes them back to master. I needed this to build a change log out of separate files, and now I have it working; it feels like a helpful thing to share how to do.

Get SSH fingerprint

You’re going to need the SSH fingerprint of your GitLab instance. If you don’t know this, you can get it by running the following command:

ssh-keyscan yourinstance.gitlab.com

# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9
# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSMqzJeV9rUzU4kWitGjeR4PWSa29SPqJ1fVkhtj3Hw9xjLVXVYrU9QlYWrOLXBpQ6KWjbjTDTdDkoohFzgbEY=
# gitlab.com:22 SSH-2.0-GitLab-SSHD
gitlab.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAfuCHKVTjquxvt6CM6tdG4SLp1Btn/nOeHHE5UOzRdf

You only want the line that contains ssh-rsa, so for instance from above:

gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9

Setup

  • Generate a SSH key, doesn’t matter where, as you just want the contents of it.
  • Add the public part of the key as a Project Deploy Key and grant it write access.
  • Create a project CI/CD variable called SSH_PUSH_KEY and add the private part of your key as the value.
  • Create a project CI/CD variable called CI_KNOWN_HOSTS and add the SSH fingerprint of your GitLab instance as the value.

GitLab CI config

You’ll then need to add a job to your .gitlab-ci.yml file like so:

build_and_commit_files:
stage: build
rules:
- if: $CI_COMMIT_BRANCH == "master" && $CI_PIPELINE_SOURCE == "push"
    when: always"
- when: never
before_script:
- mkdir ~/.ssh/
- echo "${CI_KNOWN_HOSTS}" > ~/.ssh/known_hosts
- echo "${SSH_PUSH_KEY}" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- git config user.email "gitlab@yourinstance.gitlab.com"
- git config user.name "GitLab"
- git remote remove ssh_origin || true  # May not have origin yet
- git remote add ssh_origin "git@$CI_SERVER_HOST:$CI_PROJECT_PATH.git"
script:
- touch example.txt # Or any command to change files
- git add -A && git commit -m 'Commit message' || true # May not have files to commit
- git push ssh_origin HEAD:master || true # May not have any commits to push

The above will commit & push to master, if you want to push to the current branch you should use $CI_BUILD_REF_NAME


Posted on July 26, 2022

Reflections on 2021

With the first month of 2022 ending, I thought it would make sense to reflect on what I’ve picked up in 2021 and my goals for 2022. With the last of the COVID-19 restrictions are being removed in England, I am cautiously optimistic about what the future has in store.

Flutter / App Development

In June, I decided to start learning how to create Android/iOS applications, fueled by a personal want to utilise the Health metrics from my iPhone. That was an unexpected venture for me, especially after finding out how easily PWA’s work.

I sized up two frameworks, React Native & Flutter. I decided to go with Flutter, mainly because I didn’t know the underlying programming language Dart, and I’m always keen to learn something new.

Coming from a PHP/Python/JS background Dart was initially confusing. I found myself comparing it most to ReactJS. Null-safety isn’t something I’ve worked with previously, but it wasn’t too difficult to figure out.

Flutter itself is a lot of fun. There is something very satisfying about changing a line of code and seeing it update an emulated device’s application. I’ll likely do a blog post just on my development setup for Flutter, but in case I don’t:

  • Vojtech Pavlovsky has a great article on setting up the architecture for Bottom Navigation in Flutter here, he also has an updated version here which utilises auto_route & flutter_bloc. I found this a great starting point for creating apps with bottom navigation & a simple state.
  • Flutter Platform Widgets is something else I’ve been utilising. This handy library allows you to create an application that will use either Cupertino (iOS) or Material (Android) styling depending on what type of device loads it.

Crypto

Nothing in this post should be considered financial advice. For full disclosure, I currently hold BTC, ETH, MATIC, FTM & KLIMA.

This year has certainly been interesting for cryptocurrencies. I’ve held off on mentioning cryptocurrencies at all on here, as there does seem to be a rather heated debate between “has technical value” and “another pyramid scheme” within the tech community. It is easy to look at NFT’s and think “this is a joke”, but I think the blockchain has much more potential than selling people links to images for absurd amounts of money.

I’m not going to discuss ETH or BTC, as I’m just holding those as something that may provide a return on investment over the next few years. I am interested to see how ETH switching to proof-of-stake affects its gas costs. However, I’m not particularly optimistic that it’s going reduce gas costs enough that ETH becomes reasonable to frequently trade/transfer over.

MATIC & FTM are much more interesting to me, and their networks seem to be where most of the more interesting Defi projects live. There also seems to be more developers working on creating apps on these networks.

Polygon

The Polygon Network (MATIC) aims to scale existing cryptocurrencies by providing faster staking, without sacrificing decentralization. Polygon is trying to interconnect the “layer 2” ecosystem on Ethereum. You can see some of the apps on Polygon here.

Fantom

The Fantom Foundation (FTM) is built on DAG technology, allowing faster transactions and lower fees. Fantom is a general-purpose “Layer 1” solution. Fantom doesn’t have a nice list of apps on it like Polygon does, but two I use are: Spirit Swap & Spooky Swap

Klima

Of those two networks projects, Klima (on Polygon) is one project that caught my eye this year, as it aims to drive carbon action, become a carbon-based reserve currency & facilitate the climate markets.

Their documentation lays out how they aim to achieve this much better than I can explain here, and this medium article has a great breakdown of it.

I want to get more involved in contributing to Klima DAO over 2022, as I believe climate change is the ultimate issue we face, and Klima (to me) looks to be one of the most promising projects to make positive change in that regard.

Solidity

With my goal to get more involved in contributing to Klima 2022, I’ve been learning Solidity. I found Crypto Zombies the best place to start, it’s a great resource to easily learn all about smart contracts and how development on the blockchain actually works. Ethernaut seems to be the next logical place to go, learning how to find issues with smart contracts.


Posted on January 31, 2022

Cheat Sheet: Screen

The screen command is a used to launch and arrange multiple terminal shells within a single shell.

It is mainly used for two purposes:

  1. It can be used to run long-running commands on remote servers.
  2. It can be used to organise multiple shells and allow you to navigate between them.

Commands

All screen commands need to be prefixed by an escape key, which is Ctrl-a by default.

CommandCategoryDescription
screenBasicStarts a new session
screen -S <session_name>BasicStarts a new session with session name
screen -lsBasicLists running screen sessions
screen -r <session_name>BasicAttach to a running session with name
screen -d <session_name>BasicDetach a running session with name
screen -r -d <session_name>BasicAttach to a screen that is already attached
Ctrl-a dExitingDetach
Ctrl-a D DExitingDetach and logout (fast way to exit screen)
Ctrl-a :ExitingQuit and exits all of the programs in screen
Ctrl-a escScrollingEnter scrolling mode
Ctrl-uScrollingScroll Up
Ctrl-dScrollingScroll Down
esc escScrollingExit scrolling mode
Ctrl-a cWindow ManagementCreate new window
Ctrl-a Ctrl-aWindow ManagementChange to last-visited active window
Ctrl-a <number>Window ManagementChange to window by number (only 0-9)
Ctrl-a ' <number_or_name>Window ManagementChange to window by number or name
Ctrl-a nWindow ManagementChange to next window in list
Ctrl-a pWindow ManagementChange to previous window in list
Ctrl-a "Window ManagementDisplays list of windows, allowing to select window to change into
Ctrl-a AWindow ManagementRename current window
Ctrl-a kWindow ManagementKill current window
Ctrl-a \Window ManagementKill all windows
Ctrl-a aMiscSend Ctrl-A to screen within screen, useful when working with screen within screen

Posted on September 06, 2021

Open source software I use for development

I love open source software, and I try to use it whenever possible. The ability to be able to fork the codebase and add (or fix) functionality you need means you’re not reliant on a third party for a bug fix.

In an effort to share more open source software, here are a couple of repositories for tools that I find useful day-to-day for development:

  • Super Productivity - An open source time tracking / task organization tool that integrates with Jira, GitHub and GitLab.

  • Pandoc - This is a great tool that enables easy conversion between markup formats. I mainly use it to write documentation in a markdown editor and then convert it to MediaWiki format.

  • ScreenToGif - ScreenToGif allows you to record your screen and save it as, you guessed it, a GIF. I find this incredibly useful for sharing quick demos / new functionality in video format.

  • Draw.io - It’s worth noting this project is open source, but closed to contributors. I use it to make UML diagrams and flowcharts to describe functionality and system architecture changes.

  • Git Recall - This a really useful tool that allows you to quickly browse through commit diffs in the terminal.


Posted on August 28, 2021

A demi-decade at Kobas

As of last Sunday, I’ve been working at Kobas for five years (a demi-decade 😅) , reading my post from a month in reminds me how much has changed and prompted me to provide an update.

Job Role

My job role has changed significantly throughout my time at Kobas. Initially, I was feature-focused and spent all my time programming. As time passed, I became more involved with bug fixes and technical product management: planning features, evaluating APIs and implementing technologies. We’ve also expanded our team along the way, so I’ve experienced our onboarding process from both sides. 😆

Issue Management

Previously we were using Pivotal Tracker for our issue management, while it did work great for planning features, it was lacking in terms of bug management. I worked on moving us over to YouTrack, which at the time required utilising both the Pivotal Tracker export API and the YouTrack API (a fun task 😁). YouTrack has worked great for us so far, we use their Agile Boards for sprints, project planning and issue triage.

Working Location

COVID-19 isn’t something I’ve posted about on here yet, which is strange considering that’s been the main subject of the past year. Before COVID-19, Kobas was hybrid-remote, having staff members working from home a certain % of the week, but still maintaining an office that could host all our staff members.

We’ve now transitioned to being completely remote, which I prefer. I’ve found GitLab guide on remote work fantastic for resources surrounding working remotely. Overall, I think our transition to remote work has gone smoothly as we already had a lot of technology in place to support remote work (like a VPN).

Features

We’ve made some impressive stuff for our clients since my last post, too much to really talk about without this becoming a changelog. The most recent thing that comes to mind is the Customer Interaction Centre, providing our clients with a way to support online ordering.

Other projects that come to mind when I think of my time here are: integrating PDQ terminals using socket connections (much fun), rewriting our stock system to support FIFO (not fun), and moving our API authentication over to OAuth 2.0 (easier to support third-party integrations).

Upgrades

We like to keep all our technology up to date in Kobas, and the last five years has seen a lot of upgrades.

Our back-end now runs on PHP 7.4 (currently working on upgrading to PHP 8) with CentOS 8 Stream as the OS. Our main framework is Symfony 5, after a lot of work migrating from Silex. 😬

On the front-end, we’ve added ReactJS into the mix, and new projects are now done in that. Lastly, on the automated testing front, we are still using Codeception but are moving our acceptance tests to Cypress.

Future

As evidenced by this year, it’s impossible to know what the future holds. But I hope to be able to post more about the work I’m doing at Kobas on here, as I’m aiming to add more open source projects. Stay tuned. 😉


Posted on June 08, 2021