Mastodon

oooops.dev

DevOps is hard

Cross-service “integration tests” have to go — October 17, 2022

Cross-service “integration tests” have to go

tl;dr They are often broken in pipelines and that might be because they are broken as a concept.

A typical issue in a microservices architecture is ensuring that the integration between services won’t break with the introduction of a new release. In my time as a consultant I have seen different organisations try to tackle this problem with varying degrees of success.

There is one particular approach that I have never seen fully succeed, although it is certainly popular. Organisations often try to have an “integration” test suite in a shared environment that spans several (if not all) services. It usually pokes and prods at functionality owned by multiple teams.

In this article I want to explore why I think these are a bad idea. I will also propose an alternative which is more in line with my understanding of the latest testing good practices.

First, let’s clarify the problem.

Continue reading
QAing in production — August 12, 2022

QAing in production

In this article, I want to challenge where software testing happens and suggest that most manual verification (desk checks, QA, demo) that is part of a team’s delivery lifecycle should happen in production itself, rather than in some staging or UAT environment.

I have used this approach in several teams within medium to large organisations that were already implementing Continuous Delivery and/or Deployment, as it became the obvious and natural next step in our workflow.

Is it safe?

Let’s address the elephant in the room already: yes, performing your QA in production sounds like an oxymoron. After all, isn’t the purpose of QA to prevent broken features from being deployed to production in the first place?

This might have been true in the past. However, since then, a new practice has changed the way we release new features: Feature Toggles.

The basic idea is to have a configuration file that defines a bunch of toggles for various features you have pending. The running application then uses these toggles in order to decide whether or not to show the new feature.

Martin Fowler
Continue reading
Surviving Continuous Deployment in Distributed Systems — July 30, 2021

Surviving Continuous Deployment in Distributed Systems

Note: a shorter, summarised version of this article has been published on the Thoughtworks blog. Read the full article here or watch the talk delivered at XConf Europe:

Introduction

This is an article about the day to day of software development.

The industry seems to be at a point where a lot of us are using good practices such as Trunk Based Development and/or Continuous Deployment, or we are at least hassling our managers about it working towards it. I’m also a big fan.

But, as cool and shiny as these practices are, and as much as we reassure our fellow developers and stakeholders that they are completely safe, I believe they do present some risks.

Continue reading