Pull the other one…

In my experience, estimation is the topic most re-visited in a work context in large organisations, however the question of mobbing vs pairing vs pull requests runs it a pretty close second. As with all such topics, there is no absolute right and wrong answer as it very much depends on the scenario you find yourself in. I know that many people have written about this over the years but I find I’m still having the same conversations so I thought I’d put down my thoughts in one place, even if it’s only so I can find them more easily next time!

In the spirit of full disclosure I’ll state up front that I’m generally a pairing/mobbing kinda guy for most of the situations I find myself in. Don’t get me wrong, pull requests are fine in other scenarios but I don’t think they work well in my typical scenario for the reasons laid out below.

Context, context, context

In a very Monty Python-esque way, I’ve gradually increased the number of contexts I think are relevant to software development over the years. Currently the list stands at five:

  • Tech stack. If you are working in a specific technology then you need a good level of knowledge and experience to get the most out of it. Anyone with rudimentary programming skills (I include myself in this camp) can bludgeon their way to some form of solution in a given language/library combination using a combination of previously successful approaches in procedural / OO / functional contexts combined with some smartly worded searches on Stack Overflow but the result is not going to be great. Even if you avoid just using your favourite hammer to hit the screw nail, you may still go against the grain of the environment and reduce the maintainability of a system due to a ‘quirky’ approach. My favourite description of one system I’ve worked on is that it was a Python-based system with a strong Allo Allo-style Java accent.
  • Domain. A good codebase, including its tests, will reflect the domain concepts being manipulated. The less familiar you are with the domain, the more difficult it will be for you to understand the intent. There is also the risk of you misunderstanding nuances and either duplicating existing concepts or bolting things into a concept in which they don’t fit. Quite a few codebases I’ve known have suffered from this type of technical cruft (it’s not technical debt as it wasn’t intentionally taken on) which can make changes difficult and/or necessitate a large amount of refactoring to make the codebase more malleable.
  • Codebase. The structure, layout, seams and shape of a codebase reflect the design decisions that have been made over the course of its evolution. As with the tech stack, you can go with the grain or against the grain. Going against the grain can make it harder for people to understand how things should work and will make certain changes more difficult. There is also the question of familiarity. If you are not familiar with the codebase then you may well end up duplicating code unintentionally or re-implementing something in a different way. Both of these reduce the maintainability of the code.
  • Architectural. Unless your system is a complete monolith, its code will be split out across various moving parts which conform to some overall architecture. As with the code, if you go against the architectural grain of the system then you make it more difficult to understand, maintain and support.
  • Organisational. To be effective in a large organisation and not build up a large amount of technical cruft/debt you will need to work with the grain of the organisation itself. This may take the form of governance, standards, existing services, prior art, etc. Missing any of these things may make your system more difficult to evolve and support as time goes on. It can also cause you a lot more work if you’re re-inventing existing services or tooling or pulling in different variants. To be effective you need to know who to talk to about this and how much weight to assign to their opinion.

In an ideal world, all of the paint drip people involved in designing, building and supporting the system would have lots of each of these contexts. However, in many large organisations the ebb and flow of people and work, especially if work is packaged in projects and teams are rapidly augmented with significant numbers of transient 3rd party contractors, you can find yourself very far from this situation.

So, if you find yourself closer to this less-than-ideal-world than the ideal one, what approach do you take to building and evolving software, especially existing systems which already bear the scars of a troubled evolution.

Mobbing

From my experience, if I find myself closer to this less-than-ideal-world I favour mobbing. If you consider all those different contexts, mobbing gives you the best possible combination at any given point in time. If there’s a lack of domain context amongst the developers, a BA or QA could bring this to the mob in real time. If there’s one person who is strong in the tech stack, whether that’s software or platform, they can give real time design advice and all the mob can learn together why that’s a good option. Someone with history in the system can educate the rest of the mob about the codebase or the architecture as they need to find or add some functionality and a tech lead or architect can reference the standards or governance that others may not yet be aware of. As Woody Zuill puts it, “all the brilliant minds working together on the same thing, at the same time”.

Even without the less-than-ideal-world scenario painted earlier, I’ve found mobbing to be a good approach for work in a new and unknown space. In this case, the onus is on the team to identify this type of work, whether that’s based on finger-in-the-air risk and complexity assessment or a more formal approach based on something like the Cynefin framework where suitable work for mobbing would be that classified as being in the complex or chaos domain.

The other main process benefit of mobbing from my perspective (as opposed to social benefits) is that you get immediate design review as you go. Given that all the people are working on the same thing, you are guaranteed to get the best possible review in real time so you get the best result possible with minimum rework (unless people explicitly decide to experiment and then backtrack).

An interesting point that hadn’t occurred to me when I originally wrote this blog post was that mobbing (and pairing for that matter) are a great way of addressing imposter syndrome. As Ian Cooper puts it:

One benefit of mobbing/pairing that is less talked about is that it can be a useful antidote to imposter syndrome. Because you see everyone else using StackOverflow, iterating until it works etc. alongside you, you realize that is the bar, not one you imagine from completed work.

The typical challenge to mobbing is that only one thing is being worked on at one time (single-piece flow). The assumption is that this inefficient compared to pairs or individuals working on parallel pieces of work. However, in the less-than-ideal-world scenario pairs and individuals will rapidly rack up technical cruft because they lack too much of the various contexts. The perceived increase in “speed” at the start can quickly stall due to this technical cruft. In fact, if you’re working on a codebase already containing a high level of technical cruft then you may not even get this initial burst of apparent delivery.

When the team finds that there has been a levelling up of the various contexts they may decide to do more work in pairs or mini-mobs. Similarly, where the work is in the complicated or simple Cynefin domain, a decision can be made for it to be done by pairs or even individuals.

One final word on mobbing: if you’re going to do it then do your research! Mobbing takes practice and has some common practices and etiquette. As with TDD etc. if you’re going to try it then try it properly, ideally with someone who’s done it before successfully.

Pairing

Where a team has a good level of the different contexts spread around the team, or if levelling up isn’t a primary objective for the team, pairing may be their preferred option. Obviously, it’s highly unlikely that all the different contexts are absolutely level around the team so you are already making some trade-offs.

Given that people are not evenly skilled with an equal background in the different contexts you then need to decide whether to optimise for delivery or for risk. If delivery is your primary concern you can put the same people on the same type of work all the time. This will maximise the speed of delivery for familiar work and should provide a reasonable level of quality but you implicitly increase your risk profile as your bus count for different areas of the system remains low. Alternatively, you may decide that you want to de-risk the work by spreading knowledge around and mix people who are familiar with certain types of work with those who are not. In this case delivery will be slower as learning must take place as you go (otherwise you’re not really spreading knowledge) and also there is a risk to quality as the less experienced person has less of a toolbox with which to critique the design and technology selections being made (although this may be balanced somewhat by beginners mind).

For learning to take place you need to ensure that you have the right collective mindset – that all the team is committed to this as a goal – and that you are using appropriate techniques. Given how long pair programming has been “a thing” I’m constantly surprised that people aren’t familiar with techniques such as strong pairing and regular pair rotation. If you’re skeptical about pair rotation, take a look at Arlo Belshee’s promiscuous pairing paper to see just how far you can go when experimenting with it.

One final word on pairing: if you’re going to do it then do your research! Pairing takes practice and has some common practices and etiquette. As with TDD etc. if you’re going to try it then try it properly, ideally with someone who’s done it before successfully.

Pull requests

Let me make it clear. I have no problems with pull requests in the right context. The problem I have with them is that they are often used when a better alternative is available. Taking the scenario of an open source software initiative, pull requests make a lot of sense as there is a temporally distributed group and probably trust is low as those people haven’t had an opportunity to create that trust by working closely together. However, in a typical co-located (or well connected) stable team in an organisation I believe that the overhead involved with pull requests is, in the words of a notable tweet, like making your family go through airport-style security to enter your home. I don’t say that lightly. Pull requests are sometimes symptomatic of poor trust and bad power relationships in a team but I’m putting that to one side as my concerns are around the mechanics and assumptions around the pull requests themselves. However, it’s worth stating that for a team to be effective, the people in it need to build trust and pull requests don’t typically help with that.

If you consider how a pull request should work, the person reviewing the code should have the same level of understanding of the problem to be solved as the person who wrote the code and should consider different design possibilities to decide if the one chosen is a good one. That’s quite a lot of overhead, not far short of writing the code again, which is why I’ve rarely see it happen (on this topic Keith Braithwaite drew my attention to “Fagan Inspection: the Silver Bullet No-one Wants to Fire“). My experience of pull requests in large organisations is that the review is mostly about the tech stack and syntax, a bit about the domain and a bit about the codebase. This is an order of magnitude worse than the review you would get from pairing and several orders of magnitude worse than the review you would get from mobbing.

The other issue I have with pull requests is the length of the feedback loop. If you leave it right to the end then potentially you have a huge amount of rework to do if the review highlights a better way of doing it. I’ve seen teams decide not to take on board the review as it would be too much work and so explicitly live with a worse solution. To reduce the risk of this happening you might not leave it to the end before reviewing but instead have multiple reviews as you go. However, the more often you do this, the higher the overhead so why not just pair?

An explicit decision

As I said at the start, there is no right answer here for all scenarios. However what I would ask is that teams make an informed and explicit tradeoff when deciding which approach to use.

[2021-10-17: Updated with Ian Cooper quote, Keith Braithwaite reference and link to Justin Searls twitter thread.]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s