The Impact of AI on Product Engineering teams — Part 3
This is the third part of a six part series investigating how generative AI productivity tools aimed at developers, like Github Copilot, ChatGPT and Amazon CodeWhisperer might impact the structure of entire product engineering teams.
In Part 2, we explored:
- How generative AI tools like ChatGPT are redefining the way product engineering teams approach coding — from generating user stories to writing the actual code.
- How tasks often perceived as tedious by developers, like writing tests and documentation, can now be easily and efficiently handled by AI, making the entire coding process more streamlined.
- The crucial importance of tests and the potential future of prompt-engineered applications that might be initiated with thoughtful test design.
- A vision of the future where Generative AI tools reshape the roles within product engineering teams and the profound implications for engineers and leaders.
How will Generative AI tools transform team structure?
The idea that I’ve been consumed with for the past few months is that the impact of Generative AI tools on product engineering teams is going to cause a fundamental shift in the ratio of product to technical roles. In part one of this series, I reflected that a common ratio for engineers to product managers in many product teams was five engineers to one product manager.
Now, when I’m feeling either particularly bold or particularly fatalist, that hypothesis becomes:
“Current product engineering teams require five engineers to one product manager. Next-gen product engineering teams will require only one senior engineer for each product manager’
This is what I’d like to put to the test.
In the previous articles in this series we’ve discussed the individual impact of these developer and productivity tools, but I’d now like to pull your gaze back to a more strategic view — the impact that enhanced developer productivity will have on a team, and even the whole organisation.
Suspending disbelief for the sake of a thought experiment, let’s assume that we are open to the argument that Generative AI tools, like Github Copilot, may enhance the productivity of a single developer by around 50%. Let’s also work on the assumption that in the current status quo (and as we described in Part 1) one product manager for each five engineers is a reasonable ratio. If these two things are held to be true, we can infer that the longstanding 5:1 ratio is going to need to change.
I believe that it is reasonable to expect that the advances we’ve seen since Copilot was released in 2022 will continue, especially with the firehose of VC cash being turned on GenAI startups instead of the Metaverse and Web3.0.
A flood of new tools has started to hit ProductHunt daily, like Bifrost, and Locofy (both of which turn Figma designs into working React code, like Amazon’s Amplify Studio), and it will be interesting to see how quickly, and with who, they gain traction.
It’s unlikely that developers in larger teams will drive the adoption of this new type of code automation tool because of historic suspicions about code quality and corporate risk-aversion. What’s more likely is that startup founders, innovation-focused product managers and smaller marketing teams will be the early adopters, attracted by the lower cost and faster speed to market of auto-generated websites and apps.
I can hear a sigh swelling from the engineers still reading, “Of course my application is too unique and my problem too intricate for these types of basic, Lego brick applications to take my job”. But as Kelsey Hightower (one of the original contributors to Kubernetes and formerly a Google distinguished engineer) said,
“Stick to boring architecture for as long as possible, and spend the majority of your time, and resources, building something your customers are willing to pay for”.
If you still think that your application is unique and beautiful, this followup tweet from Derek Neighbours neatly summarised exactly who that ‘boring architecture’ comment should be aimed at: almost everyone.
The number of companies needing non-boring architecture is pretty close to none. Nothing funnier than a startup with 12 customers copying scaling patterns from Netflix or Google.
Let’s get back to my hypothesis; most engineers will be more productive, most architectures shouldn’t be complex and Generative AI tools will improve considerably from their 2023 state. If these are true, then the job of engineers will shift from basic plumbing and maintenance to more serious issues like architecture, solution design, performance, security and managing increasingly complex internal ecosystems.
Instead of test driven development, we may begin to see test driven solution design, where the role of the engineer is to specify the functionality of the application in a way that the Generative AI tools can be prompted to produce both the tests and a working application. The tests created as part of the design phase will then continue to prove that the application meets the needs of the company and the customer, and self-document the application.
How code productivity tools can take on more than just obvious tasks
We’ve spoken about the impact of Generative AI tools on new product and feature development, but for most engineers this only makes up a portion of their day job. For a lot of engineers, ‘business as usual’, maintenance and operational tasks take up a lot of their working week. Refactoring, bug fixes, small change requests and minor version upgrades can consume a lot of valuable developer time. The tools we’ve been discussing are just as powerful when considering these mundane tasks as they are with the exciting new product development work.
A few years ago, I was working with a team that was confronted with an old database which needed to be upgraded. The team was now very modern and microservicey (yes, that’s now an adjective), and there was no-one with experience in archaic SQL or deprecated versions of the ancient database engine. This type of thing isn’t uncommon, and often causes a minor emergency as teams scrabble to find anyone with the mystical knowledge required. What should have been a relatively simple task suddenly overturned all of the careful roadmap planning.
Teams regularly face these types of problems — whether it’s rewriting something old and forgotten, or having to upgrade from one major version of a framework to another — they have to comb through obsolete and often poorly documented code to learn something new (or relearn something frustratingly old).
But these code assistant tools don’t really care what language they’re given (and yes I have asked ChatGPT to write both brainf*ck and machine code for giggles). The LLMs have been trained on all the docs, billions of lines of code examples and all the flame wars on Stack Overflow (probably). For an LLM, rewriting even undocumented code is not really a big deal. What’s even better is that it is easy to ask the tools to write tests for both the old and the new code.
To check that I wasn’t just hallucinating that this was possible, I went to find some random old Perl code on the internet and fed it into GPT4. After asking it to explain what the code did, I followed up by asking what language we might replace Perl with. It suggested Python, and then immediately produced the code. With one more prompt, we had some tests to check the code was running.
As a bonus for ten, I thought I’d see if we could then port it to Rust, one of Stack Overflow’s ‘most loved languages’ for the past few years. As you’d probably guess, it didn’t really have any issues with the request.
If the removal of simple but time consuming tasks like documentation and test creation, and the automation of routine engineering tasks like version upgrades and refactoring can be taken by AI tools, what is the outcome? Almost certainly a change in that 5:1 ratio.
The strange world of web and mobile app teams
In organisations that have chosen to build both a website and a mobile application, we sometimes come across a slightly odd dysfunction. Although the functionality and the customer value of both the web and mobile versions is often identical, we find ourselves having to separate the teams that build the different applications. Even in organisations that strive to align their teams around value streams, the split of expertise and systems almost always leads to the creation of a dedicated team.
This isn’t an issue for companies where the product is a redesigned web interface displayed on the mobile device (something called ‘responsive design’), but can be an issue even where companies have chosen to build ‘hybrid’ mobile apps, where the app is developed in a web-adjacent framework like React Native. At least in these hybrid applications, there can be some similarity in the code, and the skills required to release both mobile and web products are broadly transferable.
Native mobile apps — those apps that are specifically built for Android and iOS — can complicate things further. For businesses that have decided that native apps are important, and that customers with both Android and iPhones should get the same great experience, we often have an additional requirement for skills, with a requirement for engineers who have expertise in either Java or Kotlin (for Android) or Swift/Objective-C(for iOS) in the mobile team.
There’s a lot of duplication of effort and corresponding cost increases taken on by companies that have chosen to deliver native mobile apps as well as web interfaces.
As we’ve seen, our generative AI coding assistants could leverage their language agnosticism in this context, to create well designed, well tested applications in multiple languages. The potential impact of this is that we may no longer need to split mobile and app teams because of their skillsets, but rather only where the web and mobile applications have significantly different features and functionality. In some companies this could be a massive simplification and a big cost saving of up to half of their engineering budget.
What’s the impact on junior and mid level developers (especially in startups and small businesses)?
In leading up to writing this article, I’ve had the discussion about the potential impact of these coding tools with a lot of people working in tech and product. Almost unanimously, after I have described what the possible impact of the tools could be, an accusatory finger is waggled at me and I’m asked “But what’s going to happen to all the junior developers?”
Every great engineering manager or tech leader I have worked with has had an outsized concern for developing junior talent. Good engineers write good applications, but great engineers teach other people. It’s no surprise that one of the first reactions I’ve been getting to this discussion is a concern for the next generation.
To be honest, I’m worried in the short term. As we’ll see more of in Part 4, one potential impact is that companies decide to cut their budgets and team sizes, and although contractors and outsourced development teams will likely be the first to be cut, mid and junior developers will be the next in line.
I’m concerned that there is a possibility that these changes cause a generational ‘ring-barking’ of our tech talent, where an entire cohort of junior developers is unable to find work in a team. The technical skills required to build an application, at least in early-adopting companies, will be those possessed by senior engineers — experience with solution and architecture design, security and weird edge cases. Even skilled junior and mid level engineers won’t have had this experience, by definition of the role that they’re doing and how long they’ve been doing it.
If the generative AI tools are doing the basic work, it’s the skills of senior developers and solution architects that are most needed.
However, while the impact on junior developers is a real concern, it’s not all bleak. Using Copilot recently while I tried to learn React and Django, it was mind-blowing how different my experience was to when I was at the beginning of my career. The opportunity to have a real time discussion about what a piece of code did, or where an error might be, transformed my learning experience. I wish I had access to ChatGPT when I was learning to code.
But while being able to write code is obviously an important skill in a developer, it’s not the ineffable quality we really need in our teams. There is a reason that the best engineers can move with relative ease from one language to another; great engineers aren’t taught to code. They’re taught how to think. I still believe that the ability to think both creatively, and with a systems-thinking mindset will continue to be the benchmark of what engineers bring to their team.
I am worried about the impact of these changes on junior developers, but I’m also hopeful. I don’t yet see a future that doesn’t require skilled engineers working alongside skilled designers and product managers to build great products. Even if the number of developers per team is reduced, perhaps we’ll move more towards a model like some traditional artisanal industries, where a junior is apprenticed more specifically to one senior mentor for a longer period.
The German term, “Wanderjahre” referred to a practice dating back to medieval times in which a young craftsperson would move from place to place after their apprenticeship. A large component of this journey was to experience different workshops and cultures, developing skills and mindset as part of the quest to finally deliver their Meisterstück (masterpiece).
Perhaps this ancient approach to passing on engineering ‘mastery’ might become more appropriate for our next generation product teams.
In Part 4 you can read about:
- The significant shift Generative AI presents for organisations amidst economic downturns and diminished venture capital funding.
- A thought experiment examining three scenarios: potential growth, cost-cutting, and maintaining current budgets with the introduction of Generative AI productivity tools.
- Hypothetical consequences and challenges, like drastically altered team compositions and what these changes could mean for roles such as product managers and engineers.
- A reflection on the true impact of these tools on productivity and team dynamics, providing a foundation for ongoing industry discussions.
Other articles in this series:
- Part 1: How Generative AI Will Impact Product Engineering Teams
- Part 2: The proliferation of Generative AI Coding Tools and how Product Engineering teams will use them
- Part 3: If engineers start to use AI coding tools, what happens to our product teams?
- Part 4: If AI coding tools reduce the number of engineers we need, where do we spend our budgets?
- Part 5: Who wins and who loses? How different types of business could be impacted by AI tools.
- Part 6: The Best of Both Worlds: Human Developers and AI Collaborators
P.S. If you’re enjoying these articles on teams, check out my Teamcraft podcast, where my co-host Andrew Maclaren and I talk to guests about what makes teams work.