James Sulak is FlightAware’s VP of Engineering.
In early 2019, as our engineering team grew, it was becoming clear that our old ways of working were beginning to stretch and fray. Our engineering team was about 50 people, divided into 11 teams, and we were hiring at a net rate of about 1.5 engineers a month. We knew we had problems to solve to ensure that we could continue to produce high-quality work at a rapid pace.
This post is the first in a series telling the story of how we evolved our organization to enable growth, foster individuals’ personal career development, and of course, build great things.
Our challenges
We faced several challenges:
- Team size. Several of our development teams were reaching ten people in size. We think smaller teams work better (this has been codified elsewhere as Amazon’s “two-pizza rule”), so we needed to create new teams. But…
- Our teams were organized by functional competency. We had a team of web developers, a team of backend developers, a team of mobile developers, etc. This method worked at first because, at our size, this effectively organized us by product as well. But unless we wanted to have a development team of 12+ people, we now had to face down the choice of organizing by functional competency, product, or something else.
- We wanted to do more cross-functional work. We were struggling when attempting to build new products cutting across competencies—it required a lot of formal coordination between teams to make it happen (think meetings and passing Jira tickets between teams), which slowed things down and led to dropped balls.
These are interesting problems! We’d already been inching our way towards a new way of working, but it was time to look more intentionally at what we were trying to accomplish—and along the way, learn more about who we were and who we wanted to be.
We came up with a solution that captures what we value most in how we work together and evolves with us as we continue to grow.
What we created were the building blocks of Crews, Wings, and Alliances.
In this first post, I will talk about how we determined what exactly we wanted and why.
Our design process
Our first step in approaching the problem was to read.
An experienced software developer knows that the most maintainable code is code you don’t need to write. So we wanted to reuse the work of others as much as possible. Smart people have worked this problem before, and it would be irresponsible not to learn from them.
We focused our research on a set of books on the topic, and in particular, looked very closely at Spotify’s approach. I’ve included our reading list below under Agile Organizational Design Reading Program.
Our design principles
Approaching any complex, real-world problem is an exercise in choosing tradeoffs. Plans can (should!) change as the world changes.
We first focused on creating our design goals, or principles, so we had a framework to guide our decisions now and in the future. With these, we can respond to growth, changing markets, and lessons learned—while keeping sight of our true north.
Our principles:
1. Build delivery teams
“What are some of the advantages of organizing much of a company in a mission-oriented form? There is only one. It is that the individual units can stay in touch with the needs of their business or product areas and initiate changes rapidly when those needs change. That is it. All other considerations favor the functional-type of organization. But the business of any business is to respond to the demands and needs of its environment, and the need to be responsive is so important that it always leads to much of any organization being grouped in mission-oriented units.”
Andy Grove, High Output Management
- Orient our daily work outwards toward the value we deliver to our customers, not inwards toward specific activities or functions.
- Individuals work on autonomous, self-sufficient teams that include all functions to design, develop, and deliver software from idea to launch. Avoid cross-dependencies and handoffs as much as possible.
These twin ideas—outward orientation and autonomy—are foundational and complementary. They both point towards forming teams around what they produce. Teams should face outwards to connect them directly to the end product and customers. And they should be as autonomous as possible so they can serve those customers without waiting on approvals or cross-dependencies (This is also just a more fun way to work!).
A small team of engineers can pull off amazing feats. Communication is simple and the mission is clear and things just click and code flows like water.
But success leads to growth, and growth erodes the effectiveness of this easy informal communication. This isn’t because of a conspiracy of pointy-haired bosses, but because communication in large groups is exponentially more complicated. It requires meetings, emails, and formal tickets. Alignment is harder. Things slow down. An elephant has a slower metabolism than a mouse.
So, we decided to reproduce the advantages of a small company as much as possible through forming small cross-functional groups of people working side-by-side on well-defined efforts.
These groups are known as a delivery teams, which is not a concept we invented. For an in-depth discussion, see Scaling Teams, Chapter 7: Scaling the Organization: Delivery Teams.
2. Optimize communication
“The purpose of organization is to reduce the amount of communication and coordination necessary.”
Fredrick Brooks, The Mythical Man Month
- Avoid knowledge silos. Create structures and routines to ensure alignment and sharing.
- Optimize communication by getting the right people together, face-to-face.
- Encourage direct, unscripted collaboration between individuals, outside of regular meetings, and without regard to team structure or who reports to whom. Empower everyone to talk to whom they need to to get their job done
- Reward candid, respectful conversations about difficult issues. Attack problems, not people.
One thing I’ve come to believe that truly differentiates our approach from that of many engineering teams is that we value communication skills. We expect people on our team to express themselves well in writing and in person, and to have difficult conversations in a way that empowers.
Communication gets harder as a team grows. The number of possible 1-1 communication channels increases proportionally to the square of the number of people. People know each other less, so it becomes easier to miscommunicate; at the very same time, effective communication becomes more critical. You need new layers—middle managers—that introduce even more possible communication channels and complexity.
Embracing the first principle, delivery teams, helps minimize the need for formal communication (meetings, tickets, emails, etc.) by enabling people doing the work to talk about a thing informally, in person, when needed, whether they are a developer or a designer or a tester. Everyone on the team is empowered to speak to whoever they need to—including the CEO—to get things done.
This principle is a reminder that communication is a massive challenge in a growing team, and that anything that we can do to optimize communication is a huge win.
3. Respond to change
“Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.”
The Principles behind the Agile Manifesto
- Respond, with a minimum of friction, to changes in our market, strategy, and priorities.
- Our organization should be fluid and adaptable—e.g., delivery teams are spun up and down as needed—while keeping an individual’s day-to-day experience consistent and understandable.
One of FlightAware’s competitive advantages—especially in the aviation industry—is our ability to respond quickly to new opportunities. We don’t want to lose that as we grow. So we need the ability to confidently change direction and adapt.
All that said… while our organization should be fluid and adaptable—delivery teams spun up and down as needed—we aim to keep the day-to-day experience of individuals on the team consistent and understandable. Put plainly: we don’t want people to need to change managers to work on new initiatives.
4. Create a continuous learning culture
“Experience is the best teacher, but you have to do the homework.”
David H. Maister
- Create an environment that encourages and rewards learning as a team and as self-directed individuals.
- Build in opportunities for individuals to work on a variety of different projects and efforts throughout their careers.
- Engage in continuous improvement. Continually evaluate how the organization can be more effective. Conduct blameless retrospectives and incident postmortems.
- Encourage teams and individuals to experiment with new ways of working.
The work we do has two end goals:
- Deliver products to our customers.
- Deliver knowledge to ourselves.
These are co-equal! Learning, done consistently over time, is an investment that pays off with compound interest. It benefits the individual—it pushes forward skills and knowledge and prepares people to take on greater roles and responsibilities. It benefits the team—over time, we can take on more complex and higher-impact work.
This is not just about book learning; it is about ensuring we have a framework for people to work on a wide variety of projects and gain a breadth of experience to propel them in their careers.
This principle is the principle I love most. My journey into software development was a journey of self-education. We are incredibly fortunate that software is a field that doesn’t rely on formal certifications or degrees, and you can learn everything you need through a habit of self-study and an engaged curiosity.
We want to avoid the lazy temptation to switch on the intellectual autopilot, doing only what is necessary for the day to day, following the immediate course in front of us. We believe in flying the aircraft. Learning is a first-class principle.
5. Deliver business value constantly (continuous delivery)
“If it hurts, do it more often, and bring the pain forward.”
Jez Humble, Continuous Delivery
- Engage in practices and build systems to deliver high-quality software into the hands of users safely, quickly, and sustainably.
- Favor small, heavily automated releases over large, manual ones.
We have embraced the goal of continuous delivery (CD), which is the ability to deploy software in a routine, automated, and low-drama way, on demand.
By definition, the more frequently we deploy our software to production, the more value we deliver to our customers. Software that is not in production—not being used—is literally useless. It provides negative value.
It turns out (and research supports this) that deploying software more frequently is both easier (you do it more often, so you get more practice), and safer (the delta between versions is smaller, so it’s easier to figure out what went wrong when something does break). And it’s just a more fun and satisfying way to work.
Achieving the goal of CD requires adopting a host of technical practices, including continuous integration, build pipelines, configuration management, and continuous testing. Those practices are essential, but they are not sufficient. The way we work must support those practices. And so continuous delivery is a first-class principle of our team.
Conclusion
Deciding on design principles is one thing. Building something out of them is another.
In the next part of this series, I will discuss how we made these principles concrete and created the fundamental building blocks of our team: crews, wings, and alliances.
Appendix: The Agile Organizational Design Reading Program
Agile organizational design is a complex and wide-ranging topic, and it’s far from a solved problem. Some of the resources we found helpful when creating this:
Resources specific to Spotify’s journey:
- Scaling Agile at Spotify by Henrik Kniberg and Anders Ivarsson
- Spotify Engineering Culture part 1 Agile Enterprise Transition with Scrum and Kanban 1 and Spotify Engineering Culture part 2 by Andreas Tjernsli
- Thoughts on emulating Spotify’s matrix organization in other companies by Kevin Goldsmith
Extra credit reading:
