On Organizational Transformation
I have a lot of empathy for people that work at big companies. No one should be required to use a crappy ThinkPad loaded with sluggish, productivity-monitoring software. No one should be forced to communicate through a regimented, politicized hierarchy to do their job. No one should have their actions decided for them by someone else, especially because no one else has a better chance of determining necessary actions than the person who is closest to the work.
And yet, big, rigid companies are everywhere! And they're brimming with employees! Why?
After a few years of trying to change my own big company employer, I realized that I was never going to succeed—so I quit. As my friend Leon says, "you can change where you work, or you can change where you work."
But I was not content to free myself from the shackles of a big company and immediately embrace the liberation I could attain by joining a small company. That seemed like a cop-out; first, I needed to understand why changing even small aspects of a large organization was disproportionately difficult. Surely, thought I, changing large organizations isn't impossible?
To answer that question for myself, I spent about two years consulting with increasingly large, increasingly change-averse companies. Each organization was tremendously well-adapted to succeed, optimized for their own competitive environment. The only problem: over the course of their growth and evolution, they had each built software development organizations that were failing to deliver consistent results over longer periods of time and at larger scales.
In most cases, these companies had begun building an IT organization when the industrial (read: waterfall) metaphor was prominent. Because command-and-control waterfall software management is so demonstrably ineffective, I was confident that helping a company "transform" into a self-organizing, autonomy-granting agile organization would yield everyone great benefits.
We introduced companies to agile principles, values, and practices. We educated and sold agile to executives. We helped train their developers, who were always highly engaged and excited to learn. At the outset, it always seemed that everyone in the company both recognized which things weren't working and agreed that the changes prescribed would—if implemented—drastically improve their organization's effectiveness.
During each engagement, a pilot project that reflected a clean break from the past would succeed wildly. Meanwhile, incremental changes would begin to successfully cascade through the rest of the organization. The first throes of an organizational transformation are highly rewarding (if grueling) work.
But each transformation failed to take hold. Sometimes, they failed spectacularly and all of the change agents within the organization were purged. Sometimes, practices would be retained, but the broader organizational systems would remain soundly intact. And never did I witness a genuine acknowledgement of individual autonomy take root in an organization that had previously been described as "command-and-control".
And these failures were not for lack of trying hard enough! On every project, everyone always brought their "A"-game. But soon after after the initial successes, it was as if the direction of the wind changed. It was like the organization anthropomorphically woke from its slumber and—upon detecting the presence of a foreign body—quickly excised the changes before uneventfully returning to its dormant state.
I've had plenty of time to ponder this and ask myself why this pattern kept repeating.
Over the years, I've adopted (and subsequently discarded) numerous metaphors for organizational change. The only metaphor that I still consider apt is evolution. Consider Richard Dawkins' seminal work, The Selfish Gene. In it, Dawkins demonstrates that not only organisms but even individual genes undergo competition to survive via natural selection. He masterfully extrapolated this discovery: so too do ideas ("memes") and organizations fight for survival as if by natural selection.
Take dinosaurs, for example. As best I understand it, the world saw such large dinosaurs because of the parameters of their competitive environment. In order to survive and reproduce, many species got caught in a sort of arms race with others; this produced ever-larger, ever-more-powerful dinos. But when the environment changed in a fundamental way, dinosaurs' adaptations were ill-equipped to compete and they died off. The top of the food chain was (and is always), comprised of species that are highly optimized for success in the context of their environment.
But whether we're speaking of organizations or organisms, there are two catches with all optimizations:
- Optimizations are contextual. In a fundamentally different environment, a given optimization will most likely be detrimental to success.
- Optimizations are specializations. Switching from one optimization to another requires backtracking to some unoptimized point in order to find a new beginning and a fresh start.
When the competitive landscape requires an organism to change those optimizations, the path to success is typically slow and counter-intuitive. For species, the outmoded adaptations might become (evolutionarily expensive) vestigial traits. For organizations, the recovery often requires starting a brand new subordinate organization that's free from the constraints and assumptions the existing organization acquired while attaining their previous success.
##There can be no agility for the highly-optimized
Big companies aren't worse than small companies, they're just more likely to be well-adapted to succeed within their environment. For an organization to undertake operating in a fundamentally different environment, the transformation will always be more costly than it would have been to start a new venture. I believe large software companies are helping us understand this phenomenon, because of software's distinctly low—and ever-decreasing—barrier of entry (in terms of capital). In my mind, this is why RIM is doomed, why Google will take years to grok social product development, why Facebook will flounder in trying to make money from mobile devices, and why Netflix will be years behind schedule when they finally discard DVD rentals.
To prospective employees of big companies: if everything about the company is a great fit for you, then go for it! But, a warning: view any dissonance between you and the company through a very broad lens: that organization became successful by expressly not doing things the way that you wish they did.