innovation as the enemy of maturity
In my previous blog,
maturity as the enemy of innovation, I identified the destructive forces of organizational maturity. As an engineer, I cannot allow a problem to go unsolved.
To stem the tide of chronic risk adversity, we must imprint the entrepreneurial spirit into the culture, as a core value. The courage to initiate calculated risk-taking needs to be admired and rewarded. The stigma associated with failure must be removed by appreciating the knowledge gained through lessons learned.
The cost of research and experimentation must be reduced. The pace of development must be rapid. These are achieved by keeping the team size small. The most risky and unproven ideas should be attacked by an individual, not a team. Functional teams need alignment on direction and a degree of agreement. Revolutionary advancement is frequently the result of radical thinking. New directions, different techniques, and breaking well-established rules are unpopular, especially to those with emotional investments in past innovations. There are fewer disagreements in a small team, and none at all in a team of one.
Uniformity of thinking leads to stagnation. Diversity must be encouraged, and this is promoted through a free market for ideas. Organizations that rely on a command-and-control style of management to set direction eliminate the competition that nurtures a diversity of ideas. Management should empower technical decision-making to be done bottom-up rather than top-down.
Once new ideas have been adequately proven, there must be ways of teaching them to others and incorporating them into product. Product development is often done sequentially, one release at a time. The immediate release focuses on short term commitments, and the delivery is usually constrained by schedules and costs that can ill afford risk. Riskier development must be allowed to proceed concurrently on longer-term schedules, without overly constraining the process. Developments from the unstable branch should be merged into the main product release, as they become ready. This provides an environment to accommodate both risky development and risk-adverse development.
The above techniques can be effective in remedying the organizational impediments to innovation. However, there are also commercial impediments that must be overcome. That will be the topic of my next blog.
maturity as the enemy of innovation
As a software development organization matures, its culture institutionalizes practices that promote quality. The most obvious measures of quality are in terms of defects. The quest for lower defect counts is a noble one.
At the same time, complexity grows as the weight of larger feature sets and lofty requirements are incorporated into the software. With increasing complexity comes a larger development team. A larger team means more division of labor, higher cost of management overhead, and more careful planning to be able to utilize those resources. Division of labor requires the delegation of distinct responsibilities to specialized roles. Decision-making becomes more dispersed and distributed, and a decision will have wider impacts on team members, so good communication becomes a critical success factor.
Unfortunately, decision-making, communication, and the distribution of responsibility is never perfect. The more complex the software, the larger the team, the more distributed the responsibilities, the more specialized the roles, the higher the cost of development. This is not a scalable organizational model. Once a software organization has reached this stage of maturity, it is doomed by inefficiency. Creative individuals are no longer empowered to quickly realize their ideas. Two key requirements to motivate and nurture a creative mind are independence and freedom. The degree to which these environmental factors are blunted is the degree to which innovation is impeded.
Because decision-making is distributed among many people, it becomes impossible for innovative ideas to be realized without winning consensus. Innovative ideas are unconventional ideas. A novel approach often needs to break well-established rules and go against practices that have worked in the past. Innovation is destructive to the status quo. It must be disruptive in order to rise and overtake older practices. Innovation is the product of experimentation and risk-taking, usually resulting in failure and rework. Without an environment that encourages this, there will be no innovation.
The quest for quality through institutionalized practices and disciplined management are in many ways counter to the independence and freedom that motivate a creative mind. Careful project management and planning discourages experimentation with risky new ideas (technical, organizational, and commercial), where many unknowns make results difficult to predict. Risky ideas need experimentation so that many failed attempts can be discarded quickly, allowing successful ideas to be taken to production. Mature organizations respond to higher risk by applying stronger management practices for mitigation, thereby increasing the cost and slowing the pace of failure towards success. In fact, once management smells failure, they immediately work to eliminate it, thereby destroying its very value as a vehicle to learn what will be needed to succeed. This is how many mature organizations have killed off the entrepreneurial spirit. This is the reason why most innovative ideas come from small groups of individuals working in a garage or a small start-up company.
