Whence Innovation?
How IT Process is Crushing Software Engineering Progress
I was asked an interesting question once after running an internal hackathon for my organization's engineering teams; “Why can’t we always make this much progress and innovation?” It’s a fascinating observation, but sadly, the answer is simple: Agile (capital “A”) and the industry surrounding it are trying to turn software engineers into factory workers. And, they hate it.
Software engineers, those most mercurial of employees, are a wonderful mix of technologists and frustrated artists. They long to build things but simultaneously create something magical, new, and inspiring. Wrote George Sand in the 19th century, “Art for art's sake is an empty phrase. Art for the sake of truth, art for the sake of the good and the beautiful, that is the faith I am searching for.” Nothing better sums up the software engineer of the 21st century.
But, but . . . how are those talented, frustrated artists with incredible technical acumen used today? They are given lists created by people who are, at best, no more creative than they are and forced to provide those lists with “story points” and thence work on them one-by-one just like an assembly line worker doing the trade in a line. Engineers are told that product management decides the “what,” and they just do the “how,” and so they are forced, these incredibly expensive and creative people, to work their line items decided upon others.
To consider how wasteful this is, I must give an aside to the reader of my past. When I was a young man, I got my first real job at Walt Disney World, working at the world-famous Jungle Cruise. The Jungle Cruise is famous not just because of Walt Disney’s personal involvement in it or its iconic river cruise down the great rivers of the planet but because it is, effectively, the only place in any Disney Park from Paris to Hong Kong, where the Disney cast member gets to be a stand-up comic for 10 mins, over and over again, with a script that at the time was hmmmm, like traffic laws in Jakarta, mere suggestions.
The list of people who were Jungle Skippers like myself is legendary, including Robin Williams, Kevin Costner, and John Lasseter. And let me tell you, the selection process and training were quite the challenge. Brutal even. But at the end of that process, if you made it and became a skipper, and the wash-out rate was quite high back in the 80s, then you were alone, without any monitoring, with thirty Disney guests, for ten minutes, in a boat. Was I told at the tender age of 18 to be careful and stick to the guidelines and guardrails of what I could and could not say? Sure. But the biggest mandate I was given was to make sure the guests have fun and be funny. Use my creativity and inventiveness to make every experience great.
We, the software industry, are doing the opposite of how Disney treated a young creative innovator many years ago. We are stifling our expensive engineers and turning them into widget makers.
So, what is the answer to the original question? Well, hackathons have no product or release management, standups, process, formal security reviews, PR/CRs, and none of the expected lore of Agile (capital “A”). All these things still happen in the flow of the hackathon, but they aren’t the formal “ceremonies” that crush innovation. The code that is created out of a hackathon is as good or better than what comes out of “the process,” but it is created faster, with more creativity; whole new ideas for things are thought of, built, and ready to ship in a week or two, all when you take away the things that stop the frustrated engineering artists from doing great things in the first place.
It isn’t that agile is bad, or that all agile practices are bad. It is that Agile (capital “A”) is strangling the creativity out of our engineers.
Disney wanted me to be funny, so they chose me and trained me so that I could delight their guests with my imagination and creativity. We want more velocity and features in our software; stop the madness of making engineers line-item processors and let them make magic for you.
And yes, I would love to go around that river one more time and tell those jokes that I can still recite, “Rise like bread, it’s the yeast you can do for me . . . I’m on a roll.”