There is a story retold in the virtual reality community which emphasizes reaching perfection through a quantity approach over a quality approach. The text originally came from the book Art and Fear, which is about the process of making art. Derek Sivers presents the story as follows:
The ceramics teacher announced he was dividing his class into two groups. All those on the left side of the studio would be graded solely on the quantity of work they produced, all those on the right graded solely on its quality.
His procedure was simple: on the final day of class he would weigh the work of the “quantity” group: 50 pounds of pots rated an A, 40 pounds a B, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an A.
Well, come grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity!
It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
Quantity trumps a quality approach when trying to reach perfection, right? No. Not at all. It is critical to understand the story in its original context.
Quantity Versus Quality in Learning
These students aren’t exploring the cutting edge of clay pot design, finding new ways of pushing mounds of clay into innovative and problem-solving symmetrical shapes. They’re introducing themselves to a very well known skill. They gaining mastery in a known art. Nothing more.
What is the real moral of the story? It couldn’t be simpler. Practice makes perfect.
So many people have repeated this story and walked away with the wrong moral, which is unfortunate. You know what is worse than learning the wrong moral to a story? Learning the wrong moral to a story and then applying it to the wrong situation.
Quantity Versus Quality in Problem Solving
Are you looking for the best approach to solve a new problem? This probably isn’t the parable for you. Still, when someone applies this particular story to the topic of design, what they may be trying to say is that you should not become paralyzed in structuring the solution. True enough.
Equally, you should not become trapped in iterating or experimenting with a problem that you don’t fully understand. Try something different. Weigh your options. There is more than one way to reach a solution, and each way has its own advantages and disadvantages.
Quantity Versus Quality in the Metaverse
I’ve seen this quote applied to metaverse implementation, but in a different way. The idea is that time spent on designing how a metaverse should work is wasted in comparison to actually going out there and building one and iterating on it. Stupid eggheads thinking about the problem rather than going out and doing something, right?
In particular, a quote from John Carmack is used to support this view.
It is almost painful for me to watch some of the VRML initiatives. It just seems so obviously the wrong way to do something. All of this debating, committee forming, and spec writing, and in the end, there isn’t anything to show for it. Make something really cool first, and worry about the spec after you are sure it’s worth it! –John Carmack Answers (Slashdot)
Let’s keep in mind that this particular quote about a single design effort is now sixteen years old. There is little question that VRML was flawed. Shortly after that quote, John had his own opportunity to pitch an implementation of the metaverse, and it failed to gain traction within his own circle. Other implementations have since come and gone, and despite all of the experiments and iterative improvements, none have really succeeded.
Why did nobody implement the ideal solution and then publish the spec as Carmack suggests? It goes back to those students and their clay pots.
Clay Pots in the Metaverse
The story of the clay pots works in the specific scenario for which it was designed. Students learning a known craft. The student labor is free, the clay itself is free, and there is no real penalty for making a mistake. That is how education works. That isn’t how the real world works.
In the real world, we have opportunity costs, both explicit and implicit. We have the labor, hardware, and software required for platform development. We have a reputation as a steady platform to build upon. We have partners who are ready to jump ship when things go wrong.
What if the experiment fails? Well, we could back out the change, but now we’ve spent a lot of money and we are back to where we’ve started. Worse, it has cost us some of our pre-existing business and it has also cost us the trust of our community. The experimental approach comes with a price.
If the direct and indirect costs of making a new pot costs $100,000, how far does it make sense to pursue a quantity approach? Once we have the start of a good platform, does it still make sense to experiment and iterate our way to something even better?
At this point, we should be able to understand why Linden Labs is pursuing Project Sansar instead of fixing Second Life. We should also have a better understanding of two of the fundamental problems with metaverse implementations that we covered earlier:
“The more complex and integrated a platform gets, the slower its innovation becomes.” Technical complexity aside, with increased adoption comes increased opportunity cost.
“The real challenge isn’t technical.” We can’t make technical improvements in a vacuum. Social considerations (such as popularity, trust, reputation) and financial considerations (business contracts, revenue, legal considerations) come first.
Finally, we can see why people go through the trouble of building consensus, developing specifications, and publishing documents. These people also believe that a global rationalization of a metaverse might work out better than a series of individual design choices, arrived at by experimentation, all merged into a single architecture.
Really, it is a choice – be smart about it
Is design the answer? We all should be approaching a metaverse through design, right? Not necessarily. Let’s not fall for the trap that we have to follow one specific approach to arrive at a metaverse. Perhaps this topic has more than a few similarities with a comparison of linear and lateral thinking. (Neither one provides a universal solution.)
Carmack’s criticism of creating specifications and building consensus is valid when that effort is divorced from real-world considerations and specific action. A global design is never perfect, it is always going to have oversights that need addressed. On the plus side, a good design will illuminate a more efficient path to solving a problem. A great design will reshape the problem entirely with a revolutionary change.
Experimentation and iterative improvement also it own challenges. Can you evolve out of a a flawed approach? Even so, the risk and costs of evolving a successful platform will increase over time. On the plus side, experimentation can immediately demonstrate success, and find answers when we really don’t understand the problem. The right change can unlock new opportunities that were unseen up until that point.
We shouldn’t dismiss either approach.
Use the right approach, at the right time, at the right level of focus, for the right problem. Use your judgement. If you have been consistently following one path, try another. If everyone else is following another path, also try another. Balance can yield reward.
One last thing to consider. Balance is good, but do we have to arrive at balance through a string of boolean choices? No. If you can find a way to blend the two approaches together, a global design which harnesses experimentation and iterative improvement, you’re probably ahead of the game.
Now get out there and make your perfect clay pot.
