What is the most important rule of experimentation?

That’s what I discussed with Richard Browning, Englands very own Iron Man.

Of course, I didn’t phrase it that way. I’d asked Richard to explain how he went from being a rockstar commodity futures trader at British Petroleum to designing and building the worlds first rocket-propelled flying suit.

However, as the conversation progressed, he explained how he approaches experimentation.

I was expecting advice that would only be applicable for budding inventors.

Instead, I found his advice was essential for all of us, from startup founders to multinational CEO’s.

A Quick Bio on Richard

I’m not going to spend too much time introducing Richard.

This isn’t a Vanity Fair weekend bio. This post is intended to leave you with just enough information to make a judgement call about whether to apply Richards approach to your own experiments.

Having met him and watched him in action I think it’s a no brainer.

But if you’ve not heard of him here’s the rundown. I’ve copy-pasted his timeline from Tymline (and added a few extra details) to speed things up.

For hundreds of years, humankind has tried and failed to build a practical commercial jet pack. Richard Browning succeeded in less than 24 months (source)

1989 – Starts schooling at Queens College Taunton

1997 – Completes degree in Geology at Cardiff University

Initially, Richard had signed up for an engineering degree.

“But I lost interest,” he admits. “It was full of calculus and maths – we never went near a lathe or anything practical – so I shifted to something more hands-on: exploration geology.” (source)

2007 – Starts at BP trading oil and crude futures

2007 – Joins Royal Marine Reserves. After two years receives Green Beret.

2016 – Designed, built, redesigned and rebuilt Gravity’s prototype flight suit. The suit took 15 months to develop completely.

2017 – Co-founded Gravity Industries Ltd in March 2017, the company behind the ‘Daedalus Mark 1’, a flight suit that uses six miniature jet engines to achieve vertical flight. The kerosene engines are rated at 22kg of thrust each. Browning uses his arms to control the direction and speed of the flight, while a display inside the helmet gives updates on fuel consumption and other usage data

2017 – Richard gives a demo of his recording and explains the idea at TED

2017 – Browning achieves a speed of 32.02 miles per hour (51.53 km/h) with the suit during a Guinness World Records attempt for the ‘Fastest speed in a body controlled jet engine powered suit’

2019 – Richard is now an accomplished inventor, speaker, record-breaker and entrepreneur who has featured at, with or in Wired, The Telegraph, Redbull and Ted.

There are Lots of Rules for Experimentation – Aren’t There?

Before writing this post, I was curious to see if anyone else felt there was one master rule for experimentation.

So I asked my Network.

Here are just some of the great answers I received.

– Be prepared to fail. – Fail fast and get up. – Experiment again and again. – Keep it simple. – Experiment to learn. – Set hard and fast deadlines for milestones. – Make sure you test and validate with cheque writers. – Try & try again. – Believe in yourself.

All great answers but not the one I was looking for.

(If you have other suggestions please do share them on this LinkedIn thread?).

But, no one picked what I now think is the most important rule.

Richards Other Rules of Experimentation

For full transparency, Richard didn’t only share one rule. I’ve listed a couple of his absolute gems below.

Just be aware that although they are important, like the above ideas, I felt they are all subordinate to the most important advice I received from him.

Not doing these things will slow you down. But, they won’t necessarily take you down.

Don’t listen to the experts.

Despite having access to many aeronautics engineers, Richard told me that if he had listened to them, he would never have developed the rocket-suit.

He repeated this advice in an interview with Red Bull:

The big aeronautics companies would list half a dozen reasons why this would never work,” he says. “You’d never be able to carry enough fuel, never generate enough power or manage that power. The rotational forces would rip your arm off every time you moved it. The heat would be unmanageable – you’d disappear in a fireball. You’d end up needing a huge traditional jet pack with armrests, gyros and whatever. Then it’s going to be unfeasible from a power-to-weight ratio.” His solution was to ignore those voices. “Because that’s where every breakthrough lies,” he says. “Every invention mankind has ever made comes from disregarding conventional assumptions. The humble part of me acknowledges that 99 per cent of the time they’re right. But I hunt down the one per cent because that’s where you change the world.” (source)

And this certainly makes sense. Richard Branson didn’t listen to the experts from British Airways. Elon Musk didn’t listen to the engineers at BMW. James Dyson didn’t listen to the designers at Hoover.

And these are just three examples. If you look back through history, the experts seem rarely to evolve from being knowledgable to become successful inventors, innovators and entrepreneurs.

Ignorance, it would seem is a virtue, as we can think more freely about solving the impossible.

Moreover, ignorance forces you to learn from experimentation, especially if you are going to ignore the experts.

None the less, this isn’t a rule. Some experts do succeed.

Also, the Bransons, Musks and Dyson’s of the world still have to find experts at some point in their journeys.

Perhaps they were just better at filtering the opinions that were presented to them?

Mix with Supporters Not Detractors

Jim Rohn, a mentor to Anthony Robbins, said that

we are the average of the five people we spend the most time with.

So if you want to experiment. If you want to innovate. If you really want to do something different.

Make sure you surround yourself with those who support your crazy idea!

Of course, finding believers when you are trying to do something different isn’t easy. It requires planning and energy to seek out the 1% who will back you.

That’s why organisations like Abundance 360, YPO or The Entrepreneurs Organisation are so successful.

They provide an environment for like-minded people to share ideas, not so that they can be kicked down, but so they can be built up.

So great advice, but again perhaps not a rule?

It might be 100x harder, but it’s still possible to succeed if you are strong enough to ignore ‘naysayers united’.

And, if you are running experiments correctly you’ll be able to counter their ‘opinions’ with ‘fact’.

Experiment to learn

Richards story is a perfect example of how to experiment to learn.

Test, learn, iterate, repeat

Test, learn, iterate, repeat.

Or in Richards case..

Buy and a cheap rocket to see if it will fit on an arm.

Add another cheap rocket to the second arm to see what level of control an operator can have.

Change the rocket positioning multiple times to try to improve control

Add more rockets to see how many are required to generate enough thrust to take off

Change positions to test improve in-flight control.

Learn how heat is absorbed by air, helping Richard to avoid incinerating himself.

and so on, until after years of failure, you become an overnight success.

However, this isn’t a rule of experimentation.

This is the most important rule of innovation.

If you want to keep yourself at the bleeding edge of innovation, you have no choice but to run agile business models, where you test, learn and iterate continually.

Which incidentally is something all incumbent organisations find incredibly difficult to do.

So you could say the first rule of innovation is to run as many experiments as you possibly can.

But, what is the most important rule of experimentation?

The Most Important Rules of Experimentation

It’s this.

Never carry out an experiment that restricts your ability to do more experiments.

That’s it.

As Richard explained to me…

Rockets are dangerous. So I had to make sure I set ‘hard’ stops for my experiments. I would therefore never do something where I might end up killing myself, breaking the law or loosing all my money.

It is after all rather difficult doing more experiments when you are dead, in jail or destitute.

For Richard, it was perhaps obvious this is an important rule, as the consequences for not applying it would have been terminal (to him or his company).

But is this rule as important for corporates as it is for people developing rocket-propelled flying suits?

And I don’t just mean in the obvious industries like automotive, aerospace, medicine or mining where poorly conceived experiments could be fatal.

For example, It’s obvious that Tesla has to develop ‘safe’ experiments that don’t lead to situations where it’s car kill’s the passenger and/or make illegal manoeuvres and/or create so much financial risk it leads to bankruptcy.

It’s equally obvious that startups should apply this rule because the surest way to fail, aside from not experimenting enough, is by burning your limited capital on experiments that produce data which is meaningless or misinterpreted.

Yet, time and again companies ignore this rule, especially those that produce products or services which are unlikely to do harm.

Perhaps their leadership teams believe they are infallible?

Or perhaps the risk of financial loss isn’t weighted in decision making, the same way the potential loss of life would be.

Whatever the reason, time and again we see corporates running experiments that waste millions, sometimes billions of dollars, because they failed to continually ask whether running this experiment could preclude us from running more experiments?

For example, consider the backlash these corporates faced when they made significant changes to their product lines and brands.

And this is important.

The release of a new product or a new brand is most certainly an experiment. In fact it’s the ultimate experiment for all organisations.

Take New Coke from Coca Cola – In a moment of madness Coke decided to change their secret formulae to regain market share from Pepsico. Hundred of thousands of Blind tasters loved it.

Their customers? Not so much.

Within three months they were forced to revert back to their original recipe.

How about Windows Vista, one of the most hated operating systems ever. Unlike with Coke, Vista wasn’t rolled back. Users like me were forced to put up or like me give up and switch to Apple (which I still use today).

Do you remember GAP trying to change their logo? $100mill was spent on this, only for the decision to be reversed within 7 days of release!

Or, then there was YAHOO, who felt changing their brand would change customer sentiment about their company. In this case the risk to the company was minor as the change was immaterial.

In fact, it’s the immateriality of the experiment, compared to the massive challenges they should have been focused on which is interesting in there case.

These are all examples of incredibly costly experiments gone wrong, or in Yahoo’s case, failing to focus on the important experiments.

If only the CEO’s had asked their teams that one question…

Are you sure that this experiment will not reduce the likelihood we can run more experiments in the future?

If they had, I’d have expected Coke to actually run an experiment.

(Blind tastes, survey’s and focus groups are not experiments as all you observe are opinions.)

Perhaps they’d have stacked supermarket shelves in 10 pilot stores to gauge what happens. Then they’d have tried to find a way to observe whether customers become repeat customers or move to other beverages? And if that looked promising they’d have iterated again and again until they were certain about the impact of the release.

More time consuming, but ultimately a hell of a lot cheaper.

Windows Vista? Why didn’t they do what Apple has done for years (or maybe they did and ignored the responses) and offer beta versions for 12 months prior to the release?

Or GAP. They relied on ‘opinions’, instead of developing experiments to determine facts. A pop up stand here, a new line in a department store there. There must have been at least a dozen ways they could experiment with new logos to develop ‘real’ indicators about how their consumers would react.

As for Yahoo, I’d have hoped they’d have focused on designing experiments to explore solving their biggest problems.

An Experiment To Close Out

Why not run your own experiment and test the ideas I’ve shared in this post?

The next time you decide to do something new where you cannot be sure of the outcome (i.e. an experiment), consider asking your colleagues to answer these questions.

How will this experiment preclude us from being able to carry out more experiments?

And see what answers you get.

Then ask your team how they could adjust the experiment to increase the probability they’ll be able to carry out more experiments in the future?

And again, see what answers you get.

Perhaps you’ll change your decision. Perhaps it reinforces your original decision.

Either way, I’m sure it’ll improve the quality of the proposed experiments AND reduce the risks that experiments inherently create.

I’m certain your team will start thinking more actively about ways to…

-run cheaper experiments

-run experiments that generate ‘quantitative’ results (i.e. facts, not opinions)

-Prioritise experiments that test the most significant issues first

-Push for more time to run smaller, safer experiments.

-Simplify experiments, so the results are clearer and/or easier to obtain

-More frequently share their results with their sponsors (cheque writers)

-Kill the ‘bad’ projects without fear of recrimination

-Develop experiments to disprove the ideas

-Brainstorm how experiments could go wrong and adapt accordingly

-Stop relying on market surveys and focus groups and look for better ways to test consumer behaviour.

-Set clear loss limits to determine when to stop and step away.

-Move from a/b to a/b/c/d/etc testing

And if you do this, I can’t wait for feedback to know what happened?

About the Author

Colin Iles helps organisations develop breakthrough ideas and then develop cheap, fast, safe experiments to validate them.

You can contact him for keynotes, Workshops and advisory support by emailing him at colin@coliniles.com.

Postscript Want to Experiment with a Flying Suit?

Just in case you are keen to learn what it’s like flying around with 6 rockets strapped to your body…

The last time I checked you can pick up a suit for the incredibly reasonable price of $450K. And guess what. I think they even throw in a couple of free lessons to get you started.

If that is a little pricey you could consider Hiring Gravity for your next event. Or alternatively, you could visit Browning’s hangar to experience wearing the suit in a controlled safe environment.

Finally, the book Taking on Gravity will be available on Amazon in 2020.