John Howrey
Available for new work

Driving Growth Through Design  Embedding experimentation at DigitalOcean

Growth experimentation framework

Growth at DigitalOcean had been an engineering function. Tickets came in, engineers built tests, results went up on a dashboard, the next ticket came in. Design was on the deck, sometimes, when there was time.

It worked, in the way that things "work" when you're moving faster than you can think. The team shipped a lot. They learned less than they should have.

I came in to change the inputs. Not to slow the cadence, but to give every test a clearer hypothesis, a stronger hand at the page level, and a record that anyone could read three months later.

The result is a process the growth team still runs. It made design a peer of engineering and PM, and it made every test count for more than the next test.

What was broken

Lots of tests. Fewer answers.

Two months in, I went back through the experiment log. About a third of the tests had been run twice in the same calendar year. A handful had been run three times. Same setup, similar copy, inconclusive in the same way each time.

The problem was at intake. A test idea would come in as a one-line Slack message. Engineering would scope it on the fly. Design would join late, fix the type, and call it a day. Three weeks later the result would land in a thread that nobody bookmarked, and the next quarter the same idea would resurface.

Test intake was informal. Most ideas came in as Slack messages.

Hypotheses were thin. The strongest version of "what we expect to learn" was usually missing.

Decisions and outcomes lived in private threads. Three months later nobody could find them.

Designers were brought in late, when the page already had a layout to fix.

What we built

The Initializer.

A weekly intake meeting open to anyone in the company who had a growth idea. A short template they had to fill out: who is this for, what behavior do we expect to change, what's the smallest version we can ship in two weeks. Design, PM, and engineering all in the room when the idea was scoped, before any code was written.

Initializer intake template
Workshop outputPrototype review

Every test that went live had a public page. The page held the hypothesis, the screens, the result, and the call. Anyone in the company could read it. The growth team's institutional memory moved from Slack into something searchable.

Experiment dashboardFindings page

Outcome

Faster learning. Less rework.

Acquisition and retention metrics moved on the experiments that came out of the new intake.

Repeated tests dropped sharply once the prior results page existed and people learned to check it.

The growth team stopped being the team that "designs ran past." They were the team that designs ran with.

The Initializer process is still in use after I left. Other teams at DO ported pieces of it.

The trick wasn't more rigor. It was a smaller front door, with a clear template, that everyone walked through together. Once the inputs got better, the outputs followed.

Takeaways

Client

DigitalOcean

Sector

Technology, Cloud Computing, Growth

Role

Design Leadership, Process Innovation, Cross-functional Strategy

Collaborators

Product Management, Engineering, Designers

Discipline

UX/UI Design, Product Strategy, Experimentation