Product Management

My last job title at Trade Me was ‘Head of Product’.

If you say you are a ‘Software Developer’ or even a ‘Development Manager’, then most people working in technology will know what that means.

But, I’m not sure I’ve ever heard a succinct definition of what makes a ‘Product Manager’. In fact, I’m not even sure I’ve heard a succinct definition of a product, in this context.

I thought it might be useful to try and describe, at least as I’ve experienced it, for those who might be interested in this sort of role.

Some long time readers (there are a few of you left, right?) and a handful of former Trade Me employees may recognise this old diagram:

Product Management Process

I used this as part of the induction of new employees into the development team at Trade Me, to try and describe the broader product management process that we were part of.

There are six links in the infinite loop, so let’s go through them.

Actually it’s two somewhat separate loops: the smaller software development loop and the larger product development loop.

The work involved in the smaller loop is pretty well understood, I think, and widely documented elsewhere, so I won’t spend much time on that here.

Let’s pick it up at the point where you are ready to deploy a new feature out into the wild…

Release

The first important thing to realise about a release it that it’s not an end point. It’s just another link in an infinite loop.

This can be tough to understand for technical people who have predominantly worked in a project environment, for example as you would commonly experience in a consulting business. In that world projects have a defined beginning and end and then everybody moves on to the next project, and the software moves into “business-as-usual” mode.

In product management there is only business-as-usual. You are never “finished”.

This has a number of consequences, not the least being the importance of pacing yourself. Managing a product is a never ending marathon, not a sprint.

The second important thing to realise about the release cycle is how you win.

It’s tempting to think that you win by doing a good job and getting everything “right”. But, remember, I just said that you will never be finished. There is no such thing as “right”.

However, there is such a thing as late. The trade off between right and late is what makes product management more art than science. The best product managers typically have a bias to roll the dice and just try stuff – “If you’re not prepared to be wrong you’ll never come up with anything original”, “if you launch and you’re not a little embarrassed then you launched too late”, etc etc.

In other words, the challenge is not to necessarily navigate your way flawlessly once around this loop, but to navigate your way around this loop as many times as you can, getting a little bit better each time.

In order to achieve that you need to make sure that the release process is baked into your tools and process. You should be able to deploy and roll-back often and easily, ideally with one click.

I’m also a fan of a bit of release theatre, so that everybody is aware when changes are deployed and can celebrate the progress that represents.

Measure

As soon as changes are released, the next challenge is to create a feedback loop.

This is where you get to listen to what people do rather than what they say. If you can start to understand how people are really using your product then it helps you to cut through debate in the next two steps of the process with facts rather than feelings. It’s how you build confidence in your theory of what users will respond to (which doesn’t always mean what users will love, by the way – often it’s a product managers job to do what is best for the system as a whole, rather than for individual users).

I think it’s really important that everybody in the team understands how the business wins. It’s the product managers job to make sure that all of the developers and testers know what the key metrics are – for example, with a big screen on the wall showing these values and trends. But, more than that, they should be able to articulate how the features they are working on will positively impact on those numbers.

Prioritise

Broadly speaking the first job of a product manager is to keep lists.

You have to be able to take inputs from lots of different and often competing sources and constantly organise these so that the important stuff bubbles to the top, while at the same time not drowning in the long tail of less important stuff.

I recommend a “Now / Next / Later” approach.

Firstly, you need to be aware of all of the things that are underway, ideally with some idea for when they should be completed so you can keep a schedule in mind.

Secondly, and most importantly, you need to have a strong focus on what’s next. You should be able to list the next top three, four or five projects from memory, so you can stay focussed on those.

There will constantly be competing suggestions, of course. In that situation, provided you can list off the current priority projects, the question is simply: “which of these existing projects gets bumped by this new suggestion?” That usually puts new ideas in their place (on the later list).

It’s really powerful if you can have consensus from the whole team on what the next priority projects are. A useful technique for getting to that is to organise a prioritisation session where everybody is asked to bring their top two or three project to the table and advocate for them, then as a group rank them in terms of bang (i.e. expected impact on key metrics) vs buck (i.e. expected cost to implement), then pick the ideas which have the best ratio. Ideally nobody leaves until everybody has agreed what those are.

Finally, you need somewhere to dump everything else. It may be that you don’t even need to write these things down – if you’re prepared to assume that the good ideas will keep coming up. Or, maybe having a long shopping list is handy to have, so that you can fill any gaps that come up with something useful. Either way, the key is to ensure that these don’t become a constant distraction or an overwhelming background fear (remember you’ll never be finished, so getting to the end of the list isn’t the goal).

Once you have priorities agreed you also need to consider the order. There are two different approaches to scheduling projects that I’ve seen used effectively:

The Riverstone Model

Think of your schedule as a riverbed, and your job is to cover it in river stones. You’ll start by placing the big stones, picking the most important big projects to go first. Then you fill in the gaps between the big stones with some medium stones, again picking the highest priority medium projects first. Finally, in all of the little gaps between the medium stones you scatter some little pebbles – you probably don’t have to pay too much attention to which of these go first, it could be a simple as first in first out, or whatever else works. Or, you can live with the gaps and leave some slack in the system, which is often not a bad thing.

This model is recommended if you have more priority projects than development capacity.

The Train Carriage Model

Think of your schedule as a series of train carriages. On a regular timetable one will leave the station. Your job is to make sure that all of the seats on the carriage are filled. When a new project is agreed you also pick a carriage to target for release and reserve a seat in that carriage. Once a carriage is full you need to pick the next one available. And, if a carriage leaves the station with empty seats then that is a missed opportunity, so you always need to be thinking ahead to make sure that doesn’t happen – if there are no big projects ready to fill the space available then put some medium or smaller projects in there.

This model is recommended if you have more development capacity than prioritised projects.

Scope & Design

This is where a product manager will probably end up investing most of their time.

Everything should start with the user experience. I recommend having designers in the team, so you can be constantly working on this, starting with wireframes and high-level designs and later moving onto more detailed mocks which demonstrate the intent of the user interface.

There is always going to be a blurry line between design and development in any product team, and it’s important that there is a good working relationship between them. I’ve seen examples where this breaks down and developers treat the designs they have been given as a broad direction rather than a detailed specification. It’s better if the designers are responsible for the design, and developers are responsible for the code, but with a lot of communication in both directions – the developers need to loop back regularly with designers to make sure that what is implemented is as intended, and designers need to be constantly talking to developers so that specifications take into account development constraints.

I recommend putting together a project team at the very beginning of the scoping stage. This should include designers, developers and operations people as well as testers and/or support team members who bring an understanding of the current business rules and likely pot holes from dealing more directly with end users.

One of the important questions for this group to consider is: what change are we expecting, and how are we going to measure that? As we discussed above this creates a feedback loop after the feature is released, where you can confirm that the work you’ve done has had the intended impact, or not, and learn from that for future scoping and design work. If you can’t clearly articulate what the intended change or benefit is, then you probably need to go back and think about the feature some more before you start designing the user experience or cutting code.

As scope and design bleeds into development and testing the product manager will hand over to a development manager to make sure that the build runs smoothly, and will likely become a “customer” in that process. In a smaller team the product manager and development manager are often the same person, so it’s important for them to realise the two competing roles they fill in that situation.

Everything & Nothing

To do all of these things well demands a varied and interesting set of skills from a product manager, including a lot of soft skills that are not always easy to find in technical people – you need to be able to put yourself in the shoes of an end user and have empathy for who they are, how they think about your product and how they are likely to respond in different situations; you need to be able to think analytically – you’ll probably spend more time looking at spreadsheets than looking at code; but, on the other hand, you need an aesthetic judgement, a sense of style and an understanding of design trade-offs; you need to be able to write authentically and succinctly (writing is a muscle and one of the main reasons why I maintain this blog); you need to think like a marketer, because making things that people will love is hard; you need to be able to work with a variety of different people, both technical and non-technical; and, last but not least, you need to understand that you can achieve a lot in this sort of role provided you don’t need to take all of the credit for it.

In my experience the best product managers are heads-up developers who can code but don’t want to anymore.

If that sounds like you then think about how you can get involved in some of the areas that I’ve described above within your team, so you can build your skills and over time move from a software development role into taking responsibility for product management.

I’m interested in your thoughts, especially from those who have been a product manager or worked with one. What are the other jobs of a product manager that I’m missing?

Of course, the problem with an infinite loop like this is that there is no obvious beginning. This all assumes that there is an existing product that needs to be managed and developed. It doesn’t talk at all to where the idea for the product comes from in the first place, so maybe that’s where we’ll turn out attention next…

Trade Me Browser Stats

I’m pleased to see Trade Me have started posting their browser stats again, something I used to do here, way back in the day.

Here is the latest update, for May 2010: Browsers and Operating Systems

http://images.trademe.co.nz/tm/announcements/full/132922040.jpg

When people talk about Open Data they are nearly always referring to government data.  But, I think there are also lots of examples like this, where private companies have data which has a public good, and which they can open up at no material cost to themselves.

Trade Me is such a popular site that their audience can pretty much be used as a proxy for the internet in New Zealand, so this gives developers working on smaller or less popular sites a good idea of the sort of browsers they should be targeting.

Remember, if the equivalent numbers for your site are different from these there are two possible explanations:

  1. Your audience is a subset of the population which has a browser bias (e.g. if you attract more technical people you’ll probably tend to see a higher proportion of newer browsers and also some lesser known browsers that are not widely used in the mainstream)
  2. Your site makes it difficult for people with older browsers to use your site, so they choose not to.

Just about everybody assumes #1, when #2 is often more likely.

Remember that the 5% of Trade Me visitors using IE6 is still 31,500 unique visitors per day, or nearly one Westpac Stadium full.  Are you happy to turn all of those people away with a message telling them to upgrade their “browser”, what ever that means to them?

From The Archives: Usability

Enhanced metafiles, 16th April 2007

“When are those of us who build these tools going to start putting ourselves in the shoes of people that don’t speak C#?”

Visualise your audience, 9th April 2009

“I love the buzz of a big crowd.  It’s exciting to soak up the atmosphere created when lots of people are all in the same space at the same time.”

Dragging a big sack, 28th January 2008

“In the early days of any new product it’s really important that you choose your customers carefully.”

We’re not normal, 12th February 2008

“We’re not normal, but if we put our mind to it we can empathise, surely?”

Authenticity, 3rd March 2008

“Why do small businesses so often pretend to be bigger than they are?”

Don’t click here, 14th March 2007

“Even the most novice web users quickly learns that links are clickable, you don’t need to tell them where to click – unless, of course, you’re obfuscating your links by not underlining them and/or making them a colour that isn’t blue.”

Consumer Unfriendly, 7th September 2007

“Sometimes people don’t know what is good for them.”

Make it work, then make it look good, 29th May 2007

“What does it profit someone to have a site which looks like a million dollars but which doesn’t actually work?”

Thoughts about “users”, 14th June 2007

“There are only two industries that refer to their customers as users: high tech and illegal drugs.”

Getting to the third user, 29th January 2008

“The scene: some developers are observing their first usability test on some software they have built…”

Form Fail, 28th May 2009

“The quality of form design, in general, on the web continues to be a huge source of frustration for users and an embarassment for all of us who are involved in designing and building sites.”

Form Fail

The quality of form design, in general, on the web continues to be a huge source of frustration for users and an embarassment for all of us who are involved in designing and building sites.

Here is a payment page that I needed to traverse recently…

Firstly, as originally served:

Payment Page

And, here once I’d disabled CSS:

Payment Page - No CSS

It took me a few attempts to even work out what was happening. Initially I just assumed that the page had somehow not completely loaded.

Then I highlighted the area where the labels should be and realised that they were actually white text on a white background!

I’m guessing that in this case the design is incompetent rather than malicious.

Perhaps it works fine in Internet Explorer? Who knows.

I was determined, so worked it out.  But, I expect I’m an exception.

If they measured their conversion rates on this page, surely they would find that a significant number of people simply give up, unable to work it out.

So, you have to assume that they probably never bothered to measure.

Of course, none of the intelligent and experienced people who take the time to read this blog would ever make a mistake as outrageous as that. Would you?

You might be surprised …

Let me try and debunk some axioms of registration page design, and in the process highlight a simple mistake that I would guess many of you have made without even realising it (I can only say that with confidence because I’ve made it myself many times).

To start, a question… what is the difference between these two forms?

Form Fail - With Tick Box

Form A: with tick-box

Form Fail - Sans Tick Box

Form B: without tick-box

Which is easier to use?  Or, to ask the same question in a more measurable way: which would have a higher conversion rate?

As we’ll see there is evidence to suggest that Form B is a much better design.

So, why are registration pages like Form A nearly ubiquitous?

Here are some possible reasons I can think of (if there are others, please let me know)…

Legal reasons

In case it’s not obvious, I’m not a lawyer!

If there are any lawyers reading I’d be interested to get an informed opinion on this.

My guess is that it doesn’t make any material difference.

Just think about how this would sound in court: “But, your honour, they ticked the box when they registered!”. Yeah, right.

And, even so, at what cost this additional legal protection? How many fewer registrations can you afford before it’s worth taking the risk?

Because everybody else does

This is probably getting closer to the major reason for the ubiquity.

And, there is something to be said for consistency. But, surely only when it doesn’t hurt!  Lemmings!

Just because everybody else does something, that doesn’t excuse us from measuring and understanding the impact of our design decisions.

How hard is it to tick a box?

Now we’re getting to the interesting bit…

I was recently talking to Nigel about some of the work he did on improving the StarNow registration page. This is the first example I’d come across where somebody had actually taken the time to measure specifically where people were failing on this sort of registration form.

It turned out, in their case, that 27.9% of people didn’t tick the box the first time, and as a result got a validation error. What’s more this only compounded other problems. When the tick box validation error was displayed the password fields were reset (as is common, unless you explicitly set these values when the page is returned, which is frowned upon by security types). So, of course, many people would read the error message, scroll down and tick the box, only to find that the form still didn’t submit successfully as they also needed to re-enter the password.

Does this sound familiar? I know I’ve been through this exact dance myself many times, and I consider myself a pretty savvy web user.

Overall they found that over 34% of people who attempted to complete the page (i.e. pushed the submit button at least once) abandoned the site without successfully completing their registration process.

By making simple changes they decreased this to 17%. That’s a staggering 26% improvement!

So, some suggestions:

  1. Ditch the tick-box from your registration page.
  2. If your lawyer complains ask them to justify the cost, which you will now be able to quantify.
  3. Re-populate password fields on post-back. Or, if you’re not comfortable doing that for security reasons, at least highlight the fields that people will need to re-enter to avoid further error messages.
  4. Most importantly: take the time to measure and understand exactly where people are failing on your site, then do something about it!

Visualise your audience

I love the buzz of a big crowd.  It’s exciting to soak up the atmosphere created when lots of people are all in the same space at the same time.

If you’re lucky enough to be the band, or sports team or speaker who is the focus of the crowds attention, then that is quite powerful.

(Or terrifying, I suppose, depending on their mood!)

If you design or develop software it’s much harder to get feedback like this.

But, it’s still useful to think of crowds to help you visualise the audience of people who are using what you’re building.

This was the idea behind the photo I use in the header this site.

Another from the same shoot is below, for those of you reading via my feed:

Idealog Stadium

These come from a 2006 article about Trade Me in Idealog, and were taken at the Westpac Stadium in Wellington on a wet and wild day (hence the slightly damp windswept look and the Icebreaker jacket).

The stadium seats about 35,000 people when it’s full.  At the time there were about that many people online at once every evening on Trade Me.

(As I type there are over 80,000 online, showing how much it has continued to grow since then!)

When you think of that many people all in one space together, and the noise and activity they generate, it changes the way you think about the people using your site.

One of the things that software developers do all the time is dismiss a percentage of their audience as unimportant for one reason or another, without thinking of them as a distinct group of people.

If you don’t think you need to worry about people stuck using IE6 (for whatever reason) or a dial-up connection, or people who struggle to read small fonts, or people who are colour blind (or completely blind), or people who don’t know how to use a command line, or people who are still nervous (paranoid or otherwise) about entering their credit card details into a website, or people who want help from a real person, or people who use Firefox on a Mac … because on a percentage basis they are not that many … calculate just how big that group is (take your unique visitor count and multiply by the percentage, however small), and then imagine standing up in front of that group and telling them to their face that they don’t matter to you.

If you don’t think it’s a big deal for your site to be broken or off line while you make changes … think of all of the people who happen to be visiting at that point and imagine what it would feel like to have them all in the room with you while you flick the switch.  No matter how small the number it would probably feel like a lot of people.  And, you might be motivated to get the site back up more quickly if they were all standing behind you impatiently looking over your shoulder.

You can use the same technique to help put other numbers in perspective.

It’s amazing the difference it makes when you start thinking of your metrics as real people.

For example…

  • 2 people = your mum and dad!
  • 5 people = a car full
  • 15 people = a rugby team
  • 30 people = a school class
  • 100 people = a bus full
  • 120 people = a parliament full of MPs (actually 122, to be precise)
  • 380 people = all of the passengers on an Air NZ 747-400
  • 550 people = the audience at Webstock earlier this year
  • 1,500 people = capacity of the Aotea Centre in Auckland
  • 2,500 people = all of the students at Auckland Grammar school
  • 6,000 people = capacity of the TSB Arena in Wellington
  • 10,000 people = population of Gore :-)
  • 12,000 people = capacity of the Vector Arena in Auckland
  • 18,000 people = all of the students at Otago University
  • 20,000 people = population of Levin
  • 25,000 people = the crowd at Augusta National each day this week
  • 35,000 people = capacity of Westpac Stadium in Wellington
  • 60,000 people = capacity of Eden Park in Auckland, post-renovations
  • 100,000 people = capacity of MCG in Australia (also, incidentally, approx. the number of people who voted for NZ First at the last election)
  • 250,000 people = capacity of St Peter’s Square in Rome
  • 475,000 people = population of greater Wellington region
  • Etc, etc.

Help me out with some more examples…