Feature Audit, Underdoing the Competition, and Hans Rosling on UI Breakfast Podcast
The following is a transcript of an interview with the Jane Portman on the UI Breakfast Podcast.
Jane: Hello ladies and gentlemen, and welcome to the UI Breakfast podcast. I’m your host Jane Portman, and today our guest is Brian Rhea, product strategist and founder of Feature Audit. We’re going to talk about SaaS feature audits today. Hi Brian!
Brian: How’s it going, Jane?
Jane: Going great. Can you tell us a bit about yourself, what kind of work you do, and how you ended up having a software product related to features.
Brian: You bet. Right now, I’m doing product strategy and consulting for startups. I’m based in Boulder, Colorado, but I have clients in New York, California and Texas. Which is just a whole lot of fun. It gives me a chance to hop around from time to time, which I enjoy. And I love working with folks from all sorts of different areas.
But the way I got into product, was about 6 or 7 years ago. I was living in Dallas and freelancing, doing web development. My wife and I just love Colorado, and have always just loved Colorado. We found ourselves vacationing out here often, as many people do, and on one of our drives back to Dallas, we just looked at each other and said, “You know? People live in Colorado. We could try that!”
So, I built this goofy little site at the time, called “hirebrianrhea.com,” and it was my best attempt at making some jokes about how hot the temperature was in Dallas, and how much we love Colorado, and also-- here is my portfolio! Oh, I also build websites-- is there somebody I could help out?
Then I reached out to Brad Feld, who is like the Godfather of startups in the Boulder area; many people are familiar with him and he is very influential. On a total lark, I emailed him and said, “Hey Brad, if there's anyone who could make this happen, it's you. If you tweeted ‘Someone should hirebrianrhea.com’, I am sure it would happen.” I sent it off and just didn’t expect anything would come of it.
I kept grinding at sending cover letters to startups in Boulder and Denver. And then one night -- literally in the middle of the night, I was feeding our newborn while my wife got some sleep -- I looked at my iPad, and I had 100 notifications on my email. I thought, “What in the world is going on?!,” and scrolled to the bottom, and the first one was a reply back from Brad Feld that said, “Tweeted. Good luck, buddy!” or something to that effect.
The next few weeks became a whirlwind of getting to talk to a bunch of different startups in the area, and I eventually joined an incredible team at a company called Mocavo. They were a Techstars company and backed by The Foundry Group, had a brilliant leadership team, and just a great team all-around. I joined up with them, and it was through that process of working at Mocavo that I went from Front-End Developer to UX Lead and eventually into an Executive Product role. And that is where I really fell in love with what I do now, which is product strategy and Jobs to Be Done, identifying priorities, and how to design an effective roadmap that a team can follow.
Jane: Wonderful. What was your value proposition back then, when you got that score of clients, because it is typically a beginner mistake to hope that someone famous is going to tweet about you, and the world will turn upside down. I’m surprised that worked for you. So what did you do “right”?
Brian: [laughing] Yeah, totally. That’s a great question. I’ve actually written about this on Brad’s blog. He asked me to do this thing. If people would google “hirebrianrhea one year later,” I go into a little more depth on what you’re asking: why did this work? I’m under no illusion that you can definitely mastermind that sort of thing. There was a fair amount of luck and timing that went into it.
But at the time, a couple of things worked. One was that the site that I built told a story. I got some help and feedback on it from my wife, who is really good at editing. I tried to make it much more like an interesting story, like “Here is a family” and put some jokes in there about the temperature in Texas vs. Colorado and let that guide you as you scrolled through. A Story unfolded; it wasn’t just a typical “here is all my stuff”.
Of course I included my portfolio and skills. And I also some included some personality stuff, like, “Here is my reading list. What is on the office bookshelf? Are we a fit for each other?” I was trying to hit on multiple things. Warranted or not, there was enough technical ability demonstrated in the site itself, and some design integrated as well, that enough of the companies that were able to see, “This guy is a good design and development unicorn, and we have to jump on it.” I am much more skeptical of my development skills, than those companies were, thankfully! But there was definitely a positioning piece there that worked.
As far as the email to Brad Feld, I was very direct. I said, “Look, you’re a busy guy. This is what I’m asking you to do. Here is how you can help me. Will you help me?” Important people don’t have time to read a big, long thing. If you’re asking someone of influence for help, be very direct and ask them for help. They either want to help and have time, or want to help and don’t have time. Either way, you need to respect their time as much as possible. I think those are the elements of it that worked.
Jane: Totally true. As a side note, those attempts to mask your outreach attempts never work. They look miserable. So going in short, brief and direct is the best strategy.
Brian: That’s right.
Jane: So back to your product and strategy work. Who is your typical client, and what do you do for them?
Brian: My typical client is a startup who is still searching for product-market fit. I’ve had a number of clients who have a good idea of who their customer is, and they have a lot of clarity--well, that is a bit of an oxymoron. [laughing] They know if they had all the money in the world, and could fast-forward two years, they know exactly what their product would do. But, in the present day, with limited time, money, energy and no customers, they ask, “Ok Brian, what are the first four things we should build? Or, what is the one primary customer journey that we need to create?” That is my typical customer, who probably doesn’t have any users yet, or if they do, they’re not hockey-stick product-market fit raising tons and tons of money. They’ve got either a couple of interested people that they’re dying to put the first version of something in front of to start learning, or they’ve got that first thing out, but it isn’t exactly hitting yet.
What I can do is come in and help them understand their customers a bit better, give some additional perspective there, and then help them figure out how to prioritize. I help them prioritize among the things they’re imagining they’re going to do either because they’ve done some research themselves, or because they’re their own customer - they have domain expertise and know the things that it needs to do-, or because they have a handful of customers who are asking for things. The process of figuring out how to prioritize, to know what to say “yes” to and what to say “not now, not yet” to is quite hard. That is what I often end up helping folks with.
Jane: What is your “secret sauce”? Do you have any special process to help those struggling startups?
Brian: Yeah, I think the “secret sauce” is, well, consultants can get a bad rap, but I really do think there is immense value in pulling someone else in with an additional perspective. I always start with “No.” And that’s being able to help the entrepreneur say to themselves, “Yes, of course it would be better if it included *This’*, but we’re going to say, ‘No,’ and we’re going to force that feature or that additional thing you’re wanting to do to make a really, really strong case for itself.”
We can't just respond emotionally right now and say, “Let’s do this,” and figure out how to slip it in and make it work within the budget. No. We’re going to say, “NONE of these; what are the FEW things that make a very, very, very strong case for themselves, either because they keep coming up or every single customer is asking for it, telling us it is extremely important and they’re very unsatisfied with their current solution.” And so, I think, offering that push and that perspective is most helpful.
Jane: That’s wonderful. And that’s also a perfect stepping stone to our main topic, which is Feature Audit. That’s also the name of your software product. It’s obviously supposed to help software companies prioritize and figure out what exactly brings the most value for their users. Tell us more about that.
Brian: Feature Audit is a tiny little analytic tool that helps you make better product decisions in seconds. The way that it helps you do that is that it plots each of your features on a chart that will show you how many people are using a feature, and how often. So, it answers questions like: Are all of your users using something all of the time? Are all of your users using something none of the time? Are very few of your people using a feature all of the time, or very little of the time? And everything in between.
If any of the listeners have ever seen Hans Rosling’s TED talk where he’s got those beautiful bubble charts moving across time, that is what Feature Audit is modeled after, because it allows you to see the movement of a particular feature on those quadrants over time. If you go to featureaudit.com, you can click a little green button that says “See it in Action,” and you can kinda get a sense of what it does.
The original idea - the germ for where this came from - happened while I was working at Mocavo, the startup I was talking about earlier. There would be a lot of times in conversations where there would be an argument over a bug that came through. What sort of priority should that particular bug get? Even at the time, I was also coming from a position of, “We’re not going to prioritize it until it makes an argument for itself. We’ve set our priorities, we’re going to work on those things. We’re not going to do this just because a few people are trying to set off a fire alarm.”
Jane: One of my questions that I had prepared for you was, “How do you recreate similar functionality without your tool?” Because of course it is great to have a tool that solves everything, but you just mentioned that it is painful digging through inquiries and so forth.
Underdoing the Competition
Brian: Yeah, so that is my hypothesis, Jane. Analytics tools have been around for long enough that, in a lot of ways, they have overmatured. So they are very feature rich, which can be great. But, as we know, and especially, gosh, YOU -- this is your whole thing with UI, right? -- that as things become feature-rich, they become harder to use.
My hypothesis here is, “Ok, I want to dramatically under-do Google Analytics on purpose. I’m going to under-do Analytics on purpose so that I can get to a very specific answer in a few seconds.
And then -- this is the part of the product I’m still working on and developing -- allow my customer to reach out to those users in a very simple and seamless way.
So if you identify that very few of the people are using a particular feature all of the time, it would be nice to get 4 or 5 of them on the phone and get some qualitative feedback on: What is going on here? How did you adopt this feature, and what can we learn from that, so that we can get more people to adopt the feature? And then we could move it from a few of the people all of the time, to most of the people all of the time.
Jane: That’s wonderful. I love the way you put it: Dramatically under-do. It is exactly the same strategy we followed with UserList, which was to under-do the email sending part of Intercom and do it well. That approach resonates with me so much; sounds amazing.
Brian: Real quick, I’m glad you mentioned Intercom there, because part of the origin story of Feature Audit that I didn’t get a chance to get to, is that, as I was struggling with those problems, asking, “How important is this? I wish there was an easier way for us to get to it,” Des Traynor wrote an article where he describes performing a “Feature Audit.” That is where the name comes from. I read that and thought, ‘Des! This is genius. Who is currently doing this?” And I still have yet to see a simple implementation of it, and so that’s what I’m going for. [laughing] So we’ll see, Jane, how it turns out!
Jane: Isn’t that amazing, that Des Traynor has so many amazing materials and talks about making the product simple and usable. However, Intercom itself is beautiful but so complex. Oh my goodness. How can you do both?
Brian: Yes. I think there answer there, Jane, is, can you imagine how complicated Intercom would be if Des Traynor were NOT on the leadership team there? [laughing] But you’re right. And I’m sure, five years ago, Intercom was easier to use than it is today. It’s just what products and software- software especially- want to do. It WANTS to get complex over time because you have to make these, what Clayton Christensen calls, “sustaining innovations.” You have to continually incrementally improve the thing, that add a little bit of value, but not as much as everything that has come before it. And every new thing you add reduces usability. It’s a tough balance to strike, right?
Jane: Absolutely. So as a consultant, and product leader with experience with multiple products, what do you do with the results of that feature audit? Do you have any recommendations whether you can go as far as removing features from a product to make it better?
Brian: Yes. Being able to offer some visual proof and evidence that you should delete a thing is super helpful. Both at my startup and with multiple clients I’ve had as a consultant, there is such a resistance to removing a thing that very few people are using.
Most of my clients are small startups, but I have one client that has product-market fit, has been dominating their market for a decade, so they have some real big fish that are their clients. The thing was, there was this feature that we knew wasn’t used often and we knew it was only used by a few people. But it was used by some extremely important people. Certainly, that is a separate conversation about the business case.
More often than not, the resistance to removing something is emotional and reactive. My hope is to be able to provide statistical evidence and support for why we should do it-- let’s remove it, no one uses it anyway, it will clean up our UI, it will improve the user experience because less people will be confused. The other part of it, like I alluded to before, is that if you see something that has high-frequency of usage, but low adoption -- so a few of the people are using something most or all of the time -- you can ask, “Why is that? How can we get more people to adopt it, and pull this piece of the product into their daily usage?”
One of the things that definitely helps in reducing churn is to increase habits. The more often people feel like a piece of your tool is absolutely essential to their daily work flow, the harder it is going to be for them to churn out, or adopt a new thing. Habit is a strong force.
Less is More
Jane: As you mentioned, one approach would be to remove something that is not used very much, and another would be to make your strong points, stronger- to work on them and improve your UX, adoption, etc. Do you recommend grandfathering in, or maybe turning on and off certain features for certain users? That is super handy. So maybe you could switch it off for everyone, and only keep it on for those important people that need it.
Brian: Ok, if there is a really, really strong “business case” for that, you could make that argument. The trade-off you’d be starting from is that you’re guaranteeing ongoing engineering maintenance for a sliver of customers. Now again, if the business case backs that up and funds that amount of time for X amount of engineers, then so be it. That is a C-Suite decision, and fine. But I would not start with that as a recommendation. It is not going to be as easy as it sounds, you know? It is going to hamstring you on future updates for sure, and so on.
Jane: The question is, if you have a number of loyal users who have been with you from the very beginning, and then you decide to focus your app and cut-off certain functionality, how do you manage those relationships with them?
Brian: Yeah...that is a very hard question to answer. And, it’s an unsatisfying answer. But I think that it depends. One example of this that, from the outside at least, it seemed like they did it very well, was when Basecamp froze a version. I forget what they've named them now, but at one point in the past, they froze a codebase and said, “OK, people, if this is where you want to stay, then you can stay here forever. This is where your data stays, we’re not making any additional updates, here you go. But we are moving on to this new version, and it is going to do some things differently, and if you’d like to come across, then here is how you do it.”
I’d be really curious to read a case study on how that worked out, but in general, I feel like the specifics of being able to give a blanket answer to, “Should you preserve and continue to maintain features that you know are not valuable for most of your audience because you have a few people who use them?” The default answer would be “No,” not if you think your future growth is most likely to come by leaving those things behind and moving on, but in the short term, figuring out how you manage that; that has to be so contextual.
Jane: Yeah, let’s not talk about Wordpress enforcing their update onto everyone, and breaking half a million sites across the planet.
Brian: Ooooooo, Yeah…. [both laughing]
Jane: Let’s not talk about that! Let’s talk a little bit more about the technical side of your product, Feature Audit. I feel like UserList and Feature Audit are somewhat similar, because we kind of tap into the user behavior and try to record it. We do something, and analyze it and we send an email based off that, but the tracking part is pretty hard to implement and onboard users about.
Brian: [chuckling] Yes, oh me, too.
Jane: We’re struggling with that, because we require manual integration. Tell me, what are your decisions in that regard?
For sure, one thing that it is currently doing behind the scenes is automatically tracking each and every click. On the technical side, Feature Audit knows if you’re clicking on an anchor tag, an image tag, a piece of text, or just somewhere random in the document. So I’m recording all of those things.
Jane: So, what is troublesome about that approach? Is it the onboarding part or is it the technical part on your side?
Brian: It’s totally just getting it installed, for them - for the customer.
Jane: Even a snippet? Not to mention the full integration?
Brian: Yeah, what definitely increased some activation has been making the documents very, very specific with how to integrate with Google Tag Manager. Most people already have GTM set-up, and so I’m highlighting, “Look, you don’t even have to install anything new, you actually just have to log into Tag Manager and drop this in there, and Boom!, away you go.”
In terms of me being able to increase adoption on this thing, the next step is: “And also, you don’t have to worry about reporting or posting any sorts of events. Feature Audit is going to record them all, and then you’ll be able to just go in there and select ‘These click-events relate to this feature; these click-events relate to this feature, and so on’ and I will handle it all from there.”
Jane: I think that is similar to the approach that Heap takes. They record everything, and then they decide retrospectively what you want to do.
Brian: Yep, Heap and Pendo. Again, that just plays into my hypothesis. Those are great tools, but very heavy.
Jane: Another side and hardship is to decide what is actually called a feature, when you conduct that feature audit? Let's’ say you’re a consultant and you want to come into a SaaS business and use your tool to write a report. How do you figure out what goes into that grid? What makes a feature, and how many should you track, and how do you organize them?
Brian: One of the events that Feature Audit tracks that I didn’t mention is “page load,” so in terms of grouping and mapping, where you often start is at the page-load URL level. Like, is someone going into Dashboards, into Events, into Community? Let’s start there and see which of these areas of the site or the application are most popular. Ok, we can begin there. And now we can group within that category or within that high-level feature, what “featurettes” or what tiny interactions, people are using.
One of the areas I recommend starting with is, “Ok, with our current UI and our current design, what are we saying are the most important pieces? What have our assumptions said, internally or with some degree of customer-research, that led to these design decisions. Let’s measure those. Are people actually using them?” And you can start to get a better sense through the data of how close to correct you were, and modifications you can make based on some support from qualitative customer interviews based on that data.
Jane: How did you arrive at this interesting representation of these floating bubbles? You mentioned the TED talk, and we will link to it in the show notes, but generally speaking what made you feel like this is going to be a legit and useful represernation? I think the deliverable is the video, and that is definitely not something that goes into PDF. It can go into PDF, but it is not very classic-- you can’t print it.
Brian: It came to me in reading the Des Traynor post that I had mentioned, he has these different quadrants, and he is basically making the argument that as product owners, we probably want all of feature bubbles to be moving up and to the right. And I thought, “Ok, but you need to be able to see that and summarize it visually and over time.”
It wouldn’t be as instructive if you could only ever see today’s data. You would want to be able to look back over time and say, “We identified four months ago that this feature is high value to the very few people who use it. We decided we wanted to make it more valuable to more people. How have we done that? Let’s isolate that one specific feature, hit play, and watch it move.”
Jane: I can imagine that would be a moment of joy or a moment of tragedy, depending on how it moves actually.
Brian: [laughing] That’s totally true.
Product Vision & Strategy
Jane: With this product, what is your big vision for it? I know you set out to build an under-designed version of an analytic tool, but what is your grand vision for that?
Brian: My grand vision at the moment is for it to be successful enough that I can make the migration from full-time consulting over to running a software product again. I don’t want to end up raising money on it if I can bookstrap it or just get a little bit of angel investing, then great.
My total grand vision for Feature Audit is that it would never need to be more than 15 or 20 employees working remotely, or maybe a handful of us in downtown Lafayette, Colorado. It doesn’t need to be this massive juggernaut in order for me to feel like it is successful. I want us to have very high revenue per employee-- that’s the metric I’m most fascinated with-- not total MRR or anything like that. MRR per employee is most interesting.
Jane: That is perfect vision and aligns with my line of thought very much. Making a simple product and getting it into the hands of thousands of people is a success in itself. You don’t have to overcomplicate the product to achieve that. So, as we’re wrapping up this episode, if you were to give advice to someone who decides to conduct a Feature Audit in their product, what would be a few simple steps that they could follow, with or without your product
Brian: Without the tool, it can definitely be done. I would say at least begin there. Chances are good that you have Google Analytics already installed. Chances may not be as good that you are tracking specific events, and so start there. Start at the high level we spoke about earlier.
You can use a tree map or something to that affect to identify the major areas or features of your application that are getting most of your resources -- either engineering, design, marketing or otherwise. Drill into the high priority elements within each of those, and be sure that you are tracking them. Just start there. Collect a couple weeks worth of data, and observe what it tells you. Using that data, and without signing up for Feature Audit or using it, you can replicate what it does and what it shows you. Begin there.
Then, if you see a lot of value you in that and want someone to do it for you on your behalf, then check out: FeatureAudit.com. While we’re at it, you can use the coupon code: UIBREAKFAST to get 50% off your first 3 months.
Jane: That’s great! Thanks for that discount. If people want to follow your writing, your thoughts online, what would be the best place to do that.
Brian: You can find me at brianrhea.com or on Twitter at brhea. I’m not on Twitter as much these days, but you can still find me there and reach out and talk. We talked a little bit about product strategy on the podcast and didn’t get too much into Jobs to Be Done, but that is a framework that I find a ton of value in.
One of the things you’ll see on brianrhea.com is this Jobs to Be Done resource library. It’s basically my google drive folder where I’ve just put all of my resources that I use: Interview Timeline Sheet, Outcome Statements template, Forces of Progress template, Swipe file for customer interview emails that I send, workshop agendas, all sorts of things that I use, I throw into this Google Drive folder. If anyone wants access to that, there is a place where it asks for your email address, and you can get access to that.
Jane: Thank you so much for sharing that. I genuinely enjoyed our conversation today. It was awesome. I learned a ton and I hope our listeners did, too.
Brian: It was great! Thanks for having me.