The Three Skills of a “Google PM”

One of the things I’ve picked up on over the course my career is just how much respect there is for Google PMs. I don’t disagree – some of my favorite PMs have worked there. On the other hand, many of the people I really respect haven’t. So what’s the difference between a PM who worked at Google, and one who hasn’t?

I’ve been asking around. Last time I was in San Francisco, I had the chance to ask Ken Norton and he clarified it for me in a way that I wanted to share. This explanation aligns with what I’ve heard from other friends (internal and external perception) – but it was by far the best framing I’ve heard.

(0) Google helped standardize “Product Management.”

I’ve written about this before – but the origins of Product Management are nebulous. It stems from Brand Management and Program Management. Google was one of the companies in the valley who had an established discipline first. This means they set some of the standards for what we want the discipline to be. This isn’t really a difference now that it’s spread, but it is important context.

(1) Technical Respect

This has been talked about frequently, but Google is an Engineering-led culture. This means you can’t get things done if you don’t have the respect of Engineers. Having had a Google PM job for a significant chunk of time means you did figure out how to achieve this.

If you are somewhere that engineers “have” to do what you say, you haven’t achieved this.

(2) Scope

Google does not scope projects to PM-experience. Even APMs are thrown into large areas. This is different from many companies who don’t want the “risk” of having a new PM own something large. This is also different from political companies where there isn’t enough work, so there’s political infighting to “own” things.

This resonates strongly with me – at Microsoft my first project was an app where you couldn’t do anything. You just clicked a link, and it showed you PowerPoint slides.*

(3) Accountability

When you have the privilege of owning a large area, you also have to be accountable for it. That means it has to either work – or that you dealt with the consequences when it didn’t.

Some organizations will say everyone is at fault when something goes wrong. At Google it’s more likely to be specifically the PM.

I’ve seen this happen in small teams where “we’re all in this together” and people don’t want to offend each other.

I think these are three great areas to evaluate a PM on, but the only way to learn them isn’t at Google. (And while you have a great shot of learning them at Google, I do know those who haven’t).

If you’re a PM and you haven’t mastered these three skills, it’s well worth figuring out how to change your role to include them.
* To be fair, the scope of this project was originally larger. Even so, my next project had far more PMs than were necessary. Too many PMs means a lot of time in meetings with other PMs, not enough time getting things done.

You are not your culture

Over the weekend, Jodi Kantor and David Streitfelt published a piece on the culture at Amazon.

I’ve seen responses fall in three primary categories:

1) The necessity / truth in this reporting, and the need to create better conditions.
2) The idea that this isn’t true (anymore) or the story is sensationalist.
3) The idea that even if this is true, an extreme environment is necessary to do amazing work.

One of the key topics in the piece is about how forceful and direct the feedback culture is. Doing great work does require constructive feedback. Ben covered this nicely in Stratechery (paid, and highly recommended).

He particularly drew from this New Yorker article, and in particular this passage:

Jobs replied, “Why would you be vague?,” arguing that ambiguity was a form of selfishness: “You don’t care about how they feel! You’re being vain, you want them to like you.” Ive was furious, but came to agree. “It’s really demeaning to think that, in this deep desire to be liked, you’ve compromised giving clear, unambiguous feedback,” he said.

From what I can tell (article aside), Amazon does have a culture that encourages constructive and clear feedback on work. This is not surprising. That’s been true everywhere I worked. I receive and give feedback for Blade Travel every day. People who work in technology often defend the right to give clear and constructive feedback on work – preferably absent of politics.

So if we are so defensive of our right to give blunt feedback on products – why can’t we do the same with culture?

“Culture” is where we still struggle with the distinction between “critiquing my work” and “critiquing me.”

When I started my career as a PM it was hard to separate “critiquing my spec” and “critiquing me.” (You can easily substitute code/design for “spec” here). It felt like every constructive comment was something I should have seen in advance. I cried more than once after spec reviews. Like most, with more experience, I moved away from that.

Cultural critique is a more abstract version of the same concept. We have a harder time with it. It’s because we talk so much about how founders set the culture for the company. Yes, founders do help create the culture. Founders are not the culture. Jeff Bezos is not Amazon’s culture.

If the NYT published a piece talking about how the Echo was “the most useless product ever” we would not have reacted as vehemently. We would have used it as a jumping off point for the conversation about the product and how it will improve, and where it will go. It would not have been right/wrong or good/bad. It would not have felt so personal.

To build better companies, we need to get there. We need to accept that Amazon’s culture is something that can be critiqued – the same way a product can be critiqued.

It is not personal, even if it feels that way right now.

Riding the Subway

When I lived in NYC, I started suddenly seeing Tumblrs about people (mostly men) taking up too much space on the subway. Here’s one as an example.

Many of these cases show trains with plenty of space. While it’s still an expression of privilege, it has fewer practical implications if there are plenty of open seats on the train.

Unfortunately, in Boston, I frequently see this when the train is packed (even with people still waiting on the platform). In those cases an empty seat is one less person who gets on the train. Or it’s one more person crammed into the packed standing area, when there’s a seat available. For some reason, people here are too polite to sit down and force the person to make room. I assume some of the men taking up too much space don’t realize they are doing so.

I often see women look at the seat, evaluate if they think the man will move, and then walk and stand at the other side of the car – leaving that one open. Once one person has deemed it “un-sittable” no one else seems to either.

So, I’ve been sitting in all of those seats. Even when I don’t really want to sit. I don’t say anything – I just reclaim the space to make it obvious the train is full. I’ll happily offer the seat to someone elderly or pregnant after I’ve reclaimed it.

Today was one of those days. I was on a crowded train. There was a man taking up about 1.5 seats worth of space. I watched a woman walk up, look like she was going to sit down, and when he didn’t adjust to make space – walk to the back of the car to get another seat.

So I did my usual. I sat down.

The man didn’t move, or make any adjustments to provide me with more space. Instead, he left his elbow on my thigh for the entire ride. The subway is crowded – often your arm will touch another arm, your leg another leg. This wasn’t that. He left his elbow on my thigh.

I couldn’t bring myself to say anything – or figure out what to say – so I just sat there for four more stops until I got to work. This is the first time my strategy has backfired, and I’ve been doing it for about a year. I’m going to keep doing it, but I sure hope that doesn’t happen again.

“But you don’t look like…”

When I announced I was going to HBS last Spring, one of the common reactions was, “wow, the old boy’s club?” or “so.. you’re trying to find a place more sexist than tech?” I think this was particularly prevalent because of an article that came out at the time.

Transitioning to HBS was anything but easy. I was an introvert in a very social environment, and I was technical in a place where many things are high level. Many things were extremely hard for me. But at least one thing was finally easy, and that was my gender. The HBS classroom is 42% women.

Since I’m sure someone will ask: HBS has things to fix on this front, but it feels light-years ahead of tech. There was one thing in particular that stood out.

When I talk with people in technology, it’s frequently assumed that I don’t belong. People ask me “are you a recruiter?” with alarming frequency. People will ask me if I’m at a technical event because my boyfriend is. It’s not uncommon for someone to say “but you don’t look like an engineer” or give me the look that says exactly the same thing. I haven’t been mistaken for a janitor, but nearly half of black and Latina women in science have.

It was a delight that this didn’t ever happen at HBS. At HBS, a common first question is “what section are you in?” – assuming you are a student. No one ever asks me “are you a partner?”

People don’t assume what job you did before, either. You cannot, and do not, look “like a consultant” or “like a banker” or “like an engineer.” Instead, they’ll ask “what do you do?”

This was a delight, and it should not have been. It’s basic courtesy to not assume you know someone’s job based on how they look. Just as someone does not look like a banker or a consultant, they also do not look like an engineer or a recruiter.

There is nothing wrong with being a recruiter. There is something very wrong with assuming someone is a recruiter just because they’re 5’4” and have long brown hair. If you stop and think for even one minute, it’s absurd to presume to know someone’s job based on their appearance.

People try to play this down and say “it’s a compliment!”
It seems like not that big of a deal.

At points in my writings about technology, I have worried about being “too sensitive.” At points, I have wondered if micro-aggressions really matter. I have wondered if it was like this everywhere, and there was no specific problem.

But I have now been elsewhere, and this is not a compliment. It is a big deal. It doesn’t make any sense. It’s rude.

These small insinuations and questions betray huge assumptions. That the only women working in technology are non-technical. That there aren’t women in technology at all.

Each time you’re asked these small questions, you’re forced to be an ambassador: correcting the assumption, proving we belong, proving you know what you’re talking about. You no longer get to speak for yourself because you have to speak for the group.

It is exhausting to have to deal with this over and over: striking the right balance of being polite, while explaining the asker has been impolite. Trying to save the next person from the same experience. Trying to change the perception, one person at a time. Even when only one person in an event with fifty asks the question – you know that someone else is thinking the same thing. It reminds you that you’re constantly being judged in a way you didn’t sign up for, and that others see you as not “belonging.”

All because of stupid questions. Rather than having women deal with this, we should all learn:

1) Don’t be presumptuous. If you want to know what someone does, say “what do you do?”
2) For extra credit, follow up with Paul Ford’s handy primer on being polite.

At HBS my opinion still had connotations: “technical” and “startup” and “blunt” – but that’s because those things are me, and they are the things I choose to represent.

I hope that someday women in technology are granted the same relief that I had this year. I hope that some day I comment and the first things people think are “consumer product management” and “artistic process” and even “MBA.” That people will judge those categories instead of my gender.

Until then the “old boy’s club” will feel more inclusive. Watching what questions we ask will be a start.

The Coaching Fellowship

Last fall I had an interesting opportunity: to participate in The Coaching Fellowship. I wasn’t 100% sure what coaching was when I started. The program has a handy chart, but everyone’s experience is different.

Screen Shot 2015-05-31 at 10.22.25 AM

The fellowship makes eight hours of coaching available to women 25-35 at a highly reduced rate. The goal is to give women the tools to succeed early in their careers and accelerate their impact. Applications have re-opened (open until June 12th!) and I wanted to take the time to share my experience as an example.

It started with an email exchange from my mom in early June, and a little over a month later I got the news: I was accepted!

Shortly thereafter, I started working with my coach, Lindsay Jean Thomson. Lindsay was a great match for me – many of her other clients are entrepreneurs in San Francisco. She has her own new startup: Women Catalysts. In addition to coaching, she’s spent time as a yoga instructor. She’s always taking interesting trips, and she was often traveling when we talked. She’s squarely in the age bracket where should could be a participant in the program! We have a lot in common.

At first I was a little envious. Lindsay seemed so secure in her work. She’s patient, she knows what she wants to do, and she does it. She reminded me of all my favorite lifestyle bloggers. But there was no real reason to be jealous: Lindsay was completely present and ready to help me figure out how I could be more effective.

The first session was my favorite. As a Product Manager, I love to define the system and figure out what’s going on. We needed to find a baseline to see what I needed to work on, and why. We went through a few key exercises, and I found all of them valuable.

I won’t go through the exercises in detail (that ruins the fun!) but to give you an idea:

1) We assessed the different “areas” of my life and how satisfied I was with each (work, friends, home, etc). We even used numerical metrics to make sure we had a firm baseline to compare to.
2) We figured out some of the persistent things that hold me back, and ways I negatively impact myself. These were the areas we wanted to tackle in the other sessions.
3) We did some visualization exercises to explore where I see my future.

One of the things I loved about the program was having homework. Having Lindsay assign the homework, and require a verbal confirmation from me, gave me more accountability. It can be easy to push personal improvement tasks to the side when you’re busy, and the homework made things more official for me.

At the beginning, Lindsay was active in defining my homework. We started with a very uncomfortable exercise in crafting a life purpose statement. As we spent more time together, Lindsay would ask me what I thought the best assignment would be and help me refine it. We’d figure out ways to apply lessons in my life. She always sent me notes after the call to reiterate the assignments.

In each subsequent session, we’d review the homework and find ways to push it further or tweak it. We maintained focus on a few key areas throughout the program. Some of them were about amplifying my strengths, and others were about fixing problem areas. I slowly got better.

The lowest point of the experience was right after my coaching ended. I’d subconsciously expected a neat bow: “and then I got my life goal of X!” or “and suddenly I felt Y!”

Instead, we just… ended. I’d put in the hours with Lindsay, I’d put in the homework, shouldn’t something magic happen? There was never a lightning bolt moment. At the time, I was sad. Almost a year later, I’m glad there wasn’t.

Coaching didn’t give me one moment, but it did give me a suite of tools that I’m still using. If I’d had that moment, I might have closed that chapter and moved on. Instead, I keep going through the process. I’m more aware of myself and my habits. I’m now better at catching the things Lindsay and I discussed. I’ll catch myself saying “just give them an A,” and I pause to ask myself if I’m devaluing intangible skills. I’m better at holding myself accountable for prioritizing personal changes.

The Coaching Fellowship is not a magic bullet. You might not end up with a visionary moment at the end, but you will emerge with the tools you need to keep going. In my mind, that’s more valuable.

Interested in applying? The Application is open here.

pm@olin: Capstone (Class 11)

This week was our last week of PM at Olin! For me, this was the most fun class we did. First, It was the most challenging pedagogically for everyone involved. It also nicely tied together what we’d covered. Added bonus: It also gave me a whole new lens to think about my HBS education through.

As always, I couldn’t have done this alone. In particular, Professor Eisenmann was a driving force behind the class. He wrote the case, helped me learn how to teach it from his teaching plans, came to class and observed, and gave feedback and explanation of cases live for my class. A huge thanks to him for helping pull together what I think was an excellent conclusion to our class!

Goals:
1. Cement everything we’d covered so far in the course.
2. See how Product works in a company that has legacy and other complications (instead of the project examples we used).
3. Use a classic business case study instead of small exercises and projects.

Required Reading:
1. Product Development at OPOWER  (Eisenmann and Go)

Materials:
1. The Case
2. Case Discussion Plan
3. Board Plan
4. Whiteboards, markers, associated props

IMG_7550

Exercises:

The Case – 90 minutes

The typical HBS classroom case discussion takes about 80 minutes. I don’t have the ninja-skills that HBS faculty have (but I do have the luxury of a 2.5 hour class) so we went a bit over.

I can’t share the specifics of our discussion. One key point of the cases is people don’t want the answers floating around. If you buy a copy of the case and want to chat with me about it, just let me know. If you’re an HBS student, you can request a copy for free from the library (also due to Prof. Eisenmann).

We spent about 45 minutes fleshing out the specifics of the problem. That was listing and evaluating all the options available to OPOWER and the Product Manager. Prof Eisenmann said in a more experienced classroom, we could have cut that to 30-35 minutes.

Then we talked more about some background and general facets. How did OPOWER get into this situation? How does the PM role and the organizational structure play into the dilemma? What is the best software development process? Did this one work? Is this broadly applicable to other projects?

On the whole, I was happy with how the conversation with. There were a few students who I wanted to get more involved (I did do a cold call!) and I wish I’d been more proactive about that during the conversation.

Because we only had ten students, it was easy for everyone to participate. I didn’t have to use a hand-call method, everyone just jumped into the conversation naturally.

I also felt like my leading of the discussion felt apart more as time went on. At the beginning I was pretty clear on what I wanted to cover and how to get it on the board. As we got to the final sections I was afraid it felt more like “box checking” than a genuine elucidation of knowledge. If I did it again, it would be stronger.

The Postmortem of the Case – 15 minutes

After we wrapped the case, Professor Eisenmann shared what OPOWER decided. He also made some comments on the discussion.

The students, in typical Olin-fashion, asked how they could have been better. That resulted in unplanned live debrief about the discussion. We covered how it went compared to an MBA conversation, and how I could have led the conversation more effectively.

I found this part of class rewarding. First – it’s rare to get feedback on your teaching from another experienced educator. That was a treat in itself. Secondly – it was nice to get a bit more insight into the style that goes into teaching a case. I spend a lot of time in the classroom observing what my faculty do, but it was nice to go through the process myself and see what happened.

I did draw from things my faculty do: for instance, Jan Rivkin uses colored chalk to annotate pros/cons. I tried to do the same with my markers. I also tried to ask more questions and restate less, because that sometimes frustrates me in class. After having done the postmortem, I have a lot more respect for why the faculty restate and how it moves the discussion along. I’d do so more in the future.

From the student side: Wow. I was impressed with how well they handled a case study. They evoked similar ideas to an HBS classroom. The discussion went well, in part because everyone had experience in the field and participation went back and forth a lot. Our Babson student was able to give a more business-oriented perspective, helping the discussion.

The students picked up on a lot of best practices. They took strong opinions, they spoke concisely, and they built on each others comments. There was a lot of genuine listening in the room. I honestly thought the quality of conversation was higher than some of the early conversation in our section.

(It definitely helps that they’ve been together all semester, have a smaller group, and are in a lower pressure situation – but was still impressive!)

Class Wrap – 30 minutes

We finished with a general class wrap. I was exhausted after teaching the case, so this part of class was pretty amusing.

We went back through some of the learning goals in the beginning (we covered many of them). We also chatted a bit about Olin and if I’d teach the seminar again. I hope the students benefit from having taken the class and I’m looking forward to seeing what they do next!

(I’m taking it as a good sign: of all the students that enrolled and started at the beginning: no one dropped the class, and everyone attended enough seminars to get credit).

Selected Notes/Changes:
1. I’d spend more time prepping the second part of the lesson. I thought it would flow more naturally from the first.
2. I’d move to a classroom that has the moving-chalkboards the way HBS does. I felt like moving to the side of my room really changed the dynamic of the conversation.
3. I’d spend more time balancing the voices in the conversation.

pm@olin: Most Likely to Succeed (Class 10)

Class 10 was originally intended to be a recap of everything we covered before our capstone next week, as well as some “special topics” to delve deeper into a few areas.

A few hours before class started, a student forwarded me an email with the text “ :( “ added. Olin was having a screening of the documentary, “Most Likely to Succeed” specifically for the students.

The documentary is about the current state and future of education in our country, and specifically followed High Tech High in San Diego. Following the film, Richard Miller (Olin’s President) and Tony Wager (Harvard Innovation Fellow) were going to take questions.

This is why Olin is a special place. The President of the college stayed at work until 9pm, on a Wednesday, to talk to students about the future of education. If I’d been a student rather than the teacher, I would’ve skipped class, no questions. You go to Olin for those opportunities.

So, I sent an email to my class with a poll about what we should do. There was a wide consensus that we should go to the film, and tie the other content into next week’s class. So that’s what we did and will do.

The film wasn’t directly related to be PM at all.

That said, I think it still make sense. A lot of being a PM is rolling with what doesn’t cost very much, and helps make the team happy. You don’t always get the most done by optimizing.

For me this week, it was letting the class take a one-time opportunity to see a film.

For a PM, It’s figuring out how to find a little extra time for the easter egg. It’s doing the extra work to get a cool side project into the product. It’s helping someone else learn a new skill. It’s the thank you cards or the day off after shipping.

pm@olin: Presentations (Class 9)

Goals

  1. Understand what makes a presentation compelling.
  2. Practice storytelling techniques.
  3. Have frameworks to consider presentations in.
  4. Understand the range of presentation types.

Optional Reading

  1. TED Talks
  2. Onlyness at TEDxHouston: Resonate (Nilofer Merchant)
  3. slide:ology (Duarte)
  4. Presentation Zen (Reynolds)
  5. Leadership Presence  (Lubar, Halpern)

Materials
Talks
Whiteboard and markers
Paper/pen

Exercises

Watch a talk – 20 Minutes

I opened the class by having us watch a talk. In this case, I picked Nilofer Merchant’s talk from TEDxHouston on Onlyness. I picked this because: I’m familiar with it, I liked it, I could pull out a lot of specific good things, and I guessed my students hadn’t seen it.

For the first 20 minutes we watched the talk together and individually took notes on what stood out to us. Some of the questions I prompted with: “What was particularly effective?” “What surprised you the most?” “How is this similar/different to your favorite talk?”

Discussion Presentations & Frameworks- 40 minutes

After the video concluded, I prompted with more questions to see what students took away form the presentation.

I asked students about their favorite moments (how do you get rid of the staplers?!) and things that were stood out while they were watching (chasing a butt around).

We started to group things on the board for what we thought was particularly effective. This isn’t an exhaustive list, but four major ones:

1) Good content. The message was important and also authentic.
2) Left us wondering, felt relatable (the staplers. what do you do with them?)
3) Related personally to the audience (positive jokes about Texans for a presentation in Texas).
4) A little bit of humor.

At this point, one student pointed out that all of the elements we’d pointed out were related very closely to storytelling and the skills required to be a good storyteller. I’ve frequently thought of presenter as storyteller, but it was nice to organically reach the conclusion. An interesting piece was that the only bit my students viewed as a “negative” they felt had been shoved in for sake of a storyline. In that case, they valued authenticity over storyline.

I used that as a chance to introduce the “PRES” framework that is taught by the Ariel group – about leadership presence The acronym stands for Present, Reaching Out, Expressive, and Self Knowing.

In contrast, I pointed out that a historically common framework was “tell them what you’re going to say, tell them, and tell them what you said” – a talk version of the five paragraph essay. They laughed at me, but I did think it was important to point out that a story isn’t the right way to share absolutely everything. Sometimes you are conveying facts.

Story Telling Practice – 45 minutes

After this, we practiced telling stories. The students broke into pairs, and told a short story to their partner.

Then we went through a few iterations: tell your story without using words (act it out), tell your story using as much voice modulation as possible (both cadence and volume) and tell your story with a specific audience in mind.

The stories definitely got better as we went through the iterations. Some students (rightly) pointed out that telling the same story over and over may have helped more than the specific tools. That said, I think it was worthwhile practice.

Presentation Work – 15 minutes

This section is short because we abandoned it fairly quickly. I had each student bring a topic they were interested in presenting on, and then tried to teach my process for developing a talk.

I had them expand their idea to see if there was anything else in the space that was more interesting. Because many of the presentations had to be on a specific topic, this didn’t work as well as it does for me when I’m creating a new talk.

We did some expansion, and then students started working on expanding their primary point. As questions came up, we abandoned working in favor of me showing a wide array of talks.

Sharing Talks – 30 minutes

We concluded with a walkthrough of some of my talks. I showed my Olin Senior Capstone project, some of the “pitch decks” we created for Startup Lockdown, and some presentations I’d done at Microsoft (no confidential info).

Honestly, this was probably the most interesting and useful part of the class.

Most of the students had never seen a PowerPoint that primarily consisted of bullet points and text (how?!) and assumed that everyone was making beautiful slides for their internal work status reports. Not so.

Even with the pitch decks (crafted by a former VC and former consultant) they were horrified. I have great respect for the people who made the decks, and I trust that they’re the right form for the situation! I don’t know how to make them, but I wish I did. My students struggled to believe another way of presenting was possible.

This exercise was one of the first times I felt like I’d *significantly challenged* the way students thought about a topic. The feeling in the room was much more visceral. It’s unfortunate that I just learned this now, because I wish I would have been more controversial, sooner.

Selected Changes / Notes

1. I forgot to take photos of the whiteboards. I think that means I’m getting more comfortable and this is becoming more routine, but it sure makes it hard to write things up after.
2. One way of teaching is to figure out what the “sacred cows” are and challenge those. It seems to open up more emotion and discussion.

pm@olin: Metrics (Class 8)

This one is a bit of a tough one to write, because I didn’t write the class! Adam Sigel came in to do the lesson on Metrics. He’s a Product Manager at InsightSquared, a Boston-area company that provides tools for Business Analytics. I’m incredibly glad Adam taught this lesson because I spend less time with metrics than most PMs. I learned a lot too. A huge thanks to Adam for coming!!!

Goals

  1. Understand how to pick the right metrics for your project.
  2. Understand how to track metrics and some of the tools that are available.
  3. Understand how to apply metrics back to your Product in order to learn and make a better version.
  4. Understand how metrics fit in with the other parts of your product process.

Optional Reading

  1. Measuring What Matters (Yoskovitz)
  2. SMART Framework  (Wikipedia)
  3. AARRR – Pirate Metrics (McClure)
  4. More than Metrics talks at #ProductSF and Beyond the Code (second is a longer, expanded version of the first (Chisa).
  5. Blameless Postmortems (Allspaw)

Materials
Whiteboard and markers
Paper/pen
Software to demo

Exercises

This class was more of a full-group style than usual. In many of my lessons, I split the students into groups to do projects. For this class we remained together, seminar or discussion style and worked through the issues together. It was a nice variation from the usual!

Introduction – 15 minutes

Since Adam was new to the class, he took some time to explain his background:

  • Previous roles in PM / before PM
  • Why he transitioned into PM
  • The structure and responsibilities of his current PM team
  • The differences between his roles at Aereo and InsightSquared

I think it’s always valuable to know where your teacher is coming from. I try to be very up front about my background/strengths weaknesses, but I did want the class to be exposed to other perspectives.

I particularly liked that Adam was coming from a non-technical background. For technical PMs, early in your career it’s easy to miss what non-technical PMs are bringing that you can’t.

Picking Metrics – 40 minutes

Adam then transitioned into how to pick metrics. He did this by helping the students work through examples.

We started with a student’s, Ryan’s, current project. I won’t share what his project is, but we worked through how he’d measure acquisition and what other things he cares about users doing.

Then, we worked through some other examples using popular companies. In particular, we talked about Snapchat Discover. We discussed what metrics Snapchat might have been trying to optimize for when they made the feature. Then we talked about how they’d measure success of the feature.

It’s a little hard to explicitly go through everything we mentioned for this piece, so I’ll recommend the two articles in the “resources” section as possible frameworks for looking at metrics. In general, I like the “SMART” framework (specific, measurable, actionable, relevant, timely). I don’t often go through the entire framework, I just think “what do I want to know?” and “how can I measure that so I know if it’s changing?”

One of the things we particularly wanted to highlight was the idea that you should be picking a metric with a point of view. Metrics aren’t about measuring random data – it’s about having a hypothesis and testing it, or having something that you specifically want to learn.

Use of Metrics (and Funnels) – 40 minutes

To talk through use of metrics, Adam also showed us what he uses for his own projects. Since he works at InsightSquared, he uses their tools for measuring success.

He introduced the concept of a funnel, which we hadn’t previously covered in class, and how you want to measure different things at different parts of the funnel. Again, I’m not going to show the specifics. A comparable explanation is Dave McClure’s talk on the AARRR metrics.

He also talked about which specific numbers matter to him right now, why those are the numbers that matter, and what he does to impact them.

AMA – 30 minutes

We concluded with a bunch of time to just ask Adam questions about his experiences as a PM. This covered a bunch of different topics we’d already discussed, and some new areas.

Post Mortems – 30 minutes

Towards the end I also took the time to explain the post-mortem process that we didn’t get to in “building” or “feedback.”

We discussed where postmortems come from (engineering failures), and the Swiss cheese theory of things going wrong. I liked this section because it isn’t just applicable to software – you can go through the same process to consider a hardware failure.

Screen Shot 2015-04-19 at 8.52.01 AM

 

This is slide is accessible from this talk by Noah Sussman. I think it’s originally from John’s talk on the same topic as the article in materials. I’ve always found it to be a very clear explanation of how things go wrong.

After explaining the basic concept, I shared a framework I like to use for Product postmortems. My framework is to do them post-launch, and then analyze how the project went:

  1. The PM should create a detailed, factual timeline of how everything went – circulated to the attendees to add to. The important part is that this is about facts.
  2. During the meeting, everyone should walk through the timeline and add anything that’s missing, and note patterns.
  3. The group can then come up with a list of “things we did well” and “things we can improve upon next time.”

This provides learning from the project that will make future projects with the same team go more smoothly. One downside of teaching this process in schools is that students often work with different people the next time around – there isn’t the project consistency that you get at work.

Selected Changes / Notes

  1. I should’ve taken better notes while Adam was talking. I learned things, but two weeks after the fact it’s hard for me to be sure I’ve documented this as well as some of the other lessons.
  2. I liked having the more seminar-discussion style participation. With a class of 10 it isn’t too hard to get everyone talking.
  3. I also learned some of the students didn’t know about this blog, which is a pretty big “whoops” given I document everything here. HI STUDENTS!

pm@olin: Feedback (Class 7)

For this workshop, I focused on the idea of doing personal level feedback rather than Product Critique. My rational was that this is (a) harder and (b) a good foundation to build from in terms of other feedback. If I were doing it again, I would try to include both.

For this class, a huge thanks goes to Alicia Morga who took the time to share some resources and exercises that she uses when teaching this subject. She’s spent a ton of time working on this, and I actually took one of her prototype classes on giving feedback. Her work formed the core of the this class, and I made some adjustments for Olin. Anything that went wrong is purely my fault :)

Goals
1. Learn some frameworks for personal interaction and feedback.
2. Learn how to give and receive feedback in a team setting.

Optional Reading
(again, thanks to Alicia Morga).

  1. Five Levels of Communication (Francisco)
  2. Emotional Intelligence:  Why it can matter more than IQ (Goleman)
  3. Influence without Authority (Bradford and Cohen)
  4. Emotions Revealed (Ekman)
  5. Difficult Conversations (Stone)
  6. Thanks for the Feedback (Stone)
  7. How to Talk so Kids Will Listen (Faber and Mazlich) – technically a parenting book with good communication advice.
  8. What We Say Matters (Lasater and Lasater)

 

Materials
Whiteboard and markers
Paper/pen

Ground Rules

This is the first class that I set unique Ground Rules for. Given that feedback is a charged topic, at the beginning I went through these rules. I also gave anyone who wasn’t feeling open to receiving feedback the option to leave the classroom (no penalty). No one took me up on it.

1. Be 100% Present. This means no side conversations, no technology, no leaving the room when we aren’t having a break.
2. Take the most respectful possible interpretation. This means not jumping to conclusions about what someone else says, or their situation. We’re all here to help each other
3. No discussing outside. While it’s totally open to discuss the frameworks and tools we used (or use them!) no one should discuss personal stories outside of the room.

Exercises – 2.5 hours

We covered four exercises in the classroom. I tried to write down the baseline length of time the exercise took – but between each exercise we reflected and talked about what we learn, so each took a bit longer.

House Drawing & Lecture – 30+ Minutes

For the first exercise, I and each student draw a house. Then, I had the group give them feedback on the house. The idea was to get an idea of where people started in a very low pressure situation.

IMG_7104

I used this to introduce a few different frameworks:

Complimentary vs. Constructive – The first few pieces of house feedback were all what we often call “positive feedback” – “I like the house, I like the tree you included” etc.

I took this as an opportunity to introduce new language. Positive/negative gives an unfair perception of one type of feedback being good and the other bad. In reality, both types are valuable. I also shared the notion that the best time to give complimentary feedback is when the action is happening to reinforce the behavior

Then we got some constructive feedback involved. This allowed me to share:

Don’t use the “shit sandwich” – compliment/criticism/compliment – this makes things seem disingenuous. As mentioned above, it’s better to give feedback at the time it’s relevant.

Feedback should be specific – Many of the pieces of feedback were also too general. We talked about how to give specific, constructive feedback that would be actionable.

When to Give Feedback – 20 Minutes

The house exercise naturally transitioned us into some questions around “when to give feedback?” This allowed us to cover some of the reasons people are reluctant to give feedback (fear of jeopardizing the relationship in particular).

The answer here is “Ask First” – the key tenant here is to ask first and get buy in before giving feedback. There’s nothing wrong with being honest and saying “can I give you some feedback?”

This also led to the question of “well, can I say no if someone wants to give me feedback?”

We talked about some reasons why people reluctant receive feedback. One big one is that people often have the mental mis-model that if they get feedback, they need to change. A better mental model to work with is that if you get feedback, that’s just more information that you can use as you like. Feedback is a gift.

We talked about how accepting feedback can be a good way to build a relationship, and you can choose to disregard the feedback if you want to. We also discussed the five levels of communication and taking things deeper or back up a level depending on the comfort of both parties involved.

http://www.newcommglobal.com/upload/Five%20Levels%20of%20Communication%20-%20Francisco%282%29.pdf

Personal Stories – 20+ Minutes

We then applied the lecture exercise using another short framework – the students got into pairs, shared a personal story, and then gave feedback on the story using our new frameworks. Then each pair flipped and tried it the other way. This was a low pressure way to help internalize the new frameworks.

Giving Feedback to Someone else – 40+ minutes

We then took this up a notch. Each student came up with a scenario in which they’d like to give feedback to someone else.

Everyone paired up in pairs of Student A/ Student B.

Student A described the person they wanted to give feedback to to Student B.
Student A pretended to be THEIR PERSON.
Student B pretended to be Student A and gave feedback.
Student A responded as though they were their person.

This was a way to remove some emotion from the experience and see how someone else would provide feedback. Additionally, it provided some empathy for how the other student might feel while receiving feedback.

Johari Windows – 30+ minutes

We concluded with Johari windows. Students paired up with another student that they knew well. Then they did the classic Johari window exercise – picking five adjectives to describe themselves, and five to describe their partner. They created personal windows using the four traditional categories: “open, blind spot, private, and unknown.”

Giving Feedback – X minutes

If we’d had more time, I wanted the students to actually practice giving feedback in a genuine situation. The issue was that we ran out of time, and also that most students didn’t feel a need to give feedback to people in the room.

Selected Changes / Notes

1. Today was an interesting class. The feedback that I got from the students was that these exercises were less immediately helpful than some of the ones we’d done on other days. My gut feeling is that it’s because the other ones (i.e. launching and specs) were concrete skills students hadn’t seen before – this is a bit more nebulous. I also didn’t feel like many of these were helpful right when I did them, but I think it helps in the long run. I’m curious to see how students feel in a couple weeks/months.
2. I think partially that these students were also relatively aware of these concepts, even if they hadn’t formally learned them. Olin has a pretty strong feedback culture, so people tend to learn by doing – even if they don’t know the words.
3. I wish we’d had more actual conflict so we could do feedback in the classroom. If we had a class project, that would have been a good opportunity to include it.
4. I also wish I’d included more on the practical side of Product critique. I might work that in later in the class.