Posts in category: business

Many have wondered about — and feared — the impact that gen AI will have on labor markets. Some compare it to past innovations, like robots, whose effects have been relatively modest, while others have forecasted that its impacts will be more long-ranging, given gen AI’s fundamental ability to improve itself over time. New research analyzed over a million job posts for online gig workers to see what affect the introduction of tools like ChatGPT and image-generating AI have already had on the quantity of posts, job requirements, and pay of online workers — and which fields and professions have been the most impacted. The researchers offer insight into the challenges and potential opportunities of these shifting markets.

AMY GALLO: You’re listening to Women at Work from Harvard Business Review. I’m Amy Gallo.

AMY BERNSTEIN: And I’m Amy Bernstein. Some of us have gone all in on generative AI.

AMY GALLO: Like these audience members of ours.

BARBARA RUIZ: So, I created this custom GPT for my job. It was a game changer for me. Before, if I had a problem, it would take me one day to solve. Right now, with these tools, I will do it, I don’t know, in three hours.

ERIN MORAN: It drastically increases efficiency and proficiency. I am able to be a better leader, produce more work. I have definitely come to rely on it.

ANDREA PLEBANI: I’m just so much more satisfied in my work because the opportunity to step back and look at things and translating it into something that’s actionable for my teams, that’s a huge value add.

PAM BOIROS: Computers now speak our language, so we can use our own English language skills to tease what we want out of these tools.

JULIE PRICE: So, you hire a fake team of researchers, and each one of your fake researchers comes up with a research plan. And then what’s better is if you create a character who’s a behavioral economist on the team, and a behavioral economist will come in and look at all the results of this research and tell you what’s unrealistic. It’s just unbelievably different and more impactful, the stuff you can do.

AMY GALLO: Ame, have you had these revelations about AI yet? [Laughter]

AMY BERNSTEIN: Not even close. I mean, I’ve asked it to do things that I was pretty sure it would fail at, and it did not disappoint me, so that just affirmed for me that I didn’t really need to pay that much attention.

AMY GALLO: Wait, what do you mean you asked the things you knew?

AMY BERNSTEIN: Oh, maybe three months after ChatGPT debuted, I was writing a condolence letter. And I don’t know about you, but I find those really hard.

AMY GALLO: Hard to get started.

AMY BERNSTEIN: Hard to get—actually just hard to do, hard to do the middle part and hard to end. But I thought, well, okay, do it, write me my condolence letter. And I came up with several prompts, and I was tweaking the prompts, and you know, it never came up with something that I could put down on a card and send.

AMY GALLO: Yeah.

AMY BERNSTEIN: I’m kind of relieved at that though—

AMY GALLO: Yes.

AMY BERNSTEIN: —to be honest.

AMY GALLO: That relief I think is exactly where I am with this, which is that I recognize it’s powerful. I know it could be transformational for me, certainly for my work, definitely for the world, and yet I don’t want to accept it. And so when it fails, I’m so happy. I still have a job! Yay!

AMY BERNSTEIN: You see? You see?

AMY BERNSTEIN: Well, except, these are early days, and this technology is going to learn so quickly.

AMY GALLO: I agree. I know it’s going to change the way I relate to my work.

AMY BERNSTEIN: How so? Say more about that.

AMY GALLO: Well, there are just aspects of my work that I don’t like that this could help with. And we’ll get into it in a little bit, the power users we’ve heard from, because this is who I’m really basing this on, is the women who wrote in to us, to told us how they’re using it. And whether we like it or not, whether we want it to fail or not, it’s happening; and so we have to be early adopters. Although we’re late on that. We have to be middle adopters—

AMY BERNSTEIN: Yeah, we missed that boat.

AMY GALLO: [Laughter] Yeah. We have to adopt, let’s just put it that way. We have to adopt.

In listening to women like us, like you, describe how GenAI has saved them time, broadened their thinking, deepened their agency at work. Amy B and I have gone from being skeptical dabblers to being skeptical enthusiasts. Maybe listening will have the same effect on you.

AMY BERNSTEIN:nIf you haven’t yet tried your hand at GenAI, we hope these power users inspire you to finally give it a try; and if you’re already using the technology, that you come away with some new ideas.

AMY GALLO: One of the things I was really struck by was the mental and emotional relief that GenAI gave them.

AMY BERNSTEIN: Blew me away.

AMY GALLO: Yeah, because of course I expect the opposite because that’s been my experience. It makes me anxious. I don’t know how to use it. It’s going to take my job. Ah! And for them to actually be able to begin to use it in their day-to-day work, and almost all of them use it on a daily basis, but to relieve mental burdens such as burnout or task paralysis, it’s just been a huge source of relief.

AMY BERNSTEIN: Can we hear from someone?

AMY GALLO: Yeah, let’s listen to what Andrea has to say.

AMY BERNSTEIN: Yeah.

ANDREA PLEBANI: I would consider myself a middle manager. So, at the time I discovered ChatGPT and generative AI in general, I, like many people, was coming off several very difficult years: difficult personally, difficult from a work perspective, and also very difficult as a leader.

Going through the pandemic, we really wanted to be authentic and present for what was going on in people’s lives as individuals. And that’s a lot. It was draining. And I definitely had reached the stage of burnout, and that’s where the generative AI really made a huge difference in that was, how much energy do I put into things? Because there’s so much in your day that just comes through that you have to react to, react to, react to, and you definitely need to put some thought into it, but you’re not writing the U.S. Constitution.

And so rule of thumb, if I’m likely to overthink it, let me think of a prompt. Let me think of a way to respond to this that doesn’t require as much mental effort or kind of right-sizes the level of mental effort to the task at hand. Because the things that, like I said, shouldn’t require so much of our energy often do.

The power of being able to just jot down a few thoughts, make an executive summary of this…boop, here’s three bullet points. Rewrite this for a group of cross-functional stakeholders. Boop, there it is. Make this sound more professional. Make it sound more casual. Make it longer. Make it shorter. Expand on this point. Emphasize this. Define a call to action. Boop. It feels like a miracle.

AMY BERNSTEIN: So, when I think of the many, many moments of misery of I’ve experienced in my career, I cannot tell you how high a proportion of them had to do with not being able to right-size the effort and attendant anxiety to the task at hand.

AMY GALLO: Yes. Well, and to not get stuck in that moment of, how important is this thing? And if I know it’s not that important, why am I spending so much time thinking about it?

AMY BERNSTEIN: Right.

AMY GALLO: And what Andrea is describing is almost a way to short circuit your perfectionism.

AMY BERNSTEIN: Oh, absolutely. I’d love to hear from another listener who uses it in a similar way, but in a different context to focus herself in a way that is more appropriate and will get her to her goal.

JESSICA MCBRIDE: I’m Jessica McBride, the founder of Tech Savvy Assistant. I created Tech Savvy Assistant after being laid off from my EA tech job based out of New York. I used ChatGPT because it was the original one. And I realized quickly how functional and helpful it was for admin roles, especially because there’s too many nuances that go into what we’re trying to solve that cannot really be encapsulated by a Google search. And so by using ChatGPT, I was able to take these, like I said, really nuanced problems and parcel them down into something that was a much more accessible way for me to take a problem on.

A problem that we come across a lot of times in the admin field is somebody comes to us and they’re like, “Hey, I’d like for you to do an event,” and they give us this vague idea—synergy—and I would just be like a deer in the headlights. But you can workshop this idea of like, “Okay, we need to do a three-day conference based around the idea of synergy. Help me come up with a step-by-step guide on how to produce this event.” And what I love about it is that, yes, I’m perfectly capable of creating this event on my own, but what am I going to miss? What steps am I not noticing? What else would I like to take into consideration for this? And it’s going to give such a well-rounded idea of what you want to create. It’s just giving you the roadmap.

A lot of people in my industry latched on to this idea of, AI isn’t going to replace you; an EA using AI will, and I don’t like that. I think that that is anti-fellow worker. I think it’s no different than, you know, getting introduced to the internet. This is the next level of like, I just need to learn how to embrace and use it and utilize it to the best of my ability. And I can really empower my career and become the next generation of executive assistant.

There are so many things that we can do to, like, increase the culture and be strategic business partners and think and expand that I did by just talking to, you know, other leaders in our organization and solving problems. But like, you can use AI to help you solve these problems.

AMY BERNSTEIN: What I heard in that that really excited me was this idea of breaking down the daunting challenge into completable tasks. Right? I found myself wondering if LLMs are freeing admins to do higher level work, how are they doing that? And Jessica has really tackled that.

AMY GALLO: Yeah.

AMY BERNSTEIN: It’s incredibly impressive. She has figured out how to use the technology to do a better job of identifying her executives’ objectives.

AMY GALLO: And she’s actually codifying—

AMY BERNSTEIN: Codifying and making it available.

AMY GALLO: —all of this and making it available to other EAs, which is just so smart.

AMY BERNSTEIN: I have the guide in my hand, Ame, and let me just read you some of the others. Identifying resources and obstacles. I cannot tell you how many hour-long meetings I’ve sat through, participated in, where we were trying to identify resources and obstacles, and she has a bot that does that.

AMY GALLO: Yeah.

AMY BERNSTEIN: Developing an action plan. When I read this, I thought, well, this is everyone who has anything to do with formulating strategy and executing on strategy.

AMY GALLO: And she’s, again, making that available to anyone. That’s one of the things that I think is really encouraging about these power users is so many of them are not just creating these tools for themselves, but also thinking about how it can benefit their clients, their friends, their community, their network.

JESSICA MCBRIDE: You can visit techsavvyassistant.com, and on there you’ll find things like my newsletter that goes out weekly, that often includes videos and clips of how I’m utilizing ChatGPT. I also have a community that you can join. So, it’s really focused on keeping you current in what is happening in technology.

AMY GALLO: The other thing that I heard in Jessica and Andrea’s stories were that they are using it to get unstuck.

AMY BERNSTEIN: Yeah.

AMY GALLO: I want to go back to that for a second because as you said earlier, the amount of agony we’ve spent in these indecisive moments, and it’s something I actually really worry about with myself, is that I tend to be indecisive about certain types of decisions. And I’ve actually worked with a coach to help me on certain decisions. So, when an opportunity comes to me, whether I say yes or no is so overwhelming to me, and I’ve actually worked with the coach to come up with the criteria: When will I say yes? When will I say no? And I go through that criteria, I rank it. But as I’m listening to Jessica talk, I’m thinking, why don’t I make that a bot?

AMY BERNSTEIN: Why don’t you make that a bot?

AMY GALLO: And just, I already have the criteria. I can feed it decisions I made before, so it knows what I’ve said yes to. I can also feed it all of the sample language I’ve developed around saying no, because that’s often the hurdle is that I decide I’m going to say no, but now I have to write the no. And that’s really uncomfortable for me. So, I can feed it that sample language and I can say, “Should I say yes or no to this decision?” Why? Using my criteria, tell me why. Can you please write a note saying no, if the answer is no.” It often is.

AMY BERNSTEIN: Yeah. And it really brings home to me how we’re really only limited by our own imaginations how we could use this technology. And you described the steps. It’s not that complicated.

AMY GALLO: It’s not. But if someone said, “Oh, you should develop a bot that will help you make decisions,” two weeks ago, I would’ve said, “Yeah, that sounds great… Never.”

AMY BERNSTEIN: Yeah.

AMY GALLO: “I don’t have the time for that. Will the amount of time I’m going to put in actually benefit me? Like, eh, no.” But listening to these women describe how they’re using it makes it seem so much more doable.

AMY BERNSTEIN: Yeah. And it helps externalize that anxiety. If you do it right, you’re going to get to the same conclusion, but you want to put yourself through all the stress of getting there.

AMY GALLO: Right. We have some other users who have been using it as sort of thought partners, and I’d like to hear from them in terms of how are they using it to both get unstuck, but also helping them generate ideas and think creatively. Because I think that’s one of the things we’ve often thought it wouldn’t be able to do is creativity. But let’s hear from Pam first.

PAM BOIROS: So, for example, I am working on an account-based marketing program for a client, and I’m a marketing leader, so I’ve done a lot of account-based marketing programs in my day, but it’s really easy to sort of get stuck in a rut.

So, I actually put a prompt and said, “I am working on an account-based marketing program for this company. We want to save these types of messages. What are some tactics that I should think about? What are some approaches that I should think about?”

And it actually came back with a list of the personas that I might be contacting. And the kind of company that I’m working for is in the HR space. So, I’m used to speaking to chief human resources officers and heads of recruiting and heads of talent. But one of the suggestions that it came back with was heads of operations. And I said, “Oh, that’s really interesting. I’ve not really worked with that persona before.”

So, I kind of furthered that query and said, “Okay, tell me about heads of operations. Tell me about that persona.” So, the suggestion that it came back with is, heads of operations are really thinking about costs and cost savings and to consider building a calculator based on some very specific criteria and metrics related to operational costs. It would’ve never occurred to me to do that. So, whether we actually do that…and I could have, and I probably will at a separate time, continue that conversation with the AI to tease out other ideas for reaching out to that buyer persona.

AMY GALLO: What I like about that use case is that—because one of the hesitations and one of, I will now say an excuse that I’ve been using about not using GenAI as much is that it’s not accurate—that’s a great case where it doesn’t need to be accurate. It’s just suggesting something she might explore and in partnership with her continuing to explore it.

AMY BERNSTEIN: Right.

AMY GALLO: She even says, whether we actually do that or not, that’s yet to be seen. But it’s a new idea for me to think about.

AMY BERNSTEIN: Right. That’s creative.

AMY GALLO: Yeah. And to think about the sources of input I get on a project I’m working on, right? Who am I not reaching out to get that input? Or I have a target audience when I’m writing an article. Who am I not considering part of that target audience that might be it? I mean, it used to be we had to spend tens of thousands of dollars, maybe more and months to get that data about potential readers. But now we can begin to imagine who else might we include? And that’s going to change the way we think about ideas, the way we market the products we create.

AMY BERNSTEIN: The thoroughness of the consideration. Because you always worry about what you’re missing.

AMY GALLO: Yes.

AMY BERNSTEIN: Everything we’re talking about here, while we’re a little bit starry eyed. It’s also making me uncomfortable. I don’t know.

AMY GALLO: Sure.

AMY BERNSTEIN: Yeah. Because if these agents can do such a good job, then what do we need us for?

AMY GALLO: And that is a huge question.

AMY BERNSTEIN: It is. And unanswered.

AMY GALLO: Unanswered.

AMY BERNSTEIN: So, I’m going to say a year and a half ago I was recording a conversation with Karim Lakhani, who is one of the experts on AI and particularly generative AI, he’s over at Harvard Business School. And he was saying to me that it would take almost nothing for him to create an Amy bot. I edit him. He’s got a lot of correspondence for me. He’s got all of the marked-up drafts, and he said, “Yeah, it really wouldn’t be hard at all.” Just recounting this to you is giving me the chilly horrors, right?

AMY GALLO: Yeah, yeah.

AMY BERNSTEIN: Because I know he could do it.

AMY GALLO: Right. Well, but why didn’t he, I wonder?

AMY BERNSTEIN: Why didn’t he?

AMY GALLO: Is it out of respect, or is there something he knows—

AMY BERNSTEIN: Oh, he’s got way more important things to do I think.

AMY GALLO: Well, maybe that’s it. Maybe that’s it. He is like, I’ll let Amy keep her job cause I’m too busy.

AMY BERNSTEIN: Thanks, Karim.

AMY GALLO: All right. Well, let’s talk about another way that these women are using AI and in particular around professional development and career empowerment.

BARBARA RUIZ: So I am Barbara Ruiz, I’m an SAP consultant, and I started using AI because my dad taught me. It was a game changer for me because in my role as SAP consultant, there is a lot of deep analysis, complex problem solving. So, I created this custom GPT for my job. I put how I wanted the GPT to present me the information, and then also where I wanted the information to be retrieved from. And also some PDF that I had with knowledge I have been gathering through the years.

I now solve technical issues that before they were outside of my expertise. Because the GPT will guide me step by step. And even if it won’t give you the solution, it can give you very good hints at how you can get to the solution.

It’s faster. It requires less people also because then I don’t have to call all the time the developers. And I feel honestly, it’s very empowering because now I don’t need other people to help me. And just like I can do it by my own, I’m very happy to be able to improve my efficiency and to have also a bit of free time because that also have helped me to have free time.

AMY GALLO: The thing Barbara said about working more independently and not having to call the developers, I found that really both exciting and terrifying.

AMY BERNSTEIN: What happens when we don’t need other people any more Amy?

AMY GALLO: Yeah. And this is one of the things I always worry about new technologies that promise efficiency is that sometimes the most efficient way isn’t the best way. And there’s a certain amount of friction that we need to generate new ideas to form relationships, to build trust, to resolve conflicts, that I worry we’re trying to smooth the edges of the messiness of work in a way that’s going to end up dampening our humanness.

But I think that’s a good segue actually to hearing from Julie because she used to work for a company doing user research and customer development, but she recently went independent and actually has a startup that creates personalized meditations for children. Such a great idea for company.

AMY BERNSTEIN: Oh my gosh. Yeah.

AMY GALLO: And needed to create an interactive website to do that. And she wasn’t sure how to do it. She wasn’t a web developer. She didn’t have these skills. And what she did, to our point about who do you turn to, do you need actual help? She created a fake team, a multi-agent team—

AMY BERNSTEIN: Oh my gosh.

AMY GALLO: —that could help her do it, and it worked. Let’s hear. We can talk about what this makes us feel, but let’s hear her describe it.

JULIE PRICE: So, I went over to Claude and I said, “You are managing my development team. On my team, I have a software engineer, I have a code executor, I have a designer, and I have a planner. I’m going to tell you what I want, and I want you to manage these four people for me and solve my problem.”

And it was bonkers. Over the course of the next few weeks, I completely… from somebody who doesn’t know how to code…it’s not just not even knowing how to code, it’s not even knowing how do you use GitHub? How do you push to a server? What’s my tech stack? How am I doing any of this? It wrote me every single piece of code that I needed, told me where to put it, told me how to execute it, help me get set up with GitHub. I got Google Analytics.

I mean, this was a website I wanted to look a certain way. I can’t just copy and paste every single thing. So, it would create some code for me. And then I would ask it questions like, “Why did you do this? Explain this to me. How am I using this technology? Why are things working in this way?”

And so I will never be a software engineer, but I am a lot closer than I ever was. And now I can actually have conversations that I couldn’t have before. And I was honest with it, which more honest than I would be with a real person, I would say, “Look, I don’t understand this, explain it to me like a person who doesn’t understand this.” But it was bonkers. I mean, it worked. I did something I could never do.

AMY BERNSTEIN: Well, and that sound you just heard was another industry going down. Right?

AMY GALLO: The death knell. The death knell. But to our point earlier, I do think there’s something missing. I don’t know if it’s a lack of expertise, but that sort of creative energy. But then I hear in Julie’s voice, it sounds like she got it. I don’t hear anything sort of lamenting the loss of human interaction.

AMY BERNSTEIN: Well, it definitely addressed her need and did it in an efficient way, in a way she couldn’t have done on her own. To me, it highlights the importance of the smart dumb question. You know, the generalist intelligence, because we’re blinded often by our expertise.

AMY GALLO: Yeah, yeah.

AMY BERNSTEIN: And in this case, a lot of what she brought to the party was knitting together all of that work by asking the non-expert questions.

AMY GALLO: Yes. Yeah. My eighth-grade teacher, eighth-grade math teacher, had this big sign in her classroom that said, the only dumb question is the one you don’t ask.

AMY BERNSTEIN: Yes.

AMY GALLO: And I think about that a lot in the work that I do because oftentimes people don’t ask the question because they’re afraid of looking stupid. And Julie refers to that. She was asking it questions that she probably wouldn’t have asked a live human because she was afraid of looking like she didn’t know what she was doing.

AMY BERNSTEIN: Right.

AMY GALLO: It removes some of that ego too from the process. I can ask it a really silly question and not have to worry about what it thinks of me, which actually is a big part of work. How often are we thinking, should I say that? Should I ask that? What will that person think of me?

AMY BERNSTEIN: Oh, for me, it’s always, did they already say this? Because I’m so often always multitasking. [Laughter]

AMY GALLO: Right. I have started starting my sentences, “Someone may have already said this, but…” [Laughter]

AMY BERNSTEIN: So, Julie, who we’ve heard from, has been working with small groups trying to help them understand what LLMs can do for them. And she sort of poses it as, what problems can LLMs solve? And the one that I think probably is most surprising—

JULIE PRICE: People had no idea you could solve interpersonal conflict, whether it’s work related, whether it’s family related, all sorts of stuff. And just the way it approaches solving it was very surprising for people. So, you describe that person that you’re having a problem with into the—I use ChatGPT. You could use Claude.

And so you describe the problem, explain the whole situation, explain the backstory: “I keep having this conversation over and over, and I’m really frustrated. I don’t know where the actual problem lies, and I really want to fix this. Ask me questions one at a time to make sure I’m seeing it clearly. By the way, you’re an expert at interpersonal dynamics in the workplace.”

Actually, that’s something I might do is, I might say, “You’re an expert.” So, before you even describe your situation, “You’re an expert at handling interpersonal dynamics in the office when two people are having an argument. You come in all the time, and you help people who have the same conversation over and over, and you often find they’re seeing it wrong. Give me seven ways that you have helped people in the past, and give me your backstory so I know why I should trust you.” And having it go into that much depth before you ask her a question really helps it be smarter.

AMY BERNSTEIN: All right, so what Julie just described was an Amy G bot, and now I have to ask human Amy G what she thinks of that.

AMY GALLO: What do I…? I think about 45 different things at the same time. One of the obvious ones is, yep, I’ll probably be out of a job, especially if I can train a bot to do exactly what I do, which is to give that advice that she’s prompting it to give.

So, in some ways it’s a “threat,” I’m putting that in the air quotes, to what I do.

But I think more than anything, I actually find it reassuring because the person is pausing to ask the question, how do I do this differently? And that is usually anywhere from half to 80% of the battle in interpersonal conflict, which is to pause, think about, what am I doing? How do I not respond from a place of reactivity? How do I respond in a thoughtful way? And how do I respond in a way that considers who the other person is and what might matter to them?

So, the fact that we have tools to accelerate that process makes me so excited. The issue is, at some point I envision, right, you and I get into a disagreement on Slack, and we say, “Let’s just let our bots hash this out.” These better evolved versions of ourselves that won’t get emotionally worked up. Who can maybe just solve…

AMY BERNSTEIN: I prefer the trial by combat.

AMY GALLO: [Laughter] Well, you and I like a good fight, let’s be honest. But it’s forcing us to do the thing that’s so hard as humans to do. We’re almost hacking our broken brains in those moments of conflict to get to a better answer. And I do think there should be an Amy G bot.

AMY BERNSTEIN: Well…

AMY GALLO: In fact, I’m not quite there, but I’m taking an initial step of creating and training a custom GPT to be me. And actually with the help of Alex Samuel, who I think you probably know. She’s a data journalist. She’s been a long-time contributor to HBR. She’s an expert on the digital workplace. And she and I had a conversation about how I could create a bot that’s going to help me start to create content in a different way, in a way that’s easier. And that helps me with the like, Wait, what have I said about this before? I know I’ve said something about this.

AMY BERNSTEIN: Wait, what do I think of this again?

AMY GALLO: And so I’m not going to be at the point of a stranger can ask Amy G, but Amy G’s going to ask Amy G what should I think about this? What have I said about this in the past? So, the next thing you’ll hear is my conversation with Alex as she walks me through what all of this involves and helps me get over my big hesitations around it.

AMY BERNSTEIN: I’m looking forward to this. Or not.

AMY GALLO: [Laughter]

ALEXANDRA SAMUEL: I went through some of your stuff because I was thinking, what would we scrape of Amy’s to be able to make a virtual Amy? [Laughter]

AMY GALLO: Well, Alex, you and I have been writing for HBR around the same time; and pretty much every time you come out and say, you got to try this new thing, in your HBR writing, and I go, not for me. And then like two months later I’m like, oh gosh, I got to go back and find that Alex article. You have always been making me uncomfortable about what I need to do next and then helping me through the process.

ALEXANDRA SAMUEL: Thank you. I really love that. Also, I think that’s going to be my new LinkedIn byline: “Making you uncomfortable since 2009.”

AMY GALLO: That’s right. I’ll be honest, before we even started talking about this episode, there’s part of me that really wants to create an Amy GPT that can give advice. That’s what I’m in the business of doing. And so I’m not ready to be honest for a variety of reasons to do that. But I think ultimately, I do want to get there, but there’s some serious hurdles mentally for me to get there. So, I think by trying to make an internal GPT, I think it’ll help me build more trust with the technology and get me more comfortable.

One of the things I want my GPT to be able to do is help me with brain fog, which is related to menopause certainly, but also related to age, also related to overwhelm, the occasional burnout I have. So, I actually have written so much, I don’t even know what I’ve said on certain topics. And so that’s part of what I want the GPT to help me do. I also want it to help me come up with new ideas, like make connections between things I have written or said or posted about and come up with new ideas for newsletters or articles and just sort of be the better Amy. [Laughter] Like, can it do that?

ALEXANDRA SAMUEL: So, even when you customize an AI with your own content and with thoughtful instructions that reflect how you like to work, it is not you. And that’s good for those of us who still like to earn a living by being ourselves. And while as writers and thinkers, we’re always interested in what’s new and what’s next, for our readers and the audiences and the organizations we are trying to serve, often what is most useful is what we have already shared, already thought through, already said, but updated to this context, a new framework, a new structure, a new social network. And that is where these AIs are just terrific, is taking what you have already done and remixing it a thousand ways in two minutes.

AMY GALLO: Right. Okay, can I actually trust what it’s going to do for me, and can I trust it to not make me look dumb?

ALEXANDRA SAMUEL: I don’t think I’ve published a sentence that was written by an AI, except in instances where what I’m doing is saying, “Here’s something that was written by an AI.” But I also can’t think of when I last wrote something without the help of an AI. And there are a lot of mechanisms you can use to improve the quality of what it’s doing for you in the way of background research, including telling it what to draw from. And that’s what you do when you give it a collection of your own work. And that’s what you do if you ask it to summarize 10 specific PDFs rather than go and find its own sources on workplace conflict.

So, in my case, for example, I have this Claude project. I have it both as a Claude project and as a custom GPT that I call the Alexerizer. And the purpose of the Alexerizer is to make content that sounds like me, Alex. And in order for the Alexerizer to make content that sounds like Alex, it has to understand what Alex sounds like. So, I have given it a whole bunch of files that are kind of a snapshot of the breadth of my work in different contexts. And by giving it that range of content, it is able not only the sound more like me, but it understands my work, it understands what kinds of work I’ve done over many years, and it also understands how I write differently in different contexts.

AMY GALLO: So, if I want to train the Amyerizer—and I’m going to be able to pronounce that better by the end of this episode—if I want to train it on my HBR articles, my LinkedIn posts, and my newsletters, how much of each of those things would you suggest?

ALEXANDRA SAMUEL: Well, I would suggest a couple things. What I would probably do in your circumstance is I would think about your greatest hits or your foundational pieces. Like, you’re trying to configure it with three related sets of knowledge.

One is actual stuff you think and know and say. So, you need to make sure it has the core knowledge, your core principles, the things that you come back to time and again as core themes, you want to make sure those are all in there. So that’s the knowledge piece of it.

Then you want it to understand style. So, from that point of view, it doesn’t necessarily take a whole lot, right?

AMY GALLO: Yep.

ALEXANDRA SAMUEL: You want to pick the newsletters that really reflect what you like most. And again, you can have a separate file that is the voice file, and I’ve done that as well. I’ve picked, you know, there’s the ones that I have that are my body of knowledge, then I’ll pick three or four articles that are, This is, gosh, if I could always write this well, this is how I wish I always sounded. And then those are the voice files.

And then there’s a third piece beyond just the knowledge and the voice, which is process. And so part of where my Alexerizer provides value is understanding, how do we go from Alex’s 2000 word first draft to her 1200 word second draft. And because I hate that process, I love it when the AI can help me. And then also when I finished the article or finish the newsletter, how do I go from here’s my 1200-word newsletter, now I want that chunked into a series of LinkedIn posts.

So, in order to train the AI on the process, I’ve given it some kind of matched pairs of content. I have an article I wrote for the Wall Street Journal where I gave it every draft along the way, including the email exchanges with my editors and what kind of feedback they gave me. And then I gave it the final article so it can understand, here’s how we go from first draft to published article. And then I did the same thing with one of my newsletters: “Here’s a newsletter, here’s a series of LinkedIn posts based on that newsletter. I want you to use that as a model.” And that’s what goes in the custom instructions: “When you’re taking Alex’s newsletter and turning it into LinkedIn posts, use these two files,” and use that as your model.

AMY GALLO: Right.

ALEXANDRA SAMUEL: So, what I would say is in terms of volume, I would think in terms of those use cases. What do you want the AI to do for you? Do you want it to be writing your articles for you? Probably not. Do you want it to understand how to turn your notes into a first draft?

AMY GALLO: That’s what I want it to do.

ALEXANDRA SAMUEL: So, the next time you’re writing an article from notes and interviews, take the interview transcripts—if you use a recording tool that does AI transcription, take the transcript. Then take the notes where you have personally picked out the quotes that you thought were useful from those interviews. Then take your outline, take your drafts, put all those pieces together into a file called… You know, you might have one file called Interviews for Article, one file called Notes for Article, and one file that is the Final Article.

The more you kind of annotate and structure that notes file, the easier it is for the AI. So, say, outline, first draft, write these as subheads within your notes file. And then you give those to the AI and you say, “This is your model. Take a look at how I choose the quotes to use. Take a look at how I turn an outline into a draft.” You’re teaching it to think like you by giving it examples of you along all stages of the process.

AMY GALLO: Yep.

ALEXANDRA SAMUEL: And then create the greatest hits file that is your content file that gives it, here’s the essential inside of Amy’s brain. Give it your bio, your speaker profile bio, so it knows how you position yourself in the world. And then give it some samples of voice. And you can even annotate those to say, this is my voice on HBR. This is my voice as a speaker. This is my voice on LinkedIn.

And when you write the custom instructions, which is a separate file that can be quite long—I think on GPT the limit’s 8,000 characters—that’s where you tell it how to reference these different pieces and how you give it the overall instructions of what it’s doing for you, namely writing in your voice, writing first drafts, assimilating background material.

AMY GALLO: And can you change those instructions?

ALEXANDRA SAMUEL: Anytime.

AMY GALLO: Okay, good.

ALEXANDRA SAMUEL: I change them all the time.

AMY GALLO: Okay. One of the things we haven’t talked about, which I think is really important, is getting permission to do what I’m doing from whoever needs to actually know. So, in my case, I’m not a full-time employee at HBR, but because of this podcast, because of my writing, I am subject to their GenAI acceptable use policy. So that’s a piece we’ve started to figure out. And actually as we’ve been talking, we got a note from the compliance and data associate director saying that we could actually use Claude for this, which is great. So I just want to note that for anyone following along who’s like, I’m going to do this too, important check mark. What kind of advice do you give people in terms of making sure they’re complying with whatever appropriate rules, laws, policies?

ALEXANDRA SAMUEL: There’s two things. One is, and this is perhaps a slightly sensitive issue, always hold onto your copyright. We are all generating content all the time. We are creating the seeds for our own self-replacement. If you are an employee, and your work is therefore work for hire, then everything you type between nine and five belongs to your employer.

AMY GALLO: That’s right.

ALEXANDRA SAMUEL: So, I’m more worried about it to be honest, from the employee’s perspective than I am about the employee violating compliance rules. But for sure, you want to keep on top of your employer’s policies. And I think you also want to be really careful about where you create these files. So yes, there is the theoretical possibility that your employer can take all your work product and make a virtual you, but most employers aren’t there.

AMY GALLO: Yeah.

ALEXANDRA SAMUEL: But if you have meanwhile taken all your own work product and concatenated it into these giant files and built your own custom version of you that you’re using, and you have done that and stored those files on the company file server, you’re sure making it easy for them.

So, I would actually recommend to people that while you obviously don’t want to violate your company’s policies on AI, you also want to protect yourself by making sure as much as possible that when you’re creating these tools that allow you to automate portions of your own workflow. That those tools are on your computer, that those files are on your computer, again, as within the bounds of what’s consistent with your company’s policy. Because we should all be worried about protecting what remains of our own personal IP.

AMY GALLO: Yeah. So, while we stay up at night worrying that our employer is going to replace us with GenAI, we also want to make sure we don’t get fired before that even happens by violating the policy. That’s sort of how I interpret what you just said.

ALEXANDRA SAMUEL: I think that’s a good way of putting it.

AMY GALLO: Yeah. Yeah. So, I think I have what I need to get started, but I’m curious, what didn’t I ask you that I need to know just from a practical step? I know I need to think about the use cases. I need to think about voice, knowledge, process files, the instruction files. It’s a lot of work with hopefully the result that it’s going to save me time. But what else haven’t I asked?

ALEXANDRA SAMUEL: One thing I would say is really helpful to keep in mind when you are writing those custom instructions is to tell it both what you want it to do and what not to do. Let it know what your weak spots are. Say, “You’ll see in my example that I use a crazy amount of em dashes, but everybody’s always telling me not to use so many dashes, so your drafts shouldn’t use those.” Like, even small things.

AMY GALLO: Word repetition.

ALEXANDRA SAMUEL: Yeah.

AMY GALLO: Yep.

ALEXANDRA SAMUEL:

Oh, here’s a real one. It is a huge challenge to get an AI to write a conclusion that doesn’t begin “In conclusion.” They love to begin “In conclusion.” So actually I, part of my custom instructions in some of my AIs is, “Never say ‘In conclusion.’ Just don’t.”

AMY GALLO: I love that. I love that. So, wait, how do I know when I got it right? I could just seed my AI system, my custom GPT, with knowledge files and tweak the instruction. I could just do that forever. How do I know that I got an A+? That’s what I’m asking, Alex.

ALEXANDRA SAMUEL: There is no A+. I’m sorry, Amy. You will never get an A+. The finish line will move infinitely. I mean, on the one hand, I hate all the AI hype. And on the other hand, I basically want to tell you that I think this is the biggest revolution in how we think and connect and work as people. That quite apart from the tangible benefit of getting your Amyerizer working so that you can make your articles faster, that process is how you’re going to understand what is happening to our world now, what’s happening to our work, and what can happen to each of our own creative processes. And it is terrifying and overwhelming all the time. And in a way, that’s exactly why I want you personally to be doing this is because the nature of your work is such that you are one of the people who needs to help everybody else through all of the rough moments that are going to happen during this transition.

AMY GALLO: Okay. I’m excited and terrified. And that was an incredibly good pep talk, so thank you. I’m ready. Not only can I do it, not only will I do it, but I need to do it you just told me, which is, okay, here we go.

AMY BERNSTEIN: So, Amy G, it’s been a couple of weeks since you mentioned that you are programming the Amy G bot. How’s it going?

AMY GALLO: It’s going well, actually. I would say the interesting thing about the process is that it was a huge hurdle, mental hurdle for me to overcome. And I knew that. In my conversation with Alex, I knew I was going to struggle with finally taking the step to do this, and it was even bigger than I expected. And our poor producer, Amanda, kept messaging me, “Have you started? How’s it going?” And I was very honest with her and told her, “I haven’t done it yet. I’m going to do it. I’m going to do it.” And the hurdle was really about sort of trying to figure out what is this thing doing, and is the work to set it up going to be worth it?

AMY BERNSTEIN: So, what does the Amy bot do exactly?

AMY GALLO: So right now, it’s really to help be a partner in my thinking, particularly around my newsletters, my LinkedIn posts, maybe even article ideas. It’s meant to help me come up with ideas and maybe copy for short things like posts, not obviously an article, that just sort of get me started. And so it’s supposed to save me time.

One of the things, I know you can appreciate this, and I’m sure many of our listeners can, which is that one of the things that has suffered in the current busyness of my schedule is time to think. And so just having a partner who I can bounce things off of is what I was looking for.

AMY BERNSTEIN: Okay. So, give us an example of how Amy bot helped real Amy think.

AMY GALLO: Yeah, so this is the thing it’s been most helpful at. I have a newsletter for my own business that I send out twice a month. I uploaded every newsletter I’ve ever written into it and said, “Can you help me think of ideas for future newsletters?” And it came back, because I had given in the instructions, I said, “Don’t answer the question right away. Ask follow-up questions.” Which, I think I might need to change that instruction, but I’ll explain in a minute.

So, it asked me, “Do you want me to look at fresh new ideas? Do you want me to look at follow-ups to existing articles?” It prompted, what do you actually want? And I said, “Oh, I’d like a follow-up to an existing newsletter.” And it pulled one from January 2023 and said, “You mentioned gossip in this article. Maybe you want to write a newsletter all about gossip.” And it gave me three ideas, all of which were great. And I will say the reason I’m hesitating on the instructions was because that instruction to ask follow-up questions meant like, you know how so many people talk about how AI is like a smart intern? It did feel like that because I was like, the intern needs to stop asking questions and just give me an answer.

AMY BERNSTEIN: Right.

AMY GALLO: And at one point I would say something, it would ask a follow-up question or three questions, and I say, “Can I answer those one at a time?” And they’d say yes. And then they’d already have a follow-up question to my first answer, and I’d be like, wait, we were losing track of our question.

AMY BERNSTEIN: Yeah. So, it needs to be more discerning in its follow up.

AMY GALLO: Yeah, I think I need to instruct it to produce more and ask less.

AMY BERNSTEIN: So, it helps you generate ideas. What else does it do for you?

AMY GALLO: So, I named it Esme because that’s really what I wanted to name my daughter, and I couldn’t because we have a friend who had just named their daughter Esme, so I decided to name it Esme. So, I talk to it as Esme, and this is the sort of high-level instructions I gave it: “You support Amy in drafting social media posts, coming up with newsletter topics, producing rough drafts of newsletters, reminding me of advice I’ve given before, helping me respond to people who ask me for advice, and more.”

It’s a lot I’m asking it to do. And I think I will hone it a little bit more and maybe create just one that’s focused on newsletters or maybe one that’s focused on LinkedIn posts, or maybe one that’s focused on advice giving, or I sort of need to play with it more to—maybe I just have different conversations with it that do each of those things. This is where I’m a little lost in the process, and I probably will go back to Alex and say, Okay, I got pretty far. I’m really happy with what I’ve got done so far. How do I tweak this?

AMY BERNSTEIN: So what did you train Esme on?

AMY GALLO: Okay, I have a couple variations on my bio. So, I gave her all my bios so she could understand who I am. I gave her a summary of both of my books, so I didn’t give her the whole book. We’re using Claude AI for this, and I think it would’ve taken up too much of the knowledge capacity is what they call it. I’m already at 50-something percent of the knowledge capacity.

So, I gave her summaries of both my books. I gave her transcripts of our episodes I did about getting along based on my book, partly because our producer Amanda already had those. So, I was like, okay, that’s easy to get. I’ll just put those in. And then I did all of my newsletters that I’ve ever written, and then I did a sampling of some LinkedIn posts because I couldn’t get the full history.

And then I actually, last night I was talking to her about, I wanted her help me write a LinkedIn post on an article I published on Monday. So I cut and paste the copy from that article as well and put that into a text doc for her to look at. She didn’t do a great job with that, I have to say. It’s good, but especially for writing, once you start to really look at it, it’s very surface level.

AMY BERNSTEIN: So I wonder if you’ve had the same experience that I’ve had where you’ve, you ask it to write something and—or you ask Esme to write something—and usually for me, when I’m doing it, my state of mind is usually anxious and sort of a little guilty that I’m doing this, and also sort of kicking myself because God, why did I wait till the last minute again?

AMY GALLO: Yes.

AMY BERNSTEIN: So that’s usually the whirl of stuff that’s in my head. And so when Claude spits back something, I look at it very quickly, and my first response is, phew, I can work with this. And all of the noise goes away, which gets me to the point where I can look at the draft for real, and I see, Oh, no. No, no, no. But at least it’s something I can work on.

AMY GALLO: And I think I almost use it in some ways of what I don’t want to do. I do find the idea generation really interesting. So, when I asked Esme to produce a few newsletter ideas, I love the ideas, but I didn’t ask her to write about them.

AMY BERNSTEIN: Yeah.

AMY GALLO: But it just feels very, I don’t know, especially as editors, I’m like, this is like that jargon piece from a writer where I thought at first glance, it was going to be great. And then once I started editing, it all fell apart. I realized I needed to rewrite the whole thing.

AMY BERNSTEIN: Yeah. Yeah. That’s painful. I mean, for me, writing, one of the reasons I like Claude or Copilot to give me a draft to work with is that it’s much easier for me to edit than it is to write. Writing can be torture, but when you’re really in it, it’s thinking.

AMY GALLO: It’s thinking.

AMY BERNSTEIN: And it’s figuring out what you think.

AMY GALLO: Yeah. And I think I like writing more than I like editing. I wish I trusted Esme, and I haven’t tried, but I wish I trusted her to edit cause that would be fun.

AMY BERNSTEIN: I wonder what Esme would say to you if you said, “Esme, I know you want to edit, but I don’t trust you yet.” I kind of wonder what she would say. Because she already has at least some of your thought processes.

AMY GALLO: I can do that. I can do that right now. Here’s the other thing. It’s funny. I have to refer to her by name. When I start typing, I go, Esme—

AMY BERNSTEIN: And you to be polite.

AMY GALLO: I’m so polite to her, and I watched the emotional labor I do to make sure her feelings aren’t hurt when she produces something crappy is amazing.

AMY BERNSTEIN: Amy, she’s a machine. But go ahead.

AMY GALLO: I know.

AMY BERNSTEIN: I do the same thing, but I do think it’s worth reminding ourselves.

AMY GALLO: Okay, here’s what… I’m ready.

AMY BERNSTEIN: Give her the unvarnished truth.

AMY GALLO: “Esme, I would really like your help editing LinkedIn posts, but I don’t quite trust you as an editor.” She’s thinking, she’s thinking.

AMY BERNSTEIN: She’s taking a deep breath because that was a gut punch.

AMY GALLO: I know. Well, it’s funny when she’s thinking it says, “Ruminating on it.”

AMY BERNSTEIN: Yes.

AMY GALLO: No, she asked a few clarifying questions.

AMY BERNSTEIN: Oh my God…

AMY GALLO: Yeah, she wants to help.

AMY BERNSTEIN: Esme.

AMY GALLO: She says, “Please let me better understand how we can help you with LinkedIn posts while respecting your hesitation about editing.” And then she asks, “What aspects of LinkedIn post editing make you most nervous about delegating?”

AMY BERNSTEIN: Oh my gosh. It’s like talking to your therapist.

AMY GALLO: “What elements of your voice and style LinkedIn feel most important to preserve? Would you prefer to start with me reviewing existing posts and providing feedback rather than editing directly?”

AMY BERNSTEIN: Oh, I think she just gave you a path forward.

AMY GALLO: She did. I have to say, I feel like we are going to end up in a long conversation that I’m like, I still don’t trust you. So…

AMY BERNSTEIN: I think you should thank her and close your laptop.

AMY GALLO: With an exclamation, two exclamation… I’m going to do it.

AMY BERNSTEIN: Oh, go.

AMY GALLO: I’m just going to do, thank you. Let’s see what she does.

AMY BERNSTEIN: A little bubble dot under that exclamation point.

AMY GALLO: I used two exclamations. Oh, no, she has more questions. [Laughter]

AMY BERNSTEIN: I’m sure she does. Esme really needs to just back off for a sec.

AMY GALLO: All right.

AMY BERNSTEIN: Well, Amy, this has been so interesting, and I’m so curious about where you take Esme.

AMY GALLO: I will keep you posted.

AMY BERNSTEIN: Please do. And please say goodbye to Esme for me. I didn’t mean it. I swear.

AMY GALLO: I’m just closing my laptop because I can.

AMY BERNSTEIN: Bye.

Before we end this episode, there’s one more person we want you to hear from, especially those of you who are self-employed. She’s figured out how to use GenAI to make growing her business just a little easier.

TERESA RAMOS: My name is Teresa Ramos. I work as an executive coach, a technology consultant, and trainer. I love my job, but a big part of it is writing proposals, which I absolutely hate.

A client approaches me and we hit it off. We think it’s great. And then they say, “Okay, could you write a proposal?” And I’m like, oh, no. And they would go like, “Oh, you know, you just have to write what you just told me.” And I’m like, yeah, right, because I dread it. Just a request would take me maybe a couple of weeks. So, one day I thought, how about getting ChatGPT to do it?

Let’s go through the process. I go and talk to the client, and when I go and talk to the client, I take notes. In this case, I have my own GPT for writing proposals. And in the instructions, you can say, “Act as a strategic director…” “Act as a professional copywriter…” whatever. “Act as a very experienced technology consultant. You are creating proposals. You give a description, a generic description, because remember, this is something that you’re going to use for all your proposals. And then to make it even better, you can upload previous proposals that you have already created.

So now ChatGPT knows who it is and what the proposals look like and how they are structured. So, whenever I need to write a new proposal, I just say, “This time, you are giving one and a half hour training session for a group of 30 senior executives, please write a proposal. The proposal should include a description of the session, learning objectives, learning outcomes.”

And then you hit Return, and then it will write you the proposal. And not only that, those 20 minutes or that half an hour, it’s not like pulling teeth out because it’s like, whoa, let’s see what comes out now.

AMY BERNSTEIN: Women at Work’s editorial and production team is Amanda Kersey, Maureen Hoch, Tina Tobey Mack, Rob Eckhardt, Erica Truxler, Ian Fox, and Hannah Bates. Robin Moore composed this theme music. I’m Amy Bernstein. Get in touch with me and Amy G by emailing womenatwork@hbr.org.

ALISON BEARD: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Alison Beard.

For the next few Thursdays, we’re bringing you a series of interviews with some of the world’s leading tech CEOs and founders, to hear their perspectives on artificial intelligence and other top-of-mind issues. Today, we’ll listen to a conversation that Toni Townes-Whitley, CEO of SAIC, had with HBR Editor-in-Chief Adi Ignatius during our recent virtual Future of Business Conference. SAIC is a company with more than $7 billion in annual revenue, and 24,000 employees. It provides engineering, digital, AI, and mission support to defense, space, intelligence, and civilian customers. Toni took the helm about a year ago, after stints as a senior executive at Microsoft, CGI Federal, and UNISYS. She also serves on five boards, both corporate and nonprofit, and is a former US Peace Corps volunteer, so she has some useful insight on how organizations of all types can work together on our biggest challenges. In answering questions from Adi and the audience, she shares her thoughts on leading effective strategic transformation, upskilling employees, and guarding against AI bias. Here’s that discussion.

ADI IGNATIUS: Let’s dive in. Talk about how, as an incoming leader, you’re able to figure out what needs to change and what leverage you have to try to bring about that change?

TONI TOWNES-WHITLEY: A couple things were very clear. SAIC was first. It was a first in its market to have an employee-owned operating model, to operate in the world of STEM before STEM was cool. That was the history. Then I started to look at the data, the more current data relative to SAIC having gone a bit flat in its growth cycle. SAIC having to more differentiate its portfolio. And SAIC not being considered as deeply a market leader as it had in years past. It was in those inputs that I started to think about what would be the areas of pivot or tuning that would be helpful, in some key areas that would most likely correlate to shifting the growth pattern and almost resetting SAIC back in the market in a point of leadership. So I started to consider those as hypotheses, and I came up with four hypotheses of where the company needed to tune. I wanted to test those in the first year.

ADI IGNATIUS: Well, let’s talk about those four because, from the outside, it looks like you decided to move on practically everything. The product portfolio, the market strategy, the brand, the culture. How do you get buy-in across the company? How do you execute on so many different fronts?

TONI TOWNES-WHITLEY: Let’s start with tuning versus wholesale change. The magnitude of change in each area is important to understand. We were very clear about identifying where we would change and what would stay the same. In the spirit of start, stop, and continue, we had those exercises across the company to ensure that people understood that not everything was changing. We also sequenced before pivot. We started with portfolio. Leaned in heavily on the portfolio. With questions on whether our portfolio was truly enterprise scale, whether we had built point solutions for unique customers, or whether we had enterprise capability to go after large, large programs. We started to ask about whether our technology was in fact state of the art and differentiated. Whether that technology was deployed across all of our programs, as well as were we bidding that into our pipeline. So we started with portfolio. We then went to go-to-market and culture. The two came together, as we identified what kind of company, not only how did we want to go-to-market, but who do we want to be as we go to market, as individuals and collectively as SAIC. And identified the pivots within those areas. Then brand was our final move. To really start to speak to, given the shifts that we’re making inside the company, what’s the future brand of SAIC? SAIC has a phenomenal brand. Our study and research indicated that we were very well known. But it had an anachronistic brand, everyone looked back at a golden moment at SAIC, not a forward-looking brand, and understanding what we were going to be as a company in the future. How did we get the buy-in? A combination of laying what was changing and what was not, moving in sequence, looking for tuning opportunities, and quite frankly, Adi, I think probably most importantly, putting an enterprise operating model in place, with metrics, and processes, and rhythms that hardened that opportunity to change. Meaning not just words and hopes and desires, but fundamental day-to-day rhythms in the business through metrics that we use — about 14 that I track every month, but 70 that are arrayed across the company at different levels, to help us know that we’re on track, and that we’re moving, and to give signal, and to, quite frankly, create some listening systems in the organization.

ADI IGNATIUS: This is great. I hope people are taking notes. This is a masterclass in how you do strategic transformation. This is really great. Let’s dig into some of these areas. On the portfolio shift area, I assume that a lot of that is about embracing new technologies, including AI. Curious, how do you do that? How do you operate on the cutting edge while serving complex, in some cases government clients, that need the highest levels of safety, and security, and reliability?

TONI TOWNES-WHITLEY: So when I talked about the hypothesis, Adi, that I came in. On the portfolio, obviously coming from a company like Microsoft, was super focused on our capabilities and where we were differentiated, and whether we were at scale. I found, I was so pleased to find, that we actually had clear differentiation in areas of what we would call digital engineering, secured data analysis, and operational AI. You mentioned AI. The way we look at it is a very operational, gritty level of AI that happens in mission-critical environments, as well as secure cloud — our ability to broker and migrate organizations to the cloud. I also saw that we had some work in what I called Horizon Two. We were starting to turn the corner in edge computing, in understanding quantum, and starting to move into the areas that will be the future-leaning areas for the company. So I was actually quite pleased to see that we had that capability. How do we deploy it, though, within the Department of Defense, the intelligence companies, the space agencies, and the civilian government? That was really our conversation. We made some investments in the first year to harden those areas, and to create opportunities — what we would call sandboxes, to create the client environment. Ahead of deploying it to the client, testing it, modeling it, simulating it so the client could see what capabilities existed. We’re very proud of the work that we do, not just in AI. I know that’s the hot topic, I talk about it in terms of operational AI because AI is a category of capability, as you know. And we have it in every part of our company. But where we focus our AI and our data is that we spend a lot of time on the security and the accuracy of data that supports AI, so that the models are actually really robust. So that when those large language models reason, they reason over the correct data. We see the use of AI across the Department of Defense in targeting, in predictive analytics, in the ability to bring down the decision timelines, in the ability to integrate datasets. We’ve seen great receptivity to that, across the Department of Defense and the intel companies — most notably, probably with what we’ll call the combatant commands, where AI is most critical for them in their day-to-day operations.

ADI IGNATIUS: On the AI question, you talked about some of the use cases. This is a real civilian question, but anyway. I’m sure some people listening might feel a little uneasy about the idea of AI tools being used by the Department of Defense or other federal agencies. To what extent are you even able to think about so-called “responsible” use of AI?

TONI TOWNES-WHITLEY: Well, it’s core and center to how we think about AI. When you think about AI in mission-critical environments, AI in terms of training air traffic controllers that secure our air space in the US. SAIC trains every air traffic controller in this country within the Department of Transportation. AI at the border, at the southern border, within the Customs and Border Patrol. We run that capability, the technology that secures our southern border. AI that is part of decision analytics for the Department of Defense, as well as for our national security agencies and organizations. We spend a lot of time on the ethical aspects of AI. We are part of standard setting with NIST, which is the standards body within the Department of Commerce. We have our own trustworthy initiative that we do with George Washington University. We are first of its kind, as a system integrator, a mission integrator, that is really codifying what great AI, ethical AI looks like. But we think about AI in the context of the data that it is reasoning over and the security of that data. We see AI, also on the civilian side, you can use AI to improve scenarios, like what we find in our biometric center. We run, I think maybe the center is the only one of its kind in the world, where we are able to test facial recognition technology, and test the ability that AI, and facial recognition, and image capture. We test and challenge whether bias is introduced as skin tone gets darker. We know all of the concerns and the research. We do this for the Department of Homeland Security, to ensure that we understand what bias is introduced in facial capture. We’ve been able to learn and quite frankly create patent and all kinds of research, and set industry standards for not only the US, but internationally, on what does great image capture look like. That’s a concern that citizens have, that they are going to be represented correctly. We have over 4,000 volunteers that have come through to give us accurate testing on facial recognition. So I’m proud of using AI to actually address bias, using AI to improve mission, and also being held accountable for AI standards in the ways that we do, that many do across our industry.

ADI IGNATIUS: Yeah. It’s a really interesting example of public-private partnership. But I have to ask, how do you navigate a client base of government entities that are bureaucratic? They’re beholden to Congressional budget making, which isn’t always the smoothest thing in the world. With political turbulence, it can mean big swings in approach and policy, from one administration to another. That sounds like an incredible challenge. How do you handle that kind of uncertainty running the business?

TONI TOWNES-WHITLEY: No, it’s a very fair question and it’s a very real day-to-day experience. I will tell you, look, continuing resolutions have become a norm, unfortunately, in terms of how the budget locks happen. We are somewhat used to how to manage that, on a year-over-year basis. The effect really is in our customer base. Those kinds of budget stalemates create great challenges for our customers in how they do long-range planning, multi-year programs, how they handle end of year money, the signals they want to send to the private sector on where to invest. All of that assumes an ongoing and a consistent budget process that has been truncated often by continuing resolutions. Large swings in the Executive Branch, quite frankly, don’t have as much of an effect on our company. When I look at where we’re positioned across the defense and intel sectors, quite frankly, those signals come more from the conflict that’s happening around the world, and where those customers are focusing emerging tech. We happen to be in the areas that they’re driving to, which will be cloud-based command and control systems. That’s where we specialize and that’s where they’re spending. So we’re not overly concerned there. On the civilian side, it’s a fair question. Our footprint in the civilian agencies is primarily in areas, in those large federal cabinet agencies that are somewhat not affected by political swings. Think of the VA, the Veteran’s Administration, Department of Homeland Security, Department of Transportation, the State Department. That’s where we find ourselves with our largest business. If we were in agencies that were more ripe to some of the political fodder, we would probably be paying more attention in that area. But right now, we believe that our focus is continuing to drive our national imperatives around all domain war fighting, improving the citizen experience, improving on undersea dominance that this country has had for the last 50 years, really driving towards next generation space, and doing all of these activities collectively. We’re not as overly concerned on the political landscape right now.

ADI IGNATIUS: I want to go to a couple of audience questions now, and I have some more questions for you later. This is from Adam. I’m not sure where Adam is. But the question is, “What key metrics are you using to track the transformation?” Could you give some examples?

TONI TOWNES-WHITLEY: Yeah.

ADI IGNATIUS: What are the numbers that matter to you now?

TONI TOWNES-WHITLEY: Yeah, I appreciate that. For our shareholders, quite frankly, and other stakeholders, we had to return to a mid-single-digit growth company. We had been at low-single-digit for the last five years. We had to improve our growth. We had to improve our profitability. We were outside of the profitability range of our peers. And we needed to position more strongly in the market as a leader. So in the spirit of the financial growth side, we measure our growth in terms of how we bid our business and the number of submissions that we are making. We had been declining in the number of proposals we’d been submitting. We look at our win rates against those proposals. And we look at our ability to grow off of our base business. And we measure those in terms of submissions, win rate, and on-contract growth. In terms of how we grow, it’s not just that we grow but how we grow. We have four key growth vectors. Civilian is one of those. We’re rebalancing the business to be more towards a third. Right now, a fourth of our business is civilian, we’d like about a third of that business to be civilian. We have a large addressable market there that’s very profitable for us. And we also have great capabilities to support civilian agencies. That’s a growth vector. We’re moving our business into more mission and enterprise IT, so back office technology for the CIOs and front office for the mission. We measure that move of our current program revenue, as well as our future pipeline revenue. And then we measure our cultural shifts. Our culture is about enterprise mindset. Even though we have a wonderful entrepreneurial spirit, we trade as one symbol. The challenges that our customers face require all 25,000 members of our company to come together, and all of the technology of our company to come together. So we measure where enterprise mindset is showing up across the company demonstrably in terms of how we’re working together towards specific programs. Those are some of the key metrics that we look at. We’re now just starting to look at some brand metrics. But think about the four pivots that we introduced. We are measuring against each of those four pivots, and looking to see if our strategy, quite frankly, is taking hold. To date, we feel pretty good that it’s starting to take hold.

ADI IGNATIUS: Yeah. Well, let me ask you more about culture. You talked about that’s something that you’re measuring as well. Tell me more. Are there specific actions that you’ve taken that are showing results — whether on a quantifiable, or on a quality basis?

TONI TOWNES-WHITLEY: Sure, Adi. Look, there were four pivots to our culture that we introduced. One is you’ve heard me talk about moving from not just a pure entrepreneurial culture, but to a blended culture that recognized an enterprise mindset. We also talked, in our culture, about being able, and a third of our folks are military, former military. We have a very respectful, polite culture, but our ability to debate, debate topics respectfully. To get more challenging in our culture, and a little bit more bold in challenging and interrogating ideas, not interrogating each other. That’s a social, how we collaborate type of way that we wanted to change the culture. We also talked about being a more bold culture, in terms of taking risk. Calculated risk, but going after big bets and naming those. Finally, the pivot for our culture was relative to incubating talent. We were a great talent acquisition company. We have acquired talent very, very well. We use metrics like days to fill a requisition. But we needed to shift to building more talent in the organization. That meant putting metrics on managers, and giving them tools to incubate talent. We call that upskilling. We’ve run three pilots now across the company, and now we’re investing more and more in upskilling. Upskilling our individuals who are interested in moving up with more technical skills, like cloud, or data, or AI. Moving up with more consultative skills. They’ve been in a mission. They want to know how to be more business development oriented. Some people who are super technical learning more mission, understanding of what our missions do. In those four pivots, we’ve been measuring through pulse surveys how our employees feel more enterprise mindset. We track when there’s a program that, let’s say, is being supported at the enterprise level that requires each of the groups to step away from one of their objectives to support the enterprise, how often that occurs. We start to measure, as I’ve said, how much talent is being incubated through upskilling. Again, we see early indicators, our pulse surveys are up. We see greater enterprise plays, if you will. And we’re seeing early indicators that the upskilling is a big hit inside the company for career development. So that’s how we’re moving forward, and we’ll stay down that path for the next few years.

ADI IGNATIUS: You had hinted earlier, or implied earlier, about bias in AI and AI output. This is a question from somebody who’s watching named Iris, who’s interested, “From your experience, how do companies address bias in AI implementation?” in their business, and that could range from HR processes, to the content they’re putting out there in the world, or the products they’re developing. How have you come to think about that?

TONI TOWNES-WHITLEY: I can’t speak for all companies. I’ll say, at SAIC, we run an AI council. We bring all of the disciplines of AI, from our internal IT operation to our external innovation factory, every one of our functions, HR, legal, our finance function, and those that are client-facing. They all have representatives on an AI council that meets routinely to address issues of AI in the business. How we’re using AI for proposal development or recruiting or hiring. But also, AI to our customers. So AI from the business, which is how we are embedding AI in our solution sets going to our customers. I think one of the best ways, and I’ve learned this over the last few years, not only here at SAIC, but previously at Microsoft, in terms of having a multi-dimensional group that’s looking across the business to understand how AI is being designed, tested, deployed, and then reassessed. So that, what we’ll call sensitive uses of AI, are being reviewed by a broad, diverse set. It’s important that that team is not only diverse in their thinking, diverse in their background, but actually trained and have standards and metrics. That’s where I’m pleased that we are part of standard setting within the government. We have many different initiatives for standard setting for AI to make sure we are current with all ways of testing and understanding the use of AI. I think it starts with having a dedicated set of individuals that are multi-dimensional within a company, that are routinely looking at every aspect of AI — both the risk, and the opportunity.

ADI IGNATIUS: Now, bias obviously doesn’t exist only in the world of technology. I’m curious, you’re one of two Black female CEOs in the Fortune 500. I’d love your thoughts on, or if you’re willing to share, what hurdles, if any, have you faced getting to this point? And to what extent is that limited number, is that changing? Do you feel that there are more opportunities now for more people?

TONI TOWNES-WHITLEY: Have there been hurdles? Absolutely, Adi. That’s a very fair question, and supposition is correct there. And hurdles on every level. Sometimes the hurdles are there that are maybe more gender-based, sometimes they are racially-based. As I said, I’m an African-American female, I show up as both every day. There are perceptions sometimes of others. The bar can be set inappropriately high or inappropriately low. I’ve had all kinds of microaggressions throughout my career. But I also want to talk about not just the hurdles I’ve seen, but my own hurdles. There was a great book written by Becky Shambaugh 20 years ago called It’s Not Just a Glass Ceiling, It’s the Sticky Floors. There were some sticky floors where I held myself back, where I was concerned about being so representative of so many groups that I was slowing some of my decision making, and sometimes uncertain about my next steps. What I started to deploy over time was a career path that looks more like a staircase. There were horizontal and vertical aspects of my career. And I started to look for the next move after three to four years in any role, where I was on the horizontal and vertical. Which I’ll define quickly as vertically, I was looking for real stretch opportunities, almost adventurous, intellectual caffeine that would challenge me in ways I had never been challenged. Then I was looking to apply the learnings from those vertical opportunities in broader and larger opportunity roles in different companies. If you look at my career, it’s almost a staircase of vertical challenges, and then horizontal application of what I’ve learned. Some people stay in the vertical, they’re adventure seekers. Some people stay in the horizontal, they become the SMEs, and the smartest person in the room. I wanted to keep challenging, but keep applying what I’ve learned. That’s how I built my career. And as part of that, it’s given me some opportunities to take moves, like going to Microsoft was one of those vertical moments. Quite frankly, leaving Arthur Andersen as it was closing as a consulting business after Enron and coming to a business in infrastructure and technology company like UNISYS was a true vertical for me as well. But I’ve had opportunities to apply that learning. SAIC is in the horizontal phase for me, where I’m applying so much learning in this new opportunity. I will tell you, I’ve tried to address the adversity that comes — the fact that there are so few individuals that look like me — with curiosity, both in terms of understanding why aren’t there more women who look like me in this role? What do we need to do as root cause to address those issues? Helping people address the biases that they sometimes can’t see in how they think about putting women and people of color in these roles. And starting to understand that there is much more supply out there and available, if we would broaden the aperture, provide access, and start to measure how we are moving people through not just careers, but getting folks into career-moving opportunities. I’ve been blessed to have a few career-moving opportunities, and that’s why I’m sitting in the role now.

ALISON BEARD: That was SAIC CEO Toni Townes-Whitley and HBR Editor-in-Chief Adi Ignatius, speaking at our virtual Future of Business Conference. I hope you listen to all our Future of Business series, and all of the episodes we have on the HBR IdeaCast, about leadership, strategy, and the future of work. Find us at hbr.org/podcasts, or search HBR in Apple Podcast, Spotify, or wherever you listen. If you don’t already subscribe to HBR, please do. It’s the best way to support our show. Go to hbr.org/subscribe to learn more. Thanks to our team, Senior Producers Anne Saini and Mary Dooe, Associate Producer Hannah Bates, Audio Product Manager Ian Fox, and Senior Production Specialist Rob Eckhardt. Thanks to you for listening to the HBR IdeaCast. I’m Alison Beard.

Performance reviews are an important tool to help managers and organizations motivate and engage their workforce. Narrative-based feedback provides employees with more personalized analysis and can shed light on individual paths for improvement, while numerical feedback offers clear benchmarks for employees to track and meet specific targets. In a new study, researchers examined whether one format — or a combination of the two — was seen as more fair and motivating by employees. They found that while narrative feedback is perceived as the most fair, it can be especially meaningful for those employees with room to improve.

In an increasingly data-driven world, many companies, including Adobe, Morgan Stanley, and Goldman Sachs, have made the surprising move to do away with number-based performance reviews. Some have opted for more open-ended, narrative-based performance evaluations, while others have eliminated reviews completely, conducting regular “check-ins” instead. The argument for the shift away from numerical reviews is strong: narrative performance reviews allow for more context and can better offer employees ways to improve while affirming their particular strengths. At the same time, some companies who’ve eliminated numerical reviews have reverted to creating “shadow” rankings, where narrative feedback is offered to employees, but internal numbers are used in order to track growth or to have a more objective way to tie performance to bonuses or raises. This can leave employees feeling like they’re being secretly judged in ways they can’t fight or speak to.

Recent return-to-office mandates have been partially predicated on the belief that employees are able to form richer relationships in person than they can working remotely. New research, however, found that remote work can actually make coworkers feel closer by giving them authentic glimpses into each others nonwork lives through video calls. The researchers analyze their findings to offer suggestions for managers considering the benefits — and costs — of calling employees back in.

Increasingly, employers are rolling back the remote work arrangements instituted during the Covid-19 pandemic. Companies such as Starbucks, Walmart, Google, JP Morgan Chase, and Amazon have called employees back into the office — some even mandating a five-day-a-week return-to-office (RTO) schedule.