Chris Noessel – Designing Assistant Technology

Chris Noessel – Designing Assistant Technology

Podcast Power of Ten

In this episode, interaction design veteran Chris Noessel, former design principal at IBM where he ran the worldwide Design for AI Guild, and author of Make It So, Designing Agentive Technology and the new Designing Assistant Technology: AI That Makes Us Smarter, talks with me about how AI assistants risk de-skilling us and what designers can do about it. Chris explains why his books came out in the “wrong” order, why handing off the cognitive work of understanding is different from earlier waves of automation like phototypesetting and desktop publishing, and how he trained Claude Design to pair design with him rather than dump screens. The conversation covers the See, Think, Do loop and his five universal assists, the “human goes first” pattern (which turned a 48% performance collapse in a navigation study into a 19% gain), organisational over-reliance, AI as Frankfurt-style bullshitter, and why hydrofluorocarbons deserve far more design attention.



Transcript

Note: This transcript is machine-generated and may contain some errors.

Andy Polaine: [00:00:00] Hello and welcome to Power of Ten, the show about design operating at many levels of zoom from thoughtful detail through to transformation in organization, society, and the world. My name is Andy Polaine. I’m a design leadership coach, educator, and writer. My guest today is Chris Noessel, a veteran of interaction design who has helped shape the field for over thirty years, designing products and services across diverse domains.

Chris has visualized counter-terrorism futures, prototyped forthcoming technologies for Microsoft, and designed telehealth devices. He was a design principal at IBM, where he ran the worldwide Design for AI Guild and has developed core AI training for designers globally. Along with designing, running workshops, and speaking, Chris is the author of several books, including Make It So: Interaction Design Lessons from Science Fiction, the fourth edition of About Face: The Essentials of Interaction Design, Designing Agentive Technology: AI That Works for People, that was back in two thousand and seventeen, and most recently this year, Designing Assistant Technology: AI That Makes Us Smarter.

Chris, welcome to Power of Ten.

Chris Noessel: Thank you so much. I’m glad to be here.

Andy Polaine: So we are, uh, kind of contemporaries really in terms of the, the length of time we’ve both been in this field, which makes us, you know, dinosaurs on the one hand and on the other hand- … uh, able to see the repeating patterns. I’ve given you a little bit of an intro there, but what’s the sort of main thread of your journey, um, from back then to now, and particularly, I guess, how this combination of interaction design and, and the future really have threaded together throughout your career?

Chris Noessel: There– Okay. This actually immediately raises an anecdote, and I’ll try and be quick about it. But in undergraduate, I was part of… I, I, I picked a class that I needed because I was working my way through undergrad, and I needed an evening credit, and it landed me in a Photoshop course, but I was doing Photoshop professionally, like as part of my job.

So I got the class, and then afterwards, the, after the first class, I was like, “Ed,” who was the teacher, I said, “I already know this backwards and forwards.” And he was like, “Sit down and show me.” And I showed him, and he was like, “Yeah, you already know this.” So I said, “Okay, um-” There’s a group of grad students in the back.

Why don’t you join them?" And we were studying, here’s a dinosaur term, multimedia and learning Macromind Director, I think it was at the time.

Andy Polaine: Yeah.

Chris Noessel: And, uh, took the one semester course, loved the heck out of it, really found that it tickled all the right parts of my brain, and signed up for the same class the next semester, even though I wasn’t a grad student.

Uh, Ed was gonna let me in. But the thing was, and why this story relates, is that he had us, all the students who were returning to the class, write a short essay about why we wanted to take the class again. And at first I thought it was busy work, but then I wound up taking it quite seriously. And my answer, I don’t know if it was, uh, uh, what’s the term?

There’s a term where a, a, like prophetic in the sense of, uh, establishing the rule or whether it really was a reflection of something internally. But what I wrote at that time in the essay was that I want to pursue the new interesting questions. And at the time I was studying graphic design, and I felt like it was a pretty mature field.

Like, we knew a lot about what was going on in the context at the time. Um, and it was kind of down to tr- uh, fashion. And I was like, that, that’s a mark of a mature field when you’re, like, all the basics are known and you’re just saying, “Oh, what trend are you following?”

Andy Polaine: Hmm.

Chris Noessel: Um, but interaction design at the time was new.

New to me of course, but also new to the world. And I thought those are the problems that I want to pursue. So Throughout my career, I’ve been like, “Yes, I can do graphic design. Oh, but wait, there’s this new problem of interaction design.” Then I could do interaction design, which I did for a long time, and I was like, “But wait a minute, that’s, that’s not the end of the problem.

There’s a, a deeper problem here.” Um, and it’s led me to every sort of new technology. I really have a burning desire to f- understand it, what it wants, where it fails, um, and, uh, figure that out so that I can use it and not be led by it. That was a really long story.

Andy Polaine: No, that’s, uh- I could do a shorter one. No, I can, I can relate a lot.

Okay. I think, I mean, I… People who listen to this a lot will know, but I originally wanted to be a film director. Oh, nice. And I went to study pho- photography, film, and video, and then the year I joined, it became this thing called contemporary media practice because it included this new digital media thing.

And it was the same. You know, I was doing Photoshop. Yep. And then my then professor, Andy Cameron, said, “Hey, look, there’s this thing called Macromind Director.” And I was like, “Oh, that’s interesting. Non-programmers like us can make interactive things.” Uh, we’ll probably return to this, actually, a bit when we get to the AI bit.

And yeah- Yeah … similar. Similar. So you’ve got this new book out, and it’s, it’s called Designing, uh, Assistant Technology.

Chris Noessel: Yep.

Andy Polaine: And the previous one from what’s nine years ago now was called Designing- Yeah … Agentive Technology. Yes. And I expect lots of people are listening or, or watching saying, “Well, hang on, he’s got those titles the wrong way, the wrong way around.

The first one should have been assistant technology or assistive maybe, and then, uh, now we’re all about agents now.” So, you know, is that the case, or, or what was going on back then? What were you trying to do with the first book and compared to this current one?

Chris Noessel: Sure. Uh, and I, I w- I will take a brief moment to acknowledge the linguistic trap that are the two titles because agentive-

Andy Polaine: Mm-hmm

Chris Noessel: should have a parallel of assistive, but, uh, especially in the United States, but I believe throughout the EU as well, assistive is a legal category of software that helps people with handicaps.

Andy Polaine: Yeah. Yeah.

Chris Noessel: And I have no interest in diluting that term by doing something by saying, “Oh, all technology is assistive.

Oh my God.” Uh, you know, this is an underserved- Yeah, no, understood. Yeah … population, and I have no interest. So, uh, I s- I recognize that they’re not parallel, but I’m still gonna stick to calling it assistant technology to distinguish my work in that field or in an adjacent field. Okay, so the bigger question is about the order of the two books.

Um, and the order came about from Chris’s realizations and what I believed to be true about the industry. And that is at the time, I was doing a lot of to work traveling. I worked for a small Consultancy in San Francisco, and of course, firsthand research meant that we were traveling a lot. I had cats, so I had automatic cat feeders, and I was, like, thinking– I remember ve- quite specifically, I was about to leave for an around-the-world tour, um, interestingly to do some research for Autodesk, who is now my employer, and I was staring at the cat feeder, and I had set up the times when it was going to be fed, the cats were going to be fed, and I realized I had no evidence that it was set correctly because it only sh- did its thing when I was away.

And I was like, “Well, this is kinda crap.” Um, and that had stuck in my mind. At the, uh, a li- a short while later, I wound up working with Schwab to do a robo-investor, and the robo– the idea about the robo-investor is it manages a portfolio of stocks, uh, according to your personal financial goals, and it does all of its work outside of your attention.

Um, if you could just let it go and check on it once a year, but it’s still working-

Andy Polaine: Hmm …

Chris Noessel: all the time. And those two things began to sort of strike me as quite similar. So when I began to pursue that line of thought, which is h- oh, and of course, I had a Roomba at the time, again, something that does its work outside of your attention.

And those three things felt a piece to me, and I began to pursue it because it was the new interesting thing. I’m like, “I want to design this robo-investor well. I want to know how to critique and what to recommend to the cat feeders should they come and ask me.” They never did. And that– and I was like, “Well, I’ll do my due diligence and find out who has written about this.”

Um, and at the time, I didn’t have a great vocabulary because there are threads that were similar that I might have found, but I didn’t. Um, and so I was like, “Well, nobody’s written this book. I should write it.” And, um, wound up writing that book because it was what was new to me and seemed like the new big thing.

That book was about agency, models of agency between a human and a system, but also the agent itself. What is that thing? How do we design for it? What are its irreducible components? The book was published before the popularity of foundation models and large language models, so the things that fall out from using that back-end system in order to inform an agentic experience, uh, d- are not covered in the book.

But nonetheless, at the time that I wrote it, I was eager to publish something for what I thought was a new and emerging model. Turns out I was right, but I was a couple years too early.

Andy Polaine: Mm-hmm.

Chris Noessel: A lot of people had really good reception of the book, but they were like, “Yeah, there’s no technology stack for me to build on these cool ideas.”

Good on me. At the time when I was working through that model of agency, um, which in short is like if the human is doing all of it themselves, think of it as a manual tool. If the human is doing most of the work and getting a little leg up from the system, then it’s an assistant. If the system is doing most of the work and the human can, according to human preferences, and the human can occasionally get in, pause things, modify things, correct things, then it’s agentive.

And if the human’s only getting involved in a failure mode, that’s automatic. Yeah. So that’s what I’m talking about when I say the modes of agency. And I recognized that the assistant was a thing that was there, but I was like, we have fifty years of design to inform what assistants were. I looked at things like spell check, and I was like, we, we have this.

I don’t need to write about this. Um-

Andy Polaine: Yeah, or head up displays and stuff in for pilots- Yeah … and all of that stuff. So.

Chris Noessel: We’ve got it all. Um, and, and I can’t add anything new to that. But with the w- I published the agentive book. LLMs came online on around two thousand and nineteen as a sort of a popular service that you could access, use, and think about, and build around.

Then I came to the awareness of overreliance and de-skilling, and I was like, “Oh, that’s it.” AI res- is a special type of assistant that significantly exacerbates de-skilling and overreliance. That’s because they assist us in ways that de-skill us. So now I had a bead and a mission as to why I should and could talk about assistant in ways that I didn’t think we had fifty years of popular design materials, um, or, uh, best practices around.

And I was like, “Okay, now’s the time to put it out there.”

Andy Polaine: So they’ve been being used in, I’d say sort of mainstream, like for probably, what, two, three maybe years? I’d say two actually, if that. It’s currently, uh, sort of toward the end of May in twenty twenty-six. And, you know, there’s obviously been this range of usage of them from the kind of frivolous, you know, look at all the kind of funny pictures I can make or things I can do, to the sort of tragic of teenagers committing suicide, being encouraged by ChatGPT to do so and things like that, through to, you know, and now businesses going, “Everyone, w- get on the AI.

We, we need to, we need to do it or we’re gonna, you know, go under and, you know, burn as many tokens as possible, and everyone should be using AI.” And these sort of directives from the top, like if you’re, you’re, you’re not using this expensive thing we’ve bought, and you need to be using- … this expensive thing we’ve bought.

And people are going, “Well, you know, it’s not that useful.” Through to obviously people integrating into their workflows, uh, wholesale. And kind of now the other… I feel like the, you know, the last, even the last six months of, of there’s been a I mean, apart from, quite apart from all the resource consumption thing, which is a massive issue anyway, but there’s been a, a sort of rising, at least reporting of a kind of backlash to it, p-particularly from young people, but also from, I think people have now started using it quite a lot, like in the design.

So I just interviewed, um, Josh Clark and Veronika Kindred about their Sentient Design book, and there’s obviously in, in the design world, there’s, “Oh, it’s gonna replace design and designers, and we don’t need them anymore,” through to, “Hmm, it’s actually not that great,” or, “Well, actually it’s costing us quite a lot to make these prototypes, and we used to be able to do them for free, uh, in quotes, uh, with Figma or whatever.”

So there’s a kind of, I think, you know, that classic thing of all of those tools, which is everyone starts using it all the time, and then they sort of… You start to work out its affordances, I’d say, if we were talking in sort of interaction design parlance, around what’s it good for and what isn’t. I think there’s a similarity to desktop publishing, to the rise of music-making programs, you know, b-b-prior to GarageBand, but those kinds of things, uh, Cubase and everything else, and we’re just sort of seeing another cycle of that.

You’ve got a… There’s that chapter two, you do a whole thing around, like, will assistants doom us to stupidity and, uh, the de-skilling thing. There’s a common argument that I see, at least on LinkedIn, of, well, you know, but it’s just like Photoshop or any other thing. Are you telling me that I should be kind of doing Gaussian blur by hand instead of getting Photoshop?

It’s just the same. I posit that it’s, it’s different in the, in the way it kind of de-skills. So what, what’s your line on that?

Chris Noessel: I have an anecdote.

Andy Polaine: Okay. Love an anecdote.

Chris Noessel: Uh, which is only that, uh, my first job… So, uh, I mentioned my job that I was working my way through undergraduate, uh, and what I was doing is I was doing prepress for a, uh, textile printer, um, so T-shirts mostly.

Yeah. Um, but it also includes some other things. And so prepress, when I got to the shop, I guess this is my third year of undergraduate, we’re still using phototypesetting, so-

Andy Polaine: I know what that is,

Chris Noessel: yeah. Almost, almost no one will remember what this is. But back in the day, right, Letraset was really, uh, was a printed sheet of letters that you could purchase off the shelf, and they came in these little plastic sheets that you could lay down on top of a sheet of paper and rub the top of the plastic, and it would push the letter onto the paper.

But that would create horrible baselines, and if you’re setting a ton of text, it was a nightmare. So- On about the ’80s, we had phototypesetters, which were complicated machines that had these transparent disks, each of which was dedicated to a single typeface, not even a font, but a single typeface. And you would have what was essentially a markup language.

You would sit and you would code that markup language into this ancient machine, and it would tell things like, “Here’s the size. Here’s the position of this body of text. Here’s the column width.” Just like CSS now. Mm. But then you would tell it to go. You would load up some photographic paper, and then a, uh, a strong light within the machine would…

Uh, it was a pinpoint light. Um, would… Uh, the machine would position that transparent thing, turn the large disk so that the right letter was in the right position, and it would expose the paper for just that letter. And so you, you would hear it going, “Di-di-di-di, eh, uh, eh, uh, uh, uh, uh.” I remember. And it would take, like, 30 minutes to an hour depending on how long the amount of text wa- or 30 seconds to an hour depending on how long the text was.

Well, very shortly after I got there, the desktop revolution happened, where we’re like, “Yeah, we don’t need to even worry about all that.” Um, and I helped, uh, those business owners. I was like, “Hey, there’s this nifty new thing called a Mac. Let me show you what it can do.” And suddenly those, uh, needs for typography, uh, the, the markup language, uh, was much less onerous.

And Even at the time, like when I was doing color type for my undergraduate poster projects, for instance, I was still having to cut it out of Letraset. So in modern discussions, when I hear people talking about, “Oh, uh, this is gonna obviate us,” I’m like, well, that job, phototypesetter, was a full-time job when I first entered the, the workplace.

It’s not anymore. And what did we do with those people? Well, they had to join the revolution and keep going. I think the, the thing that is different now is that the types of skills that we were t- that were required then were not generally applicable. Once you get to AI, the types of skills that you risk losing by constantly asking it to assist you are the cognitive skills.

Um, uh, what’s the word I’m looking for? Skepticism. Mm-hmm. Uh, even inquiry, structuring a mental map in order to pursue the answer to a question. Losing skills in those things, uh, mean that we are losing the ability to contain the thing that threatens us, which is like the AI that’s doing the, that’s threatening the de-skilling.

So those arguments that I hear, uh, AI is going to make us stupid, I’m like, well, they can, and that’s a very important and very timely thing for us to notice. But I believe that I have shown through the course of the book that That’s only if the AI is very poorly designed, and there are tools that we have that designers can pick up on and utilize in order to ensure that, well, mitigate against that stupefaction.

Andy Polaine: Stupefaction. I mean, I, I come back to music as the thing. I mean, the, the, the, the, the desktop publishing thing, I mean, I rem- my dad had an ad, ad agency, and so they used to have- Oh … finished artists who used to hand-letter and use Letraset and stuff, and I remember that sort of shift to the, uh, the tech you were just talking about.

And I remember the smell of, you know, uh, Spray Mount glue and, uh- Oh, my God. Wow. … uh, cow gum and, uh, Letraset that had a very specific kind of smell that kind of takes me back. But I th- I think that, you know, there were a explosion of really ugly kind of posters and flyers and stuff with it, where everyone was using like, “Oh, the, we’ve got 10 fonts.

Let’s use all of them on the thing.” Um- Yeah,

Chris Noessel: we’re paying for them.

Andy Polaine: Why not? Yeah. And, you know, and then, you know, design, actually people at some point realized that, oh, oh, actually that’s kind of, that’s not very good for quite a lot of things. But we still get plenty of, you know, in-o-office things that someone’s printed out in Comic Sans and, you know, the designers will roll their eyes at Comic Sans and, um…

But, you know, for some things it doesn’t matter, right, uh, at all. I think the, the nearer comparison, um, is perhaps in, in music where you get, you know, you’ve got a whole bunch of loops. You’ve got a, you know, something like GarageBand. You can make a tune using the preexisting loops in GarageBand that kind of sounds impressive, at least from production level, um, standards of things, but kind of doesn’t really have much thought behind it at all.

And then the difference with AI is its generative aspect of it, right, I guess, which is a sort of switches it from just being, I guess, an assistive tool like any of the other ones we’re just talking about that can help you. You no longer have to write that, that markup code to, to kind of produce the type, and you can just, it’s WYSIWYG and, and you can just do it and that’s, that’s…

But the, the, you’ve still got like an intent of what you’re trying to do behind it versus, um, the it’s done the thinking for me part, or it hasn’t thought at all because, you know, it doesn’t actually have a mind. But, you know, it’s, it seems to have thought for me, and I’m just gonna use this output, and I can’t articulate why the decisions have been made, have been made about that.

That would be for, at least for me, the line. I don’t know if in 10 years’ time everyone roll their eyes and go, “Oh, do you remember when Andy said that?” But does that make sense- … as a kind of divider for you or, or is it, as you’ve gone deeper into this, am I just drawing an arbitrary line?

Chris Noessel: It, it kind o- of is arbitrary.

Uh, I’m gonna try and remember. I was part of- A group when Midjourney first came out- Mm … and a, a suite of other generative image creators that is cringey to look back on now because I’m sure we used grotesque amounts of tokens and the environmental impact that that caused, but we weren’t aware of it at the time.

But we were all trying to figure out this thing of, you know, prompting images, and there was a, a very popular science fiction author who wound up joining the group for a short while. Um, and out of respect I’m not gonna mention his name. Um, but he joined for a while, even participated in some of the challenges, um, and then went off and wrote this really embarrassing, well, I think it was cringey, um, essay sort of talking about how ridiculous it was that anyone could use text to make images, and how it’s eroding the world of the artist and aren’t, you know, I’m a writer and I would never use AI to do anything like that in my writing.

And I’m like … A- and the, the, the whole thing of it was, well, I am an artiste and, uh, I don’t wanna black box anything about my art.

Andy Polaine: Yeah.

Chris Noessel: Which was just so ridiculous because I can assure you that this author black boxed marketing, black spox– black boxed publishing. He’s not in there making paper. He’s not in there-

Andy Polaine: By which you mean hand it off to someone or something else, yeah.

Chris Noessel: And you just trust that it gets done.

Andy Polaine: Mm.

Chris Noessel: Right? You rely on those expertises. And in a very similar sense, yes, there was a time when if you wanted an image made, and here I’m just gonna go for a, a ready to hand example. My older daughter, um, plays Dungeons and Dragons. And does she- Mm-hmm, me too … prompt character, characters or monsters or scenes?

Occasionally she has. And there was a time when in order to get that, she would’ve had to either have a friend who did, did art and she could request it, or she would have to go and study it, or she would have to hire somebody. And those, that’s just not, not a market she’s gonna get into if she’s not interested in the art.

She’s not gonna try. So yeah, just generating that thing, she can treat that like a black box. And there, you could argue, well, that’s an illustrator who didn’t have a commission, but I assure you that there is no world in which she would’ve paid an illustrator for that same thing. Yeah. It opened up a, kind of a new market.

But so A, I do think that, uh, black boxing, to use, coin that term, um, it can be okay in some cases, and the question that you’re asking is, well, what is the dividing line? Um, two, three, three weeks ago, w- uh, Claude Design came out. Was it three? Three or four. Yeah, something

Andy Polaine: like that. Around then. Yeah. I’ve already seen job ads for expert in Claude Designs.

Like, really?

Chris Noessel: Are

Andy Polaine: you kidding? It’s, it’s four weeks old.

Chris Noessel: Seven years required experience. Yeah,

Andy Polaine: exactly.

Chris Noessel: Um, and the, the first thing I realized when I used Claude Design, ‘cause I was throwing a problem at it just to see what it did and how it worked, was it sat and chewed on the problem that I presented it for a while and then dumped a bunch of screens on me.

And I had this dumbfounded, stupefied moment of like, I cannot… I have no hook for understanding this, for appreciating w- what are the design trade-offs that got us here? Why is this good? And because we’re user-centered designers, why is this good for the sort of people who would be using it? Was completely absent.

Um, and- I mean, I had a great deal of sympathy for our clients or stakeholders who sometimes get just dumped screens on them, and without, you know, the hand-holding necessary to sort of n- explain why it’s good, why it’s good for users. What I wound up doing with Claude Code was I taught it about pair design, and I’m like, “Okay, don’t just dump screens on me.

That’s stupid. We’re going to pair design this thing step by step, going from user needs and technological constraints through scenarios.” And over the course… And it did it, and it started doing pair design with me, with me as the generator.

Andy Polaine: Which w- we should say this is also something you co-wrote a book on.

Chris Noessel: Oh, yes, I did. I co-wrote a book with Gretchen Anderson on pair design and- Yeah … did it, uh, at that con- uh, San Francisco consultancy for g- about 10 years. So- Yeah … uh, I’m very versed in it. And once I trained Claude to do that, well, suddenly I had a hook for understanding what it was we were designing. And that experience led me to believe that, again, it’s not just that you’re handing off You know, the cognitive work to make the thing, you’re handing off the cognitive work to understand the thing that is made, and that it is imperative upon me as a designer to make sure that that doesn’t happen to users.

Yeah. Because understanding is not output, and we just… Output machines and thinking about these things as output machines is where the fall is going to come. We have to understand that humans are understanding machines, and that understanding is critical to the outputs being successful in the present and in the future.

Andy Polaine: Yeah. So I would argue that in that situation, you pushed Claude design from generative to assistive again, in that you sort of went i- in the, in the first list, you know, you prompt it and it produces a b- Mm. … a bunch of stuff, and in the second one you’re like, “No, hang on, let’s do this together.” Right? And so you k- you kind of nudged it back into, “Don’t just sort of make some stuff for me and then that’s that job done,” it’s w- “I want you actually to be my, my pair designer, uh, on here and work alongside me.”

Is that, is that a fair call?

Chris Noessel: Not only is that fair, but it’s, it kind of, uh, brilliantly frames… Oh, oh, I want to apply it to like something like Midjourney, which like what would that experience be if you were art directing it as opposed to prompting it?

Andy Polaine: Yeah.

Chris Noessel: Like could it do… Or, or even to your filmmaker background, right?

If I was not just prompting Sora, but what if I was able to talk about the characters and the scenes and see… Oh, what do they call those drawings, uh, that describe how a sequence of film is gonna go? And

Andy Polaine: you- Oh,

Chris Noessel: it’s the

Andy Polaine: storyboard you mean? Uh

Chris Noessel: Yeah. Like you iterate the storyboard, and then you’re like, “Okay, now I’m confident that this scene is gonna go well, the character’s gonna do, and- Mm.

and the scene’s going to do what it needs to do narratively. So now let’s generate that video.” Um, that would be a really much more fascinating way to see those things work.

Andy Polaine: So on the assistant stuff, you talk about the five universal assists Um, so it seemed to be a good idea to, if you could briefly take the, you know, I know it’s the sort of 10,000-foot view of this-

Chris Noessel: Yep

Andy Polaine: but, uh, take us through what they are.

Chris Noessel: [00:28:00] Uh, happy to. Uh, and it points towards a shared model that informed both books, the Agentive book and the Assistant book, and that is the See, Think, Do model of interaction design.

Andy Polaine: Mm-hmm.

Chris Noessel: Mm-hmm. And I saw this, I believe, in 2013 at IXDA in Ireland, and I forgot to write down the person who talked about it, because as soon as she did, I was like, “Oh, yeah, this must be something that everyone knows.”

Uh, and since then I’ve looked for it elsewhere, and I can’t find the source. So if any of your listeners know this source, I need to– I owe her a great deal of debt, uh, in coining it. Anyway, the See, Think, Do loop is the atomic element for interaction design, which is, um, in a interaction, a series of steps, user sees something, they, uh, you know, some state of the system, thinks something about, “Okay, what, what am I seeing?”

Uh, then says, “Okay, great, what I want to have happen is this.” They make a plan. They execute that plan. They do it, and then the system takes it over. It becomes input on their end. Uh, they do some processing, and then it becomes output, and then whoop, look, the system’s changed again. Back to the see, think, do.

See, think, do. And that framework informed a lot of the patterns that I had named in the Agentive book, and in the case of assistance, it’s also the framework by which we can understand what those five universal assists are. Because if that is the atomic element of interaction, that’s not gonna change, um, until, you know, humans evolve differently.

So the universal assists, uh, tie, not one to one, but they break out. Uh, the first is seeing is perception. Assistants can help users perceive things, uh, in many ways, and that can include just the, the basics. A favorite example of mine is I have a profoundly colorblind friend who can’t see color, and he has an app called Red Stripe that he uses to dress himself in the morning that adds stripes onto the colors so he can go, “Ah, yes, right-leaning dark bands on this shirt match the, uh, right-leaning dark stripes on these socks, so I’m gonna put those two together.”

Um, then when you get into thinking, it breaks up into two parts, which is understanding what you’re seeing, um, and that can be just a quick reminder of this is what that part’s name is in the assembly that you’re working on. Um, or it could be, I need to understand the processes that drive why, why I’m seeing what I’m seeing.

Then the second part of thinking is planning. Okay, now that I understand what that signal meant that I perceived, what next? What are my options? Even down to, well, hey, AI, help me predict which of these options is going to be the best for my goal. Then there’s execution, and get into lots of sports things, weirdly enough, with execution, because assistants tend to involve complex actuators, so golf clubs that can help you with your swing, or electric cars that can keep you in the middle of a lane.

But it does include some software things as well, and I talk about those in the book. And so those four emerge from the see-think-do loop in a tactical level. At a strategic level, the fifth one is a reflection. I’ve read lots of Great esoteric seeming sources say that reflection is the number one differentiator to improvement over time.

How did that task go for you? Is there a different tack you would take on that task? Is the goal that drove the task still relevant, or do you need to rethink those goals? That kind of reflection is something that any system can help with, and that an LLM is pretty good at d- even helping a user work through that reflection.

And in so doing, they’re better prepared for the next set of see, think, do loops. So those are the five. In short, they are perceive, know, plan, execute, and perform.

Andy Polaine: Nice.

Chris Noessel: Oh, and reflect. Yes. Perform and reflect.

Andy Polaine: I, uh, for my PhD actually on interactivity, and so I, I did a sort of similar loop, which was about act, react, and interact.

You know, that was the kind of loop for me- Oh … that something… And you could start at either one, right? Either it was initiated by someone, or something would sense something and react. And then the human would then, you know, react, and you, you have the conversation basically, the interactive conversation, then it goes in that, in that loop, and then that’s the interaction.

As we all know, you know, we’ve become much worse, and there’s a- there’s quite a lot of, um, way finding and sort of map stuff in your, your book, ‘cause I know that’s been some stuff you worked on. You know, a lot of us have got much worse at finding our way around the place, ‘cause we slavishly follow the, the, the Google Maps and, you know, and sometimes end up, you know, there’s all those cases of people ending up in swamps and, you know, all sorts of weird things ‘cause they, they, they followed the map.

And, uh, in fact, I was in… I live near Strasbourg in France. I live in Germany, but I was in France yesterday, and, um, there were some tourists, American tourists, and they were trying to find their hotel, and there were– there was a, a, a noodle, a ramen noodle shop that we we were sitting outside. And everyone else was going, “No, look, it must be another…

Look at the numbers. This is a different number.” But the dad was, like, looking at his phone going, “No, it must be here.” And the- … and his son was like, “No, that’s number twenty-three. We’re looking for twenty-seven.” He said, “No, no, but it says it’s here.” And it was really funny, ‘cause, like, it’s so obvious. You’re not staying the night in a, in a ramen noodle shop.

Um, so there’s a, there’s a, uh, part three of the book’s all about kind of mitigating the risks and relying on it too much, and we talked about it a little bit. But you’re not just sort of out and out kind of boostering AI here. There is quite- No, no … there’s quite a sort of cautionary tale thread throughout the book actually here.

So, um, what would be some key principles for that, would you say?

Chris Noessel: So to answer your question, the key principles are certainly awareness, awareness that there is a problem. Um, de-skilling is a thing that can happen. Assistants cause it Over time and over-reliance is the thing that can lead to that de-skilling.

So principle number one, be aware that these things are risks and that there are trade-offs that users and organizations and society may not want that trade-off. Part two is, oh, what are the things in my backpack that I as a designer can use? And that is a kind of thing in the literature which is called a cognitive forcing function.

And it’s an ugly term, we’re stuck with it, but it means, hey, how do I break the habitual behavior of users and get them to stop and seriously think about what’s going on? There are I think about seven that I’ve captured in the book that I found in my survey of the literature. But because I’m also talking about de-skilling and not just over-reliance, I encourage readers, designers, and product managers to favor one of them called the human goes first pattern.

And there’s even a, like a handy little flow chart that you can find in the book all about it, and a description. Um, but the main thing about why I encourage people to privilege that cognitive forcing function above all others is it’s the only one that helps encourage practice, and practice is the thing that combats de-skilling.

Andy Polaine: Mm.

Chris Noessel: So by occasionally asking users just to pause, go first, practice that skill, remind yourself about how it is that’s supposed to be done and the way human judgment works here. Now compare your answers w- with the AI and do smart things with it. And actually it’s th- the moment where that cognitive forcing, where the user fails at that task, um, whatever task you’ve given to them to go first, where learning is, uh, the learning moment happens.

Like, wait a minute, how could I have failed at this task? I’m a professional. So that cognitive dissonance, that discomfort is the right moment for micro-learning, and so that’s what the human goes first pattern leads to.

Andy Polaine: Could you give me an example of it?

Chris Noessel: Uh, yeah, yeah, yeah. Um, there are two in the book, so those are ready to hand.

Yeah, yeah. So I’ll grab those. Um, the first is, uh, when I was at IBM leading the design for AI Guild, I also ran a small team that was looking into over-reliance, and we created a mitigation for Google Maps that we tested with a test group and a control group. And with the test group, we gave them, uh, a virtual commute that they did for four days, and then we took the phone away and I saw how well they could do, perform the commute without the, without the guidance, the little blue line.

Mm. Um, and of course there was a performance collapse, and it was to the tune of forty-eight percent-

Andy Polaine: Wow,

Chris Noessel: okay … um, as measured by decision-making time. It was a big performance drop and is sort of what we expected to find. And with the test group, we were like, well, let’s see if we can get that number close to zero.

And what we did with the human goes first was we lost the blue line and users were asked, we were, they were given the goal and asked to find their way, but they had an AI to assist them, not to think for them. So when they got to an intersection, they didn’t see the blue line, they had to make a decision.

Once they made the decision, then they got post-decision feedback. I took a left and this will get me there, but it, it’s gonna take a little bit longer than the ideal path. Users would get one kind of feedback, uh, it was a yellow stop sign, um, uh, along with a little bit of text. If they took a turn that was unrecoverable, you are going to have to take a U-turn to recover your route, they got a red stop sign with a little bit of red text.

And if they were on the ideal path, they got no feedback whatsoever. They’re like, “Oh, I must be doing pretty well So our test group had two days with regular Google Maps, an opportunity to sort of feel the route. Then two days with this modified Google Maps where they got the post-decision feedback, the human went first.

And then on the fifth day, we were like, “Oh, we’re gonna yank that away and see how well you do.” And the, this was the big number that shocked everyone, because as you’ll recall, we were hoping to minimize that performance collapse from forty-eight percent to zero. But we actually saw a performance gain on the fifth day of nineteen percent- Hmm

um, which meant that in only two days of using the human goes first intervention, users had internalized the map well enough that they were losing time double-checking the reference. So that’s one example.

Andy Polaine: Wow, okay. So then yeah, I get the, you know, I get the learning bit of that. I mean, ‘cause this is obviously how you learn, it’s how you practice.

If anyone’s a musician- Mm-hmm … you know, you do the thing where you, you can hear yourself play a wrong note, right? So you, you do the thing where you kind of slow it down and, and then you, you gradually speed it up. And part of the thing is to avoid… Like, if you keep playing it fast wrong, you end up learning the mu- you know, the muscle memory of, of the wrong notes.

And so you, you- Right … slow it down to sort of, uh, and, and practice. And that, that kind of feedback loop of practice, uh, is, is really important. And I guess that’s, you know, I should probably say I don’t play golf, but I imagine it’s the same for your observing your golf swing and getting a thing saying, you know, too much to the right or whatever.

And, and you just gradually, um, you know, improve over time.

Chris Noessel: I was in, uh, theater for a little bit in my undergraduate time, and learning monologues, very similar.

Andy Polaine: Mm.

Chris Noessel: You would go and you would go, “Ah, I stumbled on that word, and if I keep going, trying to recapture it, I’m going to lose it. So nope, gotta go back.”

Back to

Andy Polaine: the beginning, yeah. Start again. Gradually extend outwards, right? From-

Chris Noessel: Yeah, and extend outwards.

Andy Polaine: Yeah. I’ve, I’ve heard that technique too. Um, we are– Uh, we could, we could go on about this for ages, but we’re- … we’re unfortunately, uh, up against time. So there’s two things. This is the very final question.

But there’s a, there’s a thing… I think they’re related, but maybe, uh, maybe you see them as separate things, what you talked about beforehand. One of them was sort of organizational over-reliance. Now, and I think, you know, on the organizational level, I’ve talked about this thing sometimes called, um, organizational amnesia, where, you know, there’s a whole initiative and then that there’s a reorg and McKinsey come in and kind of fire a bunch of people.

And then the kind of neural path or the memory of that stuff just goes away and fades because it’s not practiced. That’s the stuff we were just talking about. But I wonder, we also, you know, obviously at the moment there’s this big push from a lot of organizations to be AI first and AI native and whatever that means and, and sort of AI everything.

And obviously as we know, right, you know, as we speak right now, there’s thousands of people laid off in different organizations, uh, Meta and Intuit. And- That there is a… I mean, I would posit that there’s the kind of same thing is going to happen to those organizations that they suddenly would have lost, you know, part of their memories.

Literally like, you know, for if people know “2001” of kind of pulling out the memory banks, right? And gradually it’s starting to kind of fade and slow down.

Chris Noessel: Yeah.

Andy Polaine: But there’s also this other thing of the conflict of bullshit in AI that we talked about, and I kind of, it sort of feels they, they come together, the kind of corporate thing.

And, um, I even saw a piece, there was a study the other day of people who use corporate bullshit and jargon more, uh, perform worse, which I thought was kind of really fascinating, um, because it’s kind of really like just covers up under- real understanding, right?

Chris Noessel: So- Yeah. Yeah, yeah …

Andy Polaine: let’s talk about bullshit and AI because it is also a very confident bullshitter, right, sometimes.

Chris Noessel: It is a confident bullshitter. I mean, like the term stochastic parrot is still so fantastic, and thank you Emily Bender for giving us that term, um, because it is a stochastic parrot. Um, now it’s a very sophisticated stochastic parrot, and the larger the foundation model, you know, the more it matches reality.

But that’s the thing. It’s just like a thing we’ve built to closely track reality, but it can’t know reality. It doesn’t have the subjective experiences that we have, so, uh, and that we build up over the course of a lifetime. Anyway, anyway, anyway, um, every mitigation that I list in the book, every cognitive forcing function that I list in the book presumes that the user, w- or rather the organization providing the software that the user uses, is vested in not bullshit And, and here I’m using the, the HG, Har- Harry Frankfurt version of that term, and I believe I read the article when you sent it.

Um, I, I, in full disclosure, I skimmed the article when you sent it because I haven’t had time to read it. Uh, but, uh, Harry G. Frankfurt in his treatise on bullshit said that bullshit is not a lie. A lie is when the user knows the truth and they have a benefit to gain from not saying that.

Andy Polaine: Mm-hmm.

Chris Noessel: Bullshit is just saying something for its effect.

It might be right, but they don’t care. All they want is the effect of what they’re saying.

Andy Polaine: Yeah, there’s absolutely no respect for the truth at all, right, yeah.

Chris Noessel: No, and that’s where the sort of danger lies. Um, and AI are fantastic bullshitters in that regard. We all know that large language models were system prompted into being sycophantic for a while, and they’ve largely undone that, but the effect was, oh, hey, users like it when they are flattered, therefore make the AI flattering.

[00:43:00] Just horrible amounts of bullshit. But when we look at it from the user’s perspec- perspective, if there is somebody who does not care about what they’re– whether what they are doing is true, and I realize that’s a load-bearing word, or whether it is just effective, is a huge problem when it comes to, uh, using AI and abusing AI and abusing the output of AI.

Because is this adding technical debt? Sh- sure, but hey, it raised our stock price for a quarter. Um, we all know the story of, uh, Google, and I don’t remember the VP who decided to put paid results up in front of real results, and now another VP, could be the same one, who put the AI results in front of real results.

Mm. But that certainly raised their stock price for a couple of quarters and has utterly ruined its credibility for users, right? Yeah. You now have to go several pages deep to get to the real references. I mean, the AI is pretty good sometimes, but nonetheless, um, it, it is a, it was a bullshit moment, or if we can borrow the Cory Doctorow term, the enshittification moment of AI.

So i- it is a big problem, and I don’t know that we have good answers for it. Uh, one, one tiny last bit is that, um, I sp- I spoke with Zhanna Buchina– Buczynska over the course of writing the book, and one of the things that is a dark fact out in the world, and please, listeners, if you can do some study to come up with this, let me know.

But we don’t know how to help people who want to over-rely Or in the parlance of, uh, the literature who are low need for cognition, meaning they don’t like thinking. We don’t know how to get them to pay attention, to think through stuff like their job may be requiring of them. Um, and it’s pointing to a future where if all of work needs to be AI augmented, then we have a category of person who will be pushed out of jobs, and that’s a scary future.

Yeah.

Andy Polaine: Mm-hmm. So,

Chris Noessel: so far we don’t have an answer for it, and, uh, we need to get one, so…

Andy Polaine: I mean, one of the other things of it is the economics of it don’t stack up. I mean, I think it’s pretty clear that, you know, the big players are not going to earn the money back that they need to in the timeframe that they have, uh, said.

So, you know, they will get– at the very worst, there’s, you know, there’s gonna be a massive collapse. Um, but also- Mm-hmm … even at the best, these things are gonna get very much more expensive and gentrified. There seems to be no kind of way out of that, um, because they’ve trained everyone to be– use it very frivolously, and really it pr- should cost, you know, at least 10 times as much as it’s currently costing people.

Which I wonder- Yeah … if it will be a bit of a forcing function for, for some people or some organizations to go, “Oh, actually You know, it’s not really worth doing that thing in AI anymore. Literally, it’s not econo- it’s, it’s not worth spending the money on it because it’s, you know, a prototype that was forty dollars is, “Oh, yeah, okay.

Well, you know, we did it quickly.” And for the prototype that costs four hundred when you’ve got sort of thirty people on their own prototyping because it’s also they’re not very collaborative tools, uh- Yeah … or not at all at the moment, uh, it starts to just cost organizations a lot. And I wonder if it’ll, if it’ll force, uh, people to use it in a more thoughtful and reflective way or whether I’m just, you know, uh, absolutely having illusions about that.

And, uh…

Chris Noessel: Well, we’re both parents, and I don’t know if you had this experience with your kids, but there is a phase, and I suspect s- most people go through a phase, where you try lying, right? Yeah. When you’re three years old, you try lying, and eventually you learn that there are a lot of good reasons why that tactic is self-destructive and destructive.

And I want to believe, because I’m a reprehensible optimist- … that culturally that is our m- this is our moment of literacy, where right, we’re using this machine that just bullshits occasionally, and people are… we’re going to have to learn generally, and it’s gonna take some companies failing, uh, you know, jobs lost, where we just realize, oh, maybe the bullshit isn’t the right way to go.

Hopefully, hopefully, that’s what happens.

Andy Polaine: No. Okay. Well, we are now definitely at the end of the show. As you know, it’s named after the Ray and Charles Eames film, uh, Powers of Ten, all about the relative size of things in, in the universe, and my one last question is, what one small thing is either overlooked or could be redesigned that would have a outsized effect on the world?

Chris Noessel: So this is a tiny soapbox I get on occasionally, so I’ll get on it again for this , uh, and that is it’s a super boring but, uh, super critical thing. So the world is facing climate change, and we’re all largely concerned about carbon dioxide, but there is a chemical out in the world on which civilization depends, uh, or a class of chemical called hydrofluorocarbons-

Andy Polaine: Mm-hmm

Chris Noessel: um, that largely powers air conditioning that, uh, you know, ounce per ounce Is 1,000 to like 9,000 times worse as a greenhouse chemical-

Andy Polaine: Mm-hmm …

Chris Noessel: uh, that’s g- that’s worsening climate change. Um, and because it’s in air conditioning and cooling systems, it’s in machines all over the world. So they can be disposed of where inevitably in landfills they leak and contribute to global warming so, uh, and climate change so much more than the other measures we’re taking.

So the simple design change is to do things like with air conditionings, you bolt that to the foundation with clear instructions, um, that this has to be handled by the government so it just doesn’t get thrown away.

Andy Polaine: Hmm.

Chris Noessel: In cars you could bolt it to the frame, but some design that makes it more difficult to simply dispose of would help us contain that particular chemical that is doing so much damage to the entire planet in nearly unimaginable ways.

That’s it. That’s

Andy Polaine: the one. Excellent. Thank you very much. I… It’s interesting with HFCs because there was a… You know, as people who are a bit older will remember the, the hole in the ozone layer, and there, there was international movement to, to ban HFC. They, they were in like spray cans and stuff like that all the time as well.

Chris Noessel: Oh, yeah, yeah.

Andy Polaine: Um, and you know, deodorant and things for people who don’t know. Um, and, and it, and it actually worked, you know- It did … and it, it fixed the hole in the ozone layer. So, you know, it was one of the few examples of, of… And I wonder why, and I guess it was the… whoever was in governments at the time, but it was one of the things where there was an interla- international cooperation, uh, to ban them and, uh, and it actually had the effect.

Chris Noessel: Well, I… Also that, that, uh, that memetic phrase, hole in the atmosphere-

Andy Polaine: Yeah …

Chris Noessel: in the ozone layer, is seems terrifying, and it should’ve been.

Andy Polaine: Yeah.

Chris Noessel: And it was- Yeah … and it did work. Yeah, yeah So the fact that we still need cooling and still use HFCs, it’s still a problem, it’s just that there’s no obvious gaping wound in our ecosystem- Yeah

like a hole in the

Andy Polaine: ozone. No, but it was like, it was like, you know, acid rain and you know, all that kind of stuff. I think we, we need to get some of that, that language back maybe.

Chris Noessel: Yep.

Andy Polaine: Uh, where can people find you online?

Chris Noessel: Uh, I used to be on Twitter before it became the Nazi hellscape that it is, so now you can find me, um, very occasionally on Blue Sky, but more often for stuff like this I’m using LinkedIn.

Andy Polaine: And your own website? I will put things in the, the show notes.

Chris Noessel: Uh, I don’t really have one. Um, I still publish to scifiinterfaces.com, um, but that’s obviously a very niche topic. Um, and otherwise, yeah… Oh, Medium. I have a Medium feed that people might be interested

Andy Polaine: in. Okay. Wonderful. Chris, thank you so much for being my guest on Power of Ten.

Chris Noessel: Of course. Thank you. It was delightful to be here.

Andy Polaine: You have been listening to Power of Ten. You can find more about the show on polaine.com, where you can also check out my leadership coaching practice, online courses, as well as sign up for my irregular newsletter, Doctor’s Note. If you have any thoughts, please put them in the comments or get in touch.

You’ll find me as Andy Polaine on Blue Sky, YouTube, LinkedIn, and of course my website. All the links are in the show notes. Thanks for listening and I’ll see you next time.