Josh Clark & Veronika Kindred - Sentient Design

Josh Clark & Veronika Kindred - Sentient Design

Podcast Power of Ten

My guests in this episode are Josh Clark and Veronika Kindred, co-authors of the book, Sentient Design: Crafting Intelligent Interfaces with AI, published by Rosenfeld Media.

Josh Clark is principal of Big Medium, a digital agency that helps complex organizations design for what’s next. Josh has over 30 years of experience in emerging technology, user experience, and design innovation. His projects include future-friendly interfaces for AI, connected devices, and websites for many of the world’s biggest companies.

Veronika Kindred is designer and researcher at Big Medium, where she defines and solves design problems alongside some of the world’s biggest companies. She travels internationally to lead Sentient Design workshops and speak to teams at startups and Fortune 100 companies alike. Her research has ranged from AI’s effect on user experience to mobile technology’s impact on African political engagement to the nuances of congressional climate hearings.

Our conversation ranged across the history of interaction design (Zork, MUDs, branching narratives), Josh and Veronika’s “four postures” of AI systems (tools, chat, agents, co-pilots), the dangers of leaning too hard on physical metaphors, the idea of “casual intelligence” as a counter to AI maximalism, and the responsibilities designers have right now to shape how this technology gets implemented before money and Big Tech make all the decisions for us.

Listeners can use the code SENTIENTPOT for 20% off the book at at rosenfeldmedia.com from June 9th to July 9th.



Transcript

Note: This transcript is machine-generated and may contain some errors.

Andy Polaine (00:09.966) Hello, welcome to Power of Ten, a show about design operating at many levels of Zoom, from thoughtful detail through to transformation in organisations, society, and the world. My name’s Andy Polaine, I’m a design leadership coach, educator, and writer. My guests today are Josh Clark and Veronika Kindred, co-authors of the book Sentient Design: Crafting Intelligent Interfaces with AI, soon to be published by Rosenfeld Media. Josh is principal of Big Medium, a digital agency that helps complex organizations design for what’s next.

Josh has over 30 years of experience in emerging technology, user experience in design innovation. His projects include future-friendly interfaces for AI, connected devices, and websites for many of the world’s biggest companies. And a thing I did not know, in 1996, Josh created the Couch to 5K, the C25K running program, which has helped millions of skeptical would-be exercisers take up the running. We can talk about this afterwards.

Josh Clark (01:04.782) I’m I’m meanwhile going from 5K to couch is seems to be my trajectory.

Andy Polaine (01:09.08) That’s the new program. Yeah. Veronika Kindred is a designer and researcher at Big Medium, where she defines and solves design problems alongside some of the world’s biggest companies. She travels internationally to lead sentient design workshops and speak to teams at startups and Fortune 100 companies alike. Her research has range from AI’s effect on user experience to mobile technology’s impact on African political engagement to the nuances of congressional climate hearings.

And she’s also Josh’s daughter. Sorry, Veronika. No escaping that one. Boswell, welcome to Power of Ten.

Josh Clark (01:44.066) Thanks, Andy. So good to be here.

Veronika Kindred (01:45.624) Thank you so much. Thank you. I always know that introduction is coming, the father-daughter stuff. And it gets me blushing every time.

Andy Polaine (01:54.302) yeah, I can see. I can see. you know, it’s it’s a nice thing, we’ll talk about it in a minute actually of of the experience of writing a book together like this. But first of all, I’d love to hear a bit more about yourself and your journey, as I’ve given you a bit in the in the intro. Veronika, tell me about your journey from where you are now or from where you were to where you are.

Veronika Kindred (02:16.406) Yeah, so I I studied political science and data science when I was at college at university. and I interned for Josh one summer and that was fun. And then when I when it was like about time to graduate, I was kind of thinking that it would be a really great career to embark on. You know, I enjoyed my summer internship.

at the at the family business. And so I was excited about continuing. so I started right after I graduated. That was in 2023. So this was also right after like OpenAI had dropped ChatGPT. That was the fall of 2022. So my the end of my schooling had LLMs as a big part of it. And certainly

my entire career has had LLMs be a big part of it. and so yeah, I’ve just been doing client work and then we started writing this book about two years ago. So that’s been my career thus far.

Andy Polaine (03:20.494) Okay, so so there was actually gonna be a question about this, which was the book is about to come out, right? It’s it’s imminent. we’re recording in the beginning of May, twenty twenty six. and so you started writing it in twenty twenty four, did you?

Veronika Kindred (03:36.364) Yes, yeah. I feel like almost maybe June twenty twenty four, almost exactly two years. Yeah. Right.

Andy Polaine (03:40.972) Okay. Now we’ll we’ll come back to that point, I think. and Josh, you are well known on the interwebs. but tell us a bit more about your journey.

Josh Clark (03:51.032) Well thanks. Yeah. I I’ve been doing interaction design for about thirty years, as you mentioned, and and running Big Media and the digital agency for twenty five of that time. And so Andy, you know, you and I are of there that kind of came up together. And when we were Veronika’s age, this thing didn’t really exist as a career as we know it. Obviously, HCI did and but it’s just sort of it is a really different animal.

And so I think my career has been about trying to see the coming and emerging waves and helping organizations adapt to those. So first it was around the web and the web two dot era and then profoundly mobile through kind of the DevOps revolution of design systems and now into AI. And, you know, I think there is a good and a bad of having

All of this experience. Veronika is very fresh in her career, obviously, three years in right now, compared to my 30. And Veronika, you don’t have my experience yet, but you’re also not burdened by it. and I think that at moments like this, when we have these scenes where there are new interaction paradigms to explore, in some ways, the best practices and

Norms that we’ve adopted from previous waves sometimes blind us to what’s possible. So I could not have written this book without Veronika, because there were constantly times where it’s like, wow, this breaks the mold. Hang on. And she’s like, this is how software always should have behaved. Let’s push farther. You know, and I think this difference of coming up with algorithmic feeds, with machine learning mediated interfaces, just

You know, growing up with that, but then coming into the field with that as an expectation brings a freshness and perspective that I as a veteran really needed. I think this is a moment, these kinds of moments like we’re confronting now, where design is changing and the form of what we make is changing really demands new perspectives. And I think one of the cool things is about this particular technology is that it’ll really allow and encourages that.

Andy Polaine (06:16.242) Mm. Yeah. I d I mean I yeah, we are we are kind of peers in that sense of, you know, when we started the roles weren’t defined. You know, I was sort of a multimedia person. I re I should remember calling myself an interactive designer. but there wasn’t that, you know, interaction designer, I I think IA was one of the first things that emerged actually as a kind of this is a a discipline. But you know, there and there was a graphic designers who were doing the the UI stuff and programmers, but it had

It was still evolving. One of the things I really enjoyed about that era was the kind of shared exploration of like what are the affordances of of this medium, right? Not not of of this particular widget or the interface thing, but what what does it mean to be able to interact with stuff now? and it kind of went away as I guess digital and product design sort of became operationalized. Veronika, when you you obviously bringing that but it reminds me actually of my my niece who’s who’s from

I shouldn’t know how old she is, but she’s 29, 30. And I remember her being really surprised when she saw a telephone that had a a cable. Because she’d grown up with cordless phones, like they were sort of cordless, like fixed line phones, but cordless phones, or mobile phones. And so the idea that you should have a phone that’s got a kind of curly wire going to it to it was kind of really weird for her. I remember this when she was kind of very young. Is there anything from, you know, when Josh pulls out his his

His old photographs of when the internet was black and white. D was there anything that you sort of heard and thought, my god, that’s that’s really interesting. I did not know that.

Veronika Kindred (07:47.402) Hmm, that’s a great question. You know, writing this book, I did kind of do like a lot of research just into like kind of the history of AI and also like some of the history of design. I I think that this is probably older than Josh, so I don’t want to date him to this, but just the the traditional command line interaction, just like going in terminal, I I think it’s so

unintuitive and so cumbersome. And obviously like it came from a place of need and it was wildly innovative at the time. but I think it now technology is so accessible from basically any standpoint, whether it’s voice to text or a cursor or using your gaze, or there’s just so many ways to interact with and kind of get what you want from a digital interface. I think just like going back to the very start of it, it’s it’s

One, I think amazing to see like how far it’s come in a relatively short amount of time. And also, the kind of I I I think maybe we take for granted now how easy it is to interact across so many modals.

Andy Polaine (08:56.086) Yeah, no, that’s very true.

Josh Clark (08:57.132) And and now here we are kind of going back temporarily, I presume, to really kind of command line interfaces is somehow sort of the state of the art question mark. I think that there’s like so much opportunity beyond that. But I mean, like Veronika, we were like looking at like Zork text adventures games from the seventies, which was my, you know, which was my kind of childhood sort of first games. And like

we were struck looking back at it at just how hard it was for the system to understand what needed. Had to use very brittle, you know, if if you didn’t say get hammer and you said pick up hammer, doesn’t know what you’re talking about. So like part of the game was like figuring out what word to use. And it’s like, look at how we’ve fast forwarded now to these systems effortlessly understand intent, no matter what slang or language you use.

Andy Polaine (09:33.72) Yeah.

Josh Clark (09:57.036) That problem that was so hard and vexing for decades. Now it’s like, wow, LLMs just get what you mean, certainly with sort of text language, but can also infer a lot from other types of context too, which is groundbreaking in what becomes possible for interaction design.

Andy Polaine (10:18.456) Sure. Yeah, so there was a bit actually in where you talk about ephemeral experiences and then you you know and and I was thinking, you know, which is these things that they well maybe you can describe what ephemeral experiences are and then I’ll I’ll carry on with my point after that.

Josh Clark (10:33.358) Sure, yeah. I w when we talk about ephemeral experiences, it it is, you know, an experience that’s invented for the moment and may never be seen again. So it’s something that, you know, I think that we’re seeing some of these things, and this is actually how we open the book, is with a so-called world model, which has strong implications for games, certainly. But what we’re talking about is see the model with an image and a prompt, just a static image, and it can create an entire world that you explore.

Invented frame by frame. So until you turn in this direction, that part of the world didn’t exist and it won’t exist for anybody else. And so there’s like this, and it feels totally natural for you in the moment. You may not be able to find your way back there again, because it’s so ephemeral and invented for the moment. And you may not be able to share it. So I think one of the things that’s interesting is like, this kind of thing is possible. What if any type of digital experience

Could be created this way? What if websites were generated this way? But also when it and I think that there are obvious utility to that. It’s like, the thing that I want and need in the moment just appears. There’s also problems with it. You know, so when is this going to be something that is ungrounding? When does it block collaboration? And so with all of these things that are possible, you know, I think we have to kind of say how how much do we crank the dial? Yeah. And what are the contexts and moments to

use this awareness and radical adaptability that’s now possible so that it is actually tuned to the moment and feels intuitive instead of like a robot fever dream just rolling out randomly and

Andy Polaine (12:15.95) Do robots dream. of electric sheet. So d the the I think it’s a good time to to say actually the book is not a book of like how to use AI in your design process and in your workflow and stuff. The book’s really a th a very, very thoughtful look at w what does it mean to be designing these things and for these things and using these elements in interfaces and in interaction paradigms rather than, you know, this is how you get Claude to

you know, pre bake your stuff.

Veronika Kindred (12:47.306) Yeah, that’s right. There’s been like so much of a focus on efficiency and process in in bringing AI into people’s works and and that into people’s work and that makes a lot of sense. It’s you know, a lot about it it’s kind of like the the first level I think is like, okay, great, now we have this very powerful tool. How can we do things faster or easier and and just that huge focus on efficiency? which is not particularly

Creative. Like it’s a very engineering focused view on integrating AI into work. It’s very much like a kind of input and output based scenario. And so what our book is really about is focused on the new kinds of experiences that people can create and the new kinds of products that are possible, the new kind of interfaces.

Andy Polaine (13:36.302) Yeah, this is what for me reminds me of those early days of in of interactive media is it is it’s a kind of it’s got two sides to it. It’s a sort of double sided thing, which is it’s not just these things help you create the stuff, which is what all the sort multimedia programs like Director and Flash did, but also it’s embedded in the stuff you put out there. And so and what does that mean? Why it’s it and why is it important? And I think it’s a, you know, really, really important the conversation to be having.

Because generally it’s like, now we can do more of the same stuff we’re already doing faster. And that’s I feel like a massively lost opportunity. Because it’s a different paradigm. Yeah. Yeah. I want to come back to the inframeworld experiences thing though, because there was this thing that you when I was thinking about it, I had the same video game thing as you. And one of the things that we all did so I I studied film and video, I wanted to be a film director originally. And one of the things we all did was try to make interactive stories straight away when we were using director and

And you get this exponential branching path problem, right? Where it very, very quickly you have you can take a choice of, you know, go left or right and then you can take another choice. And because it you know, it’s a branching thing, you’re you know, you only need well, you know, eight bit is two hundred and fifty six, right? So it’s only eight decision points and then when you get to sixteen you’re in thousands of them and it becomes impossible to sort of pre make all the assets and everything like that. Because we didn’t have the tools to or the technology to do that generative thing.

But one of the things of thinking about interactivity in a different way was, hey, you know, what you do is you you write the rules for an interactive system. and this is what we did with a lot of our stuff. And there is no sort of pre-baked choose your own adventure type thing. It it’s just it we made a thing and you interact with it and you kind of play with it. We talked about play an awful lot. That’s what my whole PhD was about. And it’s a very different way of thinking. You know, it’s the difference between placing each blade of grass and writing the rules for grass. And

And I I man, I’m immediately started thinking of video games and I think I also play Dungeons and Dragons. I’m a I’m a dungeon master. So my job is to kind of improvise the story or, you know, and I create a a set of scenarios, if you like, and loosely linked, but I don’t I d we don’t know each session how the story’s gonna unfold ‘cause it’s collaboratively made. And there’s this whole aspect of that that I started thinking about as well. And also

Andy Polaine (15:55.544) I don’t know if you’ve ever muds. Did you ever play Muds and the multi-user dungeons thing? So there was an and moose.

Josh Clark (16:03.382) I did, yeah. I was in the early nineties, like desperately addicted to a game called Moral Decay, where I played a a disco dancing ogre capitalist. We had a whole

Andy Polaine (16:17.038) And now you’re living in moral decay.

Josh Clark (16:19.438) That’s you know, so I was well prepared. Veronika, I’m I’m s we’ll have to put aside Zork again.

Andy Polaine (16:26.254) But but in the well, in those multi object oriented dunches, so there were te for people who don’t know, there were like a text based and then a kind of chat based thip space you could go into. and there was a there was a f I can’t remember the famous one actually now. but you go into these spaces and people you could be given a space. Like you could design your space and you could design the objects and you could program the objects in your space. And so given the different commands that people would s and you could clear give an object a thing. There was I remember the rumor mill.

And you could crank the rumor mill and it would spit out some kind of r you know random rumor about the the as a community. so it was a kind of like a chat room, basically, but it had this idea of like a mental space that you were in that you could program. And so a lot of this started to kind of all come together with me of like, okay, I can see how that’s sort of come really full circle in the, you know, the the chat well prompt-based interface in in setting up those rules.

And I’ve kind of found it really kind of fascinating that now we’ve finally got there. And obviously video games got there earlier in terms of generative landscapes and stuff, as you were saying before. So there is a whole bunch of stuff that I sometimes feel that the last ten, fifteen years of product design have been like this. It’s not been necessarily well, it’s obviously been progress, but it’s almost been like a little divergent fork that now we’re kind of a branch.

that we’re now we’re kind of coming back into of the exploration of of interactive media. And that’s really got that sense reading your book of all the different ways of thinking about this. that was sort of made me sort of brought me home in a way to sort of to sort of like twenty years or thirty before. I I think it’s really fascinating.

Josh Clark (18:10.51) It’s great to hear. I think these moments, you know, in sort of the modern computing era have come around only about every 15 years. And the last one was with mobile, where it was, here is something that is categorically different and wants to be different from previous computing experiences. And it takes a while to figure out, right? It took us a while to figure out, Uber is a thing that is uniquely empowered by mobile. Or you know.

Instagram type experiences, TikTok, a camera driven experience. and now we’re in this moment with AI and and we’re rusty as an industry. You know, to your point, the last 10 or 15 years have been a bit of a divergence. Once we sort of sorted mobile, the focus of the design industry, the innovation has been almost entirely in design operations. And how do large companies

metabolize design and make it fit into the structure of design at scale. And so a lot of the focus and innovation has been around design systems, with sometimes some unintended consequences, frankly, of making design a production element, of focused on those blades of grass almost exclusively, rather than thinking about what could the landscape be.

And so I think that right now we’re in a place that is very much, as Veronika was saying, focused on tooling and process. And I’m surprised, but patient that more of the design industry hasn’t been has been more interested in what happens if we take the intelligence from our tools and weave them into the interface to make extraordinary experiences that were not possible before, but can bring enormous value to both business and customers.

Andy Polaine (20:02.894) Why I mean I’d love to hear both your views from sort of either end. I’ll start with you, Veronika. And I say either end, I either either end of the sort of generation, I guess. Why do you think it is that ‘cause I I’m a g I agree, right? So I I I wrote a actually I think the other day I saw it’s my best performing post on LinkedIn of of all time. So it was it was something like, you know, big tech has you know, has suffered from a massive lack of imagination.

and I really kind of felt that everything, you know, looks the same as that wind tunnel effect of everything just for marginising. You know, and in some respects it’s you know, it’s great, right? I I’m glad that I can use like Gulf.uk’s great design system and get my passport renewed in, you know, it takes me twenty minutes to do it rather than hours of filling in forms and stuff. you know, and that I know how everyone knows how checkouts work and that’s sort of by and large a solved problem.

There’s whole lot of broken services though out there still. Why do you think, Veronika, that this hasn’t been more explored, that this conversation isn’t more present?

Veronika Kindred (21:09.366) Yeah, that’s a great question. To get very meta, I wonder if you know the past 10, 15 years have been very influenced by who’s hiring. I think a lot of people have gone in-house and large enterprises have really expanded their in-house teams. And I think especially like post-COVID agencies have shrunk or are not as many are around anymore.

And and I I wonder if that has influenced the kind of work and community building within the design community itself, whereas designers don’t have as many kind of third spaces as they once did. you know, it’s funny because I’m I’m a design Neppo baby. So I remember going to conferences like 10 years ago when I was a kid, and and and there’s there’s just not as many anymore. They’re just not around as much. and so I wonder if in that

in that loss of community, we’ve also lost some of the the exploration of the the industry and the curiosity. And and also like, you know, if you’re a designer on a team and maybe there’s one other designer and you’re in-house and it’s a very siloed organization, you’re not necessarily going to feel very empowered to advocate for the more curious or imaginative option. So I I think all of those reasons have really contributed, which is

You know, those are all still problems that are are still happening and still around and people, you know, it’s a tremendously volatile time. So I think it makes a lot of sense that people want to be in-house and have that kind of stability. But I think it is still important for us to take it upon ourselves to really push for that exploration.

Andy Polaine (22:51.19) Yeah, that’s very interesting. I think that’s very true. I mean, Josh, you will know as someone who runs an agency that when clients come and say, you know, we want something really new and innovative, that’s not actually what they want, right?

Josh Clark (23:01.406) No, that’s right. I mean I I mean I think everyone kind of comes in saying we want an industry leading solution, which is not an unreasonable thing to say. But quickly when you get into it, in many cases, the conversation is, Well, what is the competitive set doing? Well, wait, what’s the best practice here? And you realize that, a lot of people just want to be industry standard, which by the way is fine. I mean, I think that risk is sometimes

overvalued in our culture, especially the technology culture. Safety makes total sense in certain areas. But it’s good to know what you want and who you are and in what areas you’re prepared to take a risk. and you know, I think large organizations are designed to reduce risk and increase predictability, except in very specific areas where they believe they can excel and beat the competition. And often that is not

In interaction design. And so I think that this next era of design, when as I said, design has been metabolized into the culture of organizations that are designed to reduce risk, which means reducing innovation, that there’s going to be sort of a vanguard of organizations and products that really show us what’s possible. And as those things become settled and safe.

That will frame the next generation and things will follow. I mean, which is not, you know, some grand observation. There’s always sort of a leading vanguard of adopters that pave the way and make the early mistakes. but I think one of the things that we’ve seen is that there’s been enough of those in the couple of years that we’ve been writing this book that we can see the shapes emerging. And I think there’s still plenty to invent. But one of the things that we wanted to really focus on in sentient design.

Is to identify the emerging archetypes of experience that go way beyond text-based chat and text-based agent to what happens when you put intelligence into a wide variety of interfaces. We talked about ephemeral experiences, you know, dashboards or websites that build themselves. But you can also have agents show up as other users in an experience, for example, in a multiplayer experience, which changes the nature of

Josh Clark (25:29.846) of it and you’re not just doing text based chat off to the side. All to say is that look, it’s might seem crazy to try to write a book about AI right now. It is such a fast moving technology.

Andy Polaine (25:44.054) Yeah.

Josh Clark (25:47.65) Yeah. But so so if we wanted to really do that with intention, that this is not about the latest tools, the latest models, the latest prompt strategy. It is about, you know, we really wanted to write a book that would be useful in five or maybe, I don’t know, 10 years of identifying archetypes and ways to think about new postures that these systems can take toward the user. and we think that will be lasting. I mean, I think one of the things that’s reassuring is like our

initial ideas two years ago still hold up, even as much as the underlying technology has changed.

Andy Polaine (26:23.968) Yeah, it is fascinating. I mean, I I this is gonna be tooting my own horn a little bit when I kind of look back on some bits of my PhD, which I wrote. I think I finished it in two thousand and fourteen. I can’t remember. But there are bits of that and actually even sort of the very first like anti-ROM stuff, our our kind of interactive thing that we made. There’s bits of that and I look at that now and I’m still I still I still enjoy this. I still some of this is actually it still makes sense. And that’s partly what I meant, I think, by this kind of return to

As much as you know sounds a bit high following, but there’s sort of an intellectual inquiry around what does this stuff mean and how do you know what are the the shapes of it? Because, you know, the iPhone brought sort of multi-touch to the masses, but there haven’t really been there hasn’t really been an interaction paradigm shift since multi-touch, right? And and part of also the iPhone was it it it did a very clever trick of putting the the phone, making the phone bit as an app.

Because it turned it from being a phone that had some other stuff to being this blank thing that whatever app you opened is the sort of very ephemeral in that sense that whatever app you opened it became the thing, right? It fills the screen, it becomes the thing. Right.

Josh Clark (27:34.478) And the phone and the phone was everyone’s least favorite app on the thing. The one that they wanted to avoid.

Andy Polaine (27:40.066) Yeah, exactly. Yeah. And it it’s kind of interesting to see that that that whole sort of multi multi apps of on on one screen thing has never really been solved very well on the iPad, for example. It’s kind of one of the clunkiest bits of of I I would say of Apple’s UI design, having sort of multiple apps open at once. ‘Cause it just works really well, this idea of like this piece of glass is my banking app when I’m in the banking app, this piece of glass is now my, you know, email app and all the rest of it.

So in the book, you use the word postures just now. You’ve got this sentient triangle, you’ve got these different different kind of aspects of the kind of experiences you can have, and you’ve got these four postures that you’ve come up with. could you talk about that a little bit?

Veronika Kindred (28:21.07) Okay, yeah. So the four the four postures that we mentioned are tools, chat, agents, and co-pilots, which, as we looked at kind of the broad range of AI experiences out in the world and emerging, these were kind of the four categories of postures or kind of the stance that the system takes towards the user. and really we made the sentient triangle because we thought there was a need to map out these postures and these categories and really provide a framework for the kinds of experiences that

are emerging and that people are encountering. And it’s it’s just been a really great way even for us as we do our own work to really consider the needs of the task we’re trying to accomplish and its suitability versus the the capabilities that are available.

Josh Clark (29:07.16) When we think about the the postures, and you know, and Veronika, I think you said it really well that it’s the stance that the system takes toward the user, where tools are these very traditional experiences. Give an input, get an output, you use tools. Whereas chat is this turn-based exchange with a peer is the experience of it, where you do something, the system responds. We think of chat as like dialogue and often as text, but it doesn’t have to be. We’re we we’re trying to

take the the definition of the chat posture to be more metaphorically conversational. Whereas like I do something, the system responds, I respond to that. And like a conversation, you in the system shape the course of this journey in ways that can’t always be predicted and are unique to this pairing. Agents, of course, you delegate to agents, they go off and do things. And co-pilots sit in the background and wait for their moment.

to help in specific circumstances, really different models to think about things. And then as Veronika said, an explosion of specific experience patterns emerge within these and their overlaps.

Andy Polaine (30:20.416) It’s it’s hard to describe this is in in words. I’m looking at it now and I’ll I’ll put it up on the website. but there is or link to it too. Cause I was kept thinking, well, you’ve got three things here, why don’t you just have a Venn diagram? But I can see that you know, while copilots would sit in the middle of that, that you’ve sort of fenced off, you you’ve done actually within side within the ti the triangle of grounded, interoperable and radically adapted, you’ve got these other little triangles. And I see why you’ve kind of fenced off certain bits too. So a Venn diagram presumably wouldn’t work.

I’m really interested in that that thing you just said though around the turn-based of another game, a gaming paradigm as well. But it I mean, I’ve often talked about this this is the PhD thing and she we talked about this kind of action-reaction feedback loop of interaction, the that you can have a thing. And if you make an interactive installation, for example, there’s usually some kind of attractor loop, as they call it on like poker machines.

And video games. So it but there’s a thing that you know, someone walks past a thing and it reacts to you in some ways. Usually some cameras, a motion tracking thing, and reacts. And then the the passerby reacts and then the thing reacts, r responds, and then you you then they start playing together, right? And then then that’s that’s that’s what we were always looking for to try and kind of pull people in to something like that. And I noticed how my my push always against the sort of HCI thing a little bit was.

that it was very, very task-based, right? Very much around, you know, this person’s trying to achieve a thing, do a do a task, and let’s give them a tour to do that task in as efficiently as possible. There wasn’t much playfulness in it at all. And yet, I really feel that the the kind of playfulness of that, and you see the way people interact with you know, Chat GPT and and Claude, it it’s more than just functional, right? There’s definitely some some play in there. And they they are

I guess programmed or they are are trained to lean into that a little bit too, right? Mm-hmm.

Josh Clark (32:23.65) Yeah. I mean, I I think I think that’s an interesting thing of like how do you use personality and playfulness in a way that doesn’t lean too hard into anthropomorphism, which we see some of the both opportunities and this very strong risks of that in these assistant patterns like Chat GPT and Claude. And I think one of the things that is useful, I think, when we think about, what is intelligence? We go naturally

To human models of intelligence or machines that can talk or that have human personality. I think one of the things is you can have that kind of playfulness that you’re talking about, you know, in in the the physical environment of the examples you’re giving, in the digital environment as well, without having to be human style intelligence. I know I mean Veronika, like our

What I think that one of our big aha moments was when we were making a toy that we called sentient scenes.

Veronika Kindred (33:27.124) Mm-hmm. Yeah, we we we built this toy called Sentient Scenes. Yeah, it started out as just a toy, but I I I think it’s a good example of a lot of what Josh was talking about. It it’s it’s basically a little web app that you can put in a prompt in a chat box in a chat box and the browser kind of takes on the design

of what you’ve put into the chat box. So if you type something like HAL 9000 from 2001 to Space Odyssey, the whole browser kind of changes its mood so that it it’s there’s like a a red floating orb and it kind of takes that style on. And we’ve realized that in that way experiences can take on things like style and mood and tone and and and and personality without

necessarily pretending to be human. Like you would never look at a moody browser and be like, that’s y you would just never infer friendship or any type of relationship from from that experience, even though you’re still able to to capture that tone.

Josh Clark (34:27.903) Matt Webb, who also sort of like created the original triangle diagram that we riffed off of for the for the sentient triangle, has always been like really interesting about like suggesting alternate intelligences as models. He likes to use dolphins, for example. and it’s it I think it’s helpful to remember, it’s like, right, there are all kinds of intelligence in the world when we look at

anything from ants to elephants to dolphins to dogs to trees or mushrooms. There’s a kind of a community intelligence in all of these things that are really different from our own, but that we recognize as intelligent organization. And taking some of that of being like, how how can we make this aware and responsive is as an experience, but not try to make it

Act like your new BFF.

Andy Polaine (35:27.212) Yeah, and that lets you slightly off the hook of the sentient thing that I we talked about this we on on LinkedIn in the comments once when you start actually I think when you first started writing the book or started talking about it. And because I I had this I have a little bit of a kind of I can’t I don’t know what I want to say. It was not it’s not an ick thing, but I have a kind of y when we talk about these things a sentient, because as we know, they’re not, right? They’re not self aware. they don’t really I’m an agent.

do, but they don’t they don’t just do things on their own behalf without some kind of input from a human at the beginning, even if it’s this just the initial s nudge. And I know there’d be hundreds of people out there going, yes, but you once you send the agents off to do stuff, then they are well my view of it is there’s always humans at either end, right? However much tech you put in the way, there’s still humans at either end of it. but I think it’s really

important to remember that these things are then they’re not really sentient in that sense. But that kind of sense of I would say aware of the other or aware of the context is perhaps maybe a what you’re describing. and displaying a mood at least, I think is perhaps perhaps a I I learned the other day that chameleons do not change their colour to camouflage themselves. It’s based on their mood. And then I started thinking, well what colour’s a chameleon?

And and and it made me think of that when I was kind of reading or listening to you kind of think about talking about that, that there’s that sense of like a response that’s almost like an instinctual response. I don’t know how much a comedian, because none of us do, or a s a squid or any of those or oxpers, any of those things that change the colours, kind of go, Well, I’m feeling angry, I’m gonna switch on the red, or whether it’s just like blushing, right? or something else. So there’s a kind of instinctual response that’s hardwired into it, which feels to me a

perhaps a a more appropriate way of describing the kind of intelligence that’s going on, you know. That it and usually I had a thing the other day I was getting clawed to to look at a a website as I said for the university I work for and stuff and it it and I said, And who teaches service design? And just came up with some name. I thought, that’s weird. Well that’s not me. So and I looked up this guy and he’s a he’s a design professor in Hong Kong.

Andy Polaine (37:49.304) Hong Kong School of Design and I had said to Claude, Well what happened there? And it went, Well, no, I I I couldn’t actually find anything. And so because I couldn’t find any information, I just made something up. You know, and

Josh Clark (38:02.092) Very definition of Frankfurt’s bullshit, right? Sort of like the the the original essay that what bullshit is is not someone or something in this case intentionally lying, but what happens when you are expected to give an answer and don’t know it? Yeah. Bullshit happens. It’s not it’s not an attempt to mislead.

It is simply following through on the performance. Yeah. And that is what night large language models are. So I mean what’s interesting is we’ve got these this technology that can take and understand any kind of open-ended input, like we were talking about, you know, versus like the early Zork days. Total game changer. It can understand what you mean. And expert at manner, it can.

And I don’t mean just tone or style, but format, just the output can change and be really nimble. Yeah. Really bad at facts, but plays well with others enough to know how to ask for those facts of other systems. And so what all that means is like, we shouldn’t treat these things like databases or search engines, which is our previous model of computing, or really rules-based systems. But this actually is

Perfect for managing the presentation layer because they understand what you mean, can be nimble in their presentation, and know where to go elsewhere for answers if you make sure that they know that they’re supposed to. And so it’s it’s the kind of thing of like, we keep trying to fit this technology into a hole that looks like what came before with deterministic systems and were frustrated, but they were never built to do that.

Hallucinations are a feature, not a bug in terms of how this technology was created. And so it’s, but my gosh, for manner and presentation, they are great for that. And so that that’s what I think is really exciting about the sentient design concept. It’s like, the intelligence can be used for understanding and presentation in new ways while traditional systems handle the facts.

Andy Polaine (40:16.0) Hmm. There’s a section at the it’s right at the end, I think, of the book of where you talk about the sentient design sprint and this idea of a a minimal magical product, which I really liked because it’s for me it that you you just go through these different things. And again, for me it was all like how can we make these things more playful? You you call it magical, right? can you describe it a little bit, the the i the idea of it?

Veronika Kindred (40:38.818) Yeah, sure. The the idea of the minimum magical product is the the idea that you can first start by thinking about a magic wish. So this this draws on kind of the idea of the Internet of Things and So this draws on the idea of David Rose’s idea of enchanted objects and just the idea that in the Internet of Things objects became kind of imbued with magic. And and so it borrows on that thinking where we think about okay, if we

If we have a problem, what would be the magical solution to that problem? If anything were possible, what could we do? And it kind of works back from there. Obviously, there’s dangers of thinking of technology like magic because technology has real costs. But if you work backward from that idea of magical thinking, you can really get to the heart of the desire. So you you can really before you load yourself down with logistics, really kind of load up on the core of the

of the want. And and and then once you’ve kind of d distilled it down to that core, you can add in process and logistics and actual capabilities. And then and then you’ve ideally come out with something that is a bit magical just in the beginning stages of its thinking.

Josh Clark (41:54.03) Yeah, we we have this exercise called practical magic, which is exactly what Veronika is describing, where we we find it really helpful in emerging technologies because it helps to prevent immediate solutioning with kind of a technology led solution. And so the goal of this is not to pretend that AI or any technology is in fact magic, but it is the kind of thing that really when you get to what a magic wish is.

It is like, I wish there was a shortcut to this thing. You know, when I look in my magic mirror and I want to be able to see my aura of health, you know, it’s like I want to be able to sort of see how I’m actually feeling underneath my skin, for example. We can’t do that. The technology can’t do that. But we can kind of go backwards and be like, well, so what is possible? You know, one of the early ad campaigns for Uber.

And with this a marketing campaign, but I think it gets at it, was the next best thing to teleportation. And it’s true, you know, it’s like, do not get in the teleportation machine, folks. Like that technology is not ready. But the next best thing is being able to summon a car that knows exactly where you’re going in the fastest route. And it’s like, all right. It it, you know, that was a marketing thing. But what if we started with that? It’s like, I want to teleport. All right, we can’t do that.

What is the analog kind of proxy for that magical wish helps at a moment like this to sort of think through, get in touch with not how people should do things, but what they want to get done. And then you can think about, like Veronika was saying, how does today’s technology help us get us closer there than yesterday’s technology to that very user focused want?

Andy Polaine (43:43.586) Yeah, it’s fascinating. I think there’s a there’s a thing here around metaphors actually. Because obviously, you know, the whole all of our user interfaces have have some kind of metaphors. And if people know Lakoff and Johnson’s book of metaphors we live by, it’s a sort of classic for this. And, you know, we have I’ve gone looking at my desktop here and there’s a there’s a waste bin, you know, and I pull my

Well, there’s that weird thing where you pull you now, they change the icon actually when you do it, but you used to have to pull your hard drive into the waste bin to kind of eject it. And there’s always that moment of like, that doesn’t seem right to me to be doing that. Am I going to delete it? And honestly, you’re not really pulling a file into a d into a a waste bin. You’re just, you know, that triggers a a command to delete a file from the from the system at its really underlying level. And they’re unavered.

Josh Clark (44:26.434) There are no files.

Andy Polaine (44:31.936) Exactly. Somewhere there is a, you know, a a magnetic th signal changes, the state of some, you know, electrons. And and yet, so sometimes the you know, people get really wrapped up in the metaphor and it has to be ex exactly right. And I remember someone doing a a kind of prototype of a like a 3D desktop. And I I think you know, meta’s the metaverse was exactly this, which is so we’ve got this meeting space. And it’s like, wow, and and people who are not in the metaverse can

can dial in. I was yeah, but you’re you’ve made a 3D conference room with two big kind of Cisco screens at the end. You could have done anything, right? You could be meeting in Zark Mark Zuckerberg’s hair, you know, or or you could have done anything you chose a really sort of ugly kind of meeting room and you just took that metaphor and just sort of c copy and pasted it. But I’m always fascinated by metaphors or by by interaction paradigms that seem very natural, but actually actually aren’t. I mean the st stretching and zooming photos.

on you know, w you’re making that pinching, you’re wiping your fingers across a pane of glass. There’s no analogue to that in in real life and yet it feels very intuitive. Or the same as the that kind of mode I can’t remember what it’s called now. in on Apple’s desktop where you can kind of look at all your files. What’s it called? The the th the I can’t remember what it’s called now. When you

Veronika Kindred (45:53.614) I just want to press the button. I don’t know what it’s called.

Andy Polaine (45:56.692) button and you you it shows you all your files and then you can go, yeah, that one. You may got loads of windows open and stuff. And now that’s a thing that also doesn’t exist in real life. But the the magic wish of like I’ve got a whole load of stuff and clutter on my desk and I don’t know where it is. What I’d really like to be able to do is have it all sort of hover in the air and choose a thing and then it all kind of goes back and goes goes into its pile. That I think is one of the reasons why that works.

And and so I’m I’m I think you’re definitely onto a thing where there’s a a magic metaphor or something like that. That then that’s there are things that feel like they would make sense if only you could kind of do it. versus then a very practical real world metaphor where you can kind of paint yourself into a corner and go, Well, know, we can’t do that because desktops don’t do that.

Josh Clark (46:45.87) But but I think you’re right. It’s like we are we can prematurely limit our imaginations by trying to tie it too heavily to physical metaphor. I mean, obviously the desktop has been very enduring in in software design. but also in just how we’ve done digital stuff before. And I think at moments like this.

Giving yourself the permission to think a little crazy. Most of our consulting engagements now have some period of what we call mad science where we just try to shake it out a little bit, of being like, let us think about what is newly possible. Cause sometimes AI can fulfill that magic wish directly now, just in terms of being like, I can understand anything that I say and the meaning behind it. Like that is computing magic. Like it’s

Kind of amazing how quickly we’ve begun to take that for granted. That is something that we’ve been pursuing for decades and suddenly became possible. That’s magic. It’s magic in the Arthur C. Clark sense of it. That’s right. but but you know, let’s not be too carried away. I think there’s a real tendency right now to overpromise and overassume what the technology can do. So our responsibilities as designers is to constantly be playing and understanding.

what the tolerances are and what really serves customers and business and isn’t just a magic trick, a novelty. Yeah. Like, you know, we aren’t here to build novelty, although I think it’s important to play with novelty to understand what the material can do.

Andy Polaine (48:26.862) There’s I mean there’s whole section of the book of what could possibly go wrong and looking at the the dark side of things here. We talk about it a little bit, I’m conscious of the time, but it’s it’s it’s also v sort of very present. if you were to kind of distill one or two things out of that though, what what would be the key principles, if you like?

Veronika Kindred (48:49.12) Yeah, so it’s it’s it’s a tricky topic and I think a very emotional one. there’s so much fear and anxiety. I think I think people know pretty innately like what could go wrong. Like we we feel it all the time and it’s it’s in the news a lot. so I I think what I would wanna say is that I I think there is room for a lot to go right if we are really keep our eye on the ball of what could go wrong.

you know, I don’t necessarily have trust in the people or companies driving these technologies. And and thus I think it’s really, really important to pay very close attention to what they’re doing and to stay really engaged. and so I would say that if anyone has fear or anxiety about this technology and about what could happen or what feels like is happening,

to to stay very engaged and to get involved, because I think now is a really great opportunity to have a ton of influence about how this technology gets implemented into our lives, especially as designers. I I think really kind of heeding that call of of what’s possible and getting involved right now will have a very, very lasting impact for for a long time in the digital world.

Andy Polaine (50:02.862) Yeah, I I keep thinking of the sorcerer’s apprentice, you know, that that bit in and that bit in fantasia. I think it’s in fantasia, right? With Mickey Mouse and he’s you know, he’s getting the the mops and the buckets to kind of clean up for him so he doesn’t actually have to do the work and wash up and then it just gets out of control and he can’t do anything until the the sorcerer comes back and puts a stop to it. And it it feels to me, you know, if we keep on the magic thing, that that’s that’s a key aspect, right? And I feel one of the things that

about your book is it elevates the thinking about about this to beyond the stuff we were talking about at the beginning. It’s not just a kind process thing. It’s not just about doing stuff faster. It’s not just about sticking it into absolutely everything and putting little sparkles on to indicate it. But that you know it’s actually force you to think about, well, what is this thing? What does it mean? What is it wh how you know, what are the paradigms that this could be? And I

At least I would like to think that if you’re already thinking like that, you also the next question or the earlier question is, and how could this go wrong? What could is the worst case scenario for this? I feel this is one of the roles of design. As Jean Mark Curtis said this about, you know, in in movies like Shrek or You’ve got Donkey or, you know, the the guide by the side or the sidekick or the mentors or the mentor is obviously often there to

Ask the uncomfortable questions of the protagonist and sort of push back in in the ways that force them to change, rather dory in finding Nemo, that part of our job in that desire to kind of put it everywhere and and and and you know, people won’t people won’t do that. You know, why and to ask those difficult questions. And if the answer is, we don’t think people will do that, then the answer is always no, there’s going to be someone who does that.

But I think it’s partly our job is to actually ask those uncomfortable truths or uncomfortable questions to to force the conversation. And I think it’s a bit of a balancing trick for design because you know we run the risk also of just being discounted and ignored because we’re just this sort of irritant, like like we have been with sort of, what about the hu the human? What about the customer? What about the user? And and we’ve sort of often done that at the expense of talking about what’s also good for the businesses that are employing us.

Andy Polaine (52:20.398) But I do feel it’s still an important conversation to be had. And one of the things we can do, and I guess everyone could do now, is is show it, right? You you is you used to be one of the Achilles heels of design and proving it sort of ROI was we in our heads extrapolated into the future and and d decided not to do a thing. And so the ROI is that.

you didn’t do the thing that would have been a bad thing in the future. And ‘cause you can’t really sort of prove this hypothetical future bad scenario didn’t happen. Whereas now you can actually, you know, you can prototype the bad thing. Right. And I think sometimes it’s really useful to go the other way and go, well, let’s let’s prototype the worst case scenario and ‘cause you then you can show and not tell. Right.

Josh Clark (53:02.594) Though I will say some of my experience of being like, you know what, let’s just show them how terrible it is. They’re like, Yes, exactly. Exactly. So there is risk in building the thing. It’s like, yes, but but I do think, you know, I just to say out loud, it’s like I do think that designers often feel that they are naturally put into or have to adopt an antagonistic view toward business that

Veronika Kindred (53:14.862) Read it.

Josh Clark (53:27.232) Our colleagues in other areas have their own incentives of what success looks like. And for us, that looks damaging or intolerant or dismissive of the user and their concerns. And I think that the goal, you know, it’s just such a cliche now, is rather than argue about the solutions, is to agree on the problem. And that one of the things that we can as designers that I think can be a really

Core skill of ours is to name the problem, we often don’t do it enough with our colleagues. And I think that the real sign of success with design, interaction design, all of the stuff is how do we take the users’ interests and the business interests and find that narrow hidden path that connects the two and light it as brightly as we can? Like I think that is our work. And I think that.

asking those kinds of questions right now when there’s an emerging technology, when there are powerful business interests that are trying to push a certain vision, is to realize, yeah, we all do have agency in our own ways, especially in these early days, to demonstrate how this stuff should be used. Veronika, just like you were saying, you know, it the technology will decide for us if we don’t decide for it. And the future should not

Be self driving. Yeah. Like it is up to us.

Andy Polaine (54:56.054) Or the the money decides. So we are running way over time, but it’s been a f fascinating conversation. I we could go on for ages. We well, our little pre chat that was meant to be a sort of quick half an hour thing went on. We we talked for ages too. which is I think a sign, right? I think it’s a good sign that there is a lot of conversation that needs to be had here. And I really hope that people are are having the start to have these kinds of conversations in the environments they’re working in.

the show is named Power of Ten after the the Eames film Powers of Ten about the relative size of things in the universe. And so the final one small one question is is what one small thing is either overlooked or could be redesigned that would have an outsized effect upon the world? I go for you, Josh, and then give Veronika the last word.

Josh Clark (55:41.078) All right, that sounds great. She always should have the last word. you know, we talked about something in the book called Casual Intelligence, which is just a reminder that not all of this stuff has to be a wholesale change in what we do. Not everything has to be, all right, the interface now has a total mind of its own. It’s doing its own thing. And so there’s this idea of casual intelligence of what if we just made this little one little tweak a little bit better. And like I just think.

Forms, the bane of all of our existence, especially longer multi-stage things. You mentioned it, you know, with kind of gov UK affordances, you know, forms are awful. What if we can sprinkle a little bit of intelligence into those? And I mean that Veronika, you’ve got a favorite example that is not a new one, but is machine learning in an interface that does that.

Veronika Kindred (56:34.156) Yeah, Google Forms when you start to type a question immediately changes its interface to reflect the type of question you think it you’re gonna ask. So it’ll change from like multiple choice to a linear scale type of thing, depending on the phrasing of your question. And I and I think that’s just a beautiful little example. It’s so unintrusive. It’s so easy to override as a user, but it it just saves you a little bit of time and it makes the form building a bit easier.

And and so yeah, as Josh was saying, just ca the idea of casual intelligence and just adding like smart but not overweighted intelligence into experiences I think could be super powerful.

Andy Polaine (57:11.575) Amazing.

Josh Clark (57:12.504) And in that example it doesn’t it doesn’t impose the answer. It’s just the default is ready for you in a s a smarter default than it was before.

Andy Polaine (57:21.922) But it’s not clippy at the same time.

Josh Clark (57:24.654) It doesn’t have to yap at you, right? We just like so important as a takeaway. Intelligent interfaces don’t have to be talking interfaces.

Andy Polaine (57:32.664) Yeah. Wonderful. So where can people find you both online?

Josh Clark (57:37.334) We are at bigmedium.com where we share ideas and we have a a newsletter. It is an occasional newsletter, probably all too occasional, Veronika, I would say, but but that’s out there. And Veronika, you are also reluctantly

Veronika Kindred (57:54.158) Reluctantly, but very much so on LinkedIn.

Josh Clark (57:57.614) this is what’s left to us.

Andy Polaine (57:59.938) I’ll put the links in the show notes. Thank you so much for being my guest on Power of Ten.

Veronika Kindred (58:04.046) Thank you

Josh Clark (58:04.384) Yeah. Thanks so much, Andy. What a treat.

Andy Polaine (58:06.086) And if anyone would like a discount off the book, if you go to rosenfeldmedia.com and use the code sentient pot, you’ll get a discount when you order directly. You’ve been watching and listening to Power of Ten. You can find out more about the show on Polane.com, where you can also check out my leadership coaching practice, online courses, as well as sign up for my irregular newsletter, Doctors Note. If you have any thoughts, please put them in the comments or get in touch. You’ll find me as Andy Polane on BlueSky, LinkedIn, YouTube or you can get in touch with me via my website. All the links are in the show notes. Thanks for listening and watching and I’ll see you next time.