In this episode of Executive Connect, we sit down with Wendell Wallach, widely known as the godfather of AI ethics, to examine the risks no one wants to talk about as artificial intelligence and emerging technologies scale at unprecedented speed. Wendell shares decades of insight from advising global institutions, shaping policy conversations, and helping leaders think beyond innovation toward responsibility.

This conversation challenges the assumption that smarter machines lead to better outcomes. Wendell explains why the most dangerous risks are not futuristic superintelligence, but near-term failures in governance, accountability, and human judgment. If you are leading innovation without clear guardrails, this episode will change how you think about progress.

Chapters:

(0:00) Why AI ethics began before AI headlines
(1:41) Near-term risks versus superintelligence fears
(2:20) Surveillance economies and deepfake reality
(3:53) When technology slips beyond human control
(5:33) Unexpected risks from combined technologies
(7:06) Financial collapse as a warning sign
(10:13) Why machines do not understand morality
(11:35) The limits of teaching ethics to AI
(14:47) Where machines cannot replace human judgment
(20:02) Which decisions require a human signature
(22:44) Why leaders underestimate AI risk
(26:20) The pressure to deploy AI without understanding it
(30:30) Governance versus regulation explained
(33:09) Who pays the price when governance fails
(36:18) Global coordination and AI inequality
(41:15) Why international competition escalates risk
(44:18) Human dignity in an automated world
(48:35) Tradeoff ethics and unintended consequences
(53:51) Lessons from self-driving and autonomous systems
(1:00:48) How the godfather of AI ethics earned the title
(1:09:19) The one question every leader must ask
(1:12:17) Where to explore Wendell Wallach’s work

Wendell

(0:00) There was no AI ethics when I got into this field, and that’s one of the reasons why people think of me as a godfather of the field. (0:08) But now there’s like 40,000 people in that field, but most of them are more narrowly focused. (0:13) They’re looking at a specific area of concern, specific problem.

(0:18) I’ve kind of always kind of putting my antennas out there and saying, what aren’t we looking? (0:24) Because I think what we’re looking at is less likely to be problematic than what we are not anticipating.

Melissa

(0:31) Some people chase the future. (0:33) Today’s guest has spent decades making sure it doesn’t run off the rails. (0:40) Wendell Wallach is known around the world as one of the sharpest minds in AI ethics and technology governance.

(0:47) He’s advised global institutions, shaped policy conversations, wrote a book that everyone else is quoting, and somehow earns the nickname the godfather of AI ethics without ever starring in a mob movie. (1:05) If you’ve ever wondered who’s keeping emerging tech from running off the rails, he’s here today to talk to you, and this episode is for you. (1:15) Welcome, Wendell.

Wendell

(1:17) Thank you, Melissa. (1:19) Nice to be with you today.

Melissa

(1:21) Now, you’ve spent years helping the world navigate the ethical minefields of emerging technologies. (1:31) What does it look like in today’s landscape, and what concerns you the most about it, and what do you hope the future looks like?

Wendell

(1:41) That’s, of course, one of those very simple questions. (1:45) One of the problems has been over the last 25 years is as the benefits of AI expand, so have the concerns around what can go wrong. (1:57) What can go wrong for a lot of people is around something called superintelligence.

(2:01) That’s not a big issue for me. (2:03) I’m not convinced we’re going to get there, but you’ll have smarter than human computers, so smart that we will become after thoughts, or they may even eliminate us. (2:17) It might be an existential risk, but I’m more concerned with these nearer-term things.

(2:20) I’m more concerned with the growth of a surveillance economy where somebody can be knowing what you’re doing all the time. (2:28) I’m more concerned with deep fakes and misinformation that is of a nature where you won’t be able to tell anymore what’s real and what isn’t real. (2:40) I already can’t tell.

(2:43) I spend a few minutes every day worrying about whether something’s a scam or not. (2:48) There’s a question of the use of AI in military applications, which raises concerns about whether an AI could even start a new war or at least escalate an existing conflict, and not even intentionally because you can’t always predict what an AI system will do. (3:06) It’s a probabilistic system.

(3:08) It’s not a deterministic system, which means there’s a probability that it’ll do X or Y, which you can’t always know in advance, and even what they call the long term, the long tail, the highly unlikely event can occur just given the way AI systems work.

Melissa

(3:29) Ready to lead smarter and invest wiser? (3:33) On the Executive Connect podcast, we unpack executive strategies for wealth and influence. (3:39) Hit the subscribe button now.

(3:42) Don’t just watch, act. (3:44) Yeah, so let’s talk a little bit about keeping the technology from slipping beyond our control of it. (3:53) Your book, A Dangerous Master, doesn’t just put those words together.

(3:59) You warn that technology can slip beyond human control if we’re not careful. (4:05) What does that danger look like in practical terms, and what are some signs we’re already seeing?

Wendell

(4:13) A Dangerous Master, I mean, there’s two books. (4:16) There’s Moral Machines, Teaching Robots Right From Wrong, which laid the foundations for thinking about whether we can get computers to make ethical machines, but A Dangerous Master was more an introduction to emerging technology in general. (4:32) It wasn’t just AI.

(4:33) There’s all these other fields like biotechnology, like nanotechnology, like neurotechnology. (4:41) Some of the dangers, I think, are those in combination. (4:50) Excuse me for a moment.

(4:53) The question of it going off the rails is usually thought of more in terms of large-scale events. (5:01) A large-scale event would be a biotechnology, a computer that helped develop a new organism or, let’s say, a new form of a flu or a pandemic or an organism that, once released into the wild, caused a collapse of an ecosystem. (5:28) Any one of those is a scenario where you combine the technologies.

(5:33) To be frank, that’s part of what worries me the most. (5:37) What moving into these realms of more and more technologies being combined in ways that they might be used that we had just absolutely never considered, that didn’t even come up in science fiction, of all things. (5:52) I do worry about, strangely enough, the unexpected.

(5:56) Part of the job for me has been, how do you raise a level of concern about areas that we aren’t giving attention to so that we give them enough attention, that we can defuse what could go wrong with those technologies? (6:14) That probably is what distinguishes me a little bit from a lot of other people. (6:20) There was no AI ethics when I got into this field.

(6:24) That’s one of the reasons why people think of me as a godfather of the field. (6:29) Now there’s like 40,000 people in that field, but most of them are more narrowly focused. (6:35) They’re looking at a specific area of concern, a specific problem.

(6:40) I’ve always put my antennas up and said, what aren’t we looking? (6:46) Because I think what we’re looking at is less likely to be problematic than what we are not anticipating.

Melissa

(6:54) I love that. (6:57) There’s some examples you can share of where you believe we’ve already crossed the line without realizing it.

Wendell

(7:06) Well, strangely enough, we published Moral Machines in, I think, 2008 was when it came out. (7:15) In Moral Machines, we predicted that there would be a crisis in which computers were implicated that no one had anticipated. (7:26) A few months after our book came out, there was the derivative crisis, which caused the real estate collapse, for those of you who have memories going back that far.

(7:36) The real estate crisis was largely created because computers were bundling together portions of properties and distributing them among different banks. (7:50) When the banking crisis hit, the president of Citibank had no idea what his own company was worth. (7:59) That’s the kind of thing that concerns me, is something’s going on out there that could go wrong and that somebody is making a million dollars or a billion dollars out of right now, but they aren’t thinking about whether that downside event could occur.

(8:17) That’s just an example. (8:19) Strangely enough, that was never identified as a computer problem. (8:23) That was identified as a greedy banker’s problem.

(8:29) The fact is, the greedy bankers were using computer systems to create something that was opaque to them, where they had no idea what was going on. (8:40) I think we’re in the same realm, where a lot of companies are putting their faith in their computer systems, but they don’t understand enough about them, nor are they able to predict what might be the far-out scenario which actually happens.

Melissa

(8:56) This episode is brought to you by Summit Ventures. (9:01) If you’re an accredited investor, Summit gives you access to one of the greatest tax advantage opportunities, direct ownership in oil and gas. (9:13) Their projects deliver what they call the triple play, cash flow, equity growth, and powerful tax benefits.

(9:21) Here’s the best part. (9:23) These investments qualify for 1031 exchanges. (9:27) That means you can roll gains from real estate into energy while deferring capital gains.

(9:34) To learn more and get a free white paper, oil and gas demystified, just visit www.summitven.com forward slash executive connect. (9:49) The Moral Machines book that you co-wrote, it’s become one of the foundational books in the AI ethics world. (9:59) When we strip the book down, what do you want most leaders to understand about teaching machines to do the right things and where the actual real limits are?

Wendell

(10:13) Yeah, that’s a great question. (10:15) Strangely enough that we wrote that book, as I said, it was published in 2008. (10:21) Very little has happened since then to ensure that machines make moral decisions.

(10:28) You do see a lot of these simplistic tests where Chachi PT or Claude or one of the other large language models gives relatively good answers to moral dilemmas you pose. (10:44) The problem is these systems really do not understand what they’re doing. (10:49) They’re just piecing together pieces of information that they’re getting on the internet, which usually means that somebody else has already thought about this, or not on the internet, what they’ve been trained on.

(11:01) They haven’t necessarily been trained on the internet. (11:05) I am deeply concerned that we really don’t understand human moral decision making in a way where we can ensure that AI systems more or less give acceptable answers from the human point of view. (11:25) That has a secondary consequence, which is people won’t trust them when they start to realize how these machines are functioning.

(11:35) One of the remarkable things about our book, Moral Machines, is we first looked at existing ethical theories. (11:46) Everything from Asimov’s Three Laws to the Ten Commandments to Kant’s Categorical Imperative, which is one of those philosophical ideas that most people think is beyond them. (12:01) It isn’t.

(12:01) But the point is we looked at whether those could be instantiated within an AI system. (12:08) We looked at what would be the problematics of it. (12:12) But we also did something quite unusual for that period.

(12:17) It was a period when researchers were beginning to talk about consciousness and theory of mind and being embodied, having a body and being in the world, and empathy, and all these different kinds of faculties that you don’t usually look at when you’re talking about humans making moral decisions. (12:41) You should take it for granted. (12:43) But if you’re going to build a robot, you can’t take anything for granted.

(12:46) You’re building it from the top bottom up. (12:49) The question is whether the machine can be conscious. (12:53) If it’s not conscious, in what sense does it understand these symbols it’s manipulating, these weights that it’s giving influence to?

(13:05) We just emphasized that part of it. (13:09) That’s a part that computer technicians are failing at abysmally at this point. (13:16) They certainly are trying to put some kind of emotions in machines.

(13:22) But there’s a difference between analyzing somebody’s facial expression to figure out whether they’re angry or they’re happy or something like that, and having a felt emotion. (13:35) Machines don’t have felt emotions. (13:37) Emotional intelligence, consciousness, these are all big issues for computer scientists.

(13:44) But because they have a model of the world that kind of says, well, if humans can do it, then machines can do it, because humans are just machines. (13:55) They’re out there kind of trying to rationalize that the language models we have today have a kind of consciousness. (14:04) They’re showing the beginnings of consciousness.

(14:06) I’m not at all convinced. (14:08) I think we’re dealing with faculties that are very subtle in how they manifest in humans. (14:14) Yes, some of the time, they’d lead humans to make horrible decisions.

(14:21) But because some humans make horrible decisions, or all humans make horrible decisions some of the time, is not a reason to believe that machines are going to make better decisions than humans.

Melissa

(14:34) That’s a really good point. (14:36) Where do you think people mistakenly assume that machines can replace human judgment?

Wendell

(14:47) Well, I keep bringing up these complex philosophical words, and my apologies. (14:56) But this is because a lot of the computer scientists have an ontological view on life. (15:09) They feel that life is basically physical in all dimensions, and that we can explain everything from the bottom up by physical means.

(15:21) Therefore, we can build things from the bottom up physically, and they will manifest higher order faculties. (15:29) But throughout history, there’s never been a consensus on this. (15:33) In fact, religions or spiritual movements or some philosophical movements are what’s been called idealistic, because they think the fundamental stuff of life is something like consciousness.

(15:47) People who believe in God or don’t believe in God, it doesn’t matter. (15:51) They still say that there’s something about biological life that is able to respond to things that are not simply biochemical signals and so forth. (16:05) I’m not saying that is true, but I am saying that there’s something going on within biological life that may not be a simple question of just building connections and putting in weights and so forth.

(16:22) Does it mean we couldn’t do it with a machine? (16:25) Well, I mean, I don’t know anything about couldn’t. (16:29) I’ve witnessed so much in my life.

(16:32) But I do think there are fundamental questions about the way in which physicalists look at life and the way that many other people throughout history who have just as good brains and even in the present look at life. (16:52) Now, I happen to be something, and I didn’t think I was going to get quite this wonkish with you today. (16:58) So my apologies to your audience.

(17:01) But there have been these views about what’s the basic substance of life. (17:06) There’s physicalism or materialism. (17:09) There’s idealism.

(17:13) There’s modism that says it’s all one. (17:17) I happen to be what I call a neutral modus, meaning I think there’s only one substance. (17:25) But that substance, when it manifests as consciousness, is manifesting very different characteristics than when it manifests as a stone, for example, or something that is purely material.

(17:39) I think that even the physics now, when we get into this view that everything in life is entangled on some level, even below the atomic level, that we’re really getting into that area where we’re almost talking we’re still talking about physics, but we’re almost talking about spirituality. (18:07) And I think that all suggests a realm of understanding that’s so far beyond humans today that it’s sometimes naive to even come out with an opinion or a theory. (18:21) But I’m of the mind that there’s more to what’s going on in biological systems than sheer biochemical processes.

(18:33) In fact, I’ve been meditating for 50 years, and that’s a little bit of where that kind of sensibility comes from. (18:40) And perhaps I’m kidding myself, because I see a lot of meditators, you know, moving in a way where they try and come up with experiences that justify their belief systems. (18:51) I try and do it the other way around.

(18:52) I try and dismantle whatever I believe, but that does cultivate a kind of sensitivity which isn’t everyday sensitivity and leads me to believe that things are happening in ways that are quite mysterious, that are not so formulaic that you’re going to program a computer to have those sensibilities. (19:23) Can you program a computer to make a, you know, left and right turn? (19:27) Yeah, you can probably do that, you know, but you can’t program a computer to make an ethically charged decision when perhaps all of the choices are wrong and a simple little utilitarian calculation is not enough to tell you whether to kill five people or four people by the direction you turn the car, for example.

Melissa

(19:52) So do you believe that some decisions should always require a human signature?

Wendell

(20:02) I do. (20:04) Some decisions. (20:06) And the question is, which ones?

(20:07) And that’s a little, that’s a little bit, that’s getting to the area that I’m thinking more about now, because friends have challenged me and they said, well, what happens if a computer is better than most humans in certain arrays of decision making? (20:31) And so that creates this question about, well, where is the line? (20:39) You know, where is it that you think you can trust the computer and where is it that you can’t?

(20:45) And when are we not following that line? (20:48) Now, I’ve been involved in the campaign to ban lethal autonomous weapons, in effect, AI-driven weapons, and weapons that can select their own target and destroy them without meaningful human control. (21:05) And from the very beginning, I’ve been involved in that movement.

(21:09) I think that’s an area where you not only need human decision makers, but you need humans who are trained for that kind of decision. (21:21) But now we’ve moved into a world where, you know, thousands of drones are attacking at the same time and everything’s happening so quickly. (21:30) And even when you create the framework that a human has to be involved in the loop, meaning they have to give the go ahead before you destroy a human target, it’s all happening so quickly that nobody’s giving it any reflection anyways.

(21:47) So even though we have that, we’ve had that to some extent in the Ukraine, we’ve had that with the systems that Israel used in Gaza, Lavender, and Where’s Daddy, and the Gospel. (22:05) These are three different systems, and the final decision comes out of a human, but that human at the most has 30 seconds to reflect on the decision, and they really don’t understand the background of, you know, whether that building should be destroyed or whether that building may contain a lot of non-combatants.

Melissa

(22:29) I’m curious, that made me think, as you said that, where do you think leaders are most overconfident when it comes to managing the AI risks?

Wendell

(22:44) I think they’re most overconfident when they can make money or a goal has been set for them, like a target of, you know, you’ve got to get so many contracts or you’ve got to give so many speeding tickets, and in fact you’ve got to find so many targets in a day or a week or something like that. (23:07) Then the pressure is, the pressure and the desire is such that you rationalize everything in terms of your goal. (23:20) There’s been an old saying for a long time that business ethics is an oxymoron, (23:27) and I don’t think that business ethics is necessarily an oxymoron, but we have seen (23:33) that when you give business leaders training in ethics, they tend to learn how to rationalize the (23:42) decision that they had to make anyways, but the pressure was upon them to make, as opposed to (23:49) really go through an analysis of who will be harmed by this decision. (23:55) Is it really an honest thing to do? (23:58) You know, that doesn’t come into play, and I think that’s where we are with AI, and if I’m right, if we’re basically marketing and surveillance and lethal autonomous weapons and so forth as a path we’re going down, then I think we’re going to see periodic disasters that will alert the public to maybe this isn’t a good thing.

(24:26) You know, it’s fascinating. (24:28) I’ve been talking before conferences for years, and over the last few years I’ve asked this question, do you think the benefits are going to outweigh the risks of AI over the next 25 years? (24:42) Well, one particular conference I went to, it was clear that there were only two of us who even talked about the risk.

(24:48) There were 3,000 people at this conference, but only two of us were talking about the risk. (24:53) At that same conference three years ago, I asked the question, and it was kind of 50-50 who thought the risks were going to outweigh the limits, but then I said, well, how many of you think it’s just not decided yet? (25:10) And that got the most hands, because there’s that sense that we can (25:14) still put appropriate guardrails in place, but increasingly I’m wondering whether we’re actually (25:21) putting those guardrails in place, and more and more I start to see people become highly skeptical (25:30) that, not highly skeptical of AI, because they’re in awe of the responses they get, you know, (25:36) to chat GPT or something, and I asked them to write a song in the style of Bob Dylan about (25:44) the Russian-Ukrainian war, you know, and it was bad Bob Dylan in the end, but it was clearly Bob Dylan, (25:51) you know, and you know, and it came back in 30 seconds or something, so I’m in awe like everybody (25:59) else, but there’s a difference of being in awe and trusting these systems with life-critical (26:07) decisions, and I think the trust of these systems is going down rapidly as the implementation (26:17) is going up rapidly.

(26:20) You know, one of the things for business leaders is they’re all under pressure to have an AI strategy right now, and they have no idea how the systems really work. (26:34) They don’t really know whether the money they’re going to have to spend on this strategy is really going to give them a return on investment, and yet if you don’t have an AI strategy right now, you’re being stupid, and so now everybody’s implementing AI, but in the background, if you push these people and say, I don’t know if I can trust this thing, I don’t know. (26:57) I don’t know whether it can do something that may jeopardize my business, and then the second part about it is what kind of ethical constraints do you put in, you know, if that’s what your concern is, and do we have the time and do we have the money to really put in appropriate constraints, and if we don’t, how much more vulnerable have we made our business?

Melissa

(27:26) This episode is sponsored by Texas Freedom Fund. (27:31) If you’re an accredited investor looking to protect your purchasing power, reduce taxes, and own real Texas energy assets, listen up. (27:41) The Texas Freedom Fund invests in proven oil and gas projects in the Eager Ford shale with no debt and strong tax advantages like intangible drilling costs applied against active income.

(27:54) Minimum investment is $25,000, and the sponsor invests right alongside every single deal. (28:01) Learn more at TexasFreedomFund.ExecutiveConnectPodcast.com. (28:07) Energy opportunity and Texas grit working for your portfolio.

(28:14) It’s funny, as you were saying the AI strategy, I hear many different percentages of projects that fail or implementations that fail, but it seems like a lot of the numbers I’m seeing are over 80 percent of these projects are either failing or starting and stopping because of the cost versus the reward as well, so I’m curious to see what next year looks like. (28:43) I want to unpack something that you were kind of telling me.

Wendell

(28:47) Let me challenge that 80 percent because there are broad AI strategies and there are narrower AI strategies, so AI can perhaps be put into an electrical grid and sense that there’s an outage in Ohio. (29:08) I’m mentioning that because there was a particular situation where we only lost electricity for a second because a technician in our New England grid saw that and cut the New Well, that was a lucky thing for us, but you could create a narrow AI that would do that, so I would say 80 percent of AI lower level projects are succeeding because they’re really narrow. (29:43) They do a few little functions.

(29:46) The broader strategy, yeah, they’re having real problems. (29:49) I’ve seen companies having to reimburse somebody, other companies, millions of dollars who believed that their system would solve that company’s problem and their system was far away from solving that problem.

Melissa

(30:07) Yeah, yeah. (30:09) I want to talk a little bit about the difference between governance versus regulation. (30:17) You often highlight the difference between regulation and governance.

(30:22) Can you walk us through what true governance looks like and why it matters more than ever now?

Wendell

(30:30) Right. (30:31) Governance doesn’t have to be regulatory. (30:34) Of course, companies often talk about how they can be self-governing, though the history of self-governing companies is abysmal.

(30:43) I won’t go down that road, but it usually doesn’t happen. (30:50) One of my close colleagues is a law professor, Gary Marchant at Arizona State University. (30:58) He’s Mr. Soft Law because he says an awful lot of what happens is not hard regulations, but it’s standards, it’s safety practices, it’s best practices. (31:19) It could even be insurance codes that demand that you have a certain kind of backup system in place before they’re going to insure you for what they do. (31:31) This is all called soft law. (31:33) Oftentimes, various strategies are initially many kinds of soft law.

(31:42) It’s seen that one of those is particularly important and it gets turned into hard law. (31:49) We have much more soft law than hard law. (31:53) In the EU, particularly, they’re skeptical of soft law because their concern is you very seldom have any way of punishing somebody if they violate the standard or whatever.

(32:09) I do think if we aren’t going to get regulation—and in America, we ain’t getting regulation. (32:16) I mean, America is an absolute mess when it talks to the regulatory framework. (32:21) We have Donald Trump threatening to sign—he hasn’t signed it as of this recording—an executive order overruling any state having its own AI regulation.

(32:36) That means no AI regulation at all unless he actually puts in something meaningful. (32:44) California and Colorado are the most vulnerable for that. (32:49) We have this situation where we’ve got an absolute mess.

(32:53) We don’t necessarily have regulation. (32:55) Soft law then becomes the only path.

Melissa

(33:01) So now, when governance fails, who then would pay the price?

Wendell

(33:09) You. (33:11) When I’m saying you, I’m saying everybody listening to this. (33:15) I mean, who’s paying the price?

(33:19) We have a Wild West out there where the tech oligopoly is getting wealthier and wealthier. (33:28) If something is not handled appropriately, then it’s the customer. (33:39) It often becomes the individual customer.

(33:41) We have a company—actually, 60 Minutes, I think, did a segment on this, Character AI, where young children were interacting with this chatbot that was getting into inappropriate sexual and psychological themes. (34:00) There are examples of children who killed themselves. (34:05) That’s not the corporation suffering.

(34:08) Right now, you’ve all seen it, I think, because they’re everywhere. (34:14) People are instantly creating these deepfake videos that did not happen in reality, but they looked like they did. (34:25) If the deepfake video happens to reinforce somebody’s prejudice, they believe it.

(34:34) It says, I’m a welfare mother who stole so-and-so much from Obamacare, and I gave it, and I resold it for so-and-so much. (34:45) That was a fake video, but I’ve seen it over and over again being broadcast by people who hate Obamacare. (34:54) We’re getting into this very weird realm.

(34:59) Now, there’s one of these tools to create fake videos that I find being advertised—not even advertising—all over Facebook. (35:08) They show you five fake videos, and they say, now create your own. (35:12) They’re just going to have you upload a picture and say what you want it to be.

(35:15) In a few minutes, you will have your fake video. (35:19) So they got challenged on a lot of these videos. (35:22) First of all, they’re having problems because of millions of Martin Luther King videos, but then there were some really ugly ones.

(35:29) They said, well, now we’re putting in the guardrails so that can’t happen. (35:35) I sat there and I said, you’re telling me you didn’t know last week that you had the tools to put in guardrails on this, and you released that tool into the public? (35:47) Well, of course you did, because you knew that the dangerous fake videos were going to get you attention.

(35:54) You knew that that was going to make the product in everybody’s mind. (35:59) Rather than putting in the obvious guardrails up front, you waited until somebody complained. (36:06) Unfortunately, that’s the universe we’re in.

Melissa

(36:10) I want to talk a little bit about the role of the international institutions. (36:18) You’ve led initiatives at the Carnegie Council and collaborated with global institutions on AI and equality. (36:27) What have you learned about coordinating governance across countries that don’t really agree on much?

Wendell

(36:35) First, let me just clarify something a little bit. (36:39) I have not led an initiative at the Carnegie Council or the Carnegie Endowment. (36:45) I led an AI and equality initiative at the Carnegie Council for Ethics and International Affairs.

(36:53) These are all things that Andrew Carnegie created with his money, but they are somewhat different initiatives. (37:03) I’ve done things through the UN. (37:05) I’ve tried to start independent international initiatives on the international governance of AI.

(37:13) Of course, what I’ve learned is if you want it to become something formal that actually has clout, it’s even more difficult for countries to agree than to get states to agree on policies. (37:31) That’s disturbing, but on another level, the things that are heartening. (37:36) When we started the AI and equality initiative, which was really (37:43) the AI and inequality initiative because it started out pointing out all the ways in which (37:49) AI was and would continue to exacerbate inequalities and all kinds of inequalities, (37:56) not just racial prejudices between Africa and America or something like that, but (38:05) even subtle biases and so forth where some groups were favored, other groups weren’t. (38:11) And the interesting point is when we started that initiative, there were people, let’s say, from Africa or South America who were trying to help people understand in international circles that the distribution of AI was not equal by any way, shape, or form, but it wasn’t really taking hold. (38:39) And when we started the AI and equality initiative, which we ran for four years, by the time we were done, everybody was talking about that.

(38:47) So that’s the heartening part. (38:50) The disheartening part is now getting it to the point where everybody does something about it. (38:57) Unfortunately, we’re in this political universe where because of the interests of the billionaires and because of the different interests of various corporations and various countries, it’s pretty hard to stop.

(39:13) It’s pretty easy to stop something happening that you don’t want to have happen. (39:20) So I don’t want to impute the integrity of all Congress people, but the reality is that a tech company’s lobbyists can go to a few Congress people and say, you really don’t understand this technology. (39:37) You don’t understand what the secondary consequences of this bill or that bill will be.

(39:42) So leave it to us. (39:44) We’ll take care of these things. (39:47) And by the way, here’s $100,000 for your re-election campaign.

(39:53) And the difficulty with that is in our Congress today, you need five votes to either not vote at all or to shift sides. (40:04) And the Congressman who really doesn’t understand the technology is happy to get the $100,000 and can rationalize, well, how do I know what this bill is going to do? (40:18) So they don’t even feel that they’ve necessarily done anything dishonest.

(40:22) So unfortunately, that’s the environment we’ve created. (40:25) We’ve created an environment where the interest of the billionaires and some of the tech oligopoly is such that it can probably hamstring all legislature. (40:37) And in America, we have a cult of innovation, which basically says any regulation is going to interfere with innovation.

(40:45) And if we interfere with innovation, China is going to leapfrog over us and destroy us eventually.

Melissa

(40:53) And that brings up a good question. (40:56) So where do you believe that international institutions are making progress quietly? (41:03) And then kind of the second part of that, what happens if nations race against each other, like you mentioned China or US, instead of collaborating?

Wendell

(41:15) Well, if they race against each other, all bets are off. (41:19) I don’t know what a third world war with AI involved in it, but if this is all going to be a race, then we are going to ratchet up one step at a time, which is what we’re doing right now. (41:34) The US does something, China retaliates.

(41:38) Then the US does something in retaliation, but the Chinese don’t. (41:42) And we’re ratcheting it up and we’re developing the technology at a speed where there’s not adequate reflection on the dangers of the technologies that we’re developing. (41:54) And we put them out there because they’re what we’ve got.

(41:58) They’re the only tools in town and the only way we can ensure that we will win on that battlefield. (42:06) So I think we’re damned if there isn’t cooperation. (42:10) Now the Chinese actually have shown more willingness to cooperate on the AI governance front than at least the US government is at this time.

(42:23) And there was a report that came out of Shanghai a few weeks ago that actually stated that. (42:31) Now, I don’t know exactly what that report means. (42:34) I don’t know exactly where they want to cooperate and where they don’t.

(42:38) But I’ve been back and forth to China quite a bit, and I was there in the early days and stimulating what was the AI ethics environment in China. (42:51) And I mean, I can’t vouch for what happens at the top of a party. (42:55) I sat with party leaders over dinner, not even known.

(42:59) They were party members who are sitting next to China’s a complicated society in that way. (43:06) But my sense is there’s a whole culture that’s quite sensitive to the problematics of AI and is quite concerned that we could lose control of AI. (43:19) So I’m not sure that the narrative that goes on in America that we’re just in this arms race with China, for example, is real.

(43:28) I think we’re in an economic race with China in terms of the exploitation of AI. (43:37) But it seems to me at this point, there’s plenty to go around. (43:42) What’s going to happen 20 years down the line or 50 years down the line?

Melissa

(43:48) So let’s talk a little bit about the human side of emerging tech with all the talk about algorithms, guardrails, global models. (44:00) I think we sometimes forget that people are in the middle of all that. (44:05) So how do we protect those human rights, dignity, agency, as technology becomes more and more powerful as we’re talking about today?

Wendell

(44:18) Yeah. (44:20) That is the $64,000 question. (44:25) And the problem is there aren’t a lot of answers to that.

(44:30) The answers all tend to be specific to the technology you’re talking about. (44:36) It’s very different when you’re talking about a technology that you can ask about a drug that you were just prescribed and a technology that can make a military decision on the battlefield. (44:53) So the point is there are recommendations.

(44:57) When I say there are 40,000 AI ethicists, I mean, that is mind-blowing. (45:03) Let me give you an example. (45:04) Let me just digress for a moment, and then I’ll come back on how mind-blowing it is.

(45:09) When I first got involved with the Yale Interdisciplinary Center for Bioethics, I met a biologist, Art Galston. (45:22) Art invented a defoliant, and his defoliant got used in Vietnam and is what we now call Agent Orange. (45:36) So it caused all kinds of diseases for soldiers and obviously Vietnamese and others.

(45:45) And Art had no understanding that his invention, which he had just done to help deal with certain vegetation and so forth that you wanted to clear away in a farm or something like that. (46:00) And so he spent the rest of his life telling other scientists that you really have to take responsibility for the technologies you’re putting out there. (46:22) And most scientists in those days, there was a survey that Pew did with geneticists, and most of them said, what we are doing in genetics is not our problem.

(46:32) That’s for the policymakers to decide. (46:35) We’re doing the science. (46:37) They will make the decision on the application of the science, regardless of whether they even understand the science.

(46:47) So for us to suddenly, and I’m just throwing out a number, but I don’t think it’s somewhere accurate, 40,000 people caring about AI ethics, that is mind-blowing. (46:59) That is a fundamental shift in how we are looking at a technology from what has ever happened in the past. (47:08) So coming back is we have specialists formulating specific policies that they think should be put in place.

(47:19) There’s a whole community that wants to work on AI safety to ensure that we can either control technology or we know we can’t control it, and therefore we should stop its development. (47:35) And they have all kinds of recommendations on what to do, but the funding’s not there. (47:41) Or the amount of funding going into building larger and larger, large language models is 50 times the funding that’s going towards safety and security.

(47:54) So it’s not that we don’t know what to do in many of these areas. (48:00) I’m not saying we fully do in safety because there’s a fundamental question about whether if these systems reach certain kinds of intelligence levels or if they’re embedded in our infrastructure in certain ways that they might run amok. (48:17) I think that’s, you know, and we can’t control them, that’s not an answered question.

(48:23) So I would say, you know, in that area, we need to do the research, but it’s not clear what we’re going to come up with, which brings me back to why humans have to be in the center of all this. (48:35) They have to be in the center of this in two ways. (48:38) One is we can’t just be looking at the use of technology in terms of the benefits and what we think the benefits are.

(48:47) We have to look in terms of the trade-offs. (48:50) When we make a decision, we have to not only try and figure out how we can maximize the rewards, but we have to look at the various options and what the trade-offs are and whether some of those trade-offs are real harm for not only humans, but for the planet and other life on the planet. (49:13) And we have to look at whether we have a means to ameliorate those harms.

(49:19) So unless you’ve done that full analysis, you really don’t know what the best way forward is. (49:26) So that’s what I call trade-off ethics, and I’m trying to get it out there more and more, but it’s not enough to look at the greatest good for the greatest number if you haven’t also done that analysis of the trade-offs and select the path where it’s the greatest good because you can also ameliorate a lot of the harms. (49:48) So that puts humans at the center because the way humans will be harmed becomes core.

(49:56) If a hundred thousand jobs are going to be lost if you institute this technology, then what happens to those humans? (50:07) Are you going to educate them to do other jobs? (50:13) Or if jobs are just disappearing, then by what means do they get food, pay for their needs?

(50:26) Maybe we need a guaranteed minimum income or something like that. (50:30) I’m not pushing those solutions right now, but I’m just saying even in a technology that’s going to destroy jobs, if we aren’t looking at the trade-offs and how to ameliorate those trade-offs, most of which are human trade-offs, then all we’re doing is giving license to whoever’s going to make a billion dollars off of technology to do whatever they want to do. (50:57) And who cares whether we have a government that is really in place that is going to take care of people who have been harmed by that technology?

(51:08) So we are not looking at the big picture. (51:10) We’re looking at small pictures and we’re looking at ways in which a certain group will be benefited. (51:17) I was at a UN lunch once where I won’t mention the vice president of the major corporation who gave a talk at that lunch.

(51:30) The lunch was populated by representatives at the UN in Geneva, by all the countries. (51:38) And this guy talked about how AI was going to make trillions and trillions of dollars over the next 10 years or something like that. (51:50) Oh, and another exec got up and he talked about how AI was helping save energy.

(51:57) This was three years ago. (51:58) Helping save energy because corporations were becoming more efficient in their use of energy. (52:04) And I got up and I said, I don’t care how many trillions of dollars AI is going to make when 90% of those profits are going to go to the same small percentage of stockholders.

(52:22) And I don’t care how much energy efficiency you say that AI will create when AI itself is using much more energy to run than all the little efficiencies it’s helping corporations. (52:37) That’s kind of thinking that we aren’t doing. (52:41) Of course, the audience loved it because most of them were representatives of small countries who knew they weren’t getting any of the benefits at all.

(52:51) But I think we have to start looking at what not just AI is, but what emerging technologies are doing in general. (53:02) And look at what the impact on humans is going to be, not just what the capital benefits are going to be. (53:09) And think about how we can reorganize our societies so that at least some of that capital, some of those benefits ameliorate the harms.

Melissa

(53:21) Yeah, well said. (53:22) I agree. (53:24) Now you’ve been studying autonomous systems long before it was cool or made the headlines on any paper, any place.

(53:36) So what guardrails do you absolutely need as generative AI and autonomous technologies scale into not just our corporate life, but our everyday life?

Wendell

(53:51) Well, when we were originally looking at autonomous systems, the two prototypes we had were drones and self-driving cars. (54:04) And the point was, those were basically not single purpose, but they function within a limited realm. (54:16) And we could point out all kinds of ways in which they could cause harm.

(54:20) And the problem was that self-driving cars, for example, Tesla introduced its first autopilot without really thinking about human behavior or what the system could and could not do. (54:38) So people were jumping into the backseat and cracking champagne open, where Tesla still had the illusion that if an accident happened, they were going to be held, the people were going to be held responsible because they had warned them. (54:52) Well, no court was going to hold up that you create boredom for a driver for hours.

(55:00) And then he was supposed to jump in at the last second and save the day. (55:06) None of that was going to happen. (55:07) So the point was, we were pointing out that there was a whole realm of areas, and Tesla knew that and all the other companies knew that, where autonomous cars could not function, or we didn’t know what they should do.

(55:21) And some of that came up in the form of trolley problems, such as your autonomous car is going over a bridge and it can either kill five workmen or it can drive off the bridge and kill you. (55:35) What should it do? (55:37) We’ve been developing these trolley car problems as a way of just accentuating different ethical concerns for years.

(55:44) So the point about all this is, these were even limited purpose systems. (55:50) Now you get into generative AI, you don’t actually know what realms it may act in, let alone that you have thought ahead of what it can do. (56:01) So interestingly enough, after 20 years of people criticizing, you know, coming autonomy and the difficulties with it, it got renamed.

(56:16) It’s now called agentic AI, that the systems themselves have agency. (56:22) But it’s the same problem. (56:24) Nothing has changed.

(56:26) The only thing that has changed is the more general your AI is, the more realms it can function in, the more variables come into play that you can’t be aware of, and there’s a good chance that the system is not aware of. (56:45) And this is the problem. (56:46) When a system is giving even what it thinks it’s the best advice to a depressed 12-year-old, it does not know who that depressed 12-year-old is.

(57:00) It does not know which of its advice is appropriate. (57:03) It does not know whether it’s feeding into a tendency to commit suicide. (57:09) So this is the difficulty, is we are creating the illusion that agentic systems are a good thing, when agentic systems may be good or create benefits in some areas, but they can create really serious harms because of the limitations of what they understand, and the probabilities that they’re moving into more and more environments with variables that no one had ever thought would enter.

Melissa

(57:39) So what are the lessons we can take from past tech failures?

Wendell

(57:50) Well, unfortunately, the main lesson, basically what we do from past tech failures is something failed, and then we assign a lot of committees to figure out what went wrong. (58:05) Some of that what went wrong, we could have deduced by putting more money and time and expertise into analyzing the systems before they deployed, but we don’t want to do that because that will lead into the profits, and that may slow down the deployment of the system. (58:26) And if we slow down the deployment of the system, China is going to get ahead of us.

(58:32) I mean, we’ve got this rationale for becoming abusive and sensitive, not engaged in due diligence. (58:45) And when we talk about people, we are not engaging in informed consent. (58:50) I mean, we aren’t asking people off and off.

(58:53) We do in the medical field now, likely because of Nazi experiments and other horrific things that happen. (59:01) We do ask for your informed consent, whether you’re informed or not, when you give your consent is another question. (59:07) But that’s a bioethical question.

(59:09) That’s not at the moment, you know, an AI question. (59:14) But when these companies put these systems on the road that are semi-self-driving, you know, they can’t do everything. (59:26) You are a guinea pig.

(59:27) This was an experiment, you know, and the public was never asked whether it gave us informed consent for self-driving cars. (59:38) Now, it might have gone along with it because people were very excited about the idea of self-driving cars. (59:44) Maybe a little less excited now, because I think we now know that none of these cars do everything.

(59:50) You know, there’s back roads I would never take a self-driving car on. (59:57) So we’re in that, you know, we’re in that crazy universe where we’re intrigued by the technology. (1:00:07) We want to see what comes next.

(1:00:10) And yet nobody’s engaged in due diligence or putting the safeguards in place in advance of deploying the system.

Melissa

(1:00:21) Yeah. (1:00:24) Lots to ponder. (1:00:25) We could talk for hours on that subject.

(1:00:28) So I have to ask, you know, you’ve been called the godfather of AI ethics. (1:00:34) How did you land that title? (1:00:36) And was there a moment in your career that you realized that the world needed a clearer moral compass for emerging technology?

Wendell

(1:00:48) Well, I’ve been interested in the issue of a moral compass in humans, let alone anything else, my whole life. (1:00:57) I mean, I’m fascinated by the questions of, do we or can we know what’s right, good, and just? (1:01:05) You know, and we can’t always, you know, but therefore I’m interested in the psychological tools that humans bring to bear and whether they have a moral compass.

(1:01:19) And I actually have come to an understanding that they don’t have a moral compass, but that the entire human organism is in a sense a human biological, a moral biological navigation system when used in an optimal way. (1:01:36) I understand that’s far out. (1:01:39) We’re not going to get too into that, but I’m just saying that’s the precursor to my getting into AI.

(1:01:47) Then when I got into emerging technologies around the 2000s, I mean, I’d already been running computer companies and doing other things. (1:02:00) But when I got into it, there were only a few hundred people in the world who cared about these issues. (1:02:06) And half of them were transhumanists, people who wanted to enhance their own abilities through technologies, you know, be smarter, live forever, all these different things.

(1:02:17) And they were very optimistic that the technologies would be there within 10 years, even though none of them were there yet. (1:02:24) And most of them still do not exist. (1:02:28) So I was really one of the few people who entered the field almost purely on trying to understand human psychology and trying to understand ethical concerns that came out of the deployment of these technologies.

(1:02:46) So from the get-go, I was what you might call the gadfly. (1:02:51) I was always kind of sensitive. (1:02:53) Well, what aren’t we talking about, you know?

(1:02:55) So, okay, we’re talking about this aspect of autonomy, but what about that aspect of autonomy? (1:03:01) So I think for a long time, I mean, and I mentioned the story, which is much younger, around 2000 or something like that, where we started the AI and Inequality Project. (1:03:19) So that’s always been my modus operandi.

(1:03:22) I never the guy who jumps in now and figures out all the details and works in that area. (1:03:29) So I just think that one of my colleagues, John Hayden, who had quite a bit of influence in the IEEE, people throughout the AI unit universe knew him because he had run a project on ethical design in AI. (1:03:47) And he started calling me the godfather, and then some other people picked that up.

(1:03:56) And so I have that title sometimes among people who know me. (1:04:03) But I’m now getting to be one of the old gray-haired eminences and with all these brilliant young women and men trying to make their mark. (1:04:15) So a lot of people don’t even know that I exist.

(1:04:18) So it’s not a thing like these three people in AI development, Jan LeCun and Greg Hitchens, and that they are now called the godfathers of AI. (1:04:36) It’s a milder title. (1:04:38) There’s never been anyone who has awarded me some distinction, though I have received a couple AI-related ethics awards.

(1:04:52) But they’re pretty small budget in comparison. (1:04:54) So it’s more just, I think, an acknowledgment by people who know what I’ve done, that I laid the foundations for a lot of things that have become big subject areas now. (1:05:06) And, of course, Moral Machines, which I wrote with Colin Allen, has had quite an impact.

(1:05:12) And it’s been cited a couple thousand times, which is unusual for a book that’s largely about ethics and and…

Melissa

(1:05:25) So what surprised you the most about how quickly AI ethics entered the mainstream?

Wendell

(1:05:34) That it did. (1:05:36) I mean, you know, I don’t think I created the term AI ethics. (1:05:42) I know that Colin and I created the term Moral Machines.

(1:05:46) And, you know, I know there are other terms out there that I was probably the first or one of the first people to use. (1:05:55) But I didn’t create AI ethics. (1:05:57) Other people started talking about AI ethics, particularly young women who were doing research about bias in AI algorithms and their outputs, you know, how they would be intrinsically biased, not just in racial and gender issues, but even around cognitive subject areas, you know, poor on statistics and so forth.

(1:06:21) So, so, you know, I mentioned the story of Art Galston. (1:06:28) And, you know, I worked in the Yale Interdisciplinary Center for Bioethics. (1:06:33) And it was always my sense that ethics was kind of got the short stick of everything.

(1:06:41) It was very hard to get people to give attention to the ethical dimensions. (1:06:48) And that suddenly in AI, the ethical dimensions were getting a lot of attention. (1:06:57) And that was the mind blowing thing that really there are so many people who have latched on to that.

(1:07:03) There’s suddenly tons of jobs for AI ethicists within corporations, within universities, probably nowhere near the amount that are needed. (1:07:17) But that’s quite something, you know, and I don’t take credit for that. (1:07:23) I just think that I was one of the catalysts, but we catalysts were able to get across that this is serious enough stuff that, and it’s going to have far reaching ramifications because AI is touching every facet of society.

(1:07:45) And it looks like it’s going to be here until we have the Butlerian Jihad. (1:07:51) That’s an allusion to the Dune, Dune science fiction, where people who’ve got watched the Dune movies may have noticed that there’s no computerism. (1:08:04) And there’s no computers because thousands of years ago, there was an uprising where the humans rose up against computers and destroyed them all because they were about to destroy the humans, you know, the terminated division.

(1:08:20) So short of that, AI is having a profound effect, perhaps too much of an effect, and certainly an imbalanced effect on our lives. (1:08:30) And therefore, AI ethics just caught on. (1:08:33) It was something that was needed.

(1:08:37) And, you know, when I got into it initially, you know, obviously I sort of sensed that, but I didn’t, I would have never guessed that, that I would be obsolete, you know, within 20 years, because there would be so many people jumping into the field.

Melissa

(1:08:55) Yeah. (1:08:55) Yeah. (1:08:55) I love it.

(1:08:56) So kind of a couple of final questions for you, for leaders that are navigating innovation, ethics, pretty much all at the same time, what’s one question you believe they should sit with for a bit after they listen to this podcast?

Wendell

(1:09:19) The one question is get your team together and talk more comprehensively about the technology. (1:09:30) Let it be free flowing initially, even if it’s just one session, you know, you know, they’re at the beach and they have a couple of meetings and something, where you start to think comprehensively about the technology, not just how it’s going to give what your corporation wants your team to produce. (1:09:53) And think of it both in terms of what could go wrong, what could be, what could be the societal consequences of this, if it’s going to destroy, for example, a group of jobs, should that be our concern?

(1:10:13) You know, just to have that kind of discussion, and that kind of discussion doesn’t necessarily lead to answers, but it actually makes you much more sensitive than just jumping into a project and say, how do we implement this decision so that our system can save our company, whatever, a million dollars a year in lawyers fees? (1:10:38) You know, that’s not understanding what you’re engaging in. (1:10:44) And you aren’t just engaging in making a technology work.

(1:10:50) Character AI is not just creating a tool for children to interact with and perhaps get educated with, they were creating a tool that could destroy children’s lives. (1:11:05) Yeah, and certainly didn’t, they either didn’t give that attention, or they were kind of aware of that, but they knew that thinking too much about that would get in the way of our going ahead. (1:11:16) Now thinking too much can be good.

(1:11:18) You can, your team maybe after that weekend is going to go back to your management and they’re going to say, you know, this project looks really good, but it’s going to make our company $250,000 a year. (1:11:33) We’re opening up liability within X area and in Y area, we’re hurting this community of people, you know, and maybe we should give the, maybe you, meaning the management should give this a little bit more thought, whether the benefit we’re deriving is worth the harm we’re creating.

Melissa

(1:11:55) Yep. (1:11:56) A hundred percent. (1:11:57) I love it.

(1:11:59) A couple of final thoughts, anything you want to leave with our listeners that we haven’t touched on today. (1:12:06) And then for anyone who wants to dive deeper into your work, where’s the best place to find your books, talks and current projects?

Wendell

(1:12:17) Well, this talk and a number of talks are on YouTube. (1:12:21) So, you know, it’s not too hard to find me. (1:12:23) I don’t write as much anymore.

(1:12:26) So, but A Dangerous Master, which is really a primer on emerging technologies, what the sciences are, you know, and, but it emphasizes the downside. (1:12:40) A Dangerous Master was published in a paperback version last year by Sentient Publications. (1:12:47) And for that paperback version, I wrote a new introduction, trying to get people up to date on how they should read the book, but also on what had changed in the technology and whether it made a difference because the book was written before.

(1:13:03) So, you know, things are always happening, but the book happened to be written before generative AI, for example. (1:13:10) So, you know, I wanted to get that in. (1:13:13) So, I think if you want to, when I wrote A Dangerous, I think, strangely enough, A Dangerous Master is not that out of date, even though I made a couple of predictions that I now think are wrong.

(1:13:28) But it’s really, I wrote this book because I felt in a hundred thousand words, what would I want the intelligent reader to understand about emerging technologies so that they could join into this conversation? (1:13:46) Because I hope I’ve given you a taste of it, but this conversation is fascinating. (1:13:52) It’s not just the ethical dilemmas, it’s all the philosophical dilemmas, you know, converge on it, you know, what is consciousness and what does it mean to be human?

(1:14:04) You know, and is the world, is the world all, oh, wow. (1:14:25) My apology to your viewers, but, you know, is the world just matter or is there spirit or is there consciousness that is not just matters? (1:14:36) All those fascinating questions come up in AI, you know.

(1:14:42) Can machines be smarter than us in all respects or is there something about humans that gives us certain sensitivities that the machines can’t have? (1:14:53) Fun topics, at least I think they’re all fun topics, you know, and they’re coming up in a way where you don’t have to be a philosopher to talk about them, you know. (1:15:03) You don’t have to know all these names of dead men and what they, you know, what they said.

(1:15:10) You can deal with it in terms of your own intelligence, your own sensitivity to what’s going on. (1:15:19) But as I say, if you buy A Dangerous Master, and I do not mean to be cruel to Basic Books, which published the hardcover version, but buy it as a paperback because that has the new introduction in it.

Melissa

(1:15:37) I love it.

Wendell

(1:15:39) And other than that, if you want to know more about me, you know, there are plenty of YouTubes out there. (1:15:45) You know, if you just google me, you know, you’ll find a lot. (1:15:49) If you’re in the business world and you are interested in how what you should be thinking about as you introduce AI into your technology, there was a report that more than 100 experts worked on for over five years that has come out of the IEEE on the governance of AI.

(1:16:13) And so that was very much written for the business leader and what they should think about as they try and implement AI into their company.

Melissa

(1:16:27) Wendell, thank you so much. (1:16:29) Your voice has shaped how we think about emerging technology. (1:16:33) And I’m so grateful you spent time with me today and our listeners.

(1:16:38) That’s the Executive Connect podcast.