Pondering AI

[MUSIC PLAYING]

KIMBERLY NEVALA: Welcome to Pondering AI, brought to you by SAS. I'm your host, Kimberly Nevala. This episode, which marks our 100th - somewhat amazingly, to me anyway - provided an opportune time for us to ponder what is afoot in the race to innovate with the illustrious Renée Cummings.

Renée is a professor of practice in data science and a data activist in residence at the University of Virginia. She is also an active and esteemed participant in the broader AI ecosystem where she serves, amongst other things, as a nonresident fellow at the Brookings Institute and on several World Economic Forum councils.

Renée, welcome back to the show.

RENÉE CUMMINGS: Well, thank you so very much. It's an absolute honor and a pleasure to be back with you, Kimberly.

KIMBERLY NEVALA: Oh, the pleasure is all ours. I actually looked back and I realized it has been just over five years since we last had the opportunity and the honor to record with you. And as I was thinking about that, the only thing that kind of kept going through my mind was that the more things change, the more things stay the same in some way. It seems both a blink of an eye and an entirely different time way back then. So great to get you back on to talk about that.

And I thought we'd start-- and we might actually end with the same question with a slightly different slant. You said, or have said, history will not ask how intelligent our machines become, it will ask whether we remained wise enough to govern them. Looking back or casting our gaze back over the last five years give or take, I wonder how you would grade our track record thus far. So, where have we or are we exceeding expectations? And where, broadly, do we require some improvement?

RENÉE CUMMINGS: Well, I think we're still in that challenge space. We're trying to figure out how to govern AI. And while we were trying to figure out how to govern AI, generative AI came, and then agentic AI came, and then superintelligence is being pondered. So, we were left in a space of trying to govern in real time and trying to govern so many things. But trying to govern things that we cannot see. And that remains our major challenge in the space of AI.

When it comes to the grade, we have gone through so many phases since we last spoke. At one time we were speaking about diversity, equity, and inclusion and justice. Then we started to speak about responsible and trustworthy AI. Then we started to speak about AI safety. And we're still talking a lot but not generating the kinds of responses that we are hoping for. So, I think in the realm of AI governance we are yet to hit that peak. We are yet to put our hand or touch the North Star that we have been speaking about so far.

When it comes to grades, well, I'm in the business of grades, right? I teach at the University of Virginia School of Data Science, and I have, I would say, the best data ethics class in the whole world and the best data scientists in my class. When it comes to a grade, I'm going to give us an A.

You know why I'm going to give us an A? Because we have persevered and because this is so new and because none of us had this experience and because none of us understood and continued to truly understand the power of this technology. We could only imagine, and our imagination is only that much when it comes to truly understanding the imagination of AI. So I will give us an A for the attitude that we continue to bring. And for the aspirations that we have to govern wisely, to govern boldly, govern responsibly, and to govern for all of us.

KIMBERLY NEVALA: Well, I have to say, that response probably encapsulates in the most profound way why I personally adore you. But also why I think your work resonates so much with folks. Because you somehow manage to balance or deploy both radical optimism with radical honesty about the problems and issues that we are facing.

And that is not an easy path to walk or a balance to strike. I know I don't believe I have the capability of doing that in many cases. How do you strike that balance? Is that a natural attribute for you or is this something that you have to also practice?

RENÉE CUMMINGS: Well, I think it's natural with me, because I think all my life I've been committed to this question of justice and fairness. And even my career, beginning my career as a journalist and a broadcaster; working for a long time in substance abuse therapy as a therapist and then becoming a criminologist and just working in the space of criminology and terrorism studies. All the work that I've done, just from journalism to now, I've realized that justice has always been the theme that I work with.

And I believe where there is justice there is joy because there is nothing more joyous than an individual who has just received justice. And I think if we are to live in this world, equity has got to be the center, because there's so much greatness that comes from all of us.

And it's one of the things that I realized, particularly working in the criminal justice system. Working with juveniles and looking at the trajectory of a juvenile life in the criminal justice system and how much is taken away. When you incarcerate someone, you incarcerate the physical body but you don't incarcerate the mind. And we always have an opportunity, even in those types of difficult spaces, to embrace joy and to embrace the things that feed the soul.

For me, coming into AI and coming into AI about 10 years ago and coming in looking at these algorithms that were creating these zombie predictions and overestimating the risks of Black and Brown defendants, and, of course, poor white defendants as well in the criminal justice system, I always said that AI had to be the conscience of how-- we need to bring conscience to the ways in which we were thinking about AI.

And I always said that criminal justice needs to be the conscience of AI. Because in criminal justice, we came face to face with certain things. The black box that was making decisions based on algorithms and based on computation and code that no one could see. But making decisions about people's lives of course, liberty, freedom, critical decisions.

Also, in criminal justice, we came face to face with human rights and civil rights versus proprietary rights and trade secrets that were attached to the algorithms, the code, and the black box. Also in criminal justice we have this basic tenant: the ability to face your accuser. And that accuser was an algorithm. So criminal justice broke it down for us in real time. Then 2016, ProPublica came with that expose that looked at machine risk and looked at recidivism algorithms and this question of reoffending and public safety and due process.

Due process is the spirit that brought me into AI. And I think it is the spirit that continues to feed me. But also, my imagination. I've always imagined a world, or maybe a world where we can all just enjoy the benefits of it all. And I think it's that combination of imagination and justice that brought me into the space and continues to excite me in the space.

Plus, I love technology. I love AI. And I'm here and I just want to bring what I can bring to it to ensure that when I've left the space, the people who are coming into the space, they have the chapters already and they can build on those chapters.

KIMBERLY NEVALA: So, there's an interesting juxtaposition there in some ways when I think about having the imagination or even, in some cases I think, the confidence or the bravery to imagine a different way forward. And then AI as a mechanism to enable that, because in a lot of ways, we are - through these systems - and you're talking about these models who predict the future based on the past and in some cases can put you there, that are very rooted in the past.

And so how do you think about that? Is that a dichotomy? Did I just make one up just for the sake of finding a problem? Or when you say AI will be our conscience, is it because it identifies those issues that we can then think about differently? How do you think about AI in this space?

RENÉE CUMMINGS: Well definitely. To me, AI is our imagination, and we need to bring an ethical imagination into the space of something that is having a long lasting, lifelong impact on humanity.

The ability to imagine and to create the world that we deserve is a spectacular task that we have at hand. And for us to truly do that, we've got to understand the individual responsibility and the collective responsibility that we have. We are not playing only with lives, but we are playing with the future. And when you have the opportunity to hold the future in your hands you truly have to understand what are the responsibilities that are required.

And that's why I'm so passionate about AI governance. And not a static kind of governance, but a dynamic, flexible, adaptable, inclusive type of governance that we've got to create. So, as we are bringing the creativity to the technology, I think we need to bring more of that creativity to the governance, the ways in which we are thinking about governance, the ways in which we're designing the frameworks. And again, who do we bring into the room as we start to decide what governance is going to look like?

So that imagination is not only for the technology. That imagination is not only about how we innovate. That imagination has also got to come to the space of how we govern. And, of course, the imagination, the ethical imagination, which is so very powerful to ensure that we are not only accountable and transparent in the ways in which we are doing things, but we put the human being, we put us at the center of all of that work.

KIMBERLY NEVALA: Yeah. And you've said that part of the problem here is that we are trying to govern something that is invisible. What do you mean by that and how does that then show up in how we have approached the problem to date? And then let's talk about some of that moral imagination and how we might shift how we're approaching this practice of governance.

RENÉE CUMMINGS: Why, definitely. So everyone's speaking about algorithms. A few of us can define it. Some of us really don't know what it is. Some of us hope we know what it is and think we know what it is. But if you were to ask the average person, what is an algorithm? They really don't know.

But this algorithm is making decisions about us daily; hundreds if not thousands of decisions about us. And we don't know what those decisions are. We don't know what data is being used. We don't know how that data is being formulated. We don't know what sort of calculations are being used to determine our legacy, our future, the kinds of access we're going to have or not going to have. And then we deploy these algorithms in a space that we cannot see.

So we are hopeful. We are hopeful that the algorithms are going to do what we want them to do. But then we hear about things that they're not supposed to do that they're actually doing. And then we call them hallucinations or we say it's drift but it's more than that when it's impacting lives in very real ways. When an algorithm can take a child away from a family, or when an algorithm can incarcerate someone, when an algorithm can deny you a loan or a scholarship or a home. These are the things that we use to build our legacies, our future, our generational wealth.

So if we are to govern this thing that we are deploying and this thing that we cannot see, we have got to bring a really sophisticated level of risk assessment and risk management and crisis intervention into the space. Because what we don't want to do, and we've seen it happen, is continuously deploy things that are going to hurt people. And when you are deploying things into particular communities that are already challenged, already vulnerable, already high needs, we're going to have some real issues that we can deal with.

But over the last five years, I've also come to realize vulnerable communities in the world of AI means all of us. We are all vulnerable. Our age, our gender, sexuality, geography, none of us is exempt from that level of vulnerability or that level of optimization that is often weaponized.

So we have to even reimagine certain terms because when we started talking about vulnerable groups, we were speaking about Black, Brown, and poor. And now when we speak about vulnerable groups we are all in a group. If it's our age, if it's our gender, if it's our sexuality, if we're from the Global North, if we are a particular kind of worker. So now we've had to expand our own imagination when we're thinking about AI.

So how do we govern all of those things at the same time but yet maximize the benefits of the technology? How do we engage the promise of AI but protect us from the peril? How do we do that? What is the balance? And every day we've got to recalibrate because every day we're seeing new things. So, for me, it's about that ethical imagination. It's about the imagination of innovation, and it's also about the imagination of governance.

KIMBERLY NEVALA: So I want to talk about the mechanisms. But when you said we need to reimagine or maybe redefine some terms, are there other key terms that you think we need to just really challenge or redefine here?

RENÉE CUMMINGS: Well, we're still defining AI and we're still trying to figure out what AI safety looks like and we're still trying to figure out what AI governance looks like and we're still trying to figure out what superintelligence is going to look like. But as we're figuring out, we're deploying.

So I think what we've got to redefine or reposition or reimagine, it's that whole approach to what governance is supposed to look like and feel like. think we're bringing very traditional approaches to the ways in which we are governing to something that is very nontraditional and we've got to figure that out.

KIMBERLY NEVALA: So can you provide some examples or highlight how a traditional approach fails in the current instance and what a more modern, reimagined, more responsive approach might look like?

RENÉE CUMMINGS: Well, we continue to try to build these frameworks. And we're building frameworks and deploying frameworks and people still don't know what the frameworks are. One of the things that we've continued to miss when it comes to AI is how to get the conversation to the public. How to bring the public into this conversation. Because we cannot slow down or stop the rate in which this technology is being used and the rate in which individuals are being used by this technology as well.

But we've not been able to amplify awareness. We've not been able to amplify public education and public information and public engagement and public involvement. We've got to take this not only to the kitchen table, but we've got to take it beyond the boardrooms. We've got to take it somehow to the people. And where do we have those conversations? Where do we generate those other voices? So we continue, those of us in the space, to meet in different spaces globally and to have the same conversations over and over.

So since we met five years ago, we're still having the same conversations about how to govern and how to ensure there's accountability and transparency and explainability. How do we audit the technology for accuracy? How do we reduce things like discrimination? How do we bolster things like safety and privacy? And we are still having those discussions. And this technology is being deployed at a rate by individuals and organizations and countries that have no AI strategies, that have no data strategies.

And I always say we've got to bring it back to this: there is no AI without data. Data is where you begin. Data is the lifeblood of this technology. It is everything. It is the oxygen. It is the muscles. It is the skeletal structure. It is everything of AI. So for us to truly, truly govern AI, we can't continue to have these conversations about AI without conversations first about data.

So it really calls for an interdisciplinary kind of sophistication that we need to really amplify or to optimize in the governance conversation. So that interdisciplinary now we speak about it but how well are we doing it?

KIMBERLY NEVALA: And what would we have to do to then, I suppose, break the cycle of not just having the conversation? Are there concrete steps that you see that we should be taking and aren't taking at this moment?

RENÉE CUMMINGS: Well, I would start with the first one is that we need to understand that all the rooms that we go in should not be populated with all of us. So I go into a room and everyone in the room is an AI ethicist, and everyone in the room is in AI governance, and everyone in the room - it's all of us talking the same things to each other over and over and over. And that's good. That's good. That's important.

But we need to have these broader conversations across countries. In regions where there are data workers who are risking their lives and their mental health to sift through the data and the level of toxic content that they must experience. Questions around data trauma that I speak so much about. And we continue to see the proliferation of intergenerational trauma because of the ways in which we are using data.

We need to be bold enough to look beyond ourselves and we need to be bold enough to share the space. I think we're not sharing the space with enough public voices. I think we need to bring more young people into the conversation. Because young people have been using our data and this technology in ways that we could never imagine, given our age. And they're going to be using the technology much longer than most of us.

We're seeing what's happening with emotional attachment and the attachment algorithms that are really bringing young people into a space that is volatile, in which they are not prepared and creating what I call a public health challenge in the ways in which we see young people using this technology. So there are other conversations that need to be had.

So while we are innovating - and I want us to keep innovating - but I want us to innovate responsibly. I want us to optimize public trust. I want us to optimize responsible AI and to optimize things like redress and protection and awareness as well. Because these are the things that are going to make us better. These are the things that are going to make the technology better.

And these are the safeguards that are required if we want to continue building and continue deploying and to ensure that we're not undermining legacies. We're not denying dreams. We're not just extracting and exploiting and disempowering. We want to do more. I say we need to have a bigger imagination for AI than we have at the moment.

KIMBERLY NEVALA: I'm wondering if we've, in some ways, perhaps applied too much imagination in terms of how this is being hyped? Relative to looking at these solutions, these tools that are being deployed, and giving them a status that is different than any other product or service that's out there. And therefore disengaging them from the expectations we would have for things like liability or the accountability and the responsibility.

Where in some ways it's, yes, it's unlike anything, perhaps, that we've seen before, at least in the way that it operates and how we see it moving. And, on the other hand, these are still products and services and we do have existing laws. We do have existing expectations. And so I'm wondering how much the narrative - maybe it's an imaginative narrative - that presupposes AGI and all these things in the past has also contributed to this problem, if at all.

RENÉE CUMMINGS: You're quite correct. The hype, the overpromising. When we think about criminal justice and policing, AI has overpromised and under-delivered and continues to under-deliver in that space. If facial recognition technology and license plate readers are all we can give in the realm of AI, then we have not given much when it comes to reimagining justice or reimagining public safety or reimagining parole or reimagining what a life sentence is supposed to look like.

I speak often of immersive justice. We need to move to a place where we ensure when someone is incarcerated for 20 years they don't come out on the date that they were incarcerated. So if you were incarcerated in 1984, when you come out in 2007, you're still coming out in 1984 in your mind. Because all of that time you spent in there you've still been in the year that you were incarcerated. How do we use AI to change those experiences?

And you're quite correct. A lot of hype, an extraordinary amount of hype. And the hype, of course, oftentimes leads to the harm. You're quite correct. For me, even just thinking about attachment algorithms, I had to go back and study the 19- I think it was, 69- legislation that looked at questions around consumer protection and child protection and toy safety. And saying maybe this is a time for us to revisit that 1969 act and look at what is now considered hazardous and toxic for children.

Now we have children using algorithms. And I say it all the time: we childproofed the home, but we never childproof the algorithm. And we have to think about that. So it may call for a revisiting of some of those laws and a reimagining what it's supposed to look like in the future.

When we made the social contract how many years ago and created government we didn't think about an algorithm. But now we have an algorithm that's functioning as a public servant. That's functioning as a politician. That's functioning as a judge. That's functioning as an undercover police officer. That's functioning as a teacher. That's functioning as a journalist. So we've got to think about that. When we created the social contract there was no algorithm, there was no AI, there was no black box.

So it's a lot of imagining that needs to happen at the same time we are innovating. But I think what we're doing, as I said, is we are bringing the imagination to innovation and we are bringing our very traditional, dated thinking to governance and what responsibility is supposed to look like.

But then again, there's that individual and collective responsibility that I speak about. And that's where we have duty of care. I always believe we bring that duty of care to everything that we do. We need to think about that duty of care and think about the liabilities and the responsibilities and the roles that need to be played when we are designing, developing, deploying, adopting, procuring all the things that we're doing around AI. We need to think about that duty of care.

But for me, I think what continuously keeps me on, as they say, the straight and narrow is due process. Because I'm always thinking about fairness. How is it going to impact? How is it going to hit? Who is it going to hit? Who is going to remain standing up and who's going to get knocked down by this? Who's going to have a fair chance and who's going to be disadvantaged? And do we really want to deploy? Do we really want to build? Are we sure? Are we certain?

And I'm constantly, I mean, doing my own sort of mental red teaming every time I think about an algorithm. Every time someone asks me about whether we are supposed to do this or to do that. And it's a kind of eternal vigilance that I bring to the space. And that comes from that interdisciplinary thinking. That comes from my commitment to justice. And the thing is that we've got to ensure that that type of bold ethics continues to lead the space.

KIMBERLY NEVALA: And does a duty to care imply that it's really on those who are deploying, delivering, and making decisions about where this technology is deployed and how it's used to be proactive? To use their imagination, put themselves in somebody else's shoes and decide, is this how I would want it to be decided for me if I were in that situation?

Because it does strike me increasingly that the public and individuals are still at an incredible disadvantage in terms of-- you mentioned it earlier, they just don't understand it. I mean, I get it. It's the, oh my God, here Kimberly goes again around the dinner table when I'm banging on about data and privacy and these elements. Until we start to talk about why is that so expensive? And why did you get that and they didn't? And people just understanding how pervasive the information is.

And that disadvantage, to me, that seems to be growing as opposed to-- yeah, it seems to be widening as opposed to being brought together. How do we address that issue from just a public individual awareness? Because right now, as far as the duty to care, we seem to-- T and Cs are not going to cut it. Asking people to opt out is not going to cut it, in my opinion. How do we even start to break that down? And I think we can, but where do we start?

RENÉE CUMMINGS: I think we should because I always say every data decision is a governance decision. So when I think about the data pipeline that we are using to build AI, I always ask my students, when we are doing this data pipeline, what is the trust pipeline that needs to go with the data pipeline and what is the governance pipeline?

So when you're doing data you're doing data, you're doing trust, and you're doing governance at the same time. Because there's a trust ecosystem, there's a governance ecosystem, and there's a data ecosystem that must come together to build an algorithm. Mainly we focus on the data ecosystem from when we gather the data to data in the afterlife. But at every stage of the game, or every stage of that continuum, there are trust questions. There are governance questions that must be answered.

So for me, we need to focus on that trust ecosystem. We need to focus on the governance ecosystem. And once we get that right when it comes to the data ecosystem we're going to be building better algorithms. We're going to be building the algorithms that can support and sustain the algorithms that are going to be resistant and reliable and mindful and equitable and justice oriented and trauma informed.

Because those are all the things that we need to build in this technology. Because this technology is not only doing one thing. It's not about the workflow alone. It's about the workflow. It's about the work life. It's about who's going to be working and who's not going to be working. It's about who's going to get the kind of access that's required to build the future and who's not going to get. So it's a very important thing for us to get those other ecosystems right as we build and as we deploy and as we engage with the technology.

KIMBERLY NEVALA: How, though, do we make sure that individuals and us as the individual collective, as the societal collective, really have a seat at the table and the power to decide when it comes to where, when, and how? Because again, there's an asymmetry here. That even if we're looking at the data workflows and the system workflows and then we're thinking about trust, that definition of what is trusted - or what is trustworthy enough - still may be coming from those of us who deploy the technology on behalf of those who will then use it.

And I want to come back to this rather striking turn of phrase you used a little bit earlier. But where does the rubber meet the road there? I mean, are you seeing elements where we can concretely level the playing field here in such a way that individuals have a chance of having a seat at that table?

RENÉE CUMMINGS: Well, I think we cannot level the playing field. I think that playing field was taken away from us a long time ago and we are continuously trying to play catch up. And we'll never be able to catch up with this technology. When we started to share and to like and to click and to post and to drop those emojis and to just share all our content into this thing called social media we had not a clue what we were doing, or where that data was going to end up, or what was going to be inferred by that data. We did not have a clue.

So now data has become the world's greatest geopolitical asset and game changer. And we still don't know how it happened. But we do know that somewhere in that big data ocean, our data is in there as well. So I think it's this question, and it's a late question - and it's a very late question - it may be able to protect the next generation who are engaging with the technology. But just questions around data.

Back in the day, history shows us that when we started to have questions around democracy, it started with the gurus and the scholars and people who sat under trees. And they had these conversations with philosophers about the world and about the universe and all of that. So maybe it's time for us to create some algorithmic trees, other trees. Not the trees to build, but we've got to create spaces for people to have conversations.

We've got to ensure that when-- and we're seeing it, because we know that change always comes - not always, I'm sorry - change often comes from legislation and what happens in the court. And we're seeing more and more cases going to the courts. And we're seeing, unfortunately with children, suicide and mental health challenges that are now also taking place in court conversations. We're seeing that sort of forensic approach when it comes to deconstructing how these algorithms can hurt. Or the kind of harm that they create and what is the requisite level of help and intervention that's required. So maybe in examining what's happening to children, we're going to start to see some changes as well.

It's all about awareness. Because I say this: we need data. We do. We need data for the things that we want to do and for the things that we want to build. I'm fascinated by health care and longevity and the things that we can do with AI when it comes to brain disease or degenerative brain challenges. And how we can change the course of people's health and people's mind and to really bring a longer life and more prosperity. Those are the things that excite me when it comes to this technology.

Deploying it in the police service is nothing exciting. We've seen that old story before and we can do better things there. But there are great things that we could do when it comes to education. When it comes to just keeping the world engaged. When it comes to innovating intelligence. And that's what I love. How do we continue to innovate intelligence?

When I think of neurodivergent communities, when I think about individuals with disabilities, when I think about people who can now really get the access that they want with this technology. But then when you're thinking about that, you think about how much the data still excludes people with disabilities or neurodivergent populations or individuals who are elderly or people who speak another language or people who communicate in a different way. We see that this technology is still very one sided in the ways in which it's balancing the future of intelligence.

So how do we do this? More conversations are required. And I'm pleased to see that more and more universities are standing up the kinds of courses that are required around data ethics and the ways in which we reimagine data science and data justice, as we have at the University of Virginia, and that social justice perspective.

And of course, at UVA School of Data Science, we have a values approach to data science that must look at things like integrity and injustice and social impact and the ways in which we are using data science for good. We are committed to that. We are committed to ethical data science. And we're committed to bringing the most amount of interdisciplinary engagement to our students because we know the responsibility they have. I always say to my students, you are the architects of the future as data scientists. And it is a real responsibility that you have got to think about, because every data point brings with it a governance decision as well.

So what we need to do, as I say, is just expand the conversations. Because we've used social media to expand some very toxic conversations. So we can do that. We've amplified the deception and the disinformation. So now it's time to amplify the real emotion. But you have to have the will, the political will, the social will, the organizational will to do what needs to be done.

But I'm hopeful, because there are so many people now in the responsible AI space and trustworthy AI, in AI safety. And so many young people who are just resisting from the perspective of we need to know more. We need to be involved. We need to really, truly build something with this technology that really lifts up humanity. So I am hopeful. There are challenges, but I am hopeful because every day I have a class of over 100 sometimes data scientists.

And I am always hopeful when we start to speak about the ethical questions and about this individual and collective responsibility to hear how they are thinking and how they feel. And these are active individuals, adults. Many of them who are already working as data scientists at some of the top companies at the top leading industries and many of them captains of industry themselves. But truly now understanding that we've got to course correct and we've got to really see what we can do and continue to move forward at the speed - we could probably never do it at the speed is with this technology is being built - or as fast as we can move with this technology.

KIMBERLY NEVALA: Yeah, and I really appreciate that you called out the imagination around even intelligence. Which I don't think you intended, didn't mean - but you tell me if I'm wrong - just to be predicting or thinking forward to something like AGI. But the ability for us to understand that there are different forms and spectrums and ways that intelligence shows up that we need to honor within the spectrum of humanity already.

And we happen to be recording this during what I just learned recently was a Disability Pride Month. And we had a fantastic episode with Maitreya Shah from the Association for People With Disabilities; AAPD - I may have gotten that backwards. I'm going to have to go back and check. Apologies, Maitreya. But it was an eye-opening conversation in so many ways, and one that I almost think is a public service.

But it goes back to that point that you make that we have to be having conversations with really everybody involved and at the table. And listening to those different perspectives because what we might imagine for somebody who is different than us is not what they imagine for themselves. And what we each imagine for ourselves is what - as I walked out of that conversation - what we should be striving for is enabling what each of us imagines for ourselves, not what someone else imagines for us.

RENÉE CUMMINGS: Exactly.

KIMBERLY NEVALA: Now, earlier, you had a rather striking phrase that stopped my listening in its tracks. You said we also need to attend to how people are being used by the technology. And I know you are very passionate about some of these areas. You've mentioned them a little bit in terms of affective technology. Tell us a little bit about the areas right now that you are really most concerned or most interested in addressing and how that plays into the AWARE campaign that I know you have kicked off.

RENÉE CUMMINGS: Certainly, definitely. So at the moment, I would say it's about innovating intelligence. And as I said to you, I've had a very diverse journey to this space. And all the spaces that I've worked in, as I said, from journalism to working in substance abuse treatment as a substance abuse therapist, working as a rehabilitation therapist, doing a graduate degree in disabilities rights many years ago when it was just brand new. I think all of those things, as I said to you, was about justice.

And intelligence is now the new sort of justice space that we are in. And due process is so critical to intelligence because, as you said, there are diverse intelligences. Intelligence does not belong to one company or one country or one demographic. Intelligence has always been the most diverse space ever. Intelligence is creativity and we need to think about that intelligence is imagination as well.

So that is what keeps me going and keeps me excited about this technology. Because I'm seeing always the great things that could be done with this technology if we, as they say, open our hearts and open our minds at the same time. We can do some truly amazing things with AI.

The things that keep me keeping on, as they would say, in this space is that question of justice and that question of due process. And how do we get this thing right? And the thing about getting it right when it comes to AI, because this technology moves so quickly, getting it right might look like one thing yesterday and it might look like something else tomorrow. Because we've got to be flexible and we've got to be adaptable when it comes to this technology.

But as a criminologist and someone who's been a criminologist for over 20 years and someone who is committed to child protection and child wellness and development, I realized from early that this was becoming a public health question when we think about the impact of children. And I realized that the algorithm, again, was not something we thought about when we designed early child protection legislation.

When we think about child welfare, we never thought that we would have to engage with a chatbot. When we thought about stranger danger, we were expecting people to be on the streets and people to be by the schoolyards or people to be by the corner stores and bodegas and in the malls. But so many of these things are not the spaces where we're seeing children are being groomed and children being recruited into a particular kind of behavior, or children are gaining the confidence of an algorithm or a chatbot.

And it's really expanded what we need to do when it comes to understanding child safety online. It's become a catchphrase, right? Child safety online. But what is it really? What is it really? Because we've moved and many of us were those children who sat in front of the television, was doing the babysitting. Remember? That was something that we were all exposed to. And as a parent myself, I did a little bit of that. When you wanted to get something done, you put on cartoons or those great shows on PBS and you just put the child to sit there.

But now we have the tablets and now we have the phones. And now a lot of parents still don't understand, because a lot of caregivers, a lot of adults, because they're also engaging with the technology. But we're seeing more and more cases. And for me, one teen suicide based on a chatbot is all I need. I don't need 2 and 3 and 13 and 14 suicides for me to see that it's a problem. One suicide. Almost like-- it's not even almost. Just like in policing and criminal justice, one homicide is all you need to know, or one wrongful death is all you need, to know that something needs to be done.

Because these are our children, their lives. They've not developed. And they're building this kind of confidence with something that they don't understand, something the developers totally do not understand as well, and don't know why it may say something. And when you are a child and you also have an imagination that does not have the kind of maturity to be discerning.

And not only children. We have adults now. We have all these resurrection tools that are being built. We have this whole conversation now about what happens to your data in the afterlife and whether or not it's ethical. I mean, we know many cultures think about death in various ways and closure in various ways. But now we're building tools to bring people back to life, to continue engaging with people. And it's creating an extraordinary amount of ethical and emotional mental health challenges as well.

So when we think about attachment it's something that I'm very passionate about. Because I just want to ensure that people live healthy lives, that there is healthy engagement, that human wellness and child wellness and child development are things that are flourishing. Not things that are going to be destroyed or create an extraordinary and continuous trauma in families.

So the AWARE campaign, which is about algorithmic wellness and responsible engagement, is really about bringing that public health approach to this conversation of engagement. What responsible engagement is supposed to look like. And the kinds of questions that we need to start having with young people and parents also understanding what young people are doing.

You're not going to drop a 10-year-old in front of a nightclub. You're not going to do that and say, well, hey, Junior, go and have a great night. We're not doing that. But that's exactly what we're doing when we give Junior that phone that they are using in their bedroom. It's like you've dropped junior by the nightclub and say, hey, kid, see you when I see you. So we've got to think about that.

KIMBERLY NEVALA: That's a terrific analogy.

RENÉE CUMMINGS: This question of-- remember back in the day when they realized that some of the pajamas we were wearing could be flammable because of the fabric being used. That was an old school one.

KIMBERLY NEVALA: I don't know why I'm laughing. That's not funny.

RENÉE CUMMINGS: No, it's not. But when we had those '70s pajamas and it was like all these things on it. And then someone realized, wait a minute, the fabric is so flammable that the child could die in it if it's too much heat. And we started to look at the fabric. Or when we would give babies a rattler and put it in their mouth. And then one day we realized, oh, it's lead poison, we need to stop. We need to remember the old school rattler.

So we've been there when it comes to child protection and toy safety and consumer protection. And I always say the best one is this plastic bag is not a toy. So someone didn't make that up yesterday. It's because someone put it over their head or tied it around their neck thinking it was a cape or did something with it. Children can turn the simplest of things-- and we've all been there. You jump into a box as a child and then you can't get out and you're stifling inside.

And we see that's our own imagination at work. And this is why we've got to revisit those conversations, because now we have algorithms doing all the simplest of things that we did not even think about. We're seeing a kind of engagement and, bringing it back to its seriousness, when you have children committing suicide you know that you have challenges. When we think about the impact of something like Instagram on the mental health of teenage girls, very, very scary.

When we think about just this level of engagement that is not healthy, we know that we need to do something because it's not only artificial intimacy and artificial attachment. We now have children believing that relationships are frictionless and that's not what it's supposed to be. I always say to the young people around me, sometimes even love needs a break. So sometimes you need to remove yourself from that phone. And I always say to them that nobody can ever love you that much. So if you think a chatbot or something favors-- nobody loves you that much.

So it's really about letting young people understand what is healthy, letting families understand who's pulling those strings, and really understanding that awareness is the first form of protection. We've always got to think about it. So AWARE is very, very important. And more and more organizations globally are standing up similar kinds of initiatives because we've got to put our children first. We've got to do that.

So I'm very committed to that. And that really comes, again, from me as the criminologist and criminal psychologist and therapeutic jurisprudence specialist. Because I always believe that we have a responsibility and we could never renege on that. It is always incumbent on us to do the right thing when we have an opportunity to do that. So that's the social justice me, always standing up to the innovative me, as well.

KIMBERLY NEVALA: But isn't it amazing that-- I have to imagine that when you started out in criminal justice and therapeutic jurisprudence, you wouldn't have imagined this is where it would lead you. But it has so perfectly set you up for the conversation that's happening today, which we are grateful for.

RENÉE CUMMINGS: Definitely.

KIMBERLY NEVALA: So of course, we can go on and on and on, and I will always take the opportunity to do that with you. But in the interest of time and not stretching your own patience here, I'll just circle back to how we were talking earlier about how history will judge us. And so my final question for you, which is: what would you like to see us change? Or what should we do - in terms of whether it's an attitude or a mind shift or something more concrete - to make sure that those that come next do, in fact, judge us kindly?

RENÉE CUMMINGS: For me, it will be honesty. There's a lot of dishonesty in the space. And we've got to be honest. And I think if we are to confront ourselves and be honest with ourselves, we will build better tools. It begins with us. It really begins with us.

So we can talk about all the governance. We can talk about all the responsible AI, all the trustworthy AI, all the AI safety that we want to talk about. But if we don't look in that mirror and be honest with ourselves as to who we are as people, as human beings, as what do we imagine for ourselves and what do we imagine for others. And I think if we start to truly deconstruct what we imagine for others and bring more honesty into that space, we can get this brilliant piece of technology right and we can do some truly spectacular things for each other.

But we've got to care for each other as we care for ourselves. And more and more people have got to really be honest about who they are, because you cannot be dishonest about who you are and tell me that you're deploying an honest algorithm.

KIMBERLY NEVALA: Well, that is a call to action I am incredibly pleased to end on and leave ringing in folks' ears. So we'll leave that there. And thank you so much, Renée, for--

RENÉE CUMMINGS: Thank you. It's always a pleasure, and I'm so happy to see you always, Kimberly.

KIMBERLY NEVALA: Well likewise, likewise. And again, just for all the work that you do and, as I said, for being a real role model for how we can bring both radical optimism and radical honesty to bear in a very challenging world. So thank you.

RENÉE CUMMINGS: Thank you very much. Thank you. Thank you.

KIMBERLY NEVALA: Alright. So to continue learning from thinkers, doers, and advocates such as Renée, you can find us wherever you listen to podcasts and also on YouTube.

SPEAKER: This has been a SAS podcast.