Daniel Browne

Four AI trends to watch in 2024

Christopher.Sorensen — Fri, 19 Jan 2024 17:02:40 +0000

Four AI trends to watch in 2024

Christopher.Sorensen Fri, 01/19/2024 - 12:02

Cutline

AI was a hot topic at this week’s annual meeting of the World Economic Forum in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket via Getty Images)

Jovana Jankovic

Daniel Browne

Topic

Our Community

Schwartz Reisman Institute for Technology and Society

Munk School of Global Affairs & Public Policy

Artificial Intelligence

Faculty of Applied Science & Engineering

Faculty of Arts & Science

Faculty of Information

Faculty of Law

Global

Graduate Students

Research & Innovation

Rotman School of Management

�鶹��Ƶ Mississauga

Subheadline

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions”

As artificial intelligence continues to develop rapidly, the world is watching with excitement and apprehension – as evidenced by the AI buzz in Davos this week at the World Economic Forum’s annual meeting.

University of Toronto researchers are using AI to advance scientific discovery and improve health-care delivery, exploring how to mitigate potential harms and finding new ways to ensure the technology aligns with human values.

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions,” says Monique Crichlow, executive director of the Schwartz Reisman Institute for Technology and Society (SRI). “Researchers at SRI and across the university are tackling how to build and regulate AI systems for safer outcomes, as well as the social impacts of these powerful technologies.”

“From health-care delivery to accessible financial and legal services, AI has the potential to benefit society in many ways and tackle inequality around the world. But we have real work to do in 2024 to ensure that happens safely.”

As AI continues to reshape industries and challenge many aspects of society, here are four emerging themes �鶹��Ƶ researchers are keeping their eyes on in 2024:

1. AI regulation is on its way

U.S. Vice President Kamala Harris applauds as U.S. President Joe Biden signs an executive order on the safe, secure, and trustworthy development and use of artificial intelligence on Oct. 30, 2023 (photo by Brendan Simialowski/AFP/Getty Images)

As a technology with a wide range of potential applications, AI has the potential to impact all aspects of society – and regulators around the world are scrambling to catch up.

Set to pass later this year, the Artificial Intelligence and Data Act (AIDA) is the Canadian government’s first attempt to comprehensively regulate AI. Similar attempts by other governments include the European Union’s AI Act and the Algorithmic Accountability Act in the United States.

But there is still much to be done.

In the coming year, legislators and policymakers in Canada will tackle many questions, including what counts as fair use when it comes to training data and what privacy means in the 21st century. Is it illegal for companies to train AI systems on copyrighted data, as a recent lawsuit from the New York Times alleges? Who owns the rights to AI-generated artworks? Will Canada’s new privacy bill sufficiently protect citizens’ biometric data?

On top of this, AI’s entry into other sectors and industries will increasingly affect and transform how we regulate other products and services. As Gillian Hadfield, a professor in the Faculty of Law and the Schwartz Reisman Chair in Technology and Society, Policy Researcher Jamie Sandhu and Faculty of Law doctorial candidate Noam Kolt explore in a recent policy brief for CIFAR (formerly the Canadian Institute for Advanced Research), a focus on regulating AI through its harms and risks alone “obscures the bigger picture” of how these systems will transform other industries and society as a whole. For example: are current car safety regulations adequate to account for self-driving vehicles powered by AI?

2. The use of generative AI will continue to stir up controversy

Microsoft Bing Image Creator is displayed on a smartphone (photo by Jonathan Raa/NurPhoto/Getty Images)

From AI-generated text and pictures to videos and music, use of generative AI has exploded over the past year – and so have questions surrounding issues such as academic integrity, misinformation and the displacement of creative workers.

In the classroom, teachers are seeking to understand how education is evolving in the age of machine learning. Instructors will need to find new ways to embrace these tools – or perhaps opt to reject them altogether – and students will continue to discover new ways to learn alongside these systems.

At the same time, AI systems created more than 15 billion images last year by some counts – more than the entire 150-year history of photography. Online content will increasingly lack human authorship, and some researchers have proposed that by 2026 as much as 90 per cent of internet text could be generated by AI. Risks around disinformation will increase, and new methods to label content as trustworthy will be essential.

Many workers – including writers, translators, illustrators and designers – are worried about job losses. But a tidal wave of machine-generated text could also have negative impacts on AI development. In a recent study, Nicolas Papernot, an assistant professor in the Edward S. Rogers Sr. department of electrical and computer engineering in Faculty of Applied Science & Engineering and an SRI faculty affiliate, and his co-authors found training AI on machine-generated text led to the system becoming less reliable and subject to “model collapse.”

3. Public perception and trust of AI is shifting

A person walks past a temporary AI stall in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket/Getty Images)

Can we trust AI? Is our data secure?

Emerging research on public trust of AI is shedding light on changing preferences, desires and viewpoints. Peter Loewen – the director of the Munk School of Global Affairs & Public Policy, SRI’s associate director and the director of the Munk School’s Policy, Elections & Representation Lab (PEARL) – is developing an index measuring public perceptions of and attitudes towards AI technologies.

Loewen’s team conducted a representative survey of more than 23,000 people across 21 countries, examining attitudes towards regulation, AI development, perceived personal and societal economic impacts, specific emerging technologies such as ChatGPT and the use of AI by government. They plan to release their results soon.

Meanwhile, 2024 is being called “the biggest election year in history,” with more than 50 countries headed to the polls, and experts expect interference and misinformation to hit an all-time high thanks to AI. How will citizens know which information, candidates, and policies to trust?

In response, some researchers are investigating the foundations of trust itself. Beth Coleman, an associate professor at �鶹��Ƶ Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information who is an SRI research lead, is leading an interdisciplinary working group on the role of trust in interactions between humans and AI systems, examining how trust is conceptualized, earned and maintained in our interactions with the pivotal technology of our time.

4. AI will increasingly transform labour, markets and industries

A protester in London holds a placard during a rally in Leicester Square (photo by Vuk Valcic/SOPA Images/LightRocket via Getty Images)

Kristina McElheran, an assistant professor in the Rotman School of Management and an SRI researcher, and her collaborators may have recently found a gap between AI buzz in the workplace and businesses who are actually using it – but there remains a real possibility that labour, markets and industries will undergo massive transformation.

�鶹��Ƶ researchers who have published books on how AI will transform industry include: Rotman faculty members Ajay Agrawal, Joshua Gans and Avi Goldfarb, whose Power and Prediction: The Disruptive Economics of Artificial Intelligence argues that “old ways of doing things will be upended” as AI predictions improve; and the Faculty of Law’s Benjamin Alarie and Abdi Aidid, who propose in The Legal Singularity: How Artificial Intelligence Can Make Law Radically Better that AI will improve legal services by increasing ease of access and fairness for individuals.

In 2024, institutions – public and private – will be creating more guidelines and rules around how AI systems can or cannot be used in their operations. Disruptors will be challenging the hierarchy of the current marketplace.

The coming year promises to be transformative for AI as it continues to find new applications across society. Experts and citizens must stay alert to the changes AI will bring and continue to advocate that ethical and responsible practices guide the development of this powerful technology.

Power and prediction: �鶹��Ƶ's Avi Goldfarb on the disruptive economics of artificial intelligence

siddiq22 — Fri, 20 Jan 2023 16:38:57 +0000

Power and prediction: �鶹��Ƶ's Avi Goldfarb on the disruptive economics of artificial intelligence

siddiq22 Fri, 01/20/2023 - 11:38

Cutline

Avi Goldfarb, a professor at the Rotman School of Management and research lead at the Schwartz Reisman Institute for Technology and Society, says the AI revolution is well underway – but that system-level change takes time (supplied images)

Daniel Browne

Topic

Global Lens

Technology and Society

Schwartz Reisman Institute for Technology and Society

Artificial Intelligence

Business

Health

Rotman School of Management

Startups

Technology

In the new book Power and Prediction: The Disruptive Economics of Artificial Intelligence, co-author Avi Goldfarb argues we live in the “Between Times”: after discovering the potential of AI, but before its widespread adoption.

Delays in implementation are an essential part of any technology with the power to truly reshape society, says Goldfarb, a professor of marketing and the Rotman Chair in Artificial Intelligence and Healthcare at the University of Toronto's Rotman School of Management and research lead at the Schwartz Reisman Institute for Technology and Society.

He makes the case for how AI innovation will evolve in Power and Prediction, his latest book co-authored with fellow Rotman professors Ajay Agrawal and Joshua Gans. The trio, who also wrote 2018’s Prediction Machines: The Simple Economics of Artificial Intelligence, are the co-founders of the Creative Destruction Lab, a non-profit organization that helps science- and technology-based startups scale.

Goldfarb will give a talk at the Rotman School of Management on Jan. 25 as part of the SRI Seminar Series. He spoke with the Schwartz Reisman Institute to discuss how the evolution of AI innovation will require systems-level changes to the ways that organizations make decisions.

(The interview has been condensed for length and clarity.)

What changed in your understanding of the landscape of AI innovation since your last book?

We wrote Prediction Machines thinking that a revolution was about to happen, and we saw that revolution happening at a handful of companies like Google, Amazon and others. But when it came to most businesses we interacted with, by 2021 we started to feel a sense of disappointment. Yes, there was all this potential, but it hadn’t affected their bottom line yet – the uses that they’d found had been incremental, rather than transformational. And that got us trying to understand what went wrong.

One potential thing that could have gone wrong, of course, was that AI wasn’t as exciting as we thought. Another was that the technology was potentially as big a deal as the major revolutions of the past 200 years – innovations like steam, electricity, computing – and the issue was system-level implementation. For every major technological innovation, it took a long time to figure out how to make that change affect society at scale.

The core idea of Power and Prediction is that AI is an exciting technology – but it’s going to take time to see its effects, because a lot of complementary innovation has to happen as well. Now, some might respond that’s not very helpful, because we don’t want to wait. And part of our agenda in the book is to accelerate the timeline of this innovation from 40 years to 10, or even less. To get there, we then need to think through what this innovation is going to look like. We can’t just say it’s going to take time – that’s not constructive.

What sort of changes are needed for organizations to harness AI’s full potential?

Here, we lean on three key ideas. The first idea is that AI today is not artificial general intelligence (AGI) – it’s prediction technology. The second is that a prediction is useful because it helps you make decisions. A prediction without a decision is useless. So, what AI really does is allow you to unbundle the prediction from the rest of the decision, and that can lead to all sorts of transformation. Finally, the third key idea is that decisions don’t happen in isolation.

What prediction machines do is allow you to change who makes decisions and when those decisions are made. There are all sorts of examples of what seems like an automated decision, but what it actually does is take some human’s decision – typically at headquarters – and scales it. For organizations to succeed, they require a whole bunch of people working in concert. It’s not about one decision – it’s about decisions working together.

One example is health care – at the emergency department, there is somebody on triage, who gives a prediction about the severity of what’s going on. They might send a patient immediately for tests or ask them to wait. Right now, AIs are used in triage at SickKids in Toronto and other hospitals, and they are making it more effective. But to really take advantage of the prediction, they need to coordinate with the next step. If triage is sending people for a particular test more frequently, then there need to be other decisions made about staffing for those tests, and where to offer them. And, if your predictions are good enough, there’s an even different decision to be made – maybe you don’t even need the tests. If your prediction that somebody’s having a heart attack is good enough, you don’t need to send them for that extra test and waste that time or money. Instead, you’ll send them direct to treatment, and that requires coordination between what’s happening upstream on the triage side and what’s happening downstream in terms of the testing or treatment side.

AI is as exciting a technology as electricity and computing, but it will take time to see its effects, Avi Goldfarb says.

Will certain sectors have greater ease in adopting system-level changes than others?

There is a real opportunity here for startups because when building a new system from scratch, it’s often easier to start with nothing. You don’t have to convince people to come along with your changes, so it becomes a less political process – at least within your organization. If you’re trying to change a huge established company or organization, it’s going to be harder.

I’m very excited about the potential for AI and health care, but health care is complicated; there are so many different decision-makers. There are the patients, the payers – sometimes government, sometimes insurance companies, sometimes a combination of the above – and then there are doctors, who have certain interests, medical administrators who might have different interests, and nurses.

AI has potential to supercharge nurses, because a key distinction between a doctor and a nurse in terms of training is diagnosis, which is a prediction problem. If AI is helping with diagnosis, that has potential to make nurses more central to how we structure the system. But that’s going to require all sorts of changes, and we have to get used to that as patients. And so, while I think the 30-year vision for what health care could look like is extraordinary, the five-year timeline is really, really hard.

What are some of the other important barriers to AI adoption?

A lot of the challenges to AI adoption come from ambiguity about what’s allowed or not in terms of regulation. In health care contexts, we are seeing lots of people trying to identify incremental point solutions that don’t require regulatory approval. We may have an AI that can replace a human in some medical process, but to do it is going to be a 10-year, multibillion-dollar process to get approval – so they’ll implement it in an app that people can use at home with a warning that it’s not real medical advice.

The regulatory resistance to change, and the ambiguity of what’s allowed, is a real barrier. As we start thinking about system changes, there is an important role for government through legislation and regulation, as well as through its coordinating function as the country’s biggest buyer of stuff, to help push us toward new AI-based systems.

There are also real concerns about data and bias, especially in the short term. However, in the long run, I’m very optimistic about AI to help with discrimination and bias. While a lot of the resistance to AI implementation right now is coming from people who are worried about [people who will be negatively impacted by] bias [in the data], I think that pretty soon this will flip around.

There’s a story we discuss in the book, where Major League Baseball brought in a machine that could say whether a pitch was a strike or a ball, and the people who resisted it turned out to be the superstars. Why? Well, the best hitters tended to get favored by umpires and face smaller strike zones, and the best pitchers also tended to get favoured and had bigger strike zones. The superstars benefited from this human bias, and when they brought in a fairer system, the superstars got hurt. So, we should expect that people who currently benefit from bias are going to resist machine systems that can overcome it.

What do you look for to indicate where disruptions from AI innovation will occur?

We’re seeing this change already in a handful of industries tech is paying attention to, such as advertising. Advertising had a very Mad Men vibe until recently: there was a lot of seeming magic in terms of whether an ad worked, how to hire an agency and how the industry operated – a lot of charm and fancy dinners. That hasn’t completely gone away, but advertising is largely an algorithm-based industry now. The most powerful players are big tech companies – they’re no longer the historical publishers who worked on Madison Avenue. We’ve seen the disruption – it’s happened.

Think through the mission of any industry or a company. Once you understand the mission, think through all the ways that mission is compromised because of bad prediction. Once you see where the mission doesn’t align with the ways in which an organization is actually operating, those are going to be the cases where either the organization is going to need to disrupt themselves, or someone’s going to come along and do what they do better.

Read the full Q&A at the Schwartz Reisman Institute for Technology and Society

�鶹��Ƶ expert on human-centered data science – and the problem with the motto 'move fast and break things'

geoff.vendeville — Wed, 02 Mar 2022 15:53:41 +0000

�鶹��Ƶ expert on human-centered data science – and the problem with the motto 'move fast and break things'

geoff.vendeville Wed, 03/02/2022 - 10:53

Cutline

Shion Guha, a faculty affiliate at the Schwartz Residman Institute for Technology and Society, advocates incorporating human-centered design practices into data science to avoid problems like biased algorithms (photo courtesy of Guha)

Daniel Browne

Topic

Our Community

Institutional Strategic Initiatives

Schwartz Reisman Institute for Technology and Society

Artificial Intelligence

Faculty of Information

“Move fast and break things” has become a cliché in entrepreneurship and computer science circles.

But Shion Guha, an assistant professor at the University of Toronto's Faculty of Information and a faculty affiliate at the Schwartz Reisman Institute for Technology and Society, says the motto – which was once Facebook's internal motto – is a bad fit for the technology sector since algorithms are susceptible to biases that can affect human lives. Instead, Guha advocates a human-centered approach to data science that prioritizes the best outcomes for people.

“I believe in worlds where data-driven decision-making has positive outcomes, but I don't believe in a world where we do this uncritically,” he said. “I don't believe in a world where you just throw stuff at the wall and see what sticks, because that hasn’t worked out at all.”

Guha, the co-author of a new textbook on human-centered data science, spoke to the Schwartz Reisman Institute’s Daniel Browne about the need for a more deliberate and compassionate approach to data science.

Can you tell us about your background?

My academic background is primarily in statistics and machine learning. I graduated with my PhD from Cornell in 2016, and then was an assistant professor at Marquette University for five years before joining the Faculty of Information last year. �鶹��Ƶ is one of the first universities in the world to launch an academic program in human-centered data science, so I was nudged to apply.

My co-authors on the book [Human-Centered Data Science: An Introduction, MIT Press March 2022] and I are some of the first people to have talked about the concept of human-centered data science, in a workshop at one of our main conferences in 2016. We decided to write a textbook about the field because we felt there was a missing link between what is taught in the classroom and what happens in practice. In the last few years, the field has talked a lot about algorithmic biases and unforeseen consequences of technology on society. And so, we decided that instead of writing an academic monograph, we wanted to write a practical textbook for students.

What does it mean for data science to be “human-centered,” and how does this approach differ from other methodologies?

The main idea is to incorporate human-centered design practices into data science – to develop human-centered algorithms. Human-centered design is not a new thing; it’s something that has been talked about a lot in the fields of design, human-computer interaction and so on. But those fields have always been a little divorced from AI, machine learning and data science.

Now, with the advent of this tremendous growth in data science jobs came all of these criticisms around algorithmic bias, which raises the question of whether we are training students properly. Are we teaching them to be cognizant of potential critical issues down the line? Are we teaching them how to examine a system critically? Most computer scientists tend to adopt a very positivist approach. But the fact is that we need multiple approaches, and human-centered data science encourages these practices. Right now, a lot of data science is very model-centered – the conversation is always around what model can most accurately predict something. Instead, the conversation should be: “What can we do so that people have the best outcomes?” It’s a slightly different conversation; the values are different.

Human-centered data science starts off by developing a critical understanding of the socio-technical system under investigation. So, whether it’s Facebook developing a new recommendation system, or the federal government trying to decide on facial recognition policy, understanding the system critically is often the first step. And we’ve actually failed a generation of computer science and statistics students because we never trained them about any of this. I believe in worlds where data-driven decision-making has positive outcomes, but I don't believe in a world where we do this uncritically. I don't believe in a world where you just throw stuff at the wall and see what sticks, because that hasn’t worked out at all.

Next, we engage in a human-centered design process, which can be understood through three different lenses. First, there's theoretical design: the model should be drawn from existing theory – what do we know about how people are interacting in a system. For instance, a lot of my work is centered around how algorithms are used to make decisions in child welfare. So, I need to ensure whatever algorithm I develop draws from the best theories about social work and child welfare.

Second, there's something called participatory design, which means inviting all the stakeholders into the process to let them interpret the model. I might not know everything about child welfare, but my models are interpreted by specialists in that area. Participatory design ensures that the people who are affected by the system make the decisions about its interpretation and design.

The third process is called speculative design, which is about thinking outside the box. Let's think about a world where this model doesn't exist, but something else exists. How do we align this model with that world? One of the best ways to describe speculative approaches is the [British TV] series Black Mirror, which depicts technologies and systems that could happen.

Human-centered design practices are about taking these three aspects and incorporating them in the design of algorithms. But we don't stop there, because you can’t just put something into society without extensive testing, you need to do longitudinal field evaluation. And I’m not talking about six-week evaluations, which are common – I’m talking about six months to a year before putting something into practice. So, all of this is a more critical and slowed-down design process.

What helps you to collaborate successfully with researchers in other disciplines?

I think one of the major impediments to collaboration between disciplines, or even sub-disciplines, are the different values people have. For instance, in my work in child welfare, the government has a set of values – to optimize between spending money and ensuring kids have positive outcomes – while the people who work in the system have different values – they want each child to have a positive outcome. When I come in as the data scientist, I’m trying to make sure the model I build reconciles these values.

My success story has been in working with child welfare services in Wisconsin. When they came to us, I cautioned them that we needed to engage with each other through ongoing conversations to make something successful. We had many stakeholders: researchers in child welfare, department heads, and street-level case workers. I brought them together many times to figure out how to reconcile their values, and that was one of the hardest things that I ever did, because people talk about their objectives, but don't often talk about their values. It's a hard thing to say, OK, this is what I really believe how the system should work.

We conducted workshops for about a year to understand what they needed, and what we eventually realized was that they were not interested in building an algorithm that predicted risk-based probabilities, they were interested in something else: how to make sense of narratives, such as how to describe the story of a child in the system.

If a new child comes into the system, how can we look back and consider how this child displays the same features as other historical case studies? What positive outcomes can we draw upon to ensure this new child gets the services they need? It's a very different and holistic process – it’s not a number, it's not a classification model.

If I had just been given some data, I would have developed a risk-based system that would have ultimately yielded poor outcomes. But because we engaged in that difficult community building process, we figured out that what they really wanted was not what they told me they wanted. And this was because of a value mismatch.

Similarly, when I go to machine learning conferences, there’s a different kind of value mismatch. People are more interested in discussing the theoretical underpinnings of models. I am interested in that, but I’m also interested in telling the story of child welfare, I’m interested in pushing that boundary. But a lot of my colleagues are not interested in that – their part of academia values optimizing quantitative models, which is fine, but then you can't claim you're doing all these big things for society if that's really what your values are.

It's interesting to note how much initial effort is required, involving a lot of development that many wouldn't necessarily consider as part of system design.

You know, the worst slogan that I’ve ever heard in the technology sector, even though people seem to really like it for some reason, is “move fast and break things.” Maybe for product recommendations that's fine, but you don't want to do that if you've got the lives of people on the line. You can't do that. I really think we need to slow down and be critical about these things. That doesn't mean that we don't build data-driven models. It means that we do them thoughtfully, and we recognize the various risks and potential issues down the line, and how to deal with it. Not everything can be dealt with quantitatively.

Issues around algorithmic fairness have become very popular and are the hottest field of machine learning right now. The problem is that we look at this from a very positivist, quantitative perspective, by seeking to make algorithms that are mathematically fair, so different minority groups do not have disproportionate outcomes. Well, you can prove a theorem saying that and put it into practice, but here’s the problem: models are not used in isolation. If you take that model and put it where people are biased, when biased people interact with unbiased, mathematically fair algorithms it will make the algorithms also biased.

Human-AI interaction is really important. We can't pretend our systems are used in isolation. Most problems happen because the algorithmic decision-making process itself is poorly understood, and how people make a particular decision from the output of an AI system is something we don't yet understand well. This creates a lot of issues, yet the field of machine learning doesn't value that. The field values mathematical solutions, except it's a solution only if you view it in the context of a reductionist framework. It has nothing to do with reality.

What are some of the challenges around the use of algorithmic decision-making?

My co-authors and I identify three key dimensions of algorithmic decision-making. One dimension is that decisions are mediated by the specific bureaucratic laws, policies, and regulations that are inherent to that system. So, there are certain things you can do, and can’t do, that are mandated by law. The second dimension is very important, we call it human discretion. For example, police may see a minor offense like jaywalking but choose to selectively ignore it because they are focused on more significant crimes. So, while the law itself is rigid, inside the confines of the law there is discretion.

The same thing happens with algorithmically mediated systems, where an algorithm gives an output, but a person might choose to ignore it. A case worker might know more about a factor that the algorithm failed to pick up on. This works the other way too, where a person might be unsure and go along with an algorithmic decision because they trust the system. So, there’s a spectrum of discretion.

The third aspect is algorithmic literacy. How do people make decisions from numbers? Every system gives a separate visualization or output, and an average social worker on the ground might not have the training to interpret that data. What kinds of training are we going to give people who will implement these decisions?

Now, when we take these three components together, these are the main dimensions of how people make decisions from algorithms. Our group was the first group to unpack this in the case of public services, and it has major implications for AI systems going forward. For instance, how you set up the system affects what kinds of opportunities the user has for exercising discretion. Can everyone override it? Can supervisors override it? How do we look at agreements and disagreements and keep a record of that? If I have a lot of experience and think that the algorithm’s decision is wrong, I might disagree. However, I might also be afraid that if I don't agree, my supervisor will punish me.

Studying the algorithmic decision-making process has been crucial for us in setting up the next series of problems and research questions. One of the things that I’m very interested in is changes in policy. For example, my work in Wisconsin was utilized to make changes that had positive outcomes. But a critical drawback is that I haven't engaged with legal scholars or the family court system.

One of the things I like about SRI is it that brings together legal scholars and data scientists, and I’m interested in collaborating with legal scholars to think about how to write AI legislation that will affect algorithmic decision-making processes. I think it demands a radical rethinking of how laws are drafted. I don't think we can engage in the same process anymore; we need to think beyond that and engage in some speculative design.

What is the most important thing that people need to know about data science today, and what are the challenges that lie ahead for the discipline?

Obviously, I’m very invested in human-centered data science. I really think this process works well, and since �鶹��Ƶ began its program, the field has expanded to other universities and is gaining momentum. I really want to bring this to the education of our professional data science students – those who are going to immediately go out into industry and start applying these principles.

Broadly, the challenges for the discipline are the problems I've alluded to, and human-centered data science responds to these issues. We should not be moving fast, we should not be breaking things – not when it comes to making decisions about people. It doesn't have to be high stakes, like child welfare. You can imagine something like Facebook or Twitter algorithms where ostensibly you're doing recommendation systems, but that really has ramifications for democracy. There are lots of small things that have major unintended consequences down the line, even something like algorithms in the classroom to predict whether a child is doing well or not.

The other main challenge is this value mismatch problem I described. We need to teach our next generation of students to be more compassionate, to encourage them to think from other perspectives, and to center other people's values and opinions without centering their own. So how do we get better? Again, human-centered design has worked very well in other areas, and we can learn what worked well and apply it here. Why should we pretend that we have nothing to learn from other areas?