Generative AI risk assessments and enforcement outlook

This episode features a panel discussion on generative AI from both legal and technology perspectives, bringing together privacy counsel and an AI practitioner for a free-flowing conversation on risk management frameworks. The central premise is that evaluating generative AI is fundamentally a risk assessment exercise: identifying specific use cases, anticipating real-world consequences, and designing mitigation strategies accordingly. Abstract endorsement of AI adoption — without anchoring analysis to concrete deployment scenarios — does not constitute a meaningful risk assessment, and regulators are unlikely to accept it as one. From a legal standpoint, the discussion highlights the expanding role of FTC enforcement authority in the AI context. Beyond traditional deception standards, the FTC's unfairness authority — increasingly invoked in recent years — applies when business practices cause consumer harm that cannot reasonably be avoided and where the benefits do not outweigh those harms. Organizations cannot disclaim responsibility for AI outputs by treating models as black boxes; effective risk management requires pushing vendors and internal teams for genuine transparency about how models function and what they do with data. Consumer protection laws, employment and housing regulations, and privacy statutes may all be triggered depending on the specific use case. From a technology perspective, the episode frames trust as an existential precondition for AI progress. Without consumer trust in how data is collected and used, the data pipelines necessary to sustain intelligent systems will erode — making privacy not just a legal obligation but a strategic necessity for the AI economy. A key conceptual shift is also introduced: the privacy conversation is evolving from concern about data sitting in databases to concern about data embedded inside trained models, where it persists and multiplies across many systems simultaneously. This reframes how organizations should think about data governance as AI adoption accelerates.

**Jonathan:** I love the collaboration we've got here. So we've got legal expertise with Alysa and Kelly Dry, and we've got technology expertise with Yakov. And so we'll have a free flowing discussion today on generative AI. At the end of the day, it's a risk assessment, and we're balancing. Right? We have innovation. We have a lot of exciting use cases or helpful use cases. And so the question is really what are those use cases? Based on those use cases, what are the risks that you can foresee? What's anticipated? And then how do you manage for those risks? **Alysa:** And part of — I'll just — the facts that we were talking about, the scenarios, it really does mean understanding the technology and what's happening so that you can be issue spotting appropriately. And I think on the next slide, we go into some more details. But having these kinds of conversations with non lawyers is just really important. We're hearing from tons of clients because they're hearing from tons of their business clients on all different scenarios. And I think kind of with distilling without getting too complicated, you're looking at what are the consequences. That's us lawyers — whether you're a privacy lawyer or another kind of lawyer, you're trying to manage for risk. And so thinking about it's hard in the sense of the abstract. Yes, we want to use generative AI, for example. Well, using it in the abstract, it's not all that helpful of a risk assessment as opposed to using it in what instance and thinking about what are potential real world consequences as a result of that. And there's different types of laws that are gonna come up. If you're dealing with people, then you're thinking consumer protection laws. And we've heard a lot from the Federal Trade Commission, Bedoya at the summit in spring — certainly things that we can anticipate, right? What you say, you're responsible for what you say. That's just common deception one hundred one. I think we're going to see a lot more of unfairness authority, which is what we've seen the FTC already do, quite a bit more of than they have in the past. And that's thinking about consequences that injure consumers that they can't reasonably avoid. And there's not essentially a balance of what are the benefits of that to competition, to our economy, and to consumers overall. So in order to do that risk assessment, you can't say, I didn't know what the tool did. It's a black box. It's an algorithm. But in order to actually understand it really does mean pushing both either the business or the third party to get some transparency there to be able to assess. **Yakov:** Myself, you know, being a practitioner of AI and being in the space of privacy and trust as well — I mean it was obvious to me as a practitioner that we are gonna — you know while AI and machine learning all the great work is rushing ahead, we're gonna hit a wall where the lack of trust, fundamental trust in the system means that we will not get the right data we need to keep growing and doing what we need to do. And it was existential — the use cases like IoT or smart cars or smart everything is not going to happen until we solve it. So the fact that privacy flared out the way it did around data made complete sense to me. And the fact that the next data's main day job going forward is gonna be to feed into intelligent systems — I mean by far — we started this idea of privacy was a lot about that initial collection of data we did and it living in a database somewhere. Increasingly where we're gonna see data is a lot less about those individual interactions, but more about how it gets multiplied and feeds into many, many, many intelligent systems and lives inside models in many, many different places. So — just the same as we made a statement about synthetic data — the majority of the data is gonna live inside models. And so naturally, that's the area where we need to go and progress from the original thoughts around privacy and the data singularity data and pieces to now data within the context of AI. The first thing I think to realize is — and that's again to Alysa's point — it's the purpose, right? Like AI in and of itself is not an end, it's a means to an end. So you still need to decide what it is that you want to do with the data. You might be leveraging AI to do so and increasingly that would be the case, but you need to be very clear when you engage in something, what is the purpose by which you want to operate. I think folks understand that data is used to train a model. And I think that that is understood. What is maybe less understood is that when you train the model, the output of training a model is now a trained model.