This panel discussion was hosted by CIDM in January 2024, in partnership with Intuitive Stack and Fluid Topics. Lief Erickson, Fabrice Lacroix, and I spent just over an hour talking through what we had each been seeing in real generative AI projects — what's working, what's failing, and why.
The conversation covers a specific and recurring pattern: organizations kick off AI initiatives led by IT teams, assemble a technology stack, load in whatever content they can find, run some cursory tests, and declare early success — until subject matter experts start asking the hard questions. At that point the results fall apart, not because the pipes weren't connected, but because the content flowing through them was incomplete, inconsistent, or missing the context the model needed to respond accurately.
We spend a good portion of the discussion on what I'd call the "almost right" problem: a hallucination that's obviously wrong is easy to catch, but a response that sounds plausible and is mostly right is genuinely dangerous, and you need someone who knows the content well to spot it.
The panel also gets into the specific ways content professionals can add value: identifying which content belongs in the training set, designing or extending metadata schemes to handle cross-silo consistency, and building the methodology for evaluating model accuracy over time. For organizations with structured, modular content already in place, that foundation helps — but it's not sufficient on its own. The conversation is honest about what's still unresolved.
I've been in enough of these projects now to believe that the content question isn't secondary to the technology question. It is the question. Watch or listen to the full recording and hear Lief and Fabrice's take on where the gaps are and what the path forward actually looks like.
If your organization is in the middle of an AI initiative — or is about to start one — and you're trying to figure out how to position the content team's role before the project hits its first wall, I'm happy to talk through what that looks like in practice.
Book a free discovery call today.
This panel was hosted by CIDM, in partnership with Intuitive Stack and Fluid Topics. Panelists: Lief Erickson and Fabrice Lacroix.
The PoolParty Summit 2024 roundtable covers the same question from a different angle — framework, metadata, terminology, and why most AI tools are solving the wrong problem.
The practitioner case study documents what this looks like in a real implementation, including the DS team's own experience and a practical checklist.
