Presenter: Satanu Ghosh

As we continue to develop increasingly sophisticated intelligent systems—ranging from classic machine learning models to general-purpose AI agents—we face growing challenges in evaluating their capabilities meaningfully. Traditional benchmarks, once considered gold standards, are now frequently surpassed, gamed, or rendered obsolete. This fragility raises serious questions about our ability to measure progress in a fast-evolving landscape.

In this session, we’ll explore why evaluation metrics and benchmarks are struggling to keep pace with advances in AI, and why “better than human” scores often fail to reflect real-world utility or alignment with our goals. Drawing on examples from both traditional supervised models and generative systems, we’ll uncover how benchmarks can mislead, how overfitting to static tests can mask brittleness, and how evaluation metrics can be decoupled from practical success.

We’ll then turn to more promising directions, including dynamic evaluation setups, human-in-the-loop assessments, real-world task validation, and customized evaluation frameworks aligned with deployment contexts. Whether you're working on classifiers, recommendation engines, reinforcement learning agents, or generative AI, this session will help you rethink evaluation from the ground up.

By the end of this session, participants will be able to:

• Identify the limitations of traditional benchmarks in evaluating modern AI systems.
• Analyze cases where evaluation metrics failed to reflect real-world performance or robustness.
• Compare and contrast different evaluation approaches—such as static testing, task-based performance, and interactive/human-in-the-loop methods.
• Design an evaluation strategy that aligns with specific goals, stakeholders, or deployment contexts.
• Reflect on the broader ethical and societal implications of what we choose to measure—and what we ignore—when evaluating AI.