Edition 03 Monday, 28 September 2026 Amsterdam

Booking.com ML Meetup

Practical talks on building AI agents, measuring their value, and automating evaluation — with speakers from Anthropic and Booking.com. Open to everyone, inside the company and out. Free, in person, pizza included.

17:00 – 20:30 · Booking.com Amsterdam office · Free

Where Amsterdam's ML practitioners compare notes.

The meetup brings together professionals from Booking.com and other leading companies to explore the evolving landscape of machine learning. From applied use cases to research breakthroughs, we aim to foster meaningful conversations across industries and roles, whether you are building models, shaping strategy, or just curious about what comes next.

Hosted at the Booking.com office in Amsterdam, it is open to internal colleagues and external participants alike. Expect substantial talks, time to connect, and a welcoming atmosphere with pizza and drinks.

We are here to share what we have learned and push at what is possible with ML, together.

Talks, pizza, drinks, and time to talk.

17:00

Walk in

Pizza, drinks, name tags.

18:00

Welcome

by Floriana Zefi

Build the loop, not the promptDaisy Hollman · Anthropic

Interactive coding with an agent tops out at one of you: however fast the model works, you're still reading every diff. The only real multiplier is agents that run without you, several at once, for hours — and an agent runs unattended exactly as long as something other than you can tell it “wrong, try again”. That something is a verification loop, and building good ones is a skill you can get better at systematically.

This talk borrows a lens from optimization and reinforcement learning to take that loop apart into three pieces — an objective, a metric that stands in for it, and an optimizer that takes the next step — then walks through how each piece breaks in practice: agents that can't see the result they're grading, metrics that leak, signals too noisy to climb, feedback with no direction in it, and the case where it really is the model. You'll leave with a better question than “is the model smart enough?”: which part of the loop is broken?

Beyond “it works”: how we measure the quality of AI engineering at Booking.comMansi Mittal · Booking.com

We rolled out coding assistants and reusable AI workflows to practitioners. Adoption was easy. Knowing which ones actually save time, which waste money, and which quietly fail — that was hard. So we built a framework to measure it, across assistants, workflows, connected tools and prompts. Operational telemetry captures what each run cost and how it behaved: tokens, latency, retries, cache efficiency, context growth, tool paths.

Then the harder half: how do you measure whether an agent performed well? LLM-as-a-judge is widely used but usually on very different tasks, and conversation length is its own challenge. We'll cover how we adapted it to coding-agent workflows so the scores survive prompt and service changes, and what the numbers revealed once quality became visible to engineers and the business.

Judge Factory: Automating LLM Judge DevelopmentZeno Belligoli · Booking.com

LLM judges enable evaluation and monitoring of AI applications at scale, but building reliable judges takes considerable effort. In this talk, we introduce Judge Factory, Booking.com’s automated pipeline for creating customized LLM judges while minimizing the burden on human experts. We’ll share practical guidance on defining evaluation tasks, generating and annotating data, and optimizing judge prompts, highlighting where automation helps and where human expertise remains essential. We’ll close with an overview of how to validate, monitor, and maintain judges throughout their lifecycle.

until 20:30

Networking

Pizza and drinks.

The line-up.

Daisy Hollman

Member of Technical Staff · Anthropic (Claude Code)

Daisy has led the design and implementation of plugins, agent teams and skills in Claude Code. Before Anthropic she spent a decade on the ISO C++ standards committee, chairing the study group on C++ ranges and contributing to the standardization of std::mdspan, std::atomic_ref, and the executors work that became std::execution. Her areas of expertise include metaprogramming, generic programming, concurrency and library design. She holds a PhD in theoretical quantum chemistry.

Mansi Mittal

Senior Software Engineer, GenAI Engineering · Booking.com

Mansi has 7.5 years at Booking.com enabling developers across the organisation to build dependable AI systems. Her experience spans automated SDLC workflows, AI runtime platforms, agent evaluation, observability and developer tooling, with a particular emphasis on privacy-aware telemetry and measuring real-world agent value. She is an experienced international conference speaker.

Zeno Belligoli

Senior Machine Learning Scientist · Booking.com

Zeno is a Senior Machine Learning Scientist at Booking.com working on the evaluation of AI applications. His interests include developing broadly applicable, automated methods for synthetic data generation, user simulation, and prompt and harness optimization, and bringing them together to simplify AI evaluation. He holds a PhD in computational fluid mechanics, where he developed optimization techniques for reconstructing turbulent flows. As a result, he knows how to stir sugar into a cup of coffee to perfection.

What came before.

7 May 2026
Harnessing Agentic Flows in GenAI at Booking.comHai Le · GenAI Machine Learning Manager, Booking.com +

Generative AI has changed how we approach travel planning, but moving from simple chatbot calls to complex, high-accuracy travel assistants requires a shift from deterministic to agentic architectures. At Booking.com we moved from rigid pipelines to agentic flows: systems capable of planning, acting and reflecting, to handle needs like property comparisons and contextual trip support, where deterministic flows lacked adaptability and could not recover gracefully from errors.

A practical deep dive into building and scaling these agents: a modular AgentHub platform that simplifies creating them, an evaluation framework measuring dialogue quality, factual accuracy and user satisfaction at scale, and the internal tooling that carries an agent from prototype to production. Plus the lessons from moving off out-of-the-box models to in-house agentic ecosystems.

Long-Term Memory for LLM Agents: From Stateless to Stateful AIGeorge Chouliaras · Senior Machine Learning Scientist, Booking.com +

Production LLM agents are fundamentally stateless: every conversation starts from scratch, which means repetitive interactions, lost context and no ability to learn from experience. This talk presents a long-term memory framework inspired by human cognitive memory, organised into three types by abstraction level: episodic memories capturing compressed interaction summaries, semantic memories distilling persistent user facts and preferences across conversations, and procedural memories encoding behavioural rules extracted from successful and failed agent trajectories.

For each type it walks through the structure, the generation pipeline and the key design decisions, closing with offline evaluation results on the potential impact of memory-augmented agents.

24 October 2025
Tackling Interference in Promotion OptimisationBram van den Akker · Senior Machine Learning Scientist, Booking.com +

Uplift modelling is central to promotion optimisation, but its effectiveness is often limited by the Stable Unit Treatment Value Assumption. That assumption, where promoting one product does not affect another's conversion, is frequently violated in practice through product interference and cannibalisation. Ignoring those effects leads to strategies that merely cannibalise conversions rather than generating true incremental lift, and addressing interference in uplift modelling is non-trivial with limited coverage in the literature.

A practical walkthrough of how Booking.com approached it: synthetic interference simulations to model complex interactions, interference-aware Qini curves for offline evaluation, and combining state-of-the-art interference estimation with existing uplift modelling methods. Including insights from the academic collaborations that made it possible.

Experimentation that Prioritises Causal Decision-MakingOlivier Jeunen · Principal Research Scientist, Aampe +

Consumer applications offer many ways to surface content to users: promotional campaigns for new features or subscriptions, evergreen nudges for engagement, personalised recommendations, across email, push notifications and in-app surfaces. Conventional orchestration relies heavily on labour-intensive manual marketer work, and inhibits effective personalisation of content, timing, frequency and copy.

This talk formulates the task as sequential decision-making, optimising a modular policy that maximises incremental engagement for any funnel event, using a Difference-in-Differences design for Individual Treatment Effect estimation and Thompson sampling to balance explore against exploit. Results come from a multi-service application now deployed across 150 million users.

The Booking.com office, Amsterdam.

Where
Booking.com office, Amsterdam
Open in Google Maps
Format
In person only. The talks are not streamed or recorded.
Access
Open to Booking.com colleagues and external guests alike. Mention any dietary or access needs when you get in touch.
What to bring
Yourself. Bring photo ID if you are visiting as an external guest.

Answers, in fewer words than you'd expect.

Do I need to work at Booking? +

No. The meetup is open to anyone interested in machine learning. To join, contact Floriana Zefi on LinkedIn.

Is it free? +

Yes. Food, drinks, and admission are covered.

Can I speak at a future edition? +

Yes. Get in touch with one of the organisers to suggest a talk for a future edition.

What happens if I register and can't make it? +

Let one of the organisers know so we can offer your place to someone else.

Is it recorded? +

No. The talks will not be streamed or recorded. Slides are shared afterwards if the speaker opts in.

Come to Edition 03.

Places are limited, so please get in touch early. To join, contact Floriana Zefi on LinkedIn.

Organised by Floriana Zefi, Bas Vlaming and Alex Umnov.