Speaker

We measure the cost of Generative AI in cents per $1k$ tokens or milliseconds of latency. But there is a hidden bill attached to every prompt, paid for in kilowatt-hours and liters of freshwater.

As AI adoption scales, the environmental impact is becoming a critical engineering constraint. A single LLM interaction can consume up to $10x the energy of a traditional search query, and the hardware lifecycle is generating record levels of e-waste.

In this session, we will audit the "Real Cost of a Token," moving beyond the hype to analyze the hard data of AI training and inference.

We will then pivot from metrics to solutions, demonstrating that "Green AI" is often synonymous with efficient, cost-effective AI. You will walk away with a practical toolkit to better navigate our dependency on these models—learning how to right-size your architecture, optimize pipelines, and implement carbon-aware patterns to build intelligence that doesn't cost the Earth.

Semantic search promises a revolution: contextual relevance and natural language understanding with just a few lines of code. On a notebook or a POC, it’s magical. But what happens when your index exceeds a billion vectors?

The magic quickly gives way to the brutality of engineering: exploding latency, uncontrolled infrastructure costs, and RAM challenges.

In this talk, we leave marketing buzz at the door and dive into the guts of Elasticsearch and OpenSearch at very large scale. We will cover how to:

• Architect your clusters to handle a billion embeddings without failing.
• Optimize the critical trade-off between precision (recall) and performance (latency).
• Reduce costs using quantization strategies and intelligent chunking.

If you need to move from a “Hello World” semantic search to massive production, this session is your survival guide.