AI_dev Europe 2024

Large Language Models (LLMs) represent a significant leap in artificial intelligence, trained on extensive text and code datasets to perform tasks like text generation, language translation, and responsive questioning. While still evolving, LLMs are already being used in a variety of applications, including chatbots, search engines, and creative writing tools, thus monitoring and understanding AI behaviors becomes important. Users demand transparency, not a "black box," seeking insights into the AI's decision-making processes. Observability addresses this by gathering and analyzing data to refine LLM performance, uncover biases, troubleshoot problems, and guarantee AI reliability and trustworthiness. In this session, we will deep dive to LLM Observability, including metrics should be observed, like model latency/cost, model tracking etc , emerging technologies including traceloop, otel, langfuse etc, how to use those technologies to do analytics, monitoring and optimization for LLM apps.