Danny D. Leybzon

Dec 3, 2021

Back to Blog

Observability in Production: Monitoring Data Drift with WhyLabs and Valohai

AI Observability
ML Monitoring
MLOps
Data Logging
Integration

Danny D. Leybzon

Dec 3, 2021

Imagine that magical day when your machine learning model is in production. It is possibly integrated into end-user applications, serving predictions and providing real-world value. As a Data Scientist, You may think that your job is done and that you can move on to the next problem to be solved. Unfortunately, the work is just getting started.

What works today might not work tomorrow. And when a model is in real-world use, serving the faulty predictions can lead to catastrophic consequences like what happened with Zillow and their iBuying algorithm which caused the company to overpay for real estate and ultimately, lay off 25% of their workforce.

...

We will dig into how we can easily get started with observability and detect data drift using whylogs while executing your pipeline on Valohai.

Continue reading on the Valohai Blog

Danny D. Leybzon

Glassdoor Decreases Latency Overhead and Improves Data Monitoring with WhyLabs

Jamie Broomall,

Lanqi Fei,

Natalia Skaczkowska-Drabczyk

Aug 17, 2023

The Glassdoor team describes their integration latency challenges and how they were able to decrease latency overhead and improve data monitoring with WhyLabs.

Read post

WhyLabs
Machine Learning

Understanding and Monitoring Embeddings in Amazon SageMaker with WhyLabs

Andre Elizondo,

Shun Mao,

James Yi

Sep 11, 2023

WhyLabs and Amazon Web Services (AWS) explore the various ways embeddings are used, issues that can impact your ML models, how to identify those issues and set up monitors to prevent them in the future!

Read post

WhyLabs
ML Monitoring

Data Drift Monitoring and Its Importance in MLOps

Sage Elliott

Aug 29, 2023

It's important to continuously monitor and manage ML models to ensure ML model performance. We explore the role of data drift management and why it's crucial in your MLOps pipeline.

Read post

MLOps
Data Logging

Ensuring AI Success in Healthcare: The Vital Role of ML Monitoring

Kelsey Olmeim

Aug 10, 2023

Discover how ML monitoring plays a crucial role in the Healthcare industry to ensure the reliability, compliance, and overall safety of AI-driven systems.

Read post

ML Monitoring

WhyLabs Recognized by CB Insights GenAI 50 among the Most Innovative Generative AI Startups

WhyLabs Team

Aug 8, 2023

WhyLabs has been named on CB Insights’ first annual GenAI 50 list, named as one of the world’s top 50 most innovative companies developing generative AI applications and infrastructure across industries.

Read post

WhyLabs

Hugging Face and LangKit: Your Solution for LLM Observability

Sage Elliott

Jul 26, 2023

See how easy it is to generate out-of-the-box text metrics for Hugging Face LLMs and monitor them in WhyLabs to identify how model performance and user interaction are changing over time.

Read post

LLMs
WhyLabs

7 Ways to Monitor Large Language Model Behavior

Felipe Adachi

Jul 20, 2023

Discover seven ways to track and monitor Large Language Model behavior using metrics for ChatGPT’s responses for a fixed set of 200 prompts across 35 days.

Read post

LLMs
WhyLabs

Safeguarding and Monitoring Large Language Model (LLM) Applications

Felipe Adachi

Jul 11, 2023

We explore the concept of observability and validation in the context of language models, and demonstrate how to effectively safeguard them using guardrails.

Read post