The WhyLabs Blog
Our ideas and thoughts on how to run AI with certainty
FEATURED POST
We’re excited to announce our private beta release of LangKit, the first purpose-built large language model monitoring solution! Join the responsible LLM revolution by signing up for early access.
Andre Elizondo,
Alessya Visnjic,
Alex Sudbinin
May 11, 2023
- WhyLabs
- AI Observability
- WhyLabs
- AI Observability
OTHER POSTS
Filter blog posts:
Mind Your Models: 5 Ways to Implement ML Monitoring in Production
Sage Elliott
| May 17, 2023
We’ve outlined five easy ways to monitor your ML models in production to ensure they are robust and responsible by monitoring for concept drift, data drift, data quality, AI explainability and more.
- WhyLabs
- ML Monitoring
Simplifying ML Deployment: A Conversation with BentoML's Founder & CEO Chaoyu Yang
Sage Elliott
| Apr 4, 2023
A summary of the live interview with Chaoyu Yang, Founder & CEO at BentoML, on putting machine learning models in production and BentoML's role in simplifying deployment.
- WhyLabs
Data Drift vs. Concept Drift and Why Monitoring for Them is Important
Kelsey Olmeim
| Mar 28, 2023
Data drift and concept drift are two common challenges that can impact ML models on production. In this blog, we'll explore the differences between these two types of drift and why monitoring for them is crucial.
- ML Monitoring
- MLOps
Robust & Responsible AI Newsletter - Issue #5
WhyLabs Team
| Mar 10, 2023
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Detecting Financial Fraud in Real-Time: A Guide to ML Monitoring
Kelsey Olmeim
| Mar 7, 2023
Fraud is a significant challenge for financial institutions and businesses. As fraudsters constantly adapt their tactics, it’s essential to implement a robust ML monitoring system to ensure that models effectively detect fraud and minimize false positives.
- ML Monitoring
- MLOps
How to Troubleshoot Embeddings Without Eye-balling t-SNE or UMAP Plots
Bernease Herman
| Feb 23, 2023
WhyLabs' scalable approach to monitoring high dimensional embeddings data means you don’t have to eye-ball pretty UMAP plots to troubleshoot embeddings!
- AI Observability
- whylogs
Achieving Ethical AI with Model Performance Tracing and ML Explainability
Natalia Skaczkowska-Drabczyk
| Feb 2, 2023
With Model Performance Tracing and ML Explainability, we’ve accelerated our customers’ journey toward achieving the three goals of ethical AI - fairness, accountability and transparency.
- WhyLabs
- ML Monitoring
Detecting and Fixing Data Drift in Computer Vision
WhyLabs Team
| Jan 26, 2023
In this tutorial, Magdalena Konkiewicz from Toloka focuses on the practical part of data drift detection and fixing it on a computer vision example.
- ML Monitoring
- whylogs
- WhyLabs
BigQuery Data Monitoring with WhyLabs
Anthony Naddeo
| Jan 17, 2023
We’re excited to announce the release of a no-code solution for data monitoring in Google BigQuery, making it simple to monitor your data quality without writing a single line of code.
- WhyLabs
- BigQuery
- Integration
Robust & Responsible AI Newsletter - Issue #4
WhyLabs Team
| Dec 22, 2022
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
WhyLabs Private Beta: Real-time Data Monitoring on Prem
Murilo Mendonca
| Dec 21, 2022
We’re excited to announce our Private Beta release of an extension service for the Profile Store, enabling production use cases of whylogs on customers' premises.
- whylogs
- WhyLabs
Understanding Kolmogorov-Smirnov (KS) Tests for Data Drift on Profiled Data
Felipe Adachi
| Dec 21, 2022
We experiment with statistical tests, Kolmogorov-Smirnov (KS) specifically, applied to full datasets and dataset profiles and compare the results.
- Data Science
- Machine Learning
- MLOps
Re-imagine Data Monitoring with whylogs and Apache Spark
Andy Dang
| Nov 23, 2022
An overview of how the whylogs integration with Apache Spark achieves large scale data profiling, and how users can apply this integration into existing data and ML pipelines.
- Integration
- Apache Spark
- whylogs
ML Monitoring in Under 5 Minutes
Sage Elliott
| Nov 15, 2022
A quick guide to using whylogs and WhyLabs to monitor common issues with your ML models to surface data drift, concept drift, data quality, and performance issues.
- ML Monitoring
- MLOps
- Data Logging
AIShield and WhyLabs: Threat Detection and Monitoring for AI
Danny D. Leybzon,
Amit Phadke
| Nov 8, 2022
The seamless integration of AIShield’s security insights on WhyLabs AI observability platform delivers comprehensive insights into ML workloads and brings security hardening to AI-powered enterprises.
- Integration
- AI Observability
Large Scale Data Profiling with whylogs and Fugue on Spark, Ray or Dask
WhyLabs Team
| Oct 13, 2022
Profiling large-scale data for use cases such as anomaly detection, drift detection, and data validation with Fugue on Spark, Ray or Dask.
- whylogs
- Integration
Monitoring Image Data with whylogs v1
Ray Reed
| Oct 5, 2022
When operating computer vision systems, data quality and data drift issues always pose the risk of model performance degradation. Whylabs provides a simple yet highly customizable solution for maintaining observability into data to detect issues and take action sooner.
- whylogs
- Image Data
WhyLabs Private Beta: Real-time, No-code, Cloud Storage Data Profiling
WhyLabs Team
| Oct 3, 2022
We’re excited to announce our Private Beta release for a no-code integration option for WhyLabs, allowing users to bypass the need to integrate whylogs into their data pipeline.
- WhyLabs
- whylogs
Data and ML Monitoring is Easier with whylogs v1.1
Sage Elliott
| Sep 28, 2022
The release of whylogs v1.1 brings many features to the whylogs data logging API, making it even easier to monitor your data and ML models!
- whylogs
- Data Logging
- ML Monitoring
Model Monitoring for Financial Fraud Classification
Ravi Jha
| Sep 19, 2022
Model monitoring is helping the financial services industry avoid huge losses caused by performance degradation in their fraud transaction models.
- AI Observability
- whylogs
Robust & Responsible AI Newsletter - Issue #3
WhyLabs Team
| Sep 15, 2022
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Data Quality Monitoring in Apache Airflow with whylogs
Murilo Mendonca
| Sep 13, 2022
To make the most of whylogs within your existing Apache Airflow pipelines, we’ve created the whylogs Airflow provider. Using an example, we’ll show how you can use whylogs and Airflow to make your workflow more responsible, scalable, and efficient.
- Integration
- whylogs
- Data Logging
Data Logging with whylogs: Profiling for Efficiency and Speed
Murilo Mendonca
| Aug 31, 2022
Rather than sampling data, whylogs captures snapshots of the data making it fast and efficient for data logging, even if your datasets scale to larger sizes.
- whylogs
- Data Logging
- MLOps
Data Quality Monitoring for Kafka, Beyond Schema Validation
Anthony Naddeo
| Aug 23, 2022
Data quality mapped to a schema registry or data type validation is a good start, but there are a few things most data application owners don’t think about. We explore error scenarios beyond schema validation and how to mitigate them.
- Machine Learning
- Data Validation
- Kafka
- whylogs
Data + Model Monitoring with WhyLabs: simple, customizable, actionable
Danny D. Leybzon
| Aug 16, 2022
The new monitoring system maximizes the helpfulness of alerts and minimizes alert fatigue, so users can focus on improving their models instead of worrying about them in production...
- ML Monitoring
- MLOps
- WhyLabs
- DataOps
- Kafka
A Solution for Monitoring Image Data
Ray Reed
| Aug 2, 2022
A breakdown of how to monitor unstructured data such as images, the types of problems that threaten computer vision systems, and a solution for these challenges.
- ML Monitoring
- Machine Learning
- Image Data
How to Validate Data Quality for ML Monitoring
Sage Elliott
| Jul 27, 2022
Data quality is one of the most important considerations for machine learning applications—and it's one of the most frequently overlooked. We explore why it’s an essential step in the MLOps process and how to check your data quality with whylogs.
- ML Monitoring
- Data Validation
Small Changes for Big SQLite Performance Increases
Anthony Naddeo
| Jul 19, 2022
A behind-the-scenes look at how the WhyLabs engineering team improved SQLite performance to make monitoring data and machine learning models faster and easier for whylogs users.
- sqlite
- whylogs
5 Ways to Inspect Data & Models with whylogs Profile Visualizer
WhyLabs Team
| Jul 12, 2022
Understand what’s happening in your data, identify and correct issues quickly, and maintain the quality and relevance of high-performing data and ML models with whylogs profile visualizer.
- whylogs
- Data Logging
- Data Visualization
Visually Inspecting Data Profiles for Data Distribution Shifts
Felipe Adachi
| Jun 28, 2022
This short tutorial shows how to inspect data for distribution shift issues by comparing distribution metrics and applying statistical tests for drift values calculations.
- whylogs
- Data Logging
- ML Monitoring
Robust & Responsible AI Newsletter - Issue #2
WhyLabs Team
| Jun 2, 2022
Your roundup of the hottest in MLOps and Data-Centric AI including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Data Logging with whylogs
Danny D. Leybzon
| May 31, 2022
Users can detect data drift, prevent ML model performance degradation, validate the quality of their data, and more in a single, lightning-fast, easy-to-use package. The v1 release brings a simpler API, new data constraints, new profile visualizations, faster performance, and a usability refresh.
- whylogs
- Data Logging
- open source
- ML Monitoring
Choosing the Right Data Quality Monitoring Solution
Stephen Oladele,
Danny D. Leybzon
| May 18, 2022
In the second article in this series, we break down what to look for in a data quality monitoring solution, open source and Saas tools available, and how to decide on the best one for your organization.
- ML Monitoring
- Machine Learning
A Comprehensive Overview Of Data Quality Monitoring
Stephen Oladele,
Danny D. Leybzon
| Apr 29, 2022
In the first article in this series, we provide a detailed overview of why data quality monitoring is crucial for building successful data and machine learning systems and how to approach it.
- ML Monitoring
- Machine Learning
Robust & Responsible AI Newsletter - Issue #1
WhyLabs Team
| Mar 22, 2022
Your roundup of the hottest in MLOps and Data-Centric AI including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
WhyLabs Now Available in AWS Marketplace
Maria Karaivanova
| Mar 18, 2022
AWS customers worldwide can now quickly deploy the WhyLabs AI Observatory to monitor, understand, and debug their machine learning models deployed in AWS.
- AI Observability
- WhyLabs
- Machine Learning
- ML Monitoring
Deploying and Monitoring Made Easy with TeachableHub and WhyLabs
Danny D. Leybzon,
Felipe Adachi
| Mar 16, 2022
Deploying a model into production and maintaining its performance can be harrowing for many Data Scientists, especially without specialized expertise and equipment. Fortunately, TeachableHub and WhyLabs make it easy to get models out of the sandbox and into a production-ready environment.
- Integration
- ML Monitoring
- AI Observability
How Observability Uncovers the Effects of ML Technical Debt
Bernease Herman,
Alessya Visnjic,
Danny D. Leybzon
| Mar 10, 2022
Many teams test their machine learning models offline but conduct little to no online evaluation after initial deployment. These teams are flying blind—running production systems with no insight into their ongoing performance.
- AI Observability
- WhyLabs
- whylogs
- ML Monitoring
- MLOps
- Data Logging
Deploy your ML model with UbiOps and monitor it with WhyLabs
Danny D. Leybzon
| Jan 5, 2022
Machine learning models can only provide value for a business when they are brought out of the sandbox and into the real world... Fortunately, UbiOps and WhyLabs have partnered together to make deploying and monitoring machine learning models easy.
- Integration
- AI Observability
- Machine Learning
- ML Monitoring
- MLOps
- WhyLabs
AI Observability for All
Alessya Visnjic
| Jan 4, 2022
We’re excited to announce our new Starter edition: a free tier of our model monitoring solution that allows users to access all of the features of the WhyLabs AI observability platform. It is entirely self-service, meaning that users can sign up for an account and get started right away.
- AI Observability
WhyLabs Achieves SOC 2 Type 2 Certification!
Maria Karaivanova
| Dec 27, 2021
We are very happy to announce that we successfully completed our SOC 2 Type 2 examination with zero exceptions. WhyLabs is committed to ensuring our current, and future customers are well informed about the robust capabilities and security of the WhyLabs AI Observatory platform.
- WhyLabs
Observability in Production: Monitoring Data Drift with WhyLabs and Valohai
Danny D. Leybzon
| Dec 3, 2021
What works today might not work tomorrow. And when a model is in real-world use, serving the faulty predictions can lead to catastrophic consequences...
- AI Observability
- ML Monitoring
- MLOps
- Data Logging
- Integration
Why You Need ML Monitoring
Danny D. Leybzon
| Dec 2, 2021
Machine learning models are increasingly becoming key to businesses of all shapes and sizes, performing myriad functions... If a machine learning model is providing value to a business, it’s essential that the model remains performant.
- AI Observability
- Data Logging
- Machine Learning
- ML Monitoring
- MLOps
Data Labeling Meets Data Monitoring with Superb AI and WhyLabs
Danny D. Leybzon
| Nov 29, 2021
Data quality is the key to a performant machine learning model. That’s why WhyLabs and Superb AI are on a mission to ensure that data scientists and machine learning engineers have access to tools designed specifically for their needs and workflows.
- Integration
- DataOps
- Image Data
- Data Logging
- ML Monitoring
- AI Observability
Running and Monitoring Distributed ML with Ray and whylogs
Anthony Naddeo,
Danny D. Leybzon
| Nov 23, 2021
Running and monitoring distributed ML systems can be challenging. Fortunately, Ray makes parallelizing Python processes easy, and the open source whylogs enables users to monitor ML models in production, even if those models are running in a distributed environment.
- open source
- whylogs
- Integration
- AI Observability
Monitor your SageMaker model with WhyLabs
Danny D. Leybzon
| Nov 18, 2021
In this blog post, we will dive into the WhyLabs AI Observatory, a data and ML monitoring and observability platform, and show how it complements Amazon SageMaker.
- SageMaker
- AI Observability
- Machine Learning
- Integration
- WhyLabs
Deploy and Monitor your ML Application with Flask and WhyLabs
WhyLabs Team
| Nov 9, 2021
In this article, we deploy a Flask application for pattern recognition based on the well-known Iris Dataset. For the application monitoring, we’ll explore the free, starter edition of the WhyLabs Observability Platform in order to set up our own model monitoring dashboard.
- AI Observability
- Flask
- whylogs
- WhyLabs
- ML Monitoring
- MLOps
WhyLabs Raises $10M from Andrew Ng, Defy Partners to bring AI observability to every AI practitioner
WhyLabs Team
| Nov 4, 2021
SEATTLE, November 4, 2021 — WhyLabs, the leading provider of observability for AI and data applications announced today the close of a $10 million Series A co-led by Defy Partners and Andrew Ng’s AI Fund.
- MLOps
- WhyLabs
- DataOps
- AI Observability
WhyLabs, AI Observability as a Service
Alessya Visnjic
| Oct 14, 2021
The AI community is experiencing an outbreak of concerns about the robustness and reliability of AI systems, but observability is the mechanism for creating a feedback loop between the ML pipeline and human operators that builds trust and transparency.
- WhyLabs
- AI Observability
- ML Monitoring
Detecting Semantic Drift within Image Data: Monitoring Context-Full Data with whylogs
Leandro G. Almeida
| Aug 7, 2021
Concept drifts can originate in different stages of your data pipeline, even before the data collection itself.
In this article, we’ll show how whylogs can help you monitor your machine learning system’s data ingestion pipeline by enabling concept drift detection, specifically for image data.
- Data Analytics
- Data Logging
- Image Data
- whylogs
- MLOps
- ML Monitoring
Don’t Let Your Data Fail You; Continuous Data Validation with whylogs and Github Actions
WhyLabs Team
| Jul 20, 2021
Ensuring data quality should be among your top priorities when developing an ML pipeline. In this article we’ll show how whylogs constraints with Github Actions can help with data validation, as a key component in ensuring data quality.
- whylogs
- Data Logging
- Data Validation
WhyLabs' Data Geeks Unleashed
Alessya Visnjic,
Leandro G. Almeida,
Andy Dang,
Bernease Herman
| May 21, 2021
This month three members of the WhyLabs team are speaking at the Data and AI Summit. In this post you find descriptions and links to the talk by Alessya Visnjic, Leandro Almeida, and Andy Dang.
- Data Science
- Data Analytics
- Data Logging
Integrating whylogs into your Kafka ML Pipeline
Chris Warth,
Alessya Visnjic
| Apr 7, 2021
Evaluating the quality of data in the Kafka stream is a non-trivial task due to large volumes of data and latency requirements. This is an ideal job for whylogs, an open-source package for Python or Java that uses Apache DataSketches to monitor and detect statistical anomalies in streaming data.
- Machine Learning
- Kafka
- whylogs
- MLOps
Monitoring High-Performance Machine Learning Models with RAPIDS and whylogs
Andy Dang,
Bernease Herman
| Mar 1, 2021
Machine learning (ML) data is big and messy. Organizations have increasingly adopted RAPIDS and cuML to help their teams run experiments faster and achieve better model performance on larger datasets.
- Apache Spark
- Data Analytics
- Data Logging
- Machine Learning
- MLflow
- MLOps
- RAPIDS
- whylogs
Streamlining data monitoring with whylogs and MLflow
Alex Sudbinin
| Feb 8, 2021
It's hard to overstate the importance of monitoring data quality in ML pipelines. In this post we will explore an elegant solution with whylogs and MLflow, which allows for a more informed analysis of model performance.
- Machine Learning
- MLOps
- whylogs
- MLflow
- Data Logging
Data Logging: Sampling versus Profiling
Isaac Backus,
Bernease Herman
| Oct 29, 2020
In traditional software, logging and instrumentation have been adopted as standard practice to create transparency and to make sense of the health of a complex system. When it comes to AI applications, the lack of tools and standardized approaches mean that logging is often spotty and incomplete.
- MLOps
- Data Science
- Machine Learning
WhyLabs: The AI Observability Platform
Alessya Visnjic
| Sep 23, 2020
As companies across industries adopt AI applications in order to improve products and stay competitive, very few have seen a return on their investments. That’s because AI operations are expensive...
- Machine Learning
- Data Science
- Data Visualization
- AI Observability
Introducing WhyLabs, a Leap Forward in AI Reliability
Alessya Visnjic
| Sep 23, 2020
Today, we are excited to announce WhyLabs, a company that empowers AI practitioners to reap the benefits of AI without the spectacular failures that so often make the news.
- AI Observability
- WhyLabs
whylogs: Embrace Data Logging Across Your ML Systems
Andy Dang,
Bernease Herman
| Sep 23, 2020
Fire up your MLOps with a scalable, lightweight, open source data logging library
Co-author: Bernease Herman
We are thrilled to announce the open-source package whylogs. It enables data logging for any ML/AI pipeline in a few lines of code. Data logging is a critic...
- Data Science
- MLOps
- Machine Learning
- Data Logging
