The WhyLabs Blog
Our ideas and thoughts on how to run AI with certainty
FEATURED POST
The Glassdoor team describes their integration latency challenges and how they were able to decrease latency overhead and improve data monitoring with WhyLabs.
Jamie Broomall,
Lanqi Fei,
Natalia Skaczkowska-Drabczyk
Aug 17, 2023
- WhyLabs
- Machine Learning
- WhyLabs
- Machine Learning
OTHER POSTS
Filter blog posts:
Understanding and Monitoring Embeddings in Amazon SageMaker with WhyLabs
Andre Elizondo,
Shun Mao,
James Yi
| Sep 11, 2023
WhyLabs and Amazon Web Services (AWS) explore the various ways embeddings are used, issues that can impact your ML models, how to identify those issues and set up monitors to prevent them in the future!
- WhyLabs
- ML Monitoring
Data Drift Monitoring and Its Importance in MLOps
Sage Elliott
| Aug 29, 2023
It's important to continuously monitor and manage ML models to ensure ML model performance. We explore the role of data drift management and why it's crucial in your MLOps pipeline.
- MLOps
- Data Logging
Ensuring AI Success in Healthcare: The Vital Role of ML Monitoring
Kelsey Olmeim
| Aug 10, 2023
Discover how ML monitoring plays a crucial role in the Healthcare industry to ensure the reliability, compliance, and overall safety of AI-driven systems.
- ML Monitoring
WhyLabs Recognized by CB Insights GenAI 50 among the Most Innovative Generative AI Startups
WhyLabs Team
| Aug 8, 2023
WhyLabs has been named on CB Insights’ first annual GenAI 50 list, named as one of the world’s top 50 most innovative companies developing generative AI applications and infrastructure across industries.
- WhyLabs
Hugging Face and LangKit: Your Solution for LLM Observability
Sage Elliott
| Jul 26, 2023
See how easy it is to generate out-of-the-box text metrics for Hugging Face LLMs and monitor them in WhyLabs to identify how model performance and user interaction are changing over time.
- LLMs
- WhyLabs
7 Ways to Monitor Large Language Model Behavior
Felipe Adachi
| Jul 20, 2023
Discover seven ways to track and monitor Large Language Model behavior using metrics for ChatGPT’s responses for a fixed set of 200 prompts across 35 days.
- LLMs
- WhyLabs
Safeguarding and Monitoring Large Language Model (LLM) Applications
Felipe Adachi
| Jul 11, 2023
We explore the concept of observability and validation in the context of language models, and demonstrate how to effectively safeguard them using guardrails.
- LLMs
Robust & Responsible AI Newsletter - Issue #6
WhyLabs Team
| Jul 10, 2023
A quarterly roundup of the hottest LLM, ML and Data-Centric AI news, including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Monitoring LLM Performance with LangChain and LangKit
Sage Elliott
| Jul 10, 2023
In this blog post, we dive into the significance of monitoring Large Language Models (LLMs) and show how to gain insights and effectively monitor a LangChain application with LangKit and WhyLabs.
- LLMs
- Integration
BYOF: Bring Your Own Functions - Announcing UDFs in whylogs
Andre Elizondo
| Jun 30, 2023
With the release of whylogs 1.2.0, UDFs are available out-of-the-box. UDFs are the foundation for monitoring complex data, allowing you to craft custom metrics that fit your unique business or research objectives.
- whylogs
Best Practices for Monitoring Large Language Models
Kelsey Olmeim
| Jun 28, 2023
With the growing adoption of LLMs to automate and streamline NLP operations, it's crucial to establish effective monitoring practices that can detect and prevent issues.
- LLMs
Production-Ready Models with Databricks and WhyLabs
Andre Elizondo,
Alessya Visnjic
| Jun 23, 2023
Databricks and WhyLabs partner to enable a unique integration that makes it possible to compute all key telemetry data necessary for AI monitoring directly in Apache Spark.
- WhyLabs
- Integration
WhyLabs Recognized as a Leading AI Company on CB Insights' 2023 AI100 List
Kelsey Olmeim
| Jun 21, 2023
WhyLabs was included on the highly regarded CB Insights AI100 List of Most Innovative AI Companies Worldwide for 2023.
- WhyLabs
- AI
LangKit: Making Large Language Models Safe and Responsible
Andre Elizondo
| Jun 14, 2023
With LangKit, you can monitor LLM behavior and performance with a data centric approach to LLMOps. Identify and mitigate malicious prompts, toxicity, problematic topics, hallucinations, and jailbreak attempts for reliable, safe, and effective LLMs.
- LLMs
- AI Observability
- LangKit
Data Validation at Scale – Detecting and Responding to Data Misbehavior
Felipe Adachi
| Jun 6, 2023
In this tutorial, we’ll introduce the concept of data logging and discuss how to validate data at scale by creating metric constraints and generating reports based on the data’s statistical profiles using whylogs.
- whylogs
- Data Logging
Mind Your Models: 5 Ways to Implement ML Monitoring in Production
Sage Elliott
| May 17, 2023
We’ve outlined five easy ways to monitor your ML models in production to ensure they are robust and responsible by monitoring for concept drift, data drift, data quality, AI explainability and more.
- WhyLabs
- ML Monitoring
Get Early Access to the First Purpose-Built Monitoring Solution for LLMs
Andre Elizondo,
Alessya Visnjic,
Alex Sudbinin
| May 11, 2023
We’re excited to announce our private beta release of LangKit, the first purpose-built large language model monitoring solution! Join the responsible LLM revolution by signing up for early access.
- WhyLabs
- AI Observability
Simplifying ML Deployment: A Conversation with BentoML's Founder & CEO Chaoyu Yang
Sage Elliott
| Apr 4, 2023
A summary of the live interview with Chaoyu Yang, Founder & CEO at BentoML, on putting machine learning models in production and BentoML's role in simplifying deployment.
- WhyLabs
Data Drift vs. Concept Drift and Why Monitoring for Them is Important
Kelsey Olmeim
| Mar 28, 2023
Data drift and concept drift are two common challenges that can impact ML models on production. In this blog, we'll explore the differences between these two types of drift and why monitoring for them is crucial.
- ML Monitoring
- MLOps
Robust & Responsible AI Newsletter - Issue #5
WhyLabs Team
| Mar 10, 2023
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Detecting Financial Fraud in Real-Time: A Guide to ML Monitoring
Kelsey Olmeim
| Mar 7, 2023
Fraud is a significant challenge for financial institutions and businesses. As fraudsters constantly adapt their tactics, it’s essential to implement a robust ML monitoring system to ensure that models effectively detect fraud and minimize false positives.
- ML Monitoring
- MLOps
How to Troubleshoot Embeddings Without Eye-balling t-SNE or UMAP Plots
Bernease Herman
| Feb 23, 2023
WhyLabs' scalable approach to monitoring high dimensional embeddings data means you don’t have to eye-ball pretty UMAP plots to troubleshoot embeddings!
- AI Observability
- whylogs
Achieving Ethical AI with Model Performance Tracing and ML Explainability
Natalia Skaczkowska-Drabczyk
| Feb 2, 2023
With Model Performance Tracing and ML Explainability, we’ve accelerated our customers’ journey toward achieving the three goals of ethical AI - fairness, accountability and transparency.
- WhyLabs
- ML Monitoring
Detecting and Fixing Data Drift in Computer Vision
WhyLabs Team
| Jan 26, 2023
In this tutorial, Magdalena Konkiewicz from Toloka focuses on the practical part of data drift detection and fixing it on a computer vision example.
- ML Monitoring
- whylogs
- WhyLabs
BigQuery Data Monitoring with WhyLabs
Anthony Naddeo
| Jan 17, 2023
We’re excited to announce the release of a no-code solution for data monitoring in Google BigQuery, making it simple to monitor your data quality without writing a single line of code.
- WhyLabs
- BigQuery
- Integration
Robust & Responsible AI Newsletter - Issue #4
WhyLabs Team
| Dec 22, 2022
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
WhyLabs Private Beta: Real-time Data Monitoring on Prem
Murilo Mendonca
| Dec 21, 2022
We’re excited to announce our Private Beta release of an extension service for the Profile Store, enabling production use cases of whylogs on customers' premises.
- whylogs
- WhyLabs
Understanding Kolmogorov-Smirnov (KS) Tests for Data Drift on Profiled Data
Felipe Adachi
| Dec 21, 2022
We experiment with statistical tests, Kolmogorov-Smirnov (KS) specifically, applied to full datasets and dataset profiles and compare the results.
- Data Science
- Machine Learning
- MLOps
Re-imagine Data Monitoring with whylogs and Apache Spark
Andy Dang
| Nov 23, 2022
An overview of how the whylogs integration with Apache Spark achieves large scale data profiling, and how users can apply this integration into existing data and ML pipelines.
- Integration
- Apache Spark
- whylogs
ML Monitoring in Under 5 Minutes
Sage Elliott
| Nov 15, 2022
A quick guide to using whylogs and WhyLabs to monitor common issues with your ML models to surface data drift, concept drift, data quality, and performance issues.
- ML Monitoring
- MLOps
- Data Logging
AIShield and WhyLabs: Threat Detection and Monitoring for AI
Danny D. Leybzon,
Amit Phadke
| Nov 8, 2022
The seamless integration of AIShield’s security insights on WhyLabs AI observability platform delivers comprehensive insights into ML workloads and brings security hardening to AI-powered enterprises.
- Integration
- AI Observability
Large Scale Data Profiling with whylogs and Fugue on Spark, Ray or Dask
WhyLabs Team
| Oct 13, 2022
Profiling large-scale data for use cases such as anomaly detection, drift detection, and data validation with Fugue on Spark, Ray or Dask.
- whylogs
- Integration
Monitoring Image Data with whylogs v1
Ray Reed
| Oct 5, 2022
When operating computer vision systems, data quality and data drift issues always pose the risk of model performance degradation. Whylabs provides a simple yet highly customizable solution for maintaining observability into data to detect issues and take action sooner.
- whylogs
- Image Data
WhyLabs Private Beta: Real-time, No-code, Cloud Storage Data Profiling
WhyLabs Team
| Oct 3, 2022
We’re excited to announce our Private Beta release for a no-code integration option for WhyLabs, allowing users to bypass the need to integrate whylogs into their data pipeline.
- WhyLabs
- whylogs
Data and ML Monitoring is Easier with whylogs v1.1
Sage Elliott
| Sep 28, 2022
The release of whylogs v1.1 brings many features to the whylogs data logging API, making it even easier to monitor your data and ML models!
- whylogs
- Data Logging
- ML Monitoring
Model Monitoring for Financial Fraud Classification
Ravi Jha
| Sep 19, 2022
Model monitoring is helping the financial services industry avoid huge losses caused by performance degradation in their fraud transaction models.
- AI Observability
- whylogs
Robust & Responsible AI Newsletter - Issue #3
WhyLabs Team
| Sep 15, 2022
Every quarter we send out a roundup of the hottest MLOps and Data-Centric AI news including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Data Quality Monitoring in Apache Airflow with whylogs
Murilo Mendonca
| Sep 13, 2022
To make the most of whylogs within your existing Apache Airflow pipelines, we’ve created the whylogs Airflow provider. Using an example, we’ll show how you can use whylogs and Airflow to make your workflow more responsible, scalable, and efficient.
- Integration
- whylogs
- Data Logging
Data Logging with whylogs: Profiling for Efficiency and Speed
Murilo Mendonca
| Aug 31, 2022
Rather than sampling data, whylogs captures snapshots of the data making it fast and efficient for data logging, even if your datasets scale to larger sizes.
- whylogs
- Data Logging
- MLOps
Data Quality Monitoring for Kafka, Beyond Schema Validation
Anthony Naddeo
| Aug 23, 2022
Data quality mapped to a schema registry or data type validation is a good start, but there are a few things most data application owners don’t think about. We explore error scenarios beyond schema validation and how to mitigate them.
- Machine Learning
- Data Validation
- Kafka
- whylogs
Data + Model Monitoring with WhyLabs: simple, customizable, actionable
Danny D. Leybzon
| Aug 16, 2022
The new monitoring system maximizes the helpfulness of alerts and minimizes alert fatigue, so users can focus on improving their models instead of worrying about them in production...
- ML Monitoring
- MLOps
- WhyLabs
- DataOps
- Kafka
A Solution for Monitoring Image Data
Ray Reed
| Aug 2, 2022
A breakdown of how to monitor unstructured data such as images, the types of problems that threaten computer vision systems, and a solution for these challenges.
- ML Monitoring
- Machine Learning
- Image Data
How to Validate Data Quality for ML Monitoring
Sage Elliott
| Jul 27, 2022
Data quality is one of the most important considerations for machine learning applications—and it's one of the most frequently overlooked. We explore why it’s an essential step in the MLOps process and how to check your data quality with whylogs.
- ML Monitoring
- Data Validation
Small Changes for Big SQLite Performance Increases
Anthony Naddeo
| Jul 19, 2022
A behind-the-scenes look at how the WhyLabs engineering team improved SQLite performance to make monitoring data and machine learning models faster and easier for whylogs users.
- sqlite
- whylogs
5 Ways to Inspect Data & Models with whylogs Profile Visualizer
WhyLabs Team
| Jul 12, 2022
Understand what’s happening in your data, identify and correct issues quickly, and maintain the quality and relevance of high-performing data and ML models with whylogs profile visualizer.
- whylogs
- Data Logging
- Data Visualization
Visually Inspecting Data Profiles for Data Distribution Shifts
Felipe Adachi
| Jun 28, 2022
This short tutorial shows how to inspect data for distribution shift issues by comparing distribution metrics and applying statistical tests for drift values calculations.
- whylogs
- Data Logging
- ML Monitoring
Robust & Responsible AI Newsletter - Issue #2
WhyLabs Team
| Jun 2, 2022
Your roundup of the hottest in MLOps and Data-Centric AI including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
Data Logging with whylogs
Danny D. Leybzon
| May 31, 2022
Users can detect data drift, prevent ML model performance degradation, validate the quality of their data, and more in a single, lightning-fast, easy-to-use package. The v1 release brings a simpler API, new data constraints, new profile visualizations, faster performance, and a usability refresh.
- whylogs
- Data Logging
- open source
- ML Monitoring
Choosing the Right Data Quality Monitoring Solution
Stephen Oladele,
Danny D. Leybzon
| May 18, 2022
In the second article in this series, we break down what to look for in a data quality monitoring solution, open source and Saas tools available, and how to decide on the best one for your organization.
- ML Monitoring
- Machine Learning
A Comprehensive Overview Of Data Quality Monitoring
Stephen Oladele,
Danny D. Leybzon
| Apr 29, 2022
In the first article in this series, we provide a detailed overview of why data quality monitoring is crucial for building successful data and machine learning systems and how to approach it.
- ML Monitoring
- Machine Learning
Robust & Responsible AI Newsletter - Issue #1
WhyLabs Team
| Mar 22, 2022
Your roundup of the hottest in MLOps and Data-Centric AI including industry highlights, what’s brewing at WhyLabs, and more.
- WhyLabs
- Newsletter
WhyLabs Now Available in AWS Marketplace
Maria Karaivanova
| Mar 18, 2022
AWS customers worldwide can now quickly deploy the WhyLabs AI Observatory to monitor, understand, and debug their machine learning models deployed in AWS.
- AI Observability
- WhyLabs
- Machine Learning
- ML Monitoring
Deploying and Monitoring Made Easy with TeachableHub and WhyLabs
Danny D. Leybzon,
Felipe Adachi
| Mar 16, 2022
Deploying a model into production and maintaining its performance can be harrowing for many Data Scientists, especially without specialized expertise and equipment. Fortunately, TeachableHub and WhyLabs make it easy to get models out of the sandbox and into a production-ready environment.
- Integration
- ML Monitoring
- AI Observability
How Observability Uncovers the Effects of ML Technical Debt
Bernease Herman,
Alessya Visnjic,
Danny D. Leybzon
| Mar 10, 2022
Many teams test their machine learning models offline but conduct little to no online evaluation after initial deployment. These teams are flying blind—running production systems with no insight into their ongoing performance.
- AI Observability
- WhyLabs
- whylogs
- ML Monitoring
- MLOps
- Data Logging
Deploy your ML model with UbiOps and monitor it with WhyLabs
Danny D. Leybzon
| Jan 5, 2022
Machine learning models can only provide value for a business when they are brought out of the sandbox and into the real world... Fortunately, UbiOps and WhyLabs have partnered together to make deploying and monitoring machine learning models easy.
- Integration
- AI Observability
- Machine Learning
- ML Monitoring
- MLOps
- WhyLabs
AI Observability for All
Alessya Visnjic
| Jan 4, 2022
We’re excited to announce our new Starter edition: a free tier of our model monitoring solution that allows users to access all of the features of the WhyLabs AI observability platform. It is entirely self-service, meaning that users can sign up for an account and get started right away.
- AI Observability
WhyLabs Achieves SOC 2 Type 2 Certification!
Maria Karaivanova
| Dec 27, 2021
We are very happy to announce that we successfully completed our SOC 2 Type 2 examination with zero exceptions. WhyLabs is committed to ensuring our current, and future customers are well informed about the robust capabilities and security of the WhyLabs AI Observatory platform.
- WhyLabs
Observability in Production: Monitoring Data Drift with WhyLabs and Valohai
Danny D. Leybzon
| Dec 3, 2021
What works today might not work tomorrow. And when a model is in real-world use, serving the faulty predictions can lead to catastrophic consequences...
- AI Observability
- ML Monitoring
- MLOps
- Data Logging
- Integration
Why You Need ML Monitoring
Danny D. Leybzon
| Dec 2, 2021
Machine learning models are increasingly becoming key to businesses of all shapes and sizes, performing myriad functions... If a machine learning model is providing value to a business, it’s essential that the model remains performant.
- AI Observability
- Data Logging
- Machine Learning
- ML Monitoring
- MLOps
Data Labeling Meets Data Monitoring with Superb AI and WhyLabs
Danny D. Leybzon
| Nov 29, 2021
Data quality is the key to a performant machine learning model. That’s why WhyLabs and Superb AI are on a mission to ensure that data scientists and machine learning engineers have access to tools designed specifically for their needs and workflows.
- Integration
- DataOps
- Image Data
- Data Logging
- ML Monitoring
- AI Observability
