Guardrails and Active Monitoring for LLMs
Hands-on workshop on how to protect your LLMs from bad actors, abuse, misuse, and hallucinations in real time.
BERNEASE HERMAN
Senior Data Scientist
WhyLabs
As organizations move their GenAI initiatives from prototypes to real-world applications, continuous monitoring, powerful guardrails and the ability to fine-tune the LLM-powered applications becomes a top priority.
LLM applications are typically monitored after each user interaction, offering valuable insights for quick model iteration. But in some cases, the application's response needs to be fine-tuned during the user interaction - which requires assessing data in real time. This is where implementing guardrails comes in, establishing safety controls that monitor and dictate a user’s interaction with an LLM application.
Guardrails are a crucial line of defense for ensuring LLM quality and safety by protecting against bad actors, misuse, bad customer experience, hallucinations, and more. Guardrails can also reduce computational expense by invoking a cache for prompts that have been previously computed.
During this hands-on workshop, we’ll explore best practices for three common use cases for guardrails: detecting toxicity in user prompts and LLM responses, retrying for refusals and non-responses, and caching semantically similar prompts.
