Get Started

Moderation Systems, OpenAI, Perspective, Llama Guard

Production moderation systems operationalize the safety policies defined in Lessons 12-16. OpenAI Moderation API: omni-moderation-latest (2024) built on GPT-4o classifies text + images in one call; 42% better on multilingual test set than prior version; the response schema returns 13 category booleans, harassment, harassment/threatening, hate, hate/threatening, illicit, illicit/violent, self-harm, self-harm/intent, self-harm/instructions, sexual, sexual/minors, violence, violence/graphic; free for most developers. Layered patterns: Input moderation (pre-generation), Output moderation (post-generation), Custom moderation (domain rules). Async parallel calls hide latency; placeholder responses on flag. Llama Guard 3/4 (Lesson 16): 14 MLCommons hazards, Code Interpreter Abuse, 8 languages (v3), multi-image (v4). Perspective API (Google Jigsaw): toxicity scoring predating the LLM-as-moderator wave; primarily single-dimension toxicity with severe-toxicity/insult/profanity variants; baseline for content-moderation research. Deprecations: Azure Content Moderator deprecated February 2024, retired February 2027, replaced by Azure AI Content Safety.

60 minutesBuildPython (stdlib, three-layer moderation harness)Members

Learning Objectives

  • Describe the OpenAI Moderation API's category taxonomy and how it differs from Llama Guard 3's MLCommons set.
  • Describe the three moderation-layer pattern (input, output, custom) and name one failure mode of each.
  • Describe Perspective API's position as a pre-LLM-era baseline and why it remains used in research.
  • State the Azure deprecation timeline.