HarmProfile: Characterizing Harmful Distributions in Frontier LLMs

Summary

HarmProfile is a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, defining the resulting harmful-output distribution as a model-level risk profile. It contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, the authors find that frontier LLMs reliably produce harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. The source code is available at GitHub.