ToxiCN Family

§ 1About the Project

Online toxic language causes tangible harm to individuals and communities, and reliable detection underpins responsible content moderation as well as the safety alignment of language models. Progress in Chinese, however, has long been bottlenecked by the lack of large-scale, fine-grained, openly available resources — particularly for indirect phenomena (homophones, abbreviated slurs, sarcasm, dog-whistle references) and for the multimodal wild west of harmful memes.

The ToxiCN Family is our ongoing effort to close this gap along two complementary axes:

(i) Text — ToxiCN (ACL 2023): a hierarchical taxonomy (toxic / hate / targeted group / expression form) paired with a manually curated Chinese corpus covering both direct and indirect toxicity, and a knowledge-enhanced baseline (TKE).

(ii) Multimodal — ToxiCN_MM (NeurIPS 2024): a 12K image–text meme dataset annotated for harmful types and modality combinations, together with a Multimodal Knowledge-Enhancement Detector designed for Chinese cultural context.

Both resources are intended as reproducible reference points rather than final solutions, and have since been adopted by a growing line of follow-up work, summarised in the Cited By section below.

§ 2ToxiCN — ACL 2023

PaperFacilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarks

Junyu Lu, Bo Xu, Xiaokun Zhang, Changrong Min, Liang Yang, Hongfei Lin

Abstract

Hierarchical Taxonomy

1. Level 1 — Toxic vs. Non-toxic — Binary judgement on whether a post is toxic in any form.
2. Level 2 — Hate vs. General Offensive — Among toxic posts, separate group-targeted hate speech from general offensive language (insults without a protected-group target).
3. Level 3 — Targeted Group — For hate speech, the protected group attacked: gender, race, region, LGBTQ, others.
4. Level 4 — Expression Form — Direct expression vs. indirect expression (cloaked: homophones, abbreviations, irony, dog-whistles).

Dataset

ToxiCN is a manually annotated Chinese corpus of online posts spanning multiple platforms (e.g. Zhihu, Tieba). Each post is labelled under a four-level hierarchical scheme so that downstream models can be trained or evaluated at any granularity, from binary toxic detection to fine-grained targeted-group / expression-form classification.

Insult Lexicon

Beyond direct slurs harvested from prior lexicons, ToxiCN systematically derives an extended Chinese insult lexicon by tracing how online users disguise toxic intent. We catalogue six recurring derivation patterns and treat each derived form as a first-class lexicon entry, so downstream models can resolve them back to their actual referents:

Derivation Patterns

1. Homophonic substitution — Replace one or more characters with phonetically identical / similar ones to bypass keyword filters (e.g. 默 ≈ 黑·犬).
2. Compositional decomposition — Split a target character into its sub-components, then re-assemble in surface text (e.g. 仙女 = 小·仙·女).
3. Cross-lingual abbreviation — Use Pinyin initials, English / mixed-script abbreviations to encode a slur (e.g. txl → 同性恋).
4. Hybrid Chinese–English splicing — Fuse a Chinese morpheme with an English fragment to reconstruct a slur (e.g. ni + ger → ni哥).
5. Historical / cultural allusion — Re-purpose a region or historical name to demean a group (e.g. 南满 → 南蛮).
6. Conspiracy / meme reference — Invoke a memeified narrative as a coded slur (e.g. Kalergi → racial conspiracy meme).

Worked Examples

Sensitive English referents are masked (e.g. n*gger) for public display; full forms are kept inside the released lexicon files.

§ 3ToxiCN_MM — NeurIPS 2024

PaperTowards Comprehensive Detection of Chinese Harmful Memes

Junyu Lu, Bo Xu, Xiaokun Zhang, Hongbo Wang, Haohao Zhu, Dongyu Zhang, Liang Yang, Hongfei Lin

Abstract

Harmful-Type Taxonomy

1. Targeted Harmful — Memes that attack a specific individual, group, or social category — the most common harmful type on Chinese platforms.
2. General Offence — Insults, profanity, or aggressive content not directed at a particular protected group.
3. Sexual Innuendo — Implicit or explicit sexual content delivered through visual metaphor, character substitution, or suggestive composition.
4. Dispirited Culture — Memes propagating nihilism, self-harm framing, or anti-aspirational narratives that erode social well-being.

Dataset

ToxiCN_MM is a Chinese harmful-meme dataset of 12,000 image–text pairs collected from public online sources. Memes are annotated for both harmful type and modality combination, enabling fine-grained study of where toxicity actually arises (text-only, image-only, or only after fusion). Version 2.0 (Dec. 2024) re-annotates <1% ambiguous samples and additionally releases the specific attacked targets for targeted harmful memes.

§ 4Ethics Statement

§ 6Resources

ToxiCN (ACL 2023): Paper · Code

ToxiCN_MM (NeurIPS 2024): Paper · Code

§ 7BibTeX

ToxiCN ACL 2023

@inproceedings{lu-etal-2023-facilitating,
    title = "Facilitating Fine-grained Detection of {C}hinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarks",
    author = "Lu, Junyu  and  Xu, Bo  and  Zhang, Xiaokun  and  Min, Changrong  and  Yang, Liang  and  Lin, Hongfei",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.898",
    pages = "16235--16250"
}

ToxiCN_MM NeurIPS 2024

@inproceedings{lu2024towards,
    title     = {Towards Comprehensive Detection of Chinese Harmful Memes},
    author    = {Junyu Lu and Bo Xu and Xiaokun Zhang and Hongbo Wang and Haohao Zhu and Dongyu Zhang and Liang Yang and Hongfei Lin},
    booktitle = {The Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
    year      = {2024},
    url       = {https://openreview.net/forum?id=PSDXcYjrkO}
}