SOS-1K: A Fine-grained Suicide Risk Classification Dataset for Chinese Social Media Analysis

📄 arXiv: 2404.12659v1 📥 PDF

作者: Hongzhi Qi, Hanfei Liu, Jianqiang Li, Qing Zhao, Wei Zhai, Dan Luo, Tian Yu He, Shuo Liu, Bing Xiang Yang, Guanghui Fu

分类: cs.CL

发布日期: 2024-04-19

💡 一句话要点

提出SOS-1K数据集以解决中文社交媒体自杀风险分类问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自杀风险识别 社交媒体分析 深度学习 数据增强 中文数据集 MentalBERT 心理健康

📋 核心要点

  1. 现有方法在社交媒体上识别自杀意图面临隐含表达和多样化语言的挑战，缺乏针对中文的相关数据集。
  2. 本文提出了SOS-1K数据集，专注于细粒度自杀风险分类，并评估了多种深度学习模型的表现。
  3. 实验结果显示，最佳模型在高低自杀风险分类中取得88.39%的F1分数，数据增强技术提升了模型性能。

📝 摘要（中文）

在社交媒体上，用户常常表达个人情感，其中部分可能暗示潜在的自杀倾向。互联网语言的隐含性和多样性使得准确快速识别自杀意图变得复杂，给及时干预带来了挑战。为填补这一空白，本文提出了一个针对中文社交媒体的自杀风险分类数据集，重点关注自杀意图表达、自杀方法和紧迫性等指标。通过评估七个预训练模型，实验结果显示深度学习模型在高低自杀风险的区分上表现良好，最佳模型的F1分数达到88.39%。然而，细粒度自杀风险分类的结果仍不理想，权重F1分数为50.89%。为解决数据不平衡和数据集规模有限的问题，研究探讨了传统和基于大型语言模型的数据增强技术，结果表明数据增强可提升模型性能，F1分数提高了4.65%。

🔬 方法详解

问题定义：本文旨在解决中文社交媒体中自杀风险识别的困难，现有方法缺乏针对性数据集，导致识别准确性不足。

核心思路：通过构建SOS-1K数据集，聚焦于自杀意图、方法和紧迫性等细粒度指标，利用深度学习模型进行自杀风险分类。

技术框架：研究采用了七个预训练模型进行评估，分为高低自杀风险分类和0到10的细粒度分类，结合数据增强技术以应对数据不平衡问题。

关键创新：提出的SOS-1K数据集为中文社交媒体自杀风险分类提供了新的数据基础，尤其是细粒度分类的研究尚属首次。

关键设计：模型训练中采用了多种损失函数和网络结构，特别是中文MentalBERT模型在心理领域数据上进行预训练，展现出优越的性能。

📊 实验亮点

实验结果显示，最佳模型在高低自杀风险分类任务中取得了88.39%的F1分数，而细粒度自杀风险分类的权重F1分数为50.89%。通过数据增强技术，模型性能提升了4.65个百分点，表明数据增强在处理不平衡数据集中的有效性。

🎯 应用场景

该研究的潜在应用场景包括社交媒体平台的自杀风险监测和心理干预，能够帮助及时识别有自杀倾向的用户，从而提供必要的心理支持和干预措施。未来，该数据集和模型可为相关领域的研究提供基础，推动心理健康技术的发展。

📄 摘要（原文）

In the social media, users frequently express personal emotions, a subset of which may indicate potential suicidal tendencies. The implicit and varied forms of expression in internet language complicate accurate and rapid identification of suicidal intent on social media, thus creating challenges for timely intervention efforts. The development of deep learning models for suicide risk detection is a promising solution, but there is a notable lack of relevant datasets, especially in the Chinese context. To address this gap, this study presents a Chinese social media dataset designed for fine-grained suicide risk classification, focusing on indicators such as expressions of suicide intent, methods of suicide, and urgency of timing. Seven pre-trained models were evaluated in two tasks: high and low suicide risk, and fine-grained suicide risk classification on a level of 0 to 10. In our experiments, deep learning models show good performance in distinguishing between high and low suicide risk, with the best model achieving an F1 score of 88.39%. However, the results for fine-grained suicide risk classification were still unsatisfactory, with an weighted F1 score of 50.89%. To address the issues of data imbalance and limited dataset size, we investigated both traditional and advanced, large language model based data augmentation techniques, demonstrating that data augmentation can enhance model performance by up to 4.65% points in F1-score. Notably, the Chinese MentalBERT model, which was pre-trained on psychological domain data, shows superior performance in both tasks. This study provides valuable insights for automatic identification of suicidal individuals, facilitating timely psychological intervention on social media platforms. The source code and data are publicly available.