SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence

📄 arXiv: 2607.27066v1 📥 PDF

作者: Chuanzhi Xu, Zihan Deng, Huiqi Liang, Chengkun Yue, Zhanlin Cui, Pengfei Ye, Weidong Cai

分类: cs.CV, cs.AI

发布日期: 2026-07-29

💡 一句话要点

提出SciFigAlign以解决科学图形评估问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 科学图形评估 多模态评分 CLIP SciBERT 图像质量评估 同行评审 视觉内容对齐

📋 核心要点

  1. 现有方法无法有效评估科学图形是否支持论文的科学论点，传统图像质量评估方法存在局限性。
  2. 提出SciFigAlign，通过对CLIP和SciBERT进行微调，结合图形、标题和引用段落进行多模态评分。
  3. 在测试集上，SciFigAlign的宏观MAE为0.3524，相较于最佳基线MAE 0.864，提升幅度达到59%。

📝 摘要（中文）

科学图形评估在同行评审中与一般图像质量评估有根本区别：图形必须清晰可读，真实支持论文主张，并以明确的视觉层次传达证据。传统图像评估方法在科学图形质量评估中存在局限性。本文引入了一个包含3857个同行评审会议论文的科学图形注释数据集，提出了SciFigAlign，一个基于手稿证据的多模态评分器。SciFigAlign通过对CLIP和SciBERT进行端到端微调，优化了图形质量评估。实验结果显示，SciFigAlign在测试集上取得了宏观MAE为0.3524的成绩，相较于最佳基线减少了59%的相对误差。

🔬 方法详解

问题定义：论文要解决科学图形评估中的有效性问题，现有方法无法判断图形是否支持论文的科学论点，导致评估结果不准确。

核心思路：SciFigAlign通过结合图形、标题和引用段落的上下文信息，进行多模态的评分，确保图形质量评估与手稿证据紧密结合。

技术框架：整体架构包括图形裁剪、标题、引用段落和轻量级论文上下文的输入，采用CLIP和SciBERT进行端到端微调，使用交叉注意力和CubeMLP融合。

关键创新：SciFigAlign的创新在于通过学习视觉内容与手稿证据之间的对齐，而非单纯依赖提示，显著提升了评估的准确性。

关键设计：采用SmoothL1回归和论文内排名铰链损失进行优化，确保模型在不同模态间的有效融合，关键参数设置和网络结构经过多次实验验证。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，SciFigAlign在测试集上实现了宏观MAE为0.3524，论文内成对准确率达到81.64%。与最佳基线相比，SciFigAlign实现了59%的相对误差减少，证明了其在科学图形评估中的有效性。

🎯 应用场景

该研究的潜在应用领域包括科学出版、学术评审和图形设计等。通过提高科学图形的评估准确性，能够有效支持学术界对研究成果的验证与传播，促进科学交流与合作。未来，该方法可能扩展到其他领域的图像评估任务中。

📄 摘要（原文）

Scientific figure assessment in peer review differs fundamentally from general image quality evaluation: a figure must be visually legible, faithfully support the manuscript's claims, and communicate evidence with a clear visual hierarchy. However, if we apply traditional image assessment methods to scientific figure quality assessment, limitations emerge: classic IQA models capture perceptual quality or aesthetics but cannot judge whether a figure serves the paper's scientific argument; CLIP-based methods assess generic image-text correspondence, yet lack understanding of manuscript context; and zero-shot LLM/VLM judges, when repurposed for figure scoring, often yield overly concentrated scores with limited fusion of visual and textual evidence. We introduce an annotated dataset of 3,857 scientific figures from peer-reviewed conference papers, each rated along four peer-review-oriented dimensions: Clarity, Relevance, Informativeness, and Structure. We propose SciFigAlign, a fine-tuned multimodal scorer that grounds figure quality assessment in manuscript evidence. Given a figure crop, caption, citing paragraphs, and light paper context, SciFigAlign fine-tunes CLIP and SciBERT end-to-end with per-modality cross-attention and CubeMLP fusion, jointly optimizing SmoothL1 regression with a within-paper ranking hinge loss. Under paper-level splits, SciFigAlign achieves a macro MAE of 0.3524 and a within-paper pairwise accuracy of 81.64% on the test set with n = 396, a 59% relative error reduction over the best LLM-as-judge baseline with MAE 0.864. Ablations confirm that manuscript-grounded inputs, citing-context denoising, and ranking supervision are all critical, showing that scientific figure assessment requires learned alignment between visual content and manuscript evidence rather than prompting alone, even with state-of-the-art VLMs.