Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding

📄 arXiv: 2606.18101v1 📥 PDF

作者: Jingyuan Huang, Zuming Huang, Yucheng Shi, Tianze Yang, Xiaoming Zhai, Wei Chu, Ninghao Liu

分类: cs.AI

发布日期: 2026-06-16

💡 一句话要点

提出质量感知自蒸馏方法以提升GUI定位精度

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

📋 核心要点

  1. 现有的在线自蒸馏方法在GUI定位任务中存在教师信号质量下降的问题，导致定位精度不高。
  2. 本文提出的质量感知自蒸馏方法通过软正确性感知门控和教师概率缩放来提升教师信号的可靠性。
  3. 实验结果显示，该方法在六个基准测试中均显著提升了模型性能，超越了多个强基线。
  4. method_zh
  5. 问题定义：本文旨在解决图形用户界面（GUI）定位中教师信号质量不稳定的问题。现有的在线自蒸馏（OPSD）方法在学生生成的前缀偏离目标坐标时，教师信号的有效性会降低，从而影响定位精度。\n\n核心思路：本文提出的质量感知自蒸馏方法通过引入软正确性感知门控和教师概率缩放机制，来提升教师信号的质量。软正确性感知门控用于判断教师的坐标-标记预测是否仍然能够与真实框匹配，而教师概率缩放则根据教师的置信度调整信号的强度。\n\n技术框架：该方法的整体架构包括两个主要模块：首先是软正确性感知门控模块，负责筛选出可靠的教师信号；其次是教师概率缩放模块，根据教师的置信度进一步调整信号的权重。\n\n关键创新：本文的主要创新在于结合了软正确性感知门控与教师概率缩放两种机制，前者抑制了不可靠的教师信号，后者则对剩余信号的强度进行了校准。这种组合显著提升了模型的整体性能。\n\n关键设计：在设计中，软正确性感知门控通过检查教师的坐标预测是否能够与真实框匹配来决定信号的权重，而教师概率缩放则利用教师的置信度作为轻量级因子来调整信号强度。实验表明，单独使用任一机制无法提升性能，而两者结合则能显著提高效果。
  6. application_zh
  7. 该研究的潜在应用领域包括用户界面设计、自动化测试和人机交互等。通过提升GUI定位的精度，该方法能够为开发更智能的用户界面和增强现实应用提供支持，未来可能在智能助手和自动化系统中发挥重要作用。
  8. highlight_zh
  9. 实验结果显示，质量感知自蒸馏方法在六个GUI定位基准测试中均显著提升了基础模型的性能，具体提升幅度超过了X%（具体数据待补充），并且超越了多个强基线，验证了方法的有效性。
  10. tags_zh
  11. ['图形用户界面', '自蒸馏', '视觉语言模型', '坐标预测', '深度学习', '机器学习', '信号处理']

📝 摘要（中文）

图形用户界面（GUI）定位要求视觉语言模型（VLM）在高分辨率截图中识别小目标元素并预测精确的屏幕坐标。现有的在线自蒸馏（OPSD）方法在这一坐标敏感任务中表现出色，但其在GUI定位中的应用受到限制，因为教师信号的质量可能因学生生成的前缀偏离目标坐标而下降。为此，本文提出了一种质量感知自蒸馏方法，通过软正确性感知门控和教师概率缩放来提升坐标-标记教师信号的质量。实验结果表明，该方法在六个GUI定位基准测试中均显著提升了基础模型的性能，超越了强基线。

🖼️ 关键图片

fig_0
fig_1

📄 摘要（原文）

Graphical user interface (GUI) grounding requires vision-language models (VLMs) to identify small target elements in high-resolution screenshots and predict precise screen coordinates. On-policy self-distillation (OPSD) is a promising post-training approach for this coordinate-sensitive task, since it provides dense token-level teacher signals beyond hard coordinate labels. However, naive OPSD is not well suited to GUI grounding: OPSD evaluates the teacher on student-generated prefixes, the quality of coordinate-token teacher signals can degrade when the prefix has already deviated from the target coordinate, leading to unreliable teacher signal. To mitigate this, We propose quality-aware self-distillation for VLM-based GUI grounding, which improves coordinate-token teacher-signal quality through soft correctness-aware gating and teacher-probability scaling. The soft correctness-aware gate checks whether the teacher's current coordinate-token prediction can still be completed into the ground-truth box under the student-generated prefix. If not, the corresponding teacher signal is down-weighted. Teacher-probability scaling then uses the teacher's confidence as a lightweight factor to further calibrate the strength of the gated supervision. A key empirical finding is that neither component alone improves overall performance, whereas combining them consistently improves performance. This suggests that the two mechanisms play complementary roles: correctness-aware gating suppresses unreliable coordinate-token supervision, while teacher-probability scaling calibrates the strength of the remaining signals. Experiments across six GUI grounding benchmarks show that our method consistently improves the base model and outperforms strong baselines.