PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models

📄 arXiv: 2606.22958v1 📥 PDF

作者: Ruolan Sun, Pawel Polak

分类: cs.LG, cs.CV

发布日期: 2026-06-22

备注: Code: https://github.com/sophialanlan/PG-MAP

💡 一句话要点

提出PG-MAP以解决文本到图像模型推理对齐问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱四：生成式动作 (Generative Motion) 支柱七：动作重定向 (Motion Retargeting)

关键词: 文本到图像生成 推理对齐 多模态优化 Gibbs-MAP 流匹配模型

📋 核心要点

  1. 现有的文本到图像模型推理对齐方法通常仅沿单一控制轴进行，限制了条件和潜变量之间的联合建模能力。
  2. PG-MAP提出了一种无训练的框架，通过轨迹级Gibbs-MAP优化实现条件和潜状态的对齐，支持跨模态协调更新。
  3. 在多个扩散模型上，PG-MAP显著提升了对齐指标，并在流匹配模型上实现了91.9%的PickScore，超越了静态基线。

📝 摘要（中文）

在预训练的文本到图像模型中，推理时的对齐通常沿单一控制轴进行，如无分类器引导、注意力编辑或基于奖励的潜变量扰动。这种限制阻碍了条件变量与潜变量之间的联合依赖建模，影响了生成传输的迁移能力。本文提出PG-MAP，一个无训练框架，将推理时对齐形式化为条件$c$和潜状态$z_t$的轨迹级Gibbs-MAP/近端能量优化，通过前向一致性耦合实现，且可选用冻结的偏好奖励引导。该联合形式使得跨模态的协调更新成为可能，同时与扩散和流匹配模型兼容。实验结果表明，PG-MAP在多个扩散模型上显著提高了对齐指标，并在流匹配模型上表现出色。

🔬 方法详解

问题定义：本文旨在解决预训练文本到图像模型在推理时对齐的局限性，现有方法无法有效建模条件变量与潜变量之间的联合依赖，影响生成效果。

核心思路：PG-MAP通过轨迹级Gibbs-MAP/近端能量优化，将推理对齐问题转化为条件$c$和潜状态$z_t$的联合优化，允许跨模态的协调更新。

技术框架：PG-MAP的整体架构包括前向一致性耦合模块和可选的冻结偏好奖励引导，支持与扩散和流匹配模型的兼容性。

关键创新：PG-MAP的主要创新在于其训练自由的框架和联合优化策略，使得条件与潜变量的更新能够协调进行，突破了传统方法的单一控制轴限制。

关键设计：在设计中，PG-MAP采用了轨迹级优化策略，结合了特定于传输的适配，确保了在不同模型架构下的有效性，同时通过控制实验排除了噪声相关的伪影影响。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在多个扩散模型（如SD~1.5和SDXL）上，PG-MAP显著提高了对齐指标，如PickScore和美学评分，并在流匹配模型（SD3.5-medium）上实现了91.9%的PickScore和75.7%的HPS胜率，超越了静态基线，且人类评估结果进一步确认了其优越性。

🎯 应用场景

PG-MAP的研究成果在文本到图像生成、图像编辑和多模态生成任务中具有广泛的应用潜力。其优化框架能够提升生成模型的对齐性能，进而提高生成内容的质量和一致性，具有重要的实际价值和未来影响。

📄 摘要（原文）

Inference-time alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance, attention editing, or reward-based latent perturbations. This limitation prevents modeling joint dependencies between conditioning and latent variables and hinders transfer across generative transports. We propose PG-MAP, a training-free framework that formulates inference-time alignment as a trajectory-level Gibbs-MAP / proximal energy optimization over the conditioning $c$ and latent state $z_t$ via a forward-consistency coupling, optionally guided by a frozen preference reward. This joint formulation enables coordinated updates across modalities while remaining compatible with both diffusion and flow-matching models through transport-specific adaptations. Across diffusion backbones (SD~1.5, SDXL), PG-MAP consistently improves alignment metrics such as PickScore and Aesthetic, and can be effectively combined with tuned classifier-free guidance to achieve the strongest overall performance. On flow-matching models (SD3.5-medium), the framework reduces to a latent-only variant, achieving $\mathbf{91.9\%}$ PickScore and $75.7\%$ HPS win rates against a static baseline, with controlled experiments ruling out noise-related artifacts. Human evaluations further confirm consistent preference over strong baselines, including tuned CFG and compute-matched universal guidance. Finally, an oracle-routing analysis shows that the relative importance of conditioning and latent optimization depends on prompt types, surfacing further headroom that a per-prompt selector could exploit.