Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learning
作者: Teppei Suzuki
分类: cs.CV
发布日期: 2024-03-18
备注: Code: https://github.com/DensoITLab/Fed3DGS
💡 一句话要点
提出Fed3DGS以解决大规模3D重建的可扩展性问题
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱三：空间感知与语义 (Perception & Semantics)
📋 核心要点
- 现有的城市规模3D重建方法通常采用集中式架构，导致可扩展性差，服务器负载过重。
- 提出了一种基于3D高斯点云的联邦学习框架，能够利用分布式计算资源进行3D重建。
- 实验结果表明，Fed3DGS在渲染图像质量上与集中式方法相当，并能有效捕捉季节变化。
- method_zh
📝 摘要（中文）
本文提出了Fed3DGS，一个基于3D高斯点云（3DGS）和联邦学习的可扩展3D重建框架。现有的城市规模重建方法通常采用集中式方式，这种方法在处理超大规模场景时面临服务器负载过重和数据存储需求高的问题。为了解决这一可扩展性问题，我们提出了一种结合3DGS的联邦学习框架，能够利用分布式计算资源。我们为3DGS量身定制了一种基于蒸馏的模型更新方案，并引入外观建模以处理非独立同分布（non-IID）数据。通过在多个大规模基准上进行模拟，我们的方法在渲染图像质量上与集中式方法相当，并且能够反映场景的季节性变化。
🖼️ 关键图片
📄 摘要（原文）
In this work, we present Fed3DGS, a scalable 3D reconstruction framework based on 3D Gaussian splatting (3DGS) with federated learning. Existing city-scale reconstruction methods typically adopt a centralized approach, which gathers all data in a central server and reconstructs scenes. The approach hampers scalability because it places a heavy load on the server and demands extensive data storage when reconstructing scenes on a scale beyond city-scale. In pursuit of a more scalable 3D reconstruction, we propose a federated learning framework with 3DGS, which is a decentralized framework and can potentially use distributed computational resources across millions of clients. We tailor a distillation-based model update scheme for 3DGS and introduce appearance modeling for handling non-IID data in the scenario of 3D reconstruction with federated learning. We simulate our method on several large-scale benchmarks, and our method demonstrates rendered image quality comparable to centralized approaches. In addition, we also simulate our method with data collected in different seasons, demonstrating that our framework can reflect changes in the scenes and our appearance modeling captures changes due to seasonal variations.