Motion–Surface–Appearance Dynamic Gaussian Splatting
for Accurate Geometry and High-Fidelity Rendering

Seungjae Lee1,  Ho Jun Kim1,  Hak Gu Kim2,†
1Department of Imaging Science and Arts, Chung-Ang University, South Korea
2Department of Metaverse Convergence, Chung-Ang University, South Korea    †Corresponding Author
Paper (Coming Soon) Code (Coming Soon)
Our method achieves both high-fidelity rendering and geometrically accurate mesh reconstruction, resolving the trade-off inherent in prior dynamic Gaussian Splatting methods.

Existing dynamic Gaussian Splatting methods face a persistent trade-off between rendering quality and mesh accuracy. Our method resolves this by jointly improving both, yielding sharp renderings and clean mesh geometry across diverse dynamic scenes.

Abstract

Dynamic Gaussian Splatting (3DGS) has achieved impressive performance for dynamic scene reconstruction, yet jointly improving rendering fidelity and mesh reconstruction remains challenging. This trade-off arises because motion modeling, surface reconstruction, and appearance preservation are tightly coupled within a shared Gaussian representation. To address this challenge, we propose Motion–Surface–Appearance Dynamic Gaussian Splatting (MSA-DGS), a unified framework that progressively improves these three aspects of dynamic reconstruction.

First, Multi-Canonical Deformation Modeling (MCDM) partitions the sequence into multiple canonical Gaussian groups to better capture localized temporal deformation. Second, Surface-Aligned Anisotropic Densification (SAAD) initializes new Gaussian primitives according to local surface geometry, providing reliable surface support for mesh reconstruction. Finally, Residual-Based Expressiveness Restoration (RBER) progressively restores fine appearance details through geometry-space and image-space residual refinement while preserving the established geometry. Extensive experiments on DG-Mesh, D-NeRF, and Nerfies demonstrate that the proposed framework consistently achieves a better balance between rendering fidelity and mesh reconstruction than existing methods.

Video Results

Each video shows our reconstructed dynamic scene (Gaussian rendering + mesh) across time.

Girlwalk

Hook

Jumping Jacks

Stand Up

Video Comparison

Drag the divider left or right to compare our method with DG-Mesh.

Horse

DG-Mesh Ours

Beagle

DG-Mesh Ours

Bird

DG-Mesh Ours

3D Mesh Results

Drag to rotate  ·  Scroll to zoom  ·  Right-drag to pan

Bird

drag / scroll / right-drag

Duck

drag / scroll / right-drag

T-Rex

drag / scroll / right-drag

Method

Overview of the proposed MSA-DGS framework integrating MCDM, SAAD, and RBER.

MSA-DGS integrates three core modules that progressively improve the three tightly coupled aspects of dynamic Gaussian reconstruction — motion modeling, surface reconstruction, and appearance preservation. A first stage establishes the base dynamic representation through MCDM and SAAD, and a second stage applies RBER to restore fine appearance detail while preserving the established geometry.

① Multi-Canonical Deformation Modeling (MCDM)
Addresses excessive temporal deformation span by partitioning the sequence into K localized temporal groups, each assigned to an independent canonical Gaussian set. A global warm-up stage first establishes a sequence-level prior, which is then cloned into K group-specific canonical sets. Since each set covers only a restricted temporal segment, the deformation network operates over a reduced motion range, mitigating over-smoothing and improving per-frame fidelity.
② Surface-Aligned Anisotropic Densification (SAAD)
Corrects the geometry-driven bias of conventional isotropic densification under mesh supervision. Instead of inserting primitives at face centroids with isotropic scales, SAAD derives the initial covariance of each new Gaussian from the eigendecomposition of local face vertex covariance, aligning its tangent-plane support with the local surface structure. This turns densification from a purely coverage-seeking operation into a surface-aligned refinement process, improving both geometric coverage and mesh accuracy.
③ Residual-Based Expressiveness Restoration (RBER)
Recovers the fine appearance capacity attenuated by strong surface supervision through two complementary residual refinements. Geometry-space residual refinement uses a residual deformation network (ΦGR) that predicts small corrections to the deformed Gaussian parameters (position, rotation, scale), restoring appearance capacity around the geometry-consistent solution. Image-space residual refinement then employs a residual network (ΦIR) that recovers the remaining high-frequency appearance discrepancy directly in the rendered image via Fourier feature mapping on top of a frozen low-frequency base. Together they restore appearance details without modifying the surface structure established by MCDM and SAAD.

Qualitative Comparison

Qualitative comparison with DG-Mesh and Dynamic-2DGS on DG-Mesh and D-NeRF datasets.

We compare against state-of-the-art dynamic reconstruction methods — DG-Mesh and Dynamic-2DGS — on the DG-Mesh and D-NeRF datasets. For each method, we show both the direct Gaussian Splatting (GS) rendering and the extracted mesh surface. DG-Mesh yields blurry renderings, and Dynamic-2DGS produces fragmented or incomplete surfaces (e.g., missing parts in Horse). Our method reconstructs sharp appearances with topologically clean meshes, achieving superior performance in both rendering fidelity and geometric accuracy.

Ablation Study

Effectiveness of MCDM

Ablation on MCDM: baseline vs. multi-canonical deformation across an extended temporal sequence.

We compare the baseline (single canonical space) against MCDM (multiple canonical spaces) across an extended temporal sequence (T = 0, …, 10). The baseline forces a single canonical representation to span the entire motion range, causing the deformation network to over-smooth time-specific structures, leading to blurred renderings and incoherent geometry. MCDM redistributes temporal modeling burden across K canonical sets. Since each canonical set covers only a localized segment, per-frame fidelity improves substantially, yielding sharper appearances and more coherent mesh reconstruction.

Effectiveness of SAAD

Ablation on SAAD: isotropic densification vs. surface-aligned anisotropic densification.

We compare isotropic densification (baseline) against our Surface-Aligned Anisotropic Densification (SAAD). Under mesh supervision, isotropic primitives inserted at face centroids ignore local surface anisotropy, progressively degrading rendering quality as they accumulate (visible blur at 10k → 25k iterations). SAAD derives the initial covariance of each new Gaussian from local face vertex statistics via eigendecomposition, producing surface-aligned primitives that preserve structural detail. The result is sharper boundaries and cleaner local structure throughout training.

BibTeX

@article{TODO,
  author  = {Seungjae Lee and Ho Jun Kim and Hak Gu Kim},
  title   = {Motion--Surface--Appearance Dynamic Gaussian Splatting for Accurate Geometry and High-Fidelity Rendering},
  journal = {TODO},
  year    = {2025}
}