AI/ML Physics Geometric ML SE(3) Flow Matching PyTorch

SE(3)-Equivariant Flow Matching on Lie Groups: Geodesic Interpolants & Matrix Exponential Maps (FoldFlow)


In Flow Matching & Diffusion Transformers (DiT), we trained velocity fields along straight Euclidean lines $x_t = (1 - t)x_0 + t x_1$ in $\mathbb{R}^d$. What happens when the generative target is not a grid of pixels, but a 3D molecular or protein backbone parameterized by $N$ rigid residue frames $x_i = (R_i, s_i) \in \mathrm{SE}(3) = \mathrm{SO}(3) \rtimes \mathbb{R}^3$?

Taking a linear combination $(1 - t)R_0 + t R_1$ of two $3 \times 3$ rotation matrices $R_0, R_1 \in \mathrm{SO}(3)$ immediately leaves the rotation manifold ($\det R_t \neq 1$, $R_t^\top R_t \neq I_3$), crushing rigid bond lengths and dihedral angles during intermediate ODE steps. FoldFlow (Bose et al., 2024, arXiv:2310.02391; GitHub) generalizes Optimal Transport Conditional Flow Matching to Riemannian manifolds and Lie groups, evolving translations in the center-of-mass zero-mean subspace $(\mathbb{R}^3)_0^N$ and rotations along exact minimum-arclength geodesics on $\mathrm{SO}(3)^N$.


0. Geodesic Interpolants & Exponential/Logarithmic Maps on $\mathrm{SO}(3)$

The special orthogonal group $\mathrm{SO}(3) = \{R \in \mathbb{R}^{3 \times 3} : R^\top R = I,\, \det R = +1\}$ has a Lie algebra $\mathfrak{so}(3)$ consisting of $3 \times 3$ skew-symmetric matrices $\Omega = -\Omega^\top$, isomorphic to axis-angle vectors $\vec{\omega} \in \mathbb{R}^3$ via the hat operator $\hat{\cdot} : \mathbb{R}^3 \to \mathfrak{so}(3)$. Equipping $\mathrm{SO}(3)$ with the canonical bi-invariant Frobenius metric $\langle \Omega_1, \Omega_2 \rangle_{\mathrm{SO}(3)} = \frac{1}{2}\mathrm{Tr}(\Omega_1^\top \Omega_2)$ yields closed-form geodesic interpolants:

$\mathrm{SO}(3)$ Geodesic Path & Tangent Conditional Velocity (Bose et al., 2024)
$$R_t = \exp_{R_0}\!\bigl(t \cdot \log_{R_0}(R_1)\bigr) = R_0 \exp\!\bigl(t \log(R_0^\top R_1)\bigr), \qquad \dot{R}_t = u_t(R_t \mid R_0, R_1) = \frac{\log_{R_t}(R_1)}{1 - t} = R_t \frac{\log(R_t^\top R_1)}{1 - t}$$ $$\exp(\hat{\omega}) = I_3 + \frac{\sin\theta}{\theta}\hat{\omega} + \frac{1 - \cos\theta}{\theta^2}\hat{\omega}^2 \quad (\theta = \|\vec{\omega}\|_2), \qquad \log(R) = \frac{\theta}{2\sin\theta}(R - R^\top) \quad \left(\theta = \arccos\frac{\mathrm{Tr}(R) - 1}{2}\right)$$

Every intermediate frame $R_t$ along the geodesic has $\det(R_t) \equiv +1$ and constant angular velocity $\|\dot{R}_t\|_{\mathrm{SO}(3)} = d_{\mathrm{SO}(3)}(R_0, R_1)$.

1. Naive Euclidean Matrix Lerp: (1−t)R₀ + t R₁ Cuts through interior of manifold → det(R_t) = 0.44 ≠ 1 Invalid non-orthogonal frame crushes Cα–N–C bond geometry 2. Riemannian Geodesic: R₀ exp(t log(R₀^T R₁)) Stays strictly on SO(3) manifold → det(R_t) ≡ 1.000000 u_t ∈ T_{R_t}SO(3) Equivariant tangent vector field + Inference Annealing i(t) = c·t
Figure 1: Chordal Euclidean interpolation cuts inside the $\mathrm{SO}(3)$ manifold and breaks orthogonality, whereas the Riemannian exponential map $R_0 \exp(t \log(R_0^\top R_1))$ follows the exact surface geodesic.
Three $\mathrm{SE}(3)$ Geometric Flow Matching Footguns in PyTorch: (1) $\arccos$ Gradient Explosion at $\theta \to 0$ and $\theta \to \pi$: The derivative of $\arccos(x)$ is $-1/\sqrt{1 - x^2}$, which diverges to $-\infty$ when $R_0^\top R_1 \approx I$ ($\mathrm{Tr}(R) \to 3$) – always clamp $\frac{\mathrm{Tr}(R)-1}{2} \in [-1 + 10^{-6}, 1 - 10^{-6}]$ and use the Taylor series $\frac{\theta}{2\sin\theta} \approx \frac{1}{2} + \frac{\theta^2}{12}$ for $\theta \lt 10^{-4}$. (2) Center-of-Mass (CoM) Drift breaks $\mathrm{SE}(3)$ Invariance: A probability density on $\mathbb{R}^{3N}$ cannot be normalizeable and translation-invariant simultaneously; project both $s_0$ and $s_1$ onto the zero-CoM subspace $\sum_{i=1}^N s_i = \vec{0}$ at every step. (3) Divergence of $1/(1-t)$ Near $t \to 1$: Clamp the training target denominator $(1 - t)$ away from $0$ to avoid gradient spikes, and during sampling apply FoldFlow's inference annealing schedule $i(t) = c \cdot t$ ($c \approx 10$) to scale the rotational vector field.

1. Runnable PyTorch Reference: Exact $\mathrm{SO}(3)$ Rodrigues Geodesic Flow & Tangent Loss

The self-contained PyTorch script below implements singularity-safe $\mathrm{SO}(3)$ exponential and logarithmic maps and verifies that geodesic interpolation preserves $\det(R_t) = 1.000000$ and orthogonality:

💻 Python: Singularity-Safe $\mathrm{SO}(3)$ Exponential/Logarithmic Maps & Geodesic Flow MatchingPyTorch
import torch

def hat_so3(w: torch.Tensor) -> torch.Tensor:
    """Map axis-angle vector w [..., 3] to skew-symmetric matrix in so(3) [..., 3, 3]."""
    wx, wy, wz = w.unbind(dim=-1)
    z = torch.zeros_like(wx)
    return torch.stack([z, -wz, wy, wz, z, -wx, -wy, wx, z], dim=-1).reshape(*w.shape[:-1], 3, 3)

def exp_so3(Omega: torch.Tensor) -> torch.Tensor:
    """Rodrigues matrix exponential exp: so(3) -> SO(3) with Taylor expansion near theta=0."""
    w = torch.stack([Omega[..., 2, 1], Omega[..., 0, 2], Omega[..., 1, 0]], dim=-1)
    theta = torch.linalg.norm(w, dim=-1, keepdim=True).unsqueeze(-1)  # [..., 1, 1]
    theta_safe = theta.clamp_min(1e-12)
    I3 = torch.eye(3, device=Omega.device, dtype=Omega.dtype)
    c1 = torch.where(theta > 1e-4, torch.sin(theta_safe) / theta_safe, 1.0 - theta.pow(2) / 6.0)
    c2 = torch.where(theta > 1e-4, (1.0 - torch.cos(theta_safe)) / theta_safe.pow(2), 0.5 - theta.pow(2) / 24.0)
    return I3 + c1 * Omega + c2 * (Omega @ Omega)

def log_so3(R: torch.Tensor) -> torch.Tensor:
    """Riemannian matrix logarithm log: SO(3) -> so(3) with safe acos clamping."""
    tr = R[..., 0, 0] + R[..., 1, 1] + R[..., 2, 2]
    cos_theta = torch.clamp((tr - 1.0) * 0.5, -1.0 + 1e-6, 1.0 - 1e-6)
    theta = torch.acos(cos_theta).unsqueeze(-1).unsqueeze(-1)
    theta_safe = theta.clamp_min(1e-12)
    scale = torch.where(theta > 1e-4, theta_safe / (2.0 * torch.sin(theta_safe)), 0.5 + theta.pow(2) / 12.0)
    return scale * (R - R.transpose(-1, -2))

if __name__ == "__main__":
    R0 = exp_so3(hat_so3(torch.tensor([0.4, -0.7, 0.5], dtype=torch.float64)))
    R1 = exp_so3(hat_so3(torch.tensor([-0.9, 0.3, 1.1], dtype=torch.float64)))
    t = 0.5
    R_euclid = (1.0 - t) * R0 + t * R1
    R_geod = R0 @ exp_so3(t * log_so3(R0.T @ R1))

    print(f"[PASS] Naive Euclidean Lerp det(R_0.5): {torch.linalg.det(R_euclid).item():.4f} (Leaves SO(3)!)")
    print(f"[PASS] Geodesic Flow det(R_0.5)       : {torch.linalg.det(R_geod).item():.6f} (Exact SO(3)!)")
    print(f"[PASS] Orthogonality Max Error        : {(R_geod.T @ R_geod - torch.eye(3, dtype=R_geod.dtype)).abs().max().item():.2e}")
# Output:
# [PASS] Naive Euclidean Lerp det(R_0.5): 0.4365 (Leaves SO(3)!)
# [PASS] Geodesic Flow det(R_0.5)       : 1.000000 (Exact SO(3)!)
# [PASS] Orthogonality Max Error        : 2.22e-16

2. 3D Molecular & Protein Generative Frameworks: Head-to-Head

Framework Euclidean Diffusion (EDM) Score SDE on $\mathrm{SE}(3)$ (FrameDiff) Riemannian OT Flow Matching (FoldFlow)
State Space Geometry Flat $\mathbb{R}^{3N}$ atomic coordinates Curved $\mathrm{SE}(3)_0^N$ via Brownian bridge SDE Exact $\mathrm{SE}(3)_0^N$ geodesic interpolants + OT coupling
Training Target Coordinate noise $\epsilon_\theta$ Stein score $\nabla_R \log p_t(R_t | R_0)$ (requires SDE sim) Simulation-free Lie algebra velocity $\frac{\log(R_t^\top R_1)}{1 - t}$
Rigid Frame Integrity Can distort bond lengths at mid-t Preserved ($\det R_t \equiv 1$) Preserved ($\det R_t \equiv 1$) with $2\text{–}5\times$ faster convergence