NeRF & 3D Gaussian Splatting

2 min read•1 page

Fourier Feature Invariants:

1. Spectral Bias Resolution: Bypasses the lower-frequency inductive bias of ReLU networks. 2. Octave Bandwidth: Powers of two 2^0, 2^1, ..., 2^(L-1) span a logarithmic frequency hierarchy. 3. Stationary Kernel Mapping: Equivalent to transforming neural regression into a tunable Gaussian RBF kernel.

python

1import numpy as np
2
3def positional_encoding(p, num_frequencies=10):
4    """
5    Fourier Positional Encoding for Neural Radiance Fields (NeRF, Mildenhall 2020).
6    gamma(p) = [sin(2^0 * pi * p), cos(2^0 * pi * p), ..., 
7                sin(2^{L-1} * pi * p), cos(2^{L-1} * pi * p)]
8    Overcomes the 'spectral bias' of standard Multi-Layer Perceptrons (MLPs),
9    enabling networks to learn sharp edges, high-frequency textures, and complex geometry.
10    """
11    encodings = []
12    for i in range(num_frequencies):
13        freq = (2.0 ** i) * np.pi
14        encodings.append(np.sin(freq * p))
15        encodings.append(np.cos(freq * p))
16    return np.concatenate(encodings)

NeRF Positional Encoding & Spectral Bias: Standard neural networks (MLPs) suffer fromspectral bias: they inherently learn low-frequency, blurry signals first and fail to reconstruct crisp geometric boundaries or sharp highlights. By projecting low-dimensional coordinates (x, y, z) into a higher-dimensional space using sinusoidal Fourier features (sin(2^k * pi * x), cos(2^k * pi * x)), Neural Radiance Fields (NeRF) capture razor-sharp photographic details and micro-geometry.

Frequency Octaves L

3.00

2 min read•1 page

3D Gaussian Splatting (3DGS Radiance Field Rasterization): While implicit neural representations (NeRF) require slow volumetric raymarching (evaluating heavy MLPs hundreds of times per pixel), 3D Gaussian Splatting (Kerbl et al. 2023) parameterizes 3D scenes as millions of explicit 3D anisotropic Gaussian ellipsoids. Each Gaussian carries a 3D position $\mu$, full covariance matrix $\Sigma = R S S^T R^T$, opacity $\alpha$, and spherical harmonics color coefficients, rasterizing in real-time at over 100+ FPS.

3DGS Mathematical Architecture:

1. Covariance Parameterization: $\Sigma = R S S^T R^T$ enforces positive semi-definiteness via quaternion $q$ and scale $s$. 2. Projective 2D Splatting: $\Sigma' = J W \Sigma W^T J^T$ flattens the 3D ellipsoid onto screen pixels. 3. Tile-Based Alpha Blending: Parallel GPU radix sort orders splats by depth for instant $O(1)$ blending.

python

1import numpy as np
2
3def compute_gaussian_covariance(scale_factors, rotation_matrix):
4    """
5    3D Gaussian Splatting Covariance Matrix (Kerbl et al. 2023).
6    Sigma = R * S * S^T * R^T.
7    Guaranteed to be symmetric positive semi-definite.
8    Projected onto 2D image plane via Jacobian J of viewing transform W:
9    Sigma_2D = J * W * Sigma * W^T * J^T.
10    """
11    S = np.diag(scale_factors)
12    Sigma_3D = rotation_matrix @ S @ S.T @ rotation_matrix.T
13    return Sigma_3D

Ellipsoid Rotation (deg)

45.00

Anisotropy Ratio (Scale X / Y)

2.50