← Back to work
02 · Research

Flow Matching & MeanFlow

Generative Modeling with Flow-Based Methods

Flow Matching & MeanFlow result figure
Overview

Investigated Flow Matching and MeanFlow through a series of controlled experiments. Reproduced the prediction/loss comparisons from Back to Basics across different datasets and dimensions, then analyzed the degradation of v-prediction in high-dimensional settings. Tested model capacity and timestep scheduling as potential remedies, and implemented MeanFlow for one-step generation with NFE = 1.

Results
x-prediction + v-loss
Better high-dimensional performance
Larger Capacity + Schedule Shift
Cleaner v-prediction results
NFE = 1
One-step MeanFlow generation
Process
01
Baseline
Flow Matching + v-prediction at D = 2.
Setup
  • Prediction / Loss: v-prediction / v-loss
  • Architecture: 5 hidden layers · 256 units · ReLU
  • Time embedding: 128-d sinusoidal
  • Optimizer: Adam · lr = 1e-3
  • Batch size: 1024
  • Training steps: 25,000
  • Sampling: Euler ODE · 50 steps
  • Generated samples: 2,048
02
Reproduce
4 prediction/loss combinations × 3 datasets × 3 dimensions = 36 experiments.
Found:
  • → v-prediction degrades in high dimensions
  • → x-prediction scales better to high dimensions
  • → x-prediction + v-loss performs better
03
Rescue
Can we improve v-prediction in high dimensions?
  • → Tested model capacity
  • → Tested timestep scheduling
Found:
  • → Larger models reduce noise and scatter
  • → Schedule shifting further improves results at hidden widths of 512 and 768
  • → Best results come from larger capacity + schedule shift
04
MeanFlow
MeanFlow consistency training.
  • → NFE = 1 · One-step Generation
  • → Comparable quality to multi-step sampling
Found:
  • → One-step MeanFlow achieves comparable quality to multi-step Flow Matching
  • → Generation cost reduced to NFE = 1
Stack
PyTorchFlow MatchingMeanFlow
Notes

What surprised me most was how much the choice of prediction objective mattered as the data dimension increased. v-prediction worked well in low dimensions but became much harder to train at D=32, which led me to experiment with model capacity and timestep scheduling. I also found MeanFlow interesting because it approaches the sampling problem differently, allowing the model to generate a sample in a single step. This project gave me a better understanding of how seemingly small design choices can affect the behaviour of generative models.