Circulate Matching will not be a diffusion mannequin, nevertheless it’s intently associated — in truth, you possibly can consider it as a bridge between normalizing flows and diffusion fashions, and in some circumstances, it may be used to coach diffusion-like fashions extra effectively.
Circulate Matching is a current and promising strategy that may resolve the core computational drawback in normalizing flows — the necessity to compute costly Jacobian determinants — by changing it with a coaching goal that bypasses the log-determinant altogether. Circulate Matching (e.g., from the paper “Circulate Matching for Generative Modeling”, 2023) is a technique for coaching continuous-time generative fashions (like Neural ODEs) utilizing optimum transport-like targets, however while not having to judge or compute the Jacobian determinant. It trains a neural community to match a vector area that transforms noise into knowledge — instantly, with out integrating and backpropagating by means of an ODE solver or evaluating a change-of-variables components.
Conventional normalizing flows depend on the change-of-variable components:
This requires computing log of determinant of the jacobina which is:
Circulate Matching avoids this determinant completely through the use of a transport map coaching goal that instantly trains a vector area v(x,t)v(x, t)v(x,t) such that integrating it transforms a base distribution (e.g., Gaussian) to the goal distribution.
As an alternative of computing a log-likelihood with a Jacobian, it minimizes a regression loss between:
Circulate Matching resolves the normalizing circulate’s Jacobian drawback by:
Not utilizing change-of-variable log-determinants,
Changing probability coaching with vector area regression,
Coaching quick and in parallel.
It sits on the intersection of diffusion fashions, optimum transport, and CNFs, and is a deterministic various to score-based diffusion whereas sustaining a number of the similar advantages (no Jacobians, excessive expressiveness).
