In robotic manipulation planning, the ability to choose continuous values that satisfy complex geometric and physical constraints is crucial. However, traditional methods have relied on separate samplers for each type of constraint, which can be limiting for solving complex problems. To address this issue, researchers at MIT and Stanford University have proposed a unified framework that involves using constraint graphs and diffusion models.

Their approach involves expressing constraint-satisfaction problems as new combinations of learned constraint types using constraint graphs. These constraint graphs consist of nodes that represent decision variables, such as gripping stances or placement poses. The researchers then utilize diffusion models to solve for values that jointly fulfill the constraints.

The key contribution of their work is a method called the compositional diffusion constraint solver (Diffusion-CCSP). This solver learns a set of diffusion models for different constraints and combines them to find satisfying assignments. During inference, the diffusion models can be conditioned on any subset of variables to solve for the remaining values. The training process involves minimizing an implicit energy function, making the task of satisfying global constraints equivalent to minimizing the overall energy of solutions.

The researchers conducted experiments on four challenging domains to evaluate the performance of Diffusion-CCSP. These domains included dense-packing in two dimensions, form arrangement subject to qualitative restrictions, shape stacking in three dimensions with stability constraints, and item packing using robots. The results demonstrated that Diffusion-CCSP outperformed baselines in terms of inference speed and generalization to new constraint combinations.

In future work, the researchers suggest exploring constraints with variable arity and incorporating natural language instructions into their model. They also recommend using more complex shape encoders and learning constraints derived from real-world data to expand the scope of applications.

Sources: MIT, Stanford University