Under Review

DiffusionPack: Offline Bin Packing with Custom Human Preferences

Anonymous Affiliation · Anonymous Email

Paper (PDF) Code (GitHub) Live Demo ↓ arXiv

Abstract

The three-dimensional bin packing problem (3D-BPP) is fundamental in robotic palletizing and warehouse automation, where systems must generate compact, feasible layouts that also satisfy user-defined requirements. Existing exact and heuristic methods scale poorly or cannot accommodate dynamic preferences, while learning-based approaches typically require retraining for new constraints. We present DiffusionPack, a planning framework that formulates 3D-BPP as a conditional generation problem and learns the joint distribution of complete packing layouts using a diffusion model. To ensure physical feasibility, diffusion outputs are refined through a projection mechanism guided by point-based heuristics. The method incorporates user guidance at inference: hard constraints are enforced through inpainting, and soft preferences are integrated via guidance functions. Natural language instructions are supported through an LLM that translates user input into constraint specifications. Real-world validation on a Franka robot demonstrate the applicability of our approach to practical robotic packing tasks.

Overview

Key Contributions

Three core technical advances enabling preference-conditioned 3D bin packing

Hybrid Diffusion–Projection Planner

A diffusion model jointly predicts complete 3D packing configurations, refined via a keypoint-based projection operator that guarantees physical feasibility without gradient-based projections.

Inference-Time Preference Integration

Hard constraints are enforced via inpainting; soft preferences via differentiable guidance functions. No retraining required — preferences are specified at test time over any new instance.

Natural Language Conditioning via LLM

A pretrained GPT-4o model translates natural language instructions into executable constraint code, enabling intuitive human–robot interaction without manual engineering of cost functions.

Method

DiffusionPack pipeline: from noisy centroids to feasible packing configurations

PHASE 01

Training

DDPM trains the DiT denoiser with MSE loss on noisy centroid configurations

PHASE 02

Sampling

500 training-step schedule compressed to 250 sampling steps; 512 candidates run in parallel

PHASE 03

Guidance

Differentiable cost functions steer sampling toward soft user preferences

PHASE 04

Selection

512 candidates evaluated; best by Utility Rate or Packing Density returned

Denoising Objective

The model predicts the injected Gaussian noise $\epsilon_\theta(x_t, c, t)$ from noisy centroid configurations conditioned on cuboid dimensions $c$. Clean centroids are recovered iteratively from pure noise.

$$\mathcal{L}(\theta) = \mathbb{E}\left[\left\lVert \epsilon - \epsilon_\theta(x_t, c, t) \right\rVert^2\right]$$

A DiT-style transformer with 6 layers, hidden dim 256, and 8 attention heads enables global reasoning across all items simultaneously. No positional encoding ensures permutation invariance.

Keypoint-Based Feasibility

Diffusion outputs are projected onto valid placements using Empty Maximal Spaces (EMS) keypoints — the strongest strategy, directing placements into large unoccupied regions.

$$P_C(x) = \operatorname*{arg\,min}_{y \in C} \left\lVert y - x \right\rVert^2$$

Projection is applied only at the final denoising step, preserving global spatial structure formed during denoising while correcting minor boundary violations.

Hard Constraint Enforcement

Items subject to hard preferences (e.g. "always on top") have their positions overwritten after each denoising step, guaranteeing constraint satisfaction in the final layout.

$$x_{t-1} = (1 - M) \odot \tilde{x}_{t-1} + M \odot y_0$$

$M$ is a binary mask over constrained items; $y_0$ provides the reference position from the LLM-generated inpainting rule.

Soft Preference Steering

Soft preferences are encoded as differentiable cost functions. Their gradients bias the denoising trajectory without hard-fixing positions, allowing simultaneous balancing of multiple soft constraints.

$$g = -\sum_i \lambda_i \, \nabla_{x_{t-1}} c_i(x_{t-1})$$

Guidance weights $\lambda_i \in [1, 25]$ are selected via GPU-parallelised grid search across the 512 candidate batch.

Real Denoising Trajectories

Real DiT model outputs — centroids recorded at 30 DDPM keyframes across 250 denoising steps, with EMS projection at t=0

Select Example

Playback

Speed

Show unplaced ghost

Model Info

Dataset—

Items—

ProjectionEMS

Candidates512

DDPM steps250

3D Bin — Real DiT Output

Drag to orbit · scroll to zoom

Select an example

Select an example and press Play

Denoising step: — 0%

—

Utility Rate

—

Pack. Density

—

Items Placed

—

Seed

Real model output. Trajectories captured from the trained DiT denoiser running 250 DDPM steps with 512 candidates per instance; the best candidate by Utility Rate is shown. EMS projection is applied only at t=0. CSD examples tile the container perfectly; RSD examples reflect true out-of-distribution performance (~75–85% items placed). Unplaced items shown as ghosts when toggled.

Experimental Results

Evaluated on Cutting Stock Dataset (CSD) and Randomly Sampled Dataset (RSD), 16 and 32 items

Utility Rate — CSD-32

Higher is better · mean ± std over 5 seeds

Utility Rate — RSD-32

Out-of-distribution generalization · 32 items

Preference Satisfaction Rate

vs. DiffusionCCSP · % of instances satisfying all constraints

Full Comparison Table — Utility Rate (%) and Packing Density (%)

Method	Time (s)	CSD-16 UR	CSD-16 PD	CSD-32 UR	CSD-32 PD	RSD-16 UR	RSD-16 PD	RSD-32 UR	RSD-32 PD
First Fit (EP)	0.01	56.7	44.8	53.2	43.7	57.4	54.4	67.5	67.1
Key Point	0.04	61.3	53.5	60.1	53.5	63.0	66.4	69.7	69.9
BRKGA	1.20	58.9	45.4	58.1	45.3	64.7	57.6	71.1	68.3
DBLF + GA	1.44	94.5	82.0	70.8	60.3	71.4	58.1	67.5	65.3
RCQL	1.25	87.1	71.8	62.6	59.3	69.1	57.8	69.6	61.3
Ranked-Reward	360	49.3	53.8	50.9	57.6	61.9	64.3	71.8	72.0
GUROBI	63.22	97.1	81.8	77.6	69.3	74.8	67.9	73.6	68.3
Diffusion-Only	8.05	48.9	33.1	43.4	34.7	47.2	34.2	49.5	35.1
Diffusion-CP	9.95	99.2	90.1	93.5	89.1	68.3	64.0	79.6	70.7
Diffusion-EP	8.52	99.1	90.1	93.5	89.5	68.1	59.0	80.3	71.8
Diffusion-EMS (Ours)	6.22	99.8	90.1	99.4	90.1	75.8	67.3	85.7	76.3

Ablation Studies

Effect of projection timing and keypoint strategy on packing quality

Projection Timing — Utility Rate (%)

CSD-16 and CSD-32 · final-step projection is best

Why Final-Step Only Works Best

Applying projection at every step overconstrains the sampling process, reducing diversity and preventing the diffusion model from forming coherent global layouts. Early-step projection is ineffective due to high positional variance at large t.

Projection at t=0 allows the diffusion model to fully form a global spatial structure before geometric corrections are applied, preserving the quality of the learned layout prior.

Keypoint Strategy Comparison

Empty Maximal Spaces (EMS) 99.8%

Extreme Points (EP) 99.1%

Corner Points (CP) 99.2%

EMS yields the strongest performance because it directs projections into large unoccupied spaces with high structural freedom.

Human Preference Integration

Hard and soft constraints expressed in natural language, translated by GPT-4o at inference time

Hard Constraint · Inpainting

"Place the red box at the bottom of the bin"

Positions are overwritten after each denoising step. Satisfaction is guaranteed in the final layout. LLM generates a Python inpainting function.

Soft Constraint · Guidance

"Heavy boxes should be near the bottom"

A differentiable cost function biases the denoising trajectory. Allows balancing of multiple soft preferences simultaneously via gradient-based steering.

Spatial Relation

"Keep pink behind green & red above blue"

Relative spatial constraints between named items, encoded as inpainting rules with offsets derived from centroid and dimension tensors.

Grouping

"Group all black items together"

Differentiable proximity cost using cosine similarity on dimension vectors. Note: color-based grouping can fail due to non-differentiable color lookups.

Real-World Validation

Franka FR3 with suction gripper · ten physical packing trials

80%

Task success rate · 8 of 10 trials

Execution Protocol

Objects are localised at trial start using AprilTag fiducial markers. DiffusionPack generates a complete plan offline before execution begins. Items are placed in depth order along the y-axis to prevent gripper collisions with already-placed items.

All cuboid dimensions are inflated by 1 cm during planning to absorb perception and actuation uncertainties. A guarded motion lowers each item until contact is detected.

Robot armFranka FR3 (7-DoF)

End effectorSuction-cup gripper

PerceptionIntel RealSense D435

LocalisationAprilTag fiducials