ProcTHOR uses procedural generation to sample massively diverse, realistic, interactive, customizable, and performant 3D environments to train simulated embodied agents. Here is an example of sampling virtual home environments.
Customizability. ProcTHOR can construct custom scene types, such as classrooms, libraries, and offices.
Results. Pre-training with ProcTHOR improves downstream performance. Zero-Shot performance, from models pre-trained on ProcTHOR, often beats the same models trained on the training data from the benchmark it is evaluated on.
Semantic Object Navigation in Dorm Styled, 3D Artist-Designed Scenes
Semantic Object Navigation in Multi-Floor, 3D Matterport Scanned Scenes
Semantic Object Navigation in Room-Sized, 3D Artist-Designed Scenes
Semantic Object Navigation in Single-Floor House-Sized, 3D Artist-Designed Scenes
Interactive Rearrangement in Room-Sized, 3D Artist-Designed Scenes
Interactive Manipulation in Room-Sized, 3D Artist-Designed Scenes
Scale Improves Performance. Scaling the number of training houses consistently improves zero-shot performance.
Floorplan Diversity. ProcTHOR houses sample extremely diverse floorplans. Here are examples of sampled floorplans with between 1 and 10 rooms.
Object Diversity. ProcTHOR includes 1,633 interactive household objects across 108 categories. A small subset of these objects is shown below.
Material Augmentation. ProcTHOR includes 3,278 materials that can be used to visually augment objects, walls, floors, and ceilings.
Object Placement. ProcTHOR houses sample realistic and diverse placement of objects.
Lighting Variation. Environment lighting can be rendered with significantly variation to simulate real-world lighting conditions at any time of day.
Interactivity. ProcTHOR objects are highly interactive, supporting object state changes, robotic arm manipulation, and multi-agent interaction.
Get Started. ProcTHOR is fully open-source and available to the Embodied AI community. We are excited to see what you build!