CloSe: A 3D Clothing Segmentation Dataset and Model

Dimitrije Antić   Garvita Tiwari   Batuhan Ozcomlekci   Riccardo Marin   Gerard Pons-Moll

3DV 2024

 

Environment Setup

The code was tested under Ubuntu 22.04, Python 3.9, CUDA 11.6, Pytorch 1.13.0 Use the following command to create a conda environment with all the required dependencies:

git clone --recursive https://github.com/anticdimi/CloSe.git
cd CloSe
conda env create -f env.yml
conda activate close

To build the custom Open3D extension needed to run the CloSeT, see the instructions in docs/CloSeT.md.

Note

If the environment setup fails, please follow instructions on how to install Pytorch3D here, and install PyTorch from here.

CloSe-D Dataset

The dataset is hosted on the Hugging Face Hub: 🤗 anticdimi/CloSe-D.

Quick download via the CLI:

huggingface-cli download anticdimi/CloSe-D --repo-type dataset --local-dir ./data

Or programmatically:

from huggingface_hub import snapshot_download
path = snapshot_download(repo_id='anticdimi/CloSe-D', repo_type='dataset')

For file formats, label mapping, subset licensing (CloSe-Di / CloSe-Dc / CloSe-D++), and instructions on obtaining the commercial scans (Renderpeople, Twindom, AXYZ), see docs/dataset.md.

CloSe-Net Clothing Segmentation Method

The pretrained checkpoint is available here.

Inference

After setting up the environment and downloading the pretrained models, you can run the inference on the provided example scans using the following command:

python demo.py --render

And the results will be saved in the ./out folder.

Note

See the prep_scan.py script to see how the data is prepared for inference.

Training

For training CloSeNet model, you can use the following command:

python train_closenet.py cfg/closenet.yaml

See config file for more detail abot the training setup.

CloSe-T Interactive Tool

The steps for installing and using the interactive tool is described in docs/CloSeT.md.

Citation

If you find this work useful, please consider citing:

@inproceedings{antic2024close,
    title = {{CloSe}: A {3D} Clothing Segmentation Dataset and Model},
    author = {Antić, Dimitrije and Tiwari, Garvita and Ozcomlekci, Batuhan  and Marin, Riccardo  and Pons-Moll, Gerard},
    booktitle = {International Conference on 3D Vision (3DV)},
    month = {March},
    year = {2024},
}

@inproceedings{tiwari20sizer,
    title = {{SIZER}: A Dataset and Model for Parsing {3D} Clothing and Learning Size Sensitive {3D} Clothing},
    author = {Tiwari, Garvita and Bhatnagar, Bharat Lal and Tung, Tony and Pons-Moll, Gerard},
    booktitle = {European Conference on Computer Vision ({ECCV})},
    month = {August},
    organization = {{Springer}},
    year = {2020},
}