IBROC – 3D Image-Based Rendering of Clothes from Photographs

Online retailers have already tried different methods for virtual try-on. However, all of them require 3D models of the clothes or a specialized studio photography process to work. Both requirements are labor intensive and therefore prohibitively expensive in the fast moving fashion industry. Fashion photos from social networks or photographed by the users can not be used for try-on, either.

IBROC is a new intelligent system that enables users to try on clothes from just an image. Fashion photos as they are common in catalogs, webshops or social networks can be used. Such photos can either show a model wearing the piece, the piece draped on a mannequin or hanging on a hanger. Such photos are subject to a number of defects including occlusions (e.g. hands cover parts of the piece), unrealistic deformations (e.g. the garment lies on the floor and looks flat) and weak lighting situations.

IBROC uses visual computing methods to solve these problems. To extract 3D information from a set of fashion photographs, a template (shape prior) is needed. In this project, a template database of fashion images with associated 3D models will be created. Based on that database, our matching and machine learning methods compute a new 3D model for a desired piece of clothing. Occluded regions can be filled automatically by texture synthesis and projection of multiple fashion images. The resulting models can be used to simulate how a user would look when wearing the clothes.

The proposed method does not need manual 3D modeling or specialized photography methods. Therefore, it is particularly scalable and can be deployed as a mobile app or website right away. Such an application would collect an extensive data collection of fashion preferences linked with social network accounts. The data collection enables a new form of native advertising. Clothes can be advertised with the user as the model. Personalized ads can adapt to the user's taste and wardrobe and only propose relevant items.

Prof. Dieter Schmalstieg, TU Graz

Prof. Dieter Schmalstieg, TU Graz
E-Mail: schmalstieg@icg.tugraz.at
Web: http://icg.tugraz.at