Overview

The main research objective of this project is to study the problem of translucent objects perception and to develop efficient algorithms for their pose estimation in videos. The new methods will be based on deep neural networks and trained end-to-end, which will result in elegant formulation and efficient implementation. First, we will develop new backbone architectures trained on RGB images to emphasize the visual cues specific for translucent objects. The developed methods will be able to distinguish between the tracked object, surrounding background and objects that could be visually similar to the tracked target. Aparticular focus will be put into assuring high accuracy of the predicted target position. We will go beyond commonly used bounding boxes and develop a method for segmenting the tracked translucent object. Since all methods we plan to develop within this project are based on deep learning, a large amount of annotated training data will be required. We will thus develop a method for generating semi-synthetic videos using photo-realistic rendering engines. This method will be used to create a large training dataset with accurately annotated translucent objects. All developed methods will be evaluated on real data as well as new data generated by our rendering approach and compared to state-of-the-art methods for translucent and opaque object tracking.

Work packages

The work is divided into four work packages:

• WP1: Discriminative tracking architecture robust to distractors
• WP2: Segmentation of translucent objects
• WP3: Performance evaluation on translucent objects
• WP4: dissemination of the results

Online datasets

• A rendering engine and dataset for training transparent object trackers: Trans2k

Publications

Funding