Study and development of augmented reality (AR) solutions on mobile devices for virtual cosmetic product simulation using deep learning methods.

Deep learning and artificial intelligence have brought revolutionary advance to the computer vision industry. The overall subject of this project is to improve the user experience of virtual cosmetic product simulation applications (make-up, skin diag, hair coloration etc.). 

The first part of this project consists of improving the visual simulation effects of this AR application. We focus on designing deep neural network based method to improve the detection and localization of different facial components.

The second part of this project consists of improving the smootheness of the simulation on mobile devices. We aim at developping compression methods and optimised archtitectures of deep neural network to reduce the calculation and memory consumption.

To render virtual cosmetic product simulation, we need to detect several facial components such as eyes, lips, hair etc. The main challenge is the accuracy of the detection. We design several methods of semantic segmentation as well as landmark detection to localize different facial components more precisely. 

Convolutional neural networks are the most efficient solutions for image recognition and object tracking. However, better performances is always followed by more layers in the network which will also increase the computation time. Our work try to compress these models using different techniques. The classical compression methods such as pruning, quantization etc. are studied to ensure a reasonnable size, memory consumption and computation time of the models. The other aspect of this work is to construct intelligent architectures by designing intelligent blocks and using Neural Architecture Search (NAS) techniques. 

[1] Yan, Y., Berthelier, A., Duffner, S., Naturel, X., Garcia, C. and Chateau, T.  Human Hair Segmentation In The Wild Using Deep Shape Prior. In CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Long Beach, CA, 2019.

[2] Yan, Y., Berthelier, A., Duffner, S., Naturel, X., Garcia, C. and Chateau, T.  Face Parsing for Mobile AR Applications.  In demo session of International Symposium on Mixed and Augmented Reality (ISMAR) 

[3] Y. Yan, X. Naturel, T. Chateau, S. Duffner, C. Garcia, and C. Blanc. A survey of deep facial landmark detection. In Reconnaissance des Formes, Image, Apprentissage et Perception RFIAP, 2018.

[4]  A. Berthelier, P. Phutane, T. Chateau, S. Duffner, C. Garcia, and C. Blanc. Deep Model Compression for Mobile Devices : A Survey, ORASIS, 2019.