Creating Segments and Effects on Comics by Clustering Gaze Data - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue ACM Transactions on Multimedia Computing, Communications and Applications Année : 2017

Creating Segments and Effects on Comics by Clustering Gaze Data

Résumé

Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations.
Fichier non déposé

Dates et versions

hal-01650400 , version 1 (28-11-2017)

Identifiants

Citer

Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea, et al.. Creating Segments and Effects on Comics by Clustering Gaze Data. ACM Transactions on Multimedia Computing, Communications and Applications, 2017, 13 (3), pp.1 - 23. ⟨10.1145/3078836⟩. ⟨hal-01650400⟩
97 Consultations
0 Téléchargements

Altmetric

Partager

More