March 10th, 2022 – 11:00 am
According to the efficient coding hypothesis, the goal of the visual system should be to encode the information presented to the retina with as little redundancy as possible. From a signal processing point of view, the first step in removing redundancy is de-correlation, which removes the second order dependencies in the signal. This principle was explored in the context of trichromatic vision by Buchsbaum and Gottschalk (1) and later Ruderman et al. (2) who found that linear de-correlation of the LMS cone responses matches the opponent color coding in the human visual system.
And yet, there is comparatively little research in image processing and computer vision that explicitly model and incorporate color opponency into solving imaging tasks. A common perception is that “colors” are redundant and/or too correlated to be of any interest, or that they are too complex to deal with. Within deep learning frameworks, color features are rarely considered.
In this talk, I will illustrate with several examples of our research, such as on saliency and super-pixels, that considering opponent colors can significantly improve image processing and computer vision tasks not only in image enhancement but also image segmentation, image ranking, etc. We have in addition extended the concept of “color opponency” to include near-infrared. And we show that these de-correlation concepts also apply to deep learning models in rather interesting ways.
(1) http://rspb.royalsocietypublishing.org/content/220/1218/89.short
(2) http://www.opticsinfobase.org/josaa/abstract.cfm?&uri=josaa-15-8-2036
Bio:
Sabine Süsstrunk is full professor and director of the Image and Visual Representation Lab in the School of Computer and Communication Sciences (IC) at the Ecole Polytechnique Fédérale (EPFL), Lausanne, Switzerland, since 1999. From 2015-2020, she was also the first director of the Digital Humanities Institute (DHI), College of Humanities (CdH). Her main research areas are in computational photography, computational imaging, color image processing and computer vision, machine learning, and computational image quality and aesthetics. Sabine has authored and co-authored over 200 publications, of which 7 have received best paper/demo awards, and holds over 10 patents.
Sabine served as chair and/or committee member in many international conferences on image processing, computer vision, and image systems engineering. She is President of the Swiss Science Council SSC, Founding Member and Member of the Board of the EPFL-WISH (Women in Science and Humanities) Foundation, Member of the Board of the SRG SSR (Swiss Radio and Television Corporation), and Co-Founder and Member of the Board of Largo Films. She received the IS&T/SPIE 2013 Electronic Imaging Scientist of the Year Award for her contributions to color imaging, computational photography, and image quality, and the 2018 IS&T Raymond C. Bowman and the 2020 EPFL AGEPoly IC Polysphere Awards for excellence in teaching. Sabine is a Fellow of IEEE and IS&T and a Full Member of the Swiss Academy of Engineering Sciences (SATW).
Video replay:
