IST Austria main page
L3ViSU: Lifelong Learning of Visual Scene Understanding

Outline

Title: Lifelong Learning of Visual Scene Understanding

Project acronym/reference: L3ViSU / 308036

Duration: 2013-01-01 to 2017-03-31 and 2017-10-01 to 2018-12-31

EU contribution: EUR 1,464,711

Project programme: FP7 Ideas

Contract type: ERC Starting Grant (consolidator phase)

Principal Investigator

Prof. Christoph Lampert
Tel: +43 2243 9000 3101
Fax: +43 2243 9000 2000

Objective

Our goal in the project is to develop and analyse algorithms that use continuous, open-ended machine learning from visual input data (images and videos) in order to interpret visual scenes on a level comparable to humans.
L3ViSU is based on the hypothesis that we can only significantly improve the state of the art in computer vision algorithms by giving them access to background and contextual knowledge about the visual world, and that the most feasible way to obtain such knowledge is by extracting it (semi-) automatically from incoming visual stimuli. Consequently, at the core of L3ViSU lies the idea of life-long visual learning.
Sufficient data for such an effort is readily available, e.g. through digital TV-channels and media-sharing Internet platforms, but the question of how to use these resources for building better computer vision systems is wide open. In L3ViSU we will rely on modern machine learning concepts, representing task-independent prior knowledge as prior distributions and function regularizers. This functional form allows them to help solving specific tasks by guiding the solution to "reasonable" ones, and to suppress mistakes that violate "common sense". The result will not only be improved prediction quality, but also a reduction in the amount of manual supervision necessary, and the possibility to introduce more semantics into computer vision, which has recently been identified as one of the major tasks for the next decade.
L3ViSU is a project on the interface between computer vision and machine learning. Solving it requires expertise in both areas, as it is represented in my research group at IST Austria. The life-long learning concepts developed within L3ViSU, however, will have impact outside of both areas, let it be as basis of life-long learning system with a different focus, such as in bioinformatics, or as a foundation for projects of commercial value, such as more intelligent driver assistance or video surveillance systems.

Principal Investigator

  • Christoph Lampert

Team Members

  • Csaba Domokos (Postdoc, 2014-2015)
  • Sameh Khamis (Research Intern, 2013)
  • Alexander Kolesnikov (PhD Student, 2014-2018; Postdoc 2018)
  • Dmitry Kondrashkin (Research Intern, 2015)
  • Nikola Konstantinov (PhD Student, 2018)
  • Georg Martius (Postdoc, 2015-2017)
  • Emilie Morvant (Postdoc, 2013-2014)
  • Eela Nagaraj (Research Intern, 2015)
  • Saeid Naderi (Research Intern, 2016)
  • Ehsan Pajouheshgar (Research Intern, 2018)
  • Asya Pentina (PhD Student 2013-2016; Postdoc 2016-2017)
  • Mary Phuong (Phd Student, 2017-2018)
  • Marian Poljak (Research Intern, 2018)
  • Bernd Prach (Research Intern, 2018)
  • Sylvestre Rebuffi (Research Intern, 2016)
  • Amélie Royer (Research Intern 2014; PhD Student, 2016-2018)
  • Mayu Sakurada (Research Intern, 2014)
  • Neel Shah (Phd Student, 2013-2015)
  • Viktoriia Sharmanska (PhD Student, 2013-2015)
  • Jacquelyn Shelton (Research Intern, 2013)
  • Remy Sun (Research Intern, 2017)
  • Vladyslav Sydorov (Research Intern, 2014)
  • Nathaniel Ver Steeg (Research Intern, 2015)
  • Chris Wendler (Research Intern, 2014)
  • Alexander Zimin (PhD Student, 2014-2018)

Publications

Theses

Presentations

  • ...needs to updated, sorry...
  • 03/2017 Invited talk: IIT-IST Workshop: Incremental Classifier and Representation Learning, Genoa, IT.
  • 10/2016 Invited talk: TASK-CV Workshop at ECCV: Towards Principled Transfer Learning, Amsterdam, NL.
  • 09/2016 Invited talk: Microsoft Research: Multi-task and lifelong learning with unlabeled tasks, Cambridge, UK.
  • 09/2016 Invited talk: University of Oxford: Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation, Oxford, UK.
  • 09/2016 Invited talk: Yandex: Classifier Adaptation at Prediction Time, Moscow, RU.
  • 09/2016 Invited talk: Skoltech: Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation, Moscow, RU.
  • 09/2016 Invited talk: Higher School of Economics: Multi-task learning with unlabeled tasks, Moscow, RU.
  • 08/2016 Keynote talk: VS3 Workshop: Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation, Prague, CZ.
  • 06/2016 Keynote talk: CHIST-ERA Conference: Towards Lifelong Machine Learning, Vienna, AT.
  • 05/2016 Invited talk: AC Workshop of the DAGM: Lifelong Learning for Visual Scene Understanding, Hannover, DE.
  • 02/2016 Invited talk: High Visual Computing (HiVisComp) 2016: Towards lifelong visual learning, Bohemian Forest, CZ.
  • 01/2016 University of Brno: Classifier adaptation as prediction time, Brno, CZ.
  • 09/2015 Keynote talk: Netherlands Conference on Computer Vision (NCCV): Towards Lifelong Learning, Lunteren, NL.
  • 08/2015 Invited talk: VS3 Workshop: Classifier adaptation as prediction time, Prague, CZ.
  • 07/2015 Keynote talk: Symposium on Intelligent Systems in Science and Industry: Towards Lifelong Learning
  • 04/2015 Keynote talk: Dagstuhl Seminar on Machine Learning with Interdependent and Non-identically Distributed Data, Dagstuhl, DE.
  • 02/2015 Invited talk: Weizmann Workshop on Computational Challenges in Large Scale Image Analysis, Weizman Institute, Rehovot, IL.
  • 01/2015 University of Heidelberg, Image Representations and Learning, Heidelberg, DE.
  • 12/2014 University College London, Learning with Attributes for Object Recognition: Parametric and Non-parametric Views, London, UK.
  • 12/2014 University of Lübeck, Towards lifelong visual learning, Lübeck, DE.
  • 09/2014 Invited talk: ECCV Workshop on Transferring and Adapting Source Knowledge in Computer Vision, Learning with a time-evolving data distribution , Zurich, CH.
  • 06/2014 Invited talk: CVPR Workshop on Long Term Detection and Tracking, Learning with a time-evolving data distribution , Columbus, OH, USA.
  • 06/2014 Invited talk: ECCV Area Chair Workshop, Closed-form training of conditional random fields for large scale image segmentation , Zurich, CH.
  • 06/2014 University of Marburg, Machine learning for visual scene understanding, Marburg, DE.
  • 05/2014 Laboratoire d'Informatique, When PAC-Bayes meets Domain Adaptation, Grenoble, FR.
  • 05/2014 MPI for Intelligent Systems, Towards lifelong visual learning, Tübingen, DE.
  • 02/2014 Invited talk: Workshop on Recent Trends in Computer Vision, Learning with asymmetric data distributions, University of Maryland , College Park, MD, USA.
  • 02/2014 Rutgers University, Learning with asymmetric information, New Brunswick, NJ, USA.
  • 02/2014 New York University, Learning with asymmetric information, New York, NY, USA.
  • 02/2014 Memorial Sloan Kettering Cancer Center, Learning with asymmetric information, New York, NY, USA.
  • 02/2014 Massachusetts Institute of Technology, Learning with asymmetric information, Boston, MA, USA.
  • 02/2014 Laboratoire Hubert Curien, Domain Adaptation of Majority Votes via Perturbed Variation-based Label Transfer, Saint-Etienne, FR.
  • 01/2014 INRIA Rhone-Alpes, Learning with asymmetric information, Grenoble, FR.
  • 12/2013 NICTA Research Laboratory, Attribute-based classification, Sydney, AU.
  • 12/2013 Invited talk: ICCV Workshop on Visual Domain Adaptation and Dataset Bias, Towards lifelong visual learning: From practice to theory and back , Sydney, AU.
  • 12/2013 Signal Processing/Machine Learning Seminars, LATP/LIF, Aix-Marseille Université, FR
  • 10/2013 MPI for Informatics Schiele Group Retreat, Lifelong learning for visual scene understanding - from practice to theory and back, Schloss Ringberg, DE.
  • 09/2013 University of Texas, Learning using privileged information, Austin, TX, USA.
  • 05/2013 Featured talk: Workshop of the Austrian Association for Pattern Recognition, Visual scene understanding , Innsbruck, AT.
  • 05/2013 University of Illinois at Urbana-Champaign, Attribute-based classification and the dream of lifelong learning for visual scene understanding, Champaign, IL, USA.
  • 05/2013 Carnegie Mellon University, Attribute-based classification and the dream of lifelong learning for visual scene understanding, Pittsburgh, PA, USA.
  • 03/2013 University of California, Berkeley, Attribute-based classification and the dream of lifelong learning for visual scene understanding, Berkeley, CA, USA.
  • 03/2013 Stanford University, Attribute-based classification and the dream of lifelong learning for visual scene understanding, Palo Alto, CA, USA.
  • 01/2013 University of Leuven, Attribute-based classification and the dream of lifelong learning for visual scene understanding, Leuven, BE.

Sponsors