Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-settings.php on line 512

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-settings.php on line 527

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-settings.php on line 534

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-settings.php on line 570

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-includes/cache.php on line 103

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-includes/query.php on line 61

Deprecated: Assigning the return value of new by reference is deprecated in /mnt/tb/scilsdata/www/comminfo.rutgers.edu/htdocs2/conferences/mmchallenge/wp-includes/theme.php on line 1109
Google Challenge 2010: Robust, As-Accurate-As-Human Genre Classification for Video – Multimedia Grand Challenge 2010
ACM MM 2010 Header Image

Google Challenge 2010: Robust, As-Accurate-As-Human Genre Classification for Video

A notion of browsing collections is naturally associated with videos. Having videos classified into a pre-existing hierarchy of genres is one way to make the browsing task easier. The goal of this task would be to take user generated videos (along with their sparse and noisy metadata) and automatically classify them into genres. A public genre hierarchy like ODP (Open Directory Project) can be used as a target for this task.

Evaluations can be based on purely video content features as well as a combination of content and metadata features. Features that bring in information from other public data sources can also be used (eg. Object detectors trained on a separate public dataset). Thinking of new (and surprising) features is recommended!

Any dataset that reflects a breath of content is acceptable, and of course, YouTube and Google Video are a recommended source. Particularly, the data should cover most of the common video genres. If the dataset consists of web videos, sharing a list of links and corresponding labels would be ideal for researchers to compare notes. You may want to consult the The YouTube Data API for retrieving video data.

Evaluation

We propose two types of evaluations for this challenge:

  • Offline (direct evaluation): Use a labeled test set to measure precision/recall for the ODP categories.
  • Online (indirect): Allow users a browse interface for your classifiers and measure how easily they can find some target concepts (e.g., find a basketball scoring scene). Note that the errors of the classifier can be compensated here since a video can appear in multiple categories, so one could conceive of training for different loss functions here.

The ideal target in this case would match the optimal score for human agreement on the dataset.  If 5 raters categorize each video and we have agreement in 92% of the cases, we expect the automatic classifier to hit the same agreement rate.

Feel free to correspond with the challenge authors via the comments form below.

For private correspondence, consult the About page for contact details.

5 Comments on “Google Challenge 2010: Robust, As-Accurate-As-Human Genre Classification for Video”

  1. #1 Asynchron » Multimedia Grand Challenge 2009
    on Feb 11th, 2009 at 7:10 pm

    [...] Robust, As-Accurate-As-Human Genre Classification for Video A notion of browsing collections is naturally associated with videos. Having videos classified into [...]

  2. #2 Telecom ParisTech Multimedia Adaptation Team » Can we rise up the Multimedia Grand Challenge?
    on Feb 24th, 2009 at 5:23 am

    [...] Google Challenge: Robust, As-Accurate-As-Human Genre Classification for Video [...]

  3. #3 Accepted Challenge Responses – Multimedia Grand Challenge 2009
    on Sep 3rd, 2009 at 1:40 pm

    [...] Responses to Google Challenge: Video Genre Classification [...]

  4. #4 Janko Calic » ACM MM 2009
    on Sep 8th, 2009 at 10:22 am

    [...] will be presented at the Content Analysis and HCM Session. In addition, we have been accepted to compete in the Multimedia Grand Challenge, in the theme Tangible MM Products, set by CeDeWe, a major photo printing service provider in [...]

  5. #5 kevin
    on Apr 1st, 2010 at 9:17 am

    In “If the dataset consists of web videos, sharing a list of links and corresponding labels would be ideal for researchers to compare notes.”, does “a list if links” mean the links of the web videos and is the “corresponding labels” choosen by yourself or from the open ODP?

Leave a Comment