CeWe Challenge: The Next Generation of Tangible Multimedia Products

Thematic Photo Story Generation from Personal Photo Collections

With the advent of digital photography, the number of photos taken has increased tremendously. While only recently, in the analogue days, a small number of films documented a 2-weeks holiday, we are nowadays taking and storing hundreds or even thousands of digital photos. This capture rate has an enormous impact on the way users deal with their photographs. Often they are just overwhelmed with the masses of photos and defy carefully organizing and selecting them. Users may want a selection that best represents the event to browse and share with family and friends. Manually creating such a selection requires much time and effort. At the end, the precious memories reside on hard disks and are not shared with others or made into prints or other products such as calendars or photo books.

The open issue is how to help the user determine a meaningful subset of photos out of a collection, which best summarizes and represents the specific event. This is still not satisfactory solved after years of research in multimedia analysis and retrieval. However, such methods could ease the process of designing products and services from personal media significantly, and therefore attract more users to order such products from photo finishing companies like CeWe Color.

The multimedia challenge is to take realistic photo sets of users as a basis to (semi-) automatically determine those that best summarize the underlying event such as a 2-weeks holiday. This can also incorporate video snippets often taken with digital still cameras for which a suitable representation for a printed product has to be developed (e.g., extraction of suitable key frames or representative fraction). For the media selection, the system should take into account the target use of the selection, which should be oriented at commercial print products such as calendars, collages, posters or photo books. Additionally, the process could incorporate the exploitation and addition of shared media from social community platforms to augment the personal collection. The solution should not only consist of an approach for the selection but could be embedded in an authoring system the user in the loop.

Metrics/Evaluation

The primary measure for the quality of the approach will be the user’s satisfaction with the summarization result and process. Following the assumption that a user can only evaluate the summarization quality of his or her own photos, researchers should work with the users themselves to provide their own photos as data sets and evaluate the results. One aspect of this evaluation could be a questionnaire. For the evaluation, an exemplary version of a questionnaire with guideline questions will be provided and posted on the Multimedia Grand Challenge website.

The evaluation should cover qualitative questions like:

• How well does the summary reflect the personal memory of the event?
• How much is the user satisfied with the selection according to different criteria such as photo quality or presentation quality?
• How much is the user satisfied with the overall, (semi-) automatic design process? Is it too complex? Does it significantly ease the authoring process? Does it lead to results even better than with manual authoring and selection?

Additionally, several quantitative measurements can be taken by observing the user when using the system. This can be for example:

• Number of photos in the summary in relation to the costs of the product
• Number of clicks / time effort from photo selection to achieved summary
• Automatically selected photo set compared to those that would have been manually selected set by the user

In addition, the evaluation may involve:

• Performance efficiency of the underlying algorithm.
• Commercial potential, that means does the presented solution lead to an increase in sales of related digital print products such as photo books or calendars. Obviously, this cannot directly be measured, but the presented solution should have a strong focus on potential commercial exploitation and realistic estimations for this should be made.

Data set

For the evaluation, the data set has to be chosen and provided with the challenge by the participants of the challenge. Besides data sets for training and statistical evaluation and performance evaluation, we expect the researchers to use representative personal photo collections of at least 5 users. Additional metadata can be attached to this photo collection, but these have to be realistic, that means they might have been created by standard photo management tools (e.g. descriptions, tags, Exif header) or current state of the art metadata extraction.

In the evaluation, the datasets should make sure to include the following types of events:

• Birthday: Short time event (1-2 days), at least 100 photos, more than 5 persons on the different photos of the collections that reoccur on the photos of the collection.
• Vacation: A vacation of at least ten days documented by 300 photos from different places and locations. Additional metadata can be an associated such as a GPS-Track or location information attached to the photos.
• Yearbook: Should consist of at least 1000 photos of different types of events (birthday, vacation, family, fun, …) over a period of 12 months.

About CeWe

CeWe Color is the Number One services partner for first-class trade brands on the European photographic market. CeWe supplies both stores and Internet retailers (e-commerce) with photographic products.