Course description

The course explores the requirements, mathematical models, design choices and the evaluation of Information Retrieval systems (IRS). IR systems refer to automatic (computer-based, such as Web search engines) rather than "manual" systems.

Although not required, basic understanding of maths (simple probabilities, statistics, ...) and computer concepts is recommended. It is also recommended that students taking this course have taken, are taking or will be taking the following related courses:

• Human Information Behavior - for an introduction to the theory, study and practice of human interaction with information.
• Principles of Searching - to better understand what functionality is expected of IR systems.
• Organizing Information - to better understand the issues that need to be considered when designing an information system and approches taken before the time of the computers.
• Digital Libraries - for studying in detail one particular type of Information Retrieval systems.

Course objectives

• the genesis and variety of information retrieval situations;
• the variety of information retrieval models and techniques;
• design principles for information retrieval systems;
• methods for implementing information retrieval systems;
• characteristics of operational and experimental information retrieval systems;
• methods and principles for the evaluation of information retrieval systems.

Class conduct

Each meeting day, there will be a lecture on the scheduled topic. Students are expected to read the relevant chapters or sections from the recommended books before the class and to participate in discussions on that topic. The textbooks cover the accepted current wisdom in IR research and practice. In addition, students will in turn be assigned extra-readings, typically a classic or a new and promising or controversial article; in each class the student(s) with such assignments will give a presentation and submit a report. Other students will be assigned to critically discuss the presentations. The presenters have the option to adjust their report based on feedback received during the presentation, and to re-submit within 48 hours. Students will get these assignments in aphabetic order; however, you can volunteer to go early if you have a special interest in a certain topic. Websites. All the lecture notes will be available online. The course websites will contain links to students websites. Students are expected to maintain websites for the course on their eden accounts, in a subfolder of public_html called 551.They will use the websites to publish their presentations and reports, and to upload their homeworks.

Grading

Students will be graded based on reports, presentations, term project, homeworks and class participation. The final grade will be a weighted average of the partial grades. The weights will be specified if different from the deafults: 1 for a report, 1 for a presentation, 4 for the final project (including the presentation). The report and presentation of a paper (or topic) should make clear: (1) what problem the paper addresses; (2) what relation it has to prior cited literature and to the current topic discussed in class; (3) what idea it proposes to solve or improve the problem; (4) what was done to implement that idea; (5) what results were found and how well the rseults were interpreted, and (6) what suggestions were made for further work. The report should have a clear and logical structure of the topic and of the argument and an appropriate title or indication of the topic (it can help focus and structure the argument), and to display critical analysis. In addition, the presentation should make good use of visual aids to help the listeners understand the topic and the argument. Project work - is expected to display: Good understanding of the topic chosen and of the IR context Creativity Substantial work Below are suggestions of possible projects, but you can propose your own if you have a particular interest. Students are expected to choose a project soon after mid-term, to discuss their progress with the instructor, and to give a presentation at the end of the course. Class participation - apart from active, thoughtful and creative participation in class discussions, critical discussion of presentations and most homeworks are seens as class participation. Although not graded as such, class participation can determine the adjustment of the final grade by one position. Graded homeworks may be assigned occasionally in the form of exercises designed to re-inforce learning.

Grades

• A - Outstanding achievement. Student performance demonstrates full command of the course materials and evidences a high level of originality and/or creativity that surpasses course expectations.
• B+ - Very good work. Student performance demonstrates above average comprehension of the course materials and exceeds course expectations on all tasks as defined in the course syllabus.
• B - Good work. Student performance meets designated course expectations, demonstrates understanding of the course materials and performs at an acceptable level.
• C+ - Marginal work. Student performance demonstrates incomplete understanding of course materials.
• C - Poor work; not enough effort shown.
• F - Fail.
• IN - Incomplete.

The final project

• Do a literature review on an IR-related topic.
  Examples:
  • Semantic Web
  • Digital libraries
  • Agents in IR
  • Query-expansion techniques
  • Relevance feedback
  • Information extraction
  • Information filtering
  • Recommender systems
  • Personalization
  • Machine learning in IR
  • Artificial Intelligence approaches to IR
  • Multimedia / image / video / music retrieval
  Required: a 10-15-page report. Explain the topic, its structure, importance, context, foreseeable applications or evolution. Critical analysis is essential.

• Conduct a user experiment on a research problem and analyze the results.
  Examples:
  • Compare the effectiveness of two systems or user interfaces for a certain task
  • Test the usability of a system
  Required: a report discussing the research problem and or the purpose of the experiment, the experimental design and setting, the results, conclusions and implications or recommendations for designing IR systems.

  For a realistic experiments that involves 6-16 sujects, the use of questionnaires / interviews and/or the statistical analysis of the experimental results, teamwork is not only acceptable, but encouraged. On the other hand, suitable for individual students would be, for example, to compare two Web search engines based on functionality ("what we expect from an IR system") and the support that their user interfaces offers. Note that the functionality of commercial system is not well / fully documented (order and weighting of query terms, for example), so an informed guess, based on exploration such as observing the output for various inputs, is necessary.

• Give a detailed description and critique of some operational information retrieval system.
  Discuss functionality, usability advantages and diadvantages over similar systems. Draw conclusions and formulate recommendations with regard to the use of the system. Examples:
  • AntWorld
  • Onix
  • Mapuccino
  • Inquery, from CIIR, University of Massachusetts, Amherst
  • MG
    I.H. Witten and A. Moffat and T.C. Bell, “Managing Gigabytes: Compressing and Indexing Documents and Images”, 2^nd ed, 1999. Other bibliography indicated in the MG webpage.
  • Lemur, at Carnegie-Mellon University
  • Cheshire, at UC Berkeley
  • Okapi
    More info on the Okapi project at City University.

  • Lucene

  Required: Cover the installation of the system, the usefullnes and usability of the system in a range of tasks.

• A paper discussing changes taking place (or likely to occur) in operational information retrieval systems within the next five years, with specific reference to factors leading to such changes, and their likely effects.

  Required: Base your discussion on your imagination, on your perceived requirements for the IR systems of the future, on the existing systems, and on the evolution of such systems in the past.

• Constructing and documenting (a significant part of) an information retrieval system, or adding new modules to an existing system, and evaluating it by demonstration (this could be an individual or a group project for students with programming skills).

Please read the University's Policy on Academic Integrity for Undergraduate and Graduate Students.