Ph.D. Thesis

---

An Investigation of the Conditions for Effective Data Fusion in Information Retrieval

Abstract

Effective automation of the information retrieval task has long been an active area of research, leading to sophisticated retrieval models. With many IR schemes available, researchers have begun to investigate the benefits of combining the results of different IR schemes to improve performance, a process called data fusion. There are many successful data fusion experiments reported in IR literature, but there are also experiments in which the same fusion rules did not work. What is needed is a theory to tell a priori when one should use data fusion methods.

In this thesis we propose two conditions for effective data fusion: (1) The condition of efficacy, which states that fusion of two IR schemes with comparable performance tends to be effective; (2) The condition of dissimilarity, which states that fusion of two dissimilar IR schemes tends to be effective.

I use the IR systems participating in the TREC 4 routing task for training a model to predict the effectiveness of data fusion and the IR systems participating in the TREC 5 routing task to test that model. The model asks, when will fusion perform better than an oracle who uses the best scheme for each pair? I apply various statistical techniques to fit the model to the training data and use the receiver operating characteristic curve of signal detection theory to represent the power of the resulting models. Two predictive variables have been identified which predict the sign of the effectiveness of fusion of two IR schemes: (1) the ratio of precisions of the two schemes and (2) the normalized dissimilarity of the two schemes.

After training, the prediction methods predict the sign of the effectiveness of data fusion between schemes in the testing set much better than chance. For example, applying the trained models to the testing data set, when the models can predict correctly about 70% of the positive cases, they only incorrectly predict about 30% of the negative cases. The results of the experiments support the two proposed conditions for effective data fusion.