RUTGERS TREC-7 INTERACTIVE SEARCHING STUDY

Searcher #___________
Condition ___________
<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY PRE-SEARCH QUESTIONNAIRE

What high school/college/university degrees/diplomas do you have (or expect to have)?

____________________________________
degree major date

_____________________________________
degree major date

What is your occupation?

___________________________________

What is your gender?

Female

Male

What is your age? ________ years

Have you participated in previous TREC Searching Studies?

Yes

Overall, for how many years have you been doing online searching? _________ years

Please circle the number closest to your experience.......

How much experience have you had...	No experience		Some experience		A great deal of experience
1. using a point-and-click interface (e.g., Macintosh, Windows)	1	2	3	4	5
2. searching on computerized library catalogs either locally (e.g., your library) or remotely (e.g., Library of Congress)	1	2	3	4	5
3.searching on CD ROM systems (e.g., Encarta, Grolier, Infotrac)	1	2	3	4	5
4. searching on commercial online systems (e.g., BRS Afterdark, Dialog, Lexis-Nexis)	1	2	3	4	5
5. searching on world wide web search services (e.g., Alta Vista, Excite, Yahoo, HotBot, WebCrawler)	1	2	3	4	5
6. searching on other systems, please specify the system: ___________________________________________	1	2	3	4	5

Please circle the number that is closest to your searching behavior....

Never Once or twice a year Once or twice a month Once or twice a week Once or twice a day

7. How often do you conduct a search on any kind of system? 1 2 3 4 5

Circle the number that indicates to what extent you agree with the following statement.......

Strongly Disagree Disagree Neutral Agree Strongly Agree

8. I enjoy carrying out information searches. 1 2 3 4 5

Searcher #___________
Condition ___________
Topic # _____________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY POST-SEARCH QUESTIONNAIRE

Please answer the following questions, as they relate to this specific topic.

Not at all Somewhat Extremely

1. Are you familiar with this topic? 1 2 3 4 5

2. Was it easy to get started on this search? 1 2 3 4 5

3. Was it easy to do the search on this topic? 1 2 3 4 5

4. Are you satisfied with your search results? 1 2 3 4 5

5. Are you confident that you identified all the different instances for this topic? 1 2 3 4 5

6. Did you have enough time to do an effective search? 1 2 3 4 5

Searcher #___________
Condition ___________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY POST-SYSTEM QUESTIONNAIRE

Now, please consider the searching experience that you just had.

	Not at all		Somewhat		Extremely
1. How easy was it to learn to use this information system?	1	2	3	4	5
2. How easy was it to use this information system?	1	2	3	4	5
3. How well did you understand how to use the information system?	1	2	3	4	5

Please write down any other comments that you have about your searching experience with this information retrieval system here. Thank you!

Searcher #___________
Condition ___________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY TUTORIAL WORKSHEET

Searcher #___________
Condition ___________
Topic # _____________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY EXPERIMENTER NOTE FORM

Make a note of questions or comments that the participants have. You may also make a note of any problems you notice them having or anything that seems interesting to you about their searching behavior.

Searcher #___________
Condition ___________
Topic # _____________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY SEARCHER WORKSHEET (Searchers may use this form to keep track of the instances that they identify or they may use an equivalent online aid)

Topic title

As you identify instances, please keep track of them by writing down a word or short phrase to identify each one.

1. ________________________________________________________________________

2. ________________________________________________________________________

3. ________________________________________________________________________

4. ________________________________________________________________________

5. ________________________________________________________________________

6. ________________________________________________________________________

7. ________________________________________________________________________

8. ________________________________________________________________________

9. ________________________________________________________________________

10. ________________________________________________________________________

11. ________________________________________________________________________

12. ________________________________________________________________________

13. ________________________________________________________________________

14. ________________________________________________________________________

15. ________________________________________________________________________

Searcher #___________
Condition ___________

<SITE NAME/SYSTEM NAME> TREC-7 INTERACTIVE SEARCHING STUDY EXIT QUESTIONNAIRE (To minimize the potential for ordering effects, it is suggested that experimental systems be assigned neutral names)

Please consider the entire search experience that you just had when you respond to the following questions.

Not at all Somewhat Completely

1. To what extent did you understand the nature of the searching task? 1 2 3 4 5

2. To what extent did you find this task similar to other searching tasks that you typically perform? 1 2 3 4 5

3. How different did you find the systems from one another? 1 2 3 4 5

4. Please rank the two systems in order of how easy they were to learn to use. Place a "1" next to the easier system to learn and a "2" beside the more difficult system to learn.

APUS SYSTEM^*___________

PAVO SYSTEM ___________

5. Please rank the two systems in order of how easy they were to use. Place a "1" next to the easier system to use and a "2" beside the more difficult system to use.

APUS SYSTEM ___________

PAVO SYSTEM ___________

6. Please rank the two systems in order of which system you liked the best. Place a "1" next to the system that you liked the best and a "2" beside the system you liked the least.

APUS SYSTEM ___________

PAVO SYSTEM ___________

7. What did you like about each of the systems?

8. What did you dislike about each of the systems?

9. Please list any other comments that you have about your overall search experience on the back of this page. THANK YOU!!!!

	Never	Once or twice a year	Once or twice a month	Once or twice a week	Once or twice a day
7. How often do you conduct a search on any kind of system?	1	2	3	4	5

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
8. I enjoy carrying out information searches.	1	2	3	4	5

	Not at all		Somewhat		Completely
1. To what extent did you understand the nature of the searching task?	1	2	3	4	5
2. To what extent did you find this task similar to other searching tasks that you typically perform?	1	2	3	4	5
3. How different did you find the systems from one another?	1	2	3	4	5