|
|
Dushay, N. & Hillmann, D.I. (2003). Analyzing metadata for
effective use and re-use. 2003 NSDL All Projects Annual Meeting,
Washington, D.C., Oct. 12-14, 2003.
This is a poster at 2003 NSDL All Projects Annual Meeting.
The authors present the strengths of their proposed data visualization
software can improve efficiency and thoroughness of large set of
havested metadata evaluation. Without any empirical study, the approach
is supposed to enable metadata assessment in terms of correctness;
sufficiency, and error rate.
Fraser, B. & Gluck, M. (1999). Usability of geospatial metadata
or space-time matters. Bulletin of the American Society for Information
Science, 25(6): 24-28.
Having acknowledged that "little is known about how end users
actually employ metadata to evaluate relevance of geospatial information
objects.", the authors conducted a two phrase usability test to
compare three well-developed U.S. metadata format standards for
spatial data (i.e. FGDC, GILS, and MARC). The users included GIS
graduate students/faculty, and library professionals. Whereas
audio/video-taped and think aloud, and observation note techniques
are used in collecting data in experiment search session, focus
group is conducted after search session to obtain more quantitative
data. Each participant makes relevance judgement matching metadata
records to search topics. The evaluation findings show that many
participants thought the matching work was difficult due to insufficient
information in the metadata.
Hickey, T.B., O'Neill, E.T., & Toves, J. (2002). Experiments
with IFLA Functional Requirements for Bibliographic Records (FRBR).
D-Lib Magazine, 8(9).
The authors from OCLC conducted content analysis to examine how
good the developed algorithm is in terms of grouping OCLC WorldCat
records to IFLA' 4 Functional Requirements for Bibliographic Records
(FRBR). The evaluation focused one of the three groups of entities
in FRBR and did not involve the four generic user search tasks,
namely find, identify, select and obtain.
Liddy, E.D., Allen, E., & Harwell, S. et al. (2002). Automatic
metadata generation & evaluation. ACM SIGIR'02, Tampere,
Finland. Aug. 11-15, 2002, 401-402.
To examine whether metadata elements can be automatically generated
from the content of educational resources, and correctly assigned
to mathematics and science educational materials, the authors
conducted a preliminary experiment. Twenty-six education students,
teachers, and professionals were asked to evaluate 390 records
of auto-generated and human-generated, and then rate the degree
of expectation meet and satisfaction. The results demonstrate
that there is minimal difference between machine and expert generated
metadata in the participants' perspective.
Liddy, E.D., Allen, E.E., & Finneran, C.M. et al. (2003).
MetaTest: evaluation of metadata from generation to use. 2003
Joint Conference of Digital Libraries, Houston, Texas, May 1-2,
2003, 398.
In the conference poster, the authors provide a brief outline
of a series of experiments to compare the effectiveness of automatically
generated metadata and manually generated metadata in terms of
helping pre-service and in-service STEM teachers to find relevant
information. The eye-tracking technique is used to determine the
most informative metadata parts and the information scan path
a search follows to find information.
Moen, W.E. & Stewart, E. (1998). Assessing metadata quality:
Findings and methodological considerations from an evaluation of
the U.S. Government Information Locator Service (GILS). Proceedings.
IEEE International Forum on Research and Technology. Advances in
Digital Libraries, Santa Barbara, CA, USA, April, 22-24, 1998,
246-255.
Bearing the purpose of demonstrating the utility of U.S. Government
Information Locator Service (GILS), a metadata schema for improving
Internet access to publicly available government resources, the
authors conducted evaluation to assess the metadata quality. In
specific, they employed "qualitative and quantitative content
analysis techniques" to assess selected metadata records. The
analysis was carried out in two stages, within which sample records
are extracted from "the known pool of GILS implementations". Whereas
the sample records in the 1st stage were used to identify and
develop evaluation criteria, another set of sample records was
employed to apply these criteria. According the authors, about
50 measures were developed within four categories of criteria,
namely completeness, profile (i.e. aggregation and granulation
of various records and elements in the records), accuracy (i.e.
errors in format and spelling), and serviceability (including
factors affecting convenience, aesthetics, readability, and relevance
judgement, etc.). Unfortunately, the evaluation design did not
include any user studies.
Quam, E. (2001). Informing and evaluating a metadata initiative:
Usability and metadata studies in Minnesota's Foundations Project.
Government Information Quarterly, 18: 181-194.
Aiming to improve access to state environmental and natural resource
information, the author conducts a series of studies, namely needs
assessment, Web site interface and metadata scheme usability test.
Whereas the 1st study is implemented by employing three surveys
to examine citizens' information needs (i.e. what people want
to know), the 2nd and 3rd studies are carried out in a lab setting.
In particular, the metadata evaluation is comprised of three separate
passes divided by little, half and full completeness of applying
metadata to documents. The testing results show improved effectiveness
over the passes.
|
|
