Abstract

Evaluation models are being developed for assessing artificial intelligence (AI) systems in terms of similar performance by groups of people. Natural language understanding and vision systems are the areas of concentration. In simplest terms, the goal is to norm a given natural language system's performance on a sample of people. The specific program under study is a natural language query system, IRUS--an interface between the user and the information desired. IRUS is designed to serve as a general purpose interface to a broad range of databases and expert systems. A pilot study is discussed, which was conducted with early elementary school and preschool students to determine the appropriate language understanding level at which to administer the IRUS test. In the vision area, common measures of visual tasks are being analyzed in terms of their appropriateness to the vision system. This is the inverse of the language exploration that began with the tasks and created the measures. A review by the vision community of approaches they would use to compare human and machine vision will determine if looking for consistent benchmarks is a feasible approach. (SLD)

Citation

Baker, E.L. Directly Comparing Computer and Human Performance in Language Understanding and Visual Reasoning. Retrieved August 13, 2024 from https://www.learntechlib.org/p/141405/.

This record was imported from ERIC on March 21, 2014. [Original Record]

ERIC is sponsored by the Institute of Education Sciences (IES) of the U.S. Department of Education.

Copyright for this record is held by the content creator. For more details see ERIC's copyright policy.

Keywords

• artificial intelligence
• Comparative Analysis
• Computer Assisted Testing
• computer science
• Computer Software
• EVALUATION METHODS
• Language Processing
• Man Machine Systems
• Visual Perception