BYU

Abstract by Xuan Cao

Personal Infomation


Presenter's Name

Xuan Cao

Degree Level

Masters

Abstract Infomation


Department

Computer Science

Faculty Advisor

Mike Goodrich

Title

A Sample-Based Approach for Measuring Autonomous Agents Competency: Preliminary Results

Abstract

Autonomous agents are designed to achieve goals, whatever the goals are. The competency of an autonomous agent tells how likely the agent will achieve its goal given limited time. We report a sample-based method to measure competency, as well as some preliminary results derived from a basic maze environment where an agent follows the policy computed by the Value Iteration algorithm to reach a specific “goal” position from random starting positions. We ran multiple simulations for the agent and plotted curves representing the relationships between success rate and time limitation, which is referred to as “competency curves”. We applied various sigmoid curves to fit the plot and found the 3-parameter Gompertz curve was the best. We explored how the estimated parameters of Gompertz curves vary with the starting positions of the agent. These preliminary results help to build intuition about competency and enhance the basis of competency measurement for more complicated agents.