Previous: Applications Up: Applications Next: A simulation study: Air Battle

A physical implementation: the Robot Waiter

We are developing an experimental setup in which a robot arm will set a dinner table in various configurations, guided by input from a color camera and controlled by a host computer.

The physical setup includes a dinner table, a supplies table containing kitchenware, a Puma 260 robot arm, a CCD camera, a PC-based color frame grabber, and a Sun 4/260 workstation host computer. In a later phase of this project, we hope to replace the Puma 260 robot arm with a larger Puma 560 model. Figure represents the setup.

The human operator instructs the agent to set the table for dinner, breakfast, etc., specifying by named color which objects to choose from the supplies table (color is not a sufficient feature to recognize objects, as each type of object is available in several colors). The camera is mounted in a fixed position overlooking both tables and the robot. This testbed provides a dynamic, yet controllable, environment in which an agent is placed so as to facilitate empirical studies of its behavior. The places, number, kind, and color of objects is not fixed, and unannounced human intervention in the domain is allowed while the robot is carrying out its task.

We call the agent for this project the Robot Waiter (RW). RW is being developed in accordance with the principles of the GLAIR architecture. Figure schematically presents its structure. The categorizer uses domain constraints to determine what objects are in the visual field. It can also be told to look for a certain object, e.g., a red cup. The sensory memory acts as an attentional register, keeping track of the object that is being manipulated or is to be inspected.

Actions are transmitted from Knowledge level to Perceptuo-Motor level, e.g., look for a red apple and pick it up. Once the sensory memory contains an object matching the object desired, actions involving that object can be understood at the Perceptuo-Motor level. For instance, once a red apple is discovered and recorded in the sensory memory, an action like pick it up is expanded by a PMA into robot arm tasks to reach for the apple, grasp it, and lift it. A PMA is a implementation mechanism for routine activities at an ``unconscious'' level, [Hexmoor et al. 1993a][Hexmoor et al. 1992][Hexmoor \& Nute 1992]. Each task involving a robot motion is subsequently submitted to the path constructor. Some motions may have to be decomposed to visit intermediate points in order to avoid fixtures in the environment. The path constructor generates path segments for each robot motion. Each path segment generated by the path constructor is transmitted to the Sensori-Actuator level for execution.

RW incorporates an embodied model of color perception and color naming, modeled after the physiology of human color perception. This model allows the agent to (1) name colors shown to it, and express a typicality judgment, (2) point out examples of named colors in its environment, and (3) learn new names for colors. This model provides the perceptual grounding for a set of basic color terms [Berlin \& Kay 1969] represented at the Knowledge level. The color domain was chosen as a case study for embodied perception because of the relative abundance of psychological, psychophysical and neurophysiological data in the literature [Lammens 1992]. It is a complex enough domain to allow the usefulness of embodiment for computational models of perception to be demonstrated, yet feasible enough to be implemented in actual autonomous agents.

The main components of the Robot Waiter system are listed below:

• A World model: In the general KRR system, we maintain a conscious world model. This model explicitly represents the agent's knowledge (or beliefs) about the entities in its world. Representations at this level may or may not correspond to representations at the Perceptuo-Motor level, as explained in section . As much as it might be desirable to avoid building internal models of the world, having some modeling capacity is necessary, we believe.

• Specialized knowledge bases: for instance knowledge about action selection, planning, learning, experimentation, and perception.

• A Kinematic/Perceptual model: At the Perceptuo-Motor level, we maintain a model of the simple agent-level physics of the objects of interest to the agent. The kinematic/perceptual model models motor capacities and motor memory that might be implemented in different ways (e.g., purely procedurally or in a network of nodes and weighted links), but we prefer the declarative approach for its ease of interpretation and debugging. It also contains perceptual representations of perceived objects. Representations at this level are embodied and agent-centered (section ).

• Reactive processes: Also at the Perceptuo-Motor level, a number of independent processes monitor perceptual inputs, and control reactive behaviors of the agent. We need a mechanism for arbitrating among various reactive and other processes at the same level, which we have not worked out yet.

• Primitive motor and sensory actions: At the Sensori-Actuator level, a number of these primitive actions are implemented. A primitive motor action might be ``move ahead'', and a primitive sensory action might be ``look at position '' (which in turn may involve primitive motor actions). Sensing as such is not considered a primitive sensory action, and it goes on continuously.

• Reflexes: Also at the Sensori-Actuator level, a number of reflexes are implemented as low-level independent processes that monitor raw sensory data and can control actuators directly, temporarily pre-empting higher level control.

As of this writing, the Robot Waiter project is partially implemented, but not operational yet.