AAA Interface Tool

We provide a single integrated interface tool which takes an AAA factory through all the stages of its life: from design, to simulation, to operation. The integrated interface tool uses 3D rendering and modeling to assist the user in developing virtual factories, and uses monitoring tools to watch the agents in real factories as they calibrate themselves and operate.

Diagram of interface tool

In the figure above, the interface tool is shown in blue. It is the user interface to the factory, whether the factory is real or virtual. During programming and monitoring of a real factory, the agents are local with a factory-wide connectivity to the interface tool as well as high-speed inter-agent connectivity. During the design phase, agents interact over the Internet. Each AAA agent must contain a servo layer dealing with sensing,control, and actuation; a data management layer which maintains static and dynamic agent information, and a communication layer or "agent head" which can publish data for consumption by others, and subscribe to data published by others.

It is very important to realize that the interface tool has no a priori information about the nature of the agents. This information must be provided by the agents themselves.

Why an integrated interface tool?

A common usage of simulation and virtual environments in factory design is merely as an initial step, to reduce the cost of implementing the corresponding physical system. Once the physical system is built, the simulation is not used. Often the simulation and the physical system use entirely different development environments, with the only exchange being a blueprint from the simulation used to guide the physical construction.

Offline programming

To truly achieve agility---the ability to rapidly deploy factories to deliver a product to market quickly and to rapidly reconfigure factories to adapt to changing technologies and market needs---simulation and physical implementation need to be more closely coupled and done iteratively.

AAA enables this iterative approach by lowering the barriers between simulation and reality. A user can load specifications of agent behavior from agents themselves into the development environment, thus maximizing the chance of getting faithful simulations .The user can then construct and program virtual factories that perform their tasks. When the factory is assembled, the interface tool can monitor the agents' exploration of their environment, adjusting the virtual representation to precisely match the physical factory. As the physical factory runs, the integrated interface tool monitors its operation and provides a 3D rendering of the working factory.

If the factory has to be reconfigured or reprogrammed due to unforeseen problems or changing market requirements, it is straightforward to retreat back to the safe environment of simulation, since both the virtual and real factories interface with the same integrated interface tool, the same programs run in simulation as on the physical agents (see Agent programming section), and the virtual representation of the factory after exploration is in essentially perfect registration with the real factory (see Agent calibration section). Thus simulation and reality can iterate as needed, allowing solutions to problems and changing requirements to be addressed as much as possible in the world of simulation, where experimentation is cheap and mistakes are not catastrophic.

Loading factory components

A fundamental aspect of AAA agents is that they all maintain and publish descriptions of themselves: how they are to be rendered, integrated into the factory infrastructure, simulated, etc. If a factory designer wants to minimize the likelihood of inaccurate information, that designer can directly access the agent itself rather than any catalog or other mediator. In practice, there is no absolute guarantee that the agent's self-description is accurate, but it is more likely to contain accurate, up-to-date information specific to that agent than is a generic description of an agent of the appropriate class, or even a passive catalog of agent descriptions.

Obtaining models from the Internet

Factory designers can use their World Wide Web (WWW) browser to find a catalog of our prototype agents. Each catalog entry is associated with one particular agent in our laboratory. Clicking on a button next to the catalog entry starts up a helper application which sends an agent "URL'' to the interface tool. The interface uses the agent URL to contact the actual agent, and if it is active puts a reference to that agent in a component palette. The component palette is a list of iconified representations of the components along with buttons which allow the user to reserve a component and insert a representation of that component into the design environment. Designers can also use the component palette to insert "clones'' of components into the design environment, i.e. the remote components will not be reserved for use, but simulated representations of them will be available for design experiments.

Constructing Factories

The interface tool enables factory designers to quickly take the component descriptions they have found and assemble them together into a variety of different factory configurations, snapping them together and pulling them apart as they design the final factory.

Building a virtual minifactory

Each component description contains a specification of how it snaps together with other components. Currently we use a shortcut which takes advantage of the object oriented capabilities of our implementation language, C++, to "hard-code'' this specification.

Simulation and Programming

The interface tool lets a factory designer write and debug simulated agent programs while the factory is being constructed. The factory designer views a 3D rendering of the running simulated factory as a whole, and interacts with individual agents through virtual control panels, using them to observe their state variables and debug the individual agent programs.

Interface tools control panels

JayCurrently, the user specifies an agent's run-time behavior through the use of a program written in Python (a byte-coded, object-oriented programming language). These programs are similar to Java applets in that they instantiate objects with required methods rather than simply being a sequence of commands. The program objects specifically have a bind method which is used to indicate all of the global factory components the agent will use, and a run method which is the actual script that determines the agent run-time behavior. The programs are written so that the same programs developed in simulation will run on the physical agents.

Physical Instantiation

Simulation never matches reality, a fact which often limits the utility of simulation, since much of the effort is expended in the details of making a system work on physical machines well after the simulations are finished. AAA agents address this gap by being able to calibrate themselves and explore their physical environments. As long as the simulated world and the real world are topologically compatible and the real world allows the same physical motions as the simulated agents, programs should be able to run in the newly calibrated environment unchanged.

In the case of a specific example of AAA, minifactory, the same interface tool that was used to program a simulated minifactory initiates the transformation into a real factory and monitors the calibration and exploration process. After exploration, the virtual factory rendered by the interface tool is in essentially perfect correspondence with the real factory. We expect users to then switch back to simulation to test the programs using the registered virtual environment before actually trying them out on the real equipment. Any fatal discrepancies, such as swapping manipulator positions or moving components too close together to allow safe passages for couriers, should be detectable and corrected by the user at this point.

Interface tool calibration video

In the movie clip to the left, one can see two courier agents exploring their environments and locating the manipulator agent. Notice the manipulator agent (along with its bridge) suddenly snap to its correctly calibrated position in the factory.


Once a factory is instantiated, the interface tool should no longer be strictly necessary, since the programs run on each agent without need for any central control or resources. However, the interface tool can still provide a central place for monitoring factory operations in order to keep records, detect problems, and discover means of optimizing the factory.

The primary mechanism for monitoring the factory operations is watching the agents' state and monitoring the products that the agents are manipulating. The interface tool provides a 3D rendering of the whole factory as it performs assembly tasks, as well as providing virtual control panels to monitor less visible elements of an individual agent's state.

Synchronous real and virtual factory movie

In the movie clip to the left, one can see the interface tool rendering following the actual movements of the agents in the factory.