A workshop (W3) affiliated with the 22nd Conference on Artificial Intelligence (AAAI-2007)
July 22-23, 2007, Vancouver, British Columbia, Canada
Cognitive architectures form an integral part of robots and agents. Architectures structure and organize the knowledge used by the agents to select actions in dynamic environments, plan and solve problems, learn, and coordinate with others. Architectures serve to integrate general capabilities expected of an intelligent agent (e.g. planning and learning), to implement and test theories about agent cognition, and to explore domain-independent mechanisms for intelligence.
As AI research has improved in formal and empirical rigor, traditional evaluation methodologies for architectures have sometimes proved insufficient. Formal analysis has often proved elusive; we seem to be missing the notation required for proving properties of architectures. Experiments which demonstrate generality are notoriously expensive to perform, and are not sufficiently informative. And at a high-level, evaluation is difficult because the criteria are not well defined: Is it generality? Ease of programmability? Compatibility with data from biology and psychology? There are no established evaluation methodologies and only a handful established evaluation criteria.
Recognizing that scientific progress depends on the ability to conduct informative evaluation (by experiment or formal analysis), this workshop will address the methodologies needed for evaluating architectures. The focus is on methodology, rather than specific architectures. The workshop has two goals: To promote discussion and to propose evaluation criteria that will be accepted by the research community as recognized evaluation guidelines. We intend to publish the results in a special issue of an international journal and to archive presentation slides and explanatory material on an active web site.
The workshop will be composed of invited and contributed
talks on evaluation methodologies, interleaved with panels, and moderated discussions.
We seek submission of extended abstracts (2 pages) and short position papers (4 pages) that discuss evaluation methodologies for architectures. See the call for papers for submission instructions. Submission deadline is April 15, 2007.