I was Principal Investigator for this 2016-7 multidisciplinary AHRC Research Network about action-based theories of neural and artificial information processing. Focussing on my collaborator Andrew Glennerster‘s own theory, we investigated the possibility that human 3D vision is achieved without the need for representation in 3D coordinate frames.

My collaboration with Andrew is ongoing, including work on a joint article that draws on the results of the project. A draft is available here. Comments very welcome. There’s also a post about the project and the paper at the Brains blog.

Andrew’s work in cognitive neuroscience pursues the idea that perception, thought and action have more in common than traditionally assumed, as they are all explicable in terms of brain mechanisms which have long been taken to underlie the control of action (since Marr, ‘A theory of cerebellar cortex’, Journal of Physiology 1969 and Albus, ‘A theory of cerebellar function’, Mathematical Biosciences 1971). This idea has far-reaching consequences not only for our understanding of the brain, and for attempts to recreate brain functions in artificial systems, but also for our understanding of human knowledge and experience.

The project

Our Research Network of philosophers, neuroscientists, roboticists and computer vision experts focussed on an application of Andrew’s idea to 3D vision. We investigated the scope and limits of Andrew’s alternative to a standard view of spatial vision and action. According to the standard view, the brain reconstructs a representation of one’s environment in 3D coordinate frames, and performs computationally expensive transformations between different such frames. According to the alternative, spatial vision and action are instead explained by an action-based mechanism.

We found that focussing on this specific proposal facilitated a genuinely multidisciplinary discussion, obviating the risk of different disciplines’ talking past one another by providing a shared target. But we also explored points of contact and contrast between Andrew’s hypothesis and other action-based theories of perception in the philosophical and scientific literature. We think the confluence of multidisciplinary approaches helps us to make progress with broader questions about the relationship between cognition, perception and action, as they arise in philosophy, neuroscience and the theory and practice of artificial intelligence.

Meetings

The action-based alternative to 3D coordinate-frame representation was the main topic of our video meetings throughout 2016, and of our workshop at St John’s College, Oxford, in January 2017. This workshop brought together experts from diverse disciplines for a focussed multidisciplinary discussion across three days, testing the action-based hypothesis by assessing its philosophical, computational and neuroscientific consequences. You can see the final discussion of the workshop here. As well as discussions led by Andrew and me, the workshop featured talks by:

Bence Nanay (Philosophy, University of Antwerp / University of Cambridge)

Michael Milford (Computer vision, Queensland University of Technology)

Gunnar Atli Sigurdsson (Computer vision, Carnegie Mellon University)

Jenny Read (Neuroscience, Newcastle University)

Mike Gilbert (Neuroscience, University of Birmingham)

Here is Michael Milford’s blog about his trip from Australia for the workshop.

Wiki

Further discussion takes place on our wiki, which is a good place to find out more about the project.

Collaboration

Meetings of the Research Network included collaboration with various leading academic and commercial experts in robotics and computer vision from around the world. In October 2016 we also visited Tony Prescott‘s robotics lab in Sheffield, for a Research Exchange Day organised by the AHRC Science in Culture project. Here is Andrew controlling a robot: