Interface theory of perception

Perception is not a veridical representation of reality but a highly constructive process. It has been argued that perception can be compared to the graphical user interface (GUI) of a computer. The icons on your desktop do not reflect the underlying computer operations – they are icons – nothing more and nothing less. The underlying functions are invisible to the user. And the underlying binary code is invisible to the programmer of the functions, therefore the programmer uses abstract higher-order programming languages¹which oftentimes use natural language. The high-level code is then compiled² into a lower-level language. The programmer would be incapable to deal with the lower-level language (like plain binary code) due to cognitive limitations. In the same way the normal user would be unable to use the computer functions and commands which are signified by the icons. When you move an icon (say a given folder) to the “trash bin” this obviously does not mean that the folder is now in a container. It is a simplified (semiotic)^{I}Semiotics is the study of meaning-making, the study of sign process and meaningful communication. interface which allow the user to operate the system. It has been cogently argued that perception is analogous to such a interface. The object we perceive are not really what they seem to be – our perception simplifies reality (according to evolutionary principles), but perception is not a veridical representation of reality as it really is (“Das Ding an sich“^{II} “Things-in-themselves” would be objects as they are independent of observation. is imperceptible – to use Kantian language).

And we indeed, rightly considering objects of sense as mere appearances, confess thereby that they are based upon a thing in itself, though we know not this thing as it is in itself, but only know its appearances, viz., the way in which our senses are affected by this unknown something.

—Immanuel Kant, Prolegomena, § 32

Here is an example: What follows is the Base64^{III}Base64 is a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation. code for the Necker cube image you can see above. You can open this code in any browser to display it as an image. The browser compiles the image into a geometric black and white figure. However, the underlying code is based64. There are no lines and there is no black and white in the code.
Click here to open the base64 code in a new browser-window.

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAFoCAYAAAB5fq24AAAABmJLR0QA/wD/AP+gvaeTAAAAB3RJTUUH1wUDAzAo8Q+G+QAACXpJREFUeJzt27+KndcVxuFXRoUwsW7Abao0Lq0qVhNIGpMrSOciheIqSMZtQFKbVMF34SokjdQE4VtIylEvLAguDE6hkTUazd+lc863197PAwdmPHD0wazXP2aEEgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC6u7H1A9DGr5P8LsmLrR8EuLabSX6RV/t9luTprt4ULnIryRdJ/pLk9sbPAry/RxEQ9ux1OO4n+XjjZwEGJCCcdtVwPD7M4wDXcDPJJ0k+TfLRxs/CQm4luZfkKMlPp15HSb498fnDjZ4RONsm+/1gV29EW68P779J/pq3f+p4nuRPSX6Z5N+HfzTgEpvu16+w1nXRr6qe59WvqL5J8sOBnwu43BD7FZD1DHF4QMlQ+xWQdQx1eMC1DLlfAZnfkIcHXMnQ+xWQeQ19eMCFWuxXQObT4vCAM7Xar4DMo9XhAW9puV8B6a/l4QFJmu9XQPpqfXiwuCn2KyD9THF4sKip9isgfUx1eLCYKfcrIOOb8vBgEVPvV0DGNfXhweSW2K+AjGeJw4NJLbVfARnHUocHk1lyvwKyvSUPDyax9H4FZDtLHx40Z78RkC04POjLfk8QkMNxeNCX/Z5BQPbP4UFf9nsBAdkfhwd92e8VCMjuOTzoy36vQUB2x+FBX/ZbICDvz+FBX/b7HgSkzuFBX/a7AwJyfQ4P+rLfHRKQq3N40Jf97oGAXM7hQV/2u0cCcj6HB33Z7wEIyLscHvRlvwckIG84POjLfjcgIA4POrPfDa0cEIcHfdnvAFYMiMODvux3ICsFxOFBX/Y7oBUC4vCgL/sd2MwBcXjQl/02MGNAHB70Zb+NzBQQhwd92W9DMwTE4UFf9ttY54A4POjLfifQMSAOD/qy34l0CojDg77sd0IdAuLwoC/7ndjIAXF40Jf9LmDEgDg86Mt+FzJSQBwe9GW/CxohIA4P+rLfhW0ZEIcHfdkvmwTE4UFf9svPDhkQhwd92S/vOERAHB70Zb+ca58BcXjQl/1yqX0ExOFBX/bLle0yIHeS/DHJ75PcPvW175M8SfJdkg+TfLnDP5fD+OzEx3fy6n8wzOVXOXu/wsGZdhmQu0n+cM7Xbif5/PhFf3ePX8xNOLjQCP+QEBjLyyRfRzi4xD4D8niP783h3cmbX2M9S/J0w2dh907+SvLvSf621YPQx74C8ijJV3t6b7bxIG8C8iS+v7N5keTh8cc/bvkg9PHB1g8AQE8CAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQcnPrB6CNk7dyP8mLrR6EvfjsxMd38up7zDwe7ONNBYTL3EryRZI/n/hvN5I82uZxOIC7xy+4kIBwntfhuJ/k442fBRiQgHDaReF4meRfSf5z6Idi7+7kza+xniV5uuGzUHczySdJPk3y0amvvd7vkN/bB0l+On493PhZuL5bSe4lOcqb7+Pr19Hx125t9nTsm/32tsl+/QTCRT9xPE/yOMk3SX448HMBl9t0vwKyLuGAvobYr4CsZ4jDA0qG2q+ArGOowwOuZcj9Csj8hjw84EqG3q+AzGvowwMu1GK/AjKfFocHnKnVfgVkHq0OD3hLy/0KSH8tDw9I0ny/AtJX68ODxU2xXwHpZ4rDg0VNtV8B6WOqw4PFTLlfARnflIcHi5h6vwIyrqkPDya3xH4FZDxLHB5Maqn9Csg4ljo8mMyS+xWQ7S15eDCJpfcrINtZ+vCgOfuNgGzB4UFf9nuCgByOw4O+7PcMArJ/Dg/6st8LCMj+ODzoy36vQEB2z+FBX/Z7DQKyOw4P+rLfAgF5fw4P+rLf9yAgdQ4P+rLfHRCQ63N40Jf97pCAXJ3Dg77sdw8E5HIOD/qy3z0SkPM5POjLfg9AQN7l8KAv+z0gAXnD4UFf9rsBAXF40Jn9bmjlgDg86Mt+B7BiQBwe9GW/A1kpIA4P+rLfAa0QEIcHfdnvwGYOiMODvuy3gRkD4vCgL/ttZKaAODzoy34bmiEgDg/6st/GOgfE4UFf9juBjgFxeNCX/U6kU0AcHvRlvxPqEBCHB33Z78RGDojDg77sdwEjBsThQV/2u5CRAuLwoC/7XdAIAXF40Jf9LmzLgDg86Mt+2SQgDg/6sl9+dsiAODzoy355xyEC4vCgL/vlXPsMiMODvuyXg3qQ5Kfj1z+THJ34/PXrKMm9vDpOYBz2y7Xt6yeQ35z6/PskT5J8l+TDJF/u6c8Fau6f+Pj0fv3EwZkO9Zfot5N8fvwCehAOLjTCPyQExvIyydcRDi5xY4fvdTfJb/Pqp43/Jflxh+8NHMbtJP9I8u3WDwIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7N//Aepc59H5ES0uAAAAAElFTkSuQmCC

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAFoCAYAAAB5fq24AAAABmJLR0QA/wD/AP+gvaeTAAAAB3RJTUUH1wUDAzAo8Q+G+QAACXpJREFUeJzt27+KndcVxuFXRoUwsW7Abao0Lq0qVhNIGpMrSOciheIqSMZtQFKbVMF34SokjdQE4VtIylEvLAguDE6hkTUazd+lc863197PAwdmPHD0wazXP2aEEgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC6u7H1A9DGr5P8LsmLrR8EuLabSX6RV/t9luTprt4ULnIryRdJ/pLk9sbPAry/RxEQ9ux1OO4n+XjjZwEGJCCcdtVwPD7M4wDXcDPJJ0k+TfLRxs/CQm4luZfkKMlPp15HSb498fnDjZ4RONsm+/1gV29EW68P779J/pq3f+p4nuRPSX6Z5N+HfzTgEpvu16+w1nXRr6qe59WvqL5J8sOBnwu43BD7FZD1DHF4QMlQ+xWQdQx1eMC1DLlfAZnfkIcHXMnQ+xWQeQ19eMCFWuxXQObT4vCAM7Xar4DMo9XhAW9puV8B6a/l4QFJmu9XQPpqfXiwuCn2KyD9THF4sKip9isgfUx1eLCYKfcrIOOb8vBgEVPvV0DGNfXhweSW2K+AjGeJw4NJLbVfARnHUocHk1lyvwKyvSUPDyax9H4FZDtLHx40Z78RkC04POjLfk8QkMNxeNCX/Z5BQPbP4UFf9nsBAdkfhwd92e8VCMjuOTzoy36vQUB2x+FBX/ZbICDvz+FBX/b7HgSkzuFBX/a7AwJyfQ4P+rLfHRKQq3N40Jf97oGAXM7hQV/2u0cCcj6HB33Z7wEIyLscHvRlvwckIG84POjLfjcgIA4POrPfDa0cEIcHfdnvAFYMiMODvux3ICsFxOFBX/Y7oBUC4vCgL/sd2MwBcXjQl/02MGNAHB70Zb+NzBQQhwd92W9DMwTE4UFf9ttY54A4POjLfifQMSAOD/qy34l0CojDg77sd0IdAuLwoC/7ndjIAXF40Jf9LmDEgDg86Mt+FzJSQBwe9GW/CxohIA4P+rLfhW0ZEIcHfdkvmwTE4UFf9svPDhkQhwd92S/vOERAHB70Zb+ca58BcXjQl/1yqX0ExOFBX/bLle0yIHeS/DHJ75PcPvW175M8SfJdkg+TfLnDP5fD+OzEx3fy6n8wzOVXOXu/wsGZdhmQu0n+cM7Xbif5/PhFf3ePX8xNOLjQCP+QEBjLyyRfRzi4xD4D8niP783h3cmbX2M9S/J0w2dh907+SvLvSf621YPQx74C8ijJV3t6b7bxIG8C8iS+v7N5keTh8cc/bvkg9PHB1g8AQE8CAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQIiAAlAgIACUCAkCJgABQcnPrB6CNk7dyP8mLrR6EvfjsxMd38up7zDwe7ONNBYTL3EryRZI/n/hvN5I82uZxOIC7xy+4kIBwntfhuJ/k442fBRiQgHDaReF4meRfSf5z6Idi7+7kza+xniV5uuGzUHczySdJPk3y0amvvd7vkN/bB0l+On493PhZuL5bSe4lOcqb7+Pr19Hx125t9nTsm/32tsl+/QTCRT9xPE/yOMk3SX448HMBl9t0vwKyLuGAvobYr4CsZ4jDA0qG2q+ArGOowwOuZcj9Csj8hjw84EqG3q+AzGvowwMu1GK/AjKfFocHnKnVfgVkHq0OD3hLy/0KSH8tDw9I0ny/AtJX68ODxU2xXwHpZ4rDg0VNtV8B6WOqw4PFTLlfARnflIcHi5h6vwIyrqkPDya3xH4FZDxLHB5Maqn9Csg4ljo8mMyS+xWQ7S15eDCJpfcrINtZ+vCgOfuNgGzB4UFf9nuCgByOw4O+7PcMArJ/Dg/6st8LCMj+ODzoy36vQEB2z+FBX/Z7DQKyOw4P+rLfAgF5fw4P+rLf9yAgdQ4P+rLfHRCQ63N40Jf97pCAXJ3Dg77sdw8E5HIOD/qy3z0SkPM5POjLfg9AQN7l8KAv+z0gAXnD4UFf9rsBAXF40Jn9bmjlgDg86Mt+B7BiQBwe9GW/A1kpIA4P+rLfAa0QEIcHfdnvwGYOiMODvuy3gRkD4vCgL/ttZKaAODzoy34bmiEgDg/6st/GOgfE4UFf9juBjgFxeNCX/U6kU0AcHvRlvxPqEBCHB33Z78RGDojDg77sdwEjBsThQV/2u5CRAuLwoC/7XdAIAXF40Jf9LmzLgDg86Mt+2SQgDg/6sl9+dsiAODzoy355xyEC4vCgL/vlXPsMiMODvuyXg3qQ5Kfj1z+THJ34/PXrKMm9vDpOYBz2y7Xt6yeQ35z6/PskT5J8l+TDJF/u6c8Fau6f+Pj0fv3EwZkO9Zfot5N8fvwCehAOLjTCPyQExvIyydcRDi5xY4fvdTfJb/Pqp43/Jflxh+8NHMbtJP9I8u3WDwIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7N//Aepc59H5ES0uAAAAAElFTkSuQmCC

The same image can also be converted into plain binary code (inverted: 0=white, 1=black).

Click here to open the binary code as a *.txt document in new window

Show/hide publication abstract

“Perception is a product of evolution. our perceptual systems, like our limbs and livers, have been shaped by natural selection. the effects of selection on perception can be studied using evolutionary games and genetic algorithms. to this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. we find that veridical perceptions-strategies tuned to the true structure of the world-are routinely dominated by nonveridical strategies tuned to fitness. veridical perceptions escape extinction only if fitness varies monotonically with truth. thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a pc. just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the invention of space-time theorem) that objective reality has the structure of space-time and objects. an interface serves to guide useful actions, not to resemble truth. indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. for the perceptions of h. sapiens, space-time is the desktop and physical objects are the icons. our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. perception is an adaptive interface.”

Further References

Hoffman, D. D.. (2009). The interface theory of perception: Natural selection drives true perception to swift extinction. In Object Categorization: Computer and Human Vision Perspectives

Plain numerical DOI: 10.1017/CBO9780511635465.009
DOI URL
directSciHub download

Show/hide publication abstract

“We perceive the shapes and material properties of objects quickly and reliably despite the complexity and objective ambiguities of natural images. typical images are highly complex because they consist of many objects embedded in background clutter. moreover, the image features of an object are extremely variable and ambiguous owing to the effects of projection, occlusion, background clutter, and illumination. the very success of everyday vision implies neural mechanisms, yet to be understood, that discount irrelevant information and organize ambiguous or noisy local image features into objects and surfaces. recent work in bayesian theories of visual perception has shown how complexity may be managed and ambiguity resolved through the task-dependent, probabilistic integration of prior object knowledge with image features.”

Hoffman, D. D., Singh, M., & Prakash, C.. (2015). Probing the interface theory of perception: Reply to commentaries. Psychonomic Bulletin and Review

Plain numerical DOI: 10.3758/s13423-015-0931-3
DOI URL
directSciHub download

Show/hide publication abstract

“We propose that selection favors nonveridical perceptions that are tuned to fitness. current textbooks assert, to the contrary, that perception is useful because, in the normal case, it is veridical. intuition, both lay and expert, clearly sides with the textbooks. we thus expected that some commentators would reject our proposal and provide counterarguments that could stimulate a productive debate. we are pleased that several commentators did indeed rise to the occasion and have argued against our proposal. we are also pleased that several others found our proposal worth exploring and have offered ways to test it, develop it, and link it more deeply to the history of ideas in the science and philosophy of perception. to both groups of commentators: thank you. point and counterpoint, backed by data and theory, is the essence of science. we hope that the exchange recorded here will advance the scientific understanding of perception and its evolution. in what follows, we respond to the commentaries in alphabetical order.”

Hoffman, D. D.. (2008). Conscious realism and the mind-body problem. Mind and Matter

Plain numerical DOI: 10.1097/ALN.0b013e3182217167
DOI URL
directSciHub download

Show/hide publication abstract

“Despite substantial efforts by many researchers, we still have no scientific theory of how brain activity can create, or be, con-scious experience. this is troubling, since we have a large body of correlations between brain activity and consciousness, correlations normally assumed to entail that brain activity creates conscious experience. here i explore a solution to the mind-body problem that starts with the converse assumption: these correlations arise because consciousness creates brain activity, and indeed creates all objects and properties of the physical world. to this end, i develop two theses. the multimodal user interface theory of perception states that perceptual experiences do not match or approximate properties of the objective world, but instead provide a simplified, species-specific, user interface to that world. conscious realism states that the objective world consists of conscious agents and their experiences; these can be mathematically modeled and em-pirically explored in the normal scientific manner.”

Hickok, G.. (2015). The interface theory of perception: the future of the science of the mind?. Psychonomic Bulletin and Review

Plain numerical DOI: 10.3758/s13423-015-0930-4
DOI URL
directSciHub download

Show/hide publication abstract

Hoffman, D. D.. (2016). The Interface Theory of Perception. Current Directions in Psychological Science

Plain numerical DOI: 10.1177/0963721416639702
DOI URL
directSciHub download

Show/hide publication abstract

For instance: Wolfram Language, C#, Java, ECMAScript, ActionScript, JavaScript, JScript, F#, Haskell, Lisp, Lua, Oz, Perl, Prolog, Python, Ruby, Smalltalk, Scala, ML, Erlang, etc.pp. [↩]
A compiler is computer software that transforms computer code written in one programming language (the source language) into another programming language (the target language). Compilers are a type of translator that support digital devices, primarily computers. The name compiler is primarily used for programs that translate source code from a high-level programming language to a lower level language (e.g., assembly language, object code, or machine code) to create an executable program. [↩]

Footnotes [ + ]

I.	↥	Semiotics is the study of meaning-making, the study of sign process and meaningful communication.
II.	↥	“Things-in-themselves” would be objects as they are independent of observation.
III.	↥	Base64 is a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation.