Spatio-temporal blended identities

by Alvaro Cassinelli, 2008~

In the film "A Scanner Darkly" [1] (Richard Linklater, 2006) based on the novel of Philip K. Dick, special agents use a disguise that constantly morph their appearance: identity protection is achieved not by wearing a mask, but by constantly changing recognition cues, such as color of the hair, eyebrows, shape of the face, etc, all taken from a huge database of people. In other words, annonimity is not achieved by using an avatar or an "average" face, but by actually oversaturating the cognitive process of recognition. Today this technology could apply to video-conferencing for instance, using images extracted automatically from the web.

This is an interesting observation: one can remember a face (even if it is fake mask), but it is impossible to remember a thousand faces. In fact, the approach has a parallel in the domain of secure telecommunications: using a unique carrier not only makes the system more sensitive to interference and jamming, but simplifies eveasdroping. In a sense, spatio-temporal blending of faces is reminiscent of frequency hopping or spread-spectrum techniques [2]. A face-to-face conversation provides two information channels: one is the face itself that reveals the identity of the speaker; the other channel is the sequence of gestures that the speaker does, providing additional non-verbal information. Here, we want to secure the identity, i.e. avoid using a particular carrier frequency, while letting the partner to recognize the gestures. A particular face correspond to a single carrier frequency, and the gestures modulate that carrier. "Frequency hopping" means transmitting the information contained in the "modulation" (i.e., the sequence of gestures), over a broad range of hopping frequencies (i.e., different faces).

 

I wondered then what would be the "hopping frequency threshold" at which one could still have the feeling that we are switching from one (known or unknown) face to another; and what would be the percentage of the face that, while modified, would render recognition impossible (a threshold measured in percentage of change/second, and "change" having both a spatial and temporal component). I think there are serious interesting experiments to do in this area. In the following, I show some preliminary experiments done with the "khronos projector" interface. Here is an example: [WMV-6MB].

You can try by yourself below using a Processing applet. It's an eerie feeling... Try it by fixating the face for a while, while changing the flipping speed. Why fixation is needed? I don't know (this process of fixation reminds me of the old tradition of scrying in mirrors in a half-lited room: the scrying end up seeing morphing faces and "visions"). Perhaps it enables to concentrate all the power of observation on the visual changes taking place before the eyes, and never attributing these changes to modifications of the observer position.

applet blend

Informal observations

The following are some preliminary observations as well as explanatory hypothesis obtained by just playing with the speed of transition between two whole faces, i.e. having a change rate defined only in the temporal domain (in the future I plan to study the effects of mosaicing these):

tourne moni

"Also, the result of visualizing simple gestures using the Khronos projector can be strikingly weird, probably because our visual system is fed with information that almost make sense - but for some subtle, continuous disturbance on the temporal coordinates of each pixel. The result of this reconstruction effort is that sometimes the deformation is no longer interpreted as the result of an isometric rotation or translation of the whole subject, but instead as a tremendously deformed gesture. There are some interesting things to explore here concerning the emotional response to such "impossible" gestures, and perhaps the Khronos Projector can effectively be used to explore some cognitive aspects concerning our visual system (for instance, how motion is integrated in the process of face recognition). " [3]

Some interesting experiments to do in the future may involve:

edgar allan poe symmetry  
edgar_palindrome
"In 1930 William E. Benton patented the "Duality Mirror," which showed how faces would look with two perfectly symmetrical right or left halves." (from [4]).  

(click on the image to launch applet)


References

  1. "A Scanner Darkly", by Richard Linklater (2006), adapted from the novel of the same name by Philip K. Dick.
  2. Frequency-hopping spread spectrum (FHSS) is a method of transmitting radio signals by rapidly switching a carrier among many frequency channels, using a pseudorandom sequence known to both transmitter and receiver (from Wikipedia). Of course, this is just an analogy: there is no "known sequence" here, but this is unnecessary, as the observer is capable of "listening" to all the "frequencies" at the same time (i.e., she or he is able to recognize any face as a face, and then "demodulate" the gesture, even though he cannot identify these faces).
  3. The Khronos Projector web page, 2005
  4. "Mirror, Mirror: a history of human love affair with reflection", by Mark Pendergrast.

Exhibition History

  • Blended identities using the "Khronos Projector" featured at Fuji-terebi, ホルスの好奇心, 11.4.2010 (smoothly blended identities, see left)
  • Utakophane, Exhibited at "SuperDeluxe" in Tokyo (2009).