Researchers at the University of Helsinki have developed a technique
in which a computer models visual perception by monitoring human brain
signals. In a way, it is as if the computer tries to imagine what a
human is thinking about. As a result of this imagining, the computer is
able to produce entirely new information, such as fictional images that
were never before seen.
The technique is based on a novel
brain-computer interface. Previously, similar brain-computer interfaces
have been able to perform one-way communication from brain to computer,
such as spell individual letters or move a cursor.
As far as is
known, the new study is the first where both the computer’s presentation
of the information and brain signals were modelled simultaneously using
artificial intelligence methods. Images that matched the visual
characteristics that participants were focusing on were generated
through interaction between human brain responses and a generative
neural network.
The study was published in the Scientific Reports journal in September. Scientific Reports is an online multidisciplinary, open-access journal from the publishers of Nature.
Neuroadaptive generative modelling
The
researchers call this method neuroadaptive generative modelling. A
total of 31 volunteers participated in a study that evaluated the
effectiveness of the technique. Participants were shown hundreds of
AI-generated images of diverse-looking people while their EEG was
recorded.
The subjects were asked to concentrate on certain
features, such as faces that looked old or were smiling. While looking
at a rapidly presented series of face images, the EEGs of the subjects
were fed to a neural network, which inferred whether any image was
detected by the brain as matching what the subjects were looking for.
Based
on this information, the neural network adapted its estimation as to
what kind of faces people were thinking of. Finally, the images
generated by the computer were evaluated by the participants and they
nearly perfectly matched with the features the participants were
thinking of. The accuracy of the experiment was 83 per cent.
“The
technique combines natural human responses with the computer’s ability
to create new information. In the experiment, the participants were only
asked to look at the computer-generated images. The computer, in turn,
modelled the images displayed and the human reaction toward the images
by using human brain responses. From this, the computer can create an
entirely new image that matches the user’s intention,” says Tuukka Ruotsalo,
Academy of Finland Research Fellow at the University of Helsinki,
Finland and Associate Professor at the University of Copenhagen,
Denmark.
Unconscious attitudes may be exposed
Generating
images of the human face is only one example of the technique’s
potential uses. One practical benefit of the study may be that computers
can augment human creativity.
“If you want to draw or illustrate
something but are unable to do so, the computer may help you to achieve
your goal. It could just observe the focus of attention and predict what
you would like to create,” Ruotsalo says. However, the researchers
believe that the technique may be used to gain understanding of
perception and the underlying processes in our mind.
“The
technique does not recognise thoughts but rather responds to the
associations we have with mental categories. Thus, while we are not able
to find out the identity of a specific ‘old person’ a participant was
thinking of, we may gain an understanding of what they associate with
old age. We, therefore, believe it may provide a new way of gaining
insight into social, cognitive and emotional processes,” says Senior
Researcher Michiel Spapé.
According to Spapé, this is also interesting from a psychological perspective.
“One
person’s idea of an elderly person may be very different from
another’s. We are currently uncovering whether our technique might
expose unconscious associations, for example by looking if the computer
always renders old people as, say, smiling men.”
