Department+of+Brain+and+Cognitive+Sciences
-
KAIST Proposes a New Way to Circumvent a Long-time Frustration in Neural Computing
The human brain begins learning through spontaneous random activities even before it receives sensory information from the external world. The technology developed by the KAIST research team enables much faster and more accurate learning when exposed to actual data by pre-learning random information in a brain-mimicking artificial neural network, and is expected to be a breakthrough in the development of brain-based artificial intelligence and neuromorphic computing technology in the future.
KAIST (President Kwang-Hyung Lee) announced on the 23rd of October that Professor Se-Bum Paik 's research team in the Department of Brain Cognitive Sciences solved the weight transport problem*, a long-standing challenge in neural network learning, and through this, explained the principles that enable resource-efficient learning in biological brain neural networks.
*Weight transport problem: This is the biggest obstacle to the development of artificial intelligence that mimics the biological brain. It is the fundamental reason why large-scale memory and computational work are required in the learning of general artificial neural networks, unlike biological brains.
Over the past several decades, the development of artificial intelligence has been based on error backpropagation learning proposed by Geoffery Hinton, who won the Nobel Prize in Physics this year. However, error backpropagation learning was thought to be impossible in biological brains because it requires the unrealistic assumption that individual neurons must know all the connected information across multiple layers in order to calculate the error signal for learning.
< Figure 1. Illustration depicting the method of random noise training and its effects >
This difficult problem, called the weight transport problem, was raised by Francis Crick, who won the Nobel Prize in Physiology or Medicine for the discovery of the structure of DNA, after the error backpropagation learning was proposed by Hinton in 1986. Since then, it has been considered the reason why the operating principles of natural neural networks and artificial neural networks will forever be fundamentally different.
At the borderline of artificial intelligence and neuroscience, researchers including Hinton have continued to attempt to create biologically plausible models that can implement the learning principles of the brain by solving the weight transport problem.
In 2016, a joint research team from Oxford University and DeepMind in the UK first proposed the concept of error backpropagation learning being possible without weight transport, drawing attention from the academic world. However, biologically plausible error backpropagation learning without weight transport was inefficient, with slow learning speeds and low accuracy, making it difficult to apply in reality.
KAIST research team noted that the biological brain begins learning through internal spontaneous random neural activity even before experiencing external sensory experiences. To mimic this, the research team pre-trained a biologically plausible neural network without weight transport with meaningless random information (random noise).
As a result, they showed that the symmetry of the forward and backward neural cell connections of the neural network, which is an essential condition for error backpropagation learning, can be created. In other words, learning without weight transport is possible through random pre-training.
< Figure 2. Illustration depicting the meta-learning effect of random noise training >
The research team revealed that learning random information before learning actual data has the property of meta-learning, which is ‘learning how to learn.’ It was shown that neural networks that pre-learned random noise perform much faster and more accurate learning when exposed to actual data, and can achieve high learning efficiency without weight transport.
< Figure 3. Illustration depicting research on understanding the brain's operating principles through artificial neural networks >
Professor Se-Bum Paik said, “It breaks the conventional understanding of existing machine learning that only data learning is important, and provides a new perspective that focuses on the neuroscience principles of creating appropriate conditions before learning,” and added, “It is significant in that it solves important problems in artificial neural network learning through clues from developmental neuroscience, and at the same time provides insight into the brain’s learning principles through artificial neural network models.”
This study, in which Jeonghwan Cheon, a Master’s candidate of KAIST Department of Brain and Cognitive Sciences participated as the first author and Professor Sang Wan Lee of the same department as a co-author, will be presented at the 38th Neural Information Processing Systems (NeurIPS), the world's top artificial intelligence conference, to be held in Vancouver, Canada from December 10 to 15, 2024. (Paper title: Pretraining with random noise for fast and robust learning without weight transport)
This study was conducted with the support of the National Research Foundation of Korea's Basic Research Program in Science and Engineering, the Information and Communications Technology Planning and Evaluation Institute's Talent Development Program, and the KAIST Singularity Professor Program.
2024.10.23 View 830 -
KAIST researchers discovers the neural circuit that reacts to alarm clock
KAIST (President Kwang Hyung Lee) announced on the 20th that a research team led by Professor Daesoo Kim of the Department of Brain and Cognitive Sciences and Dr. Jeongjin Kim 's team from the Korea Institute of Science and Technology (KIST) have identified the principle of awakening animals by responding to sounds even while sleeping.
Sleep is a very important physiological process that organizes brain activity and maintains health. During sleep, the function of sensory nerves is blocked, so the ability to detect danger in the proximity is reduced. However, many animals detect approaching predators and respond even while sleeping. Scientists thought that animals ready for danger by alternating between deep sleep and light sleep.
A research team led by Professor Daesoo Kim at KAIST discovered that animals have neural circuits that respond to sounds even during deep sleep. While awake, the medial geniculate thalamus responds to sounds, but during deep sleep, or Non-REM sleep, the Mediodorsal thalamus responds to sounds to wake up the brain.
As a result of the study, when the rats fell into deep sleep, the nerves of the medial geniculate thalamus were also sleeping, but the nerves of mediodorsal thalamus were awake and responded immediately to sounds. In addition, it was observed that when mediodorsal thalamus was inhibited, the rats could not wake up even when a sound was heard, and when the mediodorsal thalamus was stimulated, the rats woke up within a few seconds without sound.
This is the first study to show that sleep and wakefulness can transmit auditory signals through different neural circuits, and was reported in the international journal, Current Biology on February 7, and was highlighted by Nature. (https://www.nature.com/articles/d41586-023-00354-0)
Professor Daesoo Kim explained, “The findings of this study can used in developing digital healthcare technologies to be used to improve understanding of disorders of senses and wakefulness seen in various brain diseases and to control the senses in the future.”
This research was carried out with the support from the National Research Foundation of Korea's Mid-Career Research Foundation Program.
Figure 1. Traditionally, sound signals were thought to be propagated from the auditory nerve to the auditory thalamus. However, while in slow-wave sleep, the auditory nerve sends sound signals to the mediodorsal thalamic neurons via the brainstem nerve to induce arousal in the brain.
Figure 2. GRIK4 dorsomedial nerve in response to sound stimulation. The awakening effect is induced as the activity of the GRIK4 dorsal medial nerve increases based on the time when sound stimulation is given.
2023.03.03 View 3745 -
KAIST Offers Hope to Musicians with Dystonia
< Photo 1. Conductor and Pianist João Carlos Martins before the Recital at the Carnegie Hall preparing with his bionic gloves >
KAIST’s neuroscientist and professor, Dr. Daesoo Kim attended the “Conference for Musicians with Dystonia” supported by the World Health Organization (WHO) and the Carnegie Hall concert of legendary pianist João Carlos Martins, who is also a dystonia patient, to announce his team’s recent advancements toward finding a cure for dystonia.
On November 19, 2022, a “miracle concert” was held in Carnegie Hall. João Carlos Martins was a renowned world-class pianist in the 70s and 80s, but he had to put an end to his musical career due to focal dystonia in his fingers. But in 2020, he began using a bionic glove developed by industrial designer Ubiratã Bizarro Costa and after years of hard work he was back in Carnegie Hall as an 82-year-old man.
During the concert, he conducted the NOVUS NY orchestra in a performance of Bach, and later even played the piano himself. In particular, between his performances, he gave shout-outs to scientists studying dystonia including KAIST Professor Daesoo Kim, asking them to continue working towards curing rare diseases for musicians.
< Photo 2. Professor Daesoo Kim with Conductor and Pianist João Carlos Martins >
Musician’s dystonia affects 1-3% of musicians around the world and musicians make up approximately 5% of the total number of dystonia patients. Musicians who are no longer able to practice music due to the disease often experience stress and depression, which may even lead to suicide in extreme cases. Musicians are known to be particularly prone to such diseases due to excessive practice regimens, perfectionism, and even genetics. Currently, botulinum toxin (Botox) is used to suppress abnormal muscles, but muscle function suppression ultimately means that the musician is no longer able to play the instrument. João Carlos Martins himself underwent several Botox procedures and three brain surgeries, but saw no therapeutic results. This is why a new treatment was necessary.
Professor Daesoo Kim’s research team at KAIST took note of the fact that abnormal muscle tension is caused by excessive stress, and developed NT-1, a treatment that blocks the development of the symptoms of dystonia from the brain, allowing patients to use their muscles as they normally would. The research team published their findings in Science Advances in 2021, and João Carlos Martins invited Professor Daesoo Kim to the UN conference and his concert after reading this paper.
< Photo 3. Professor Daesoo Kim (3rd from the left) photographed with other guests at the recital including Dr. Dévora Kestel, the Director of the Mental Health and Substance Use at WHO, sharing the center with Conductor and Pianist João Carlos Martins >
During the UN conference held the day prior to the Carnegie Hall concert, Dr. Dévora Kestel, Director of the Mental Health and Substance Use at WHO, said, “Although dystonia is not as well-known, it is a common disease around the world, and needs our society’s attention and the devotion of many researchers.” Professor Daesoo Kim said, “NT-1 is a drug that blocks the cause of dystonia in the brain, and will allow musicians to continue practicing music. We aim to attain clinical approval in Korea by 2024.”
NT-1 is currently under development by NeuroTobe, a faculty-led start-up company at KAIST, headed by Professor Daesoo Kim as the CEO. The synthesis of the drug for clinical testing has been successfully completed, and it has shown excellent efficacy and safety through various rounds of animal testing. Unlike Botox, which takes a few days to show its therapeutic effects after receiving the procedure from a hospital, NT-1 shows its therapeutic effects within an hour after taking it. As a so-called “edible Botox”, it is expected to help treat various muscular diseases and ailments.
2022.12.27 View 8300