On March 11, Du Jiulin's research group and Mu Yu's research group of the Center of Excellence for Brain Science and Intelligent Technology of the Chinese Academy of Sciences, together with Hao Jie's research group of the Institute of Automation, published a research paper entitled "Real time analysis of large-scale neuroimaging to achieve closed-loop research on neural dynamics" online on Nature Neuroscience. The relevant achievements have been granted an invention patent "Optical Brain Computer Interface System and Method".

This study utilizes data processing techniques in the field of astronomy and employs a hybrid FPGA GPU architecture to perform real-time registration, signal extraction, and analysis of neural function data in big data streams up to 500MB/s. Through this technological breakthrough, the team achieved real-time monitoring and analysis of 100000 level neurons in the entire brain of zebrafish for the first time, and then decoded the activity of any selected neuron cluster to control external devices. This achievement marks a crucial step in the application of virtual reality, optogenetic regulation, and other technologies based on whole brain single-cell optical imaging in closed-loop research in neuroscience.

Whole brain single neuron activity imaging is a tool for analyzing the principles of parallel distributed computing in the brain, but the huge data processing requirements have become a technical bottleneck that is difficult to overcome, making it difficult to analyze in real-time and closed-loop control and study brain function on a large scale. Inspired by the rapid radio burst detection technology in the field of astronomy, researchers drew inspiration from the FX system design strategy and utilized the flexibility of FPGA programming to establish an optical neural signal preprocessing system. The signals from optical sensors were regularized and sent to a GPU based real-time processing system for high-speed nonlinear registration. The neural signals from each channel were extracted and decoded according to coding rules, To obtain feedback signals for controlling external devices. The system generates feedback signals through real-time monitoring of the activity of neurons in the entire brain of zebrafish, with a feedback interval of less than 70.5 milliseconds.

The system performance is demonstrated in three closed-loop research scenarios of neuroscience, namely real-time optogenetic stimulation that is phase-locked with any specific neuron cluster activity, real-time visual stimulation that is phase-locked with specific brain functional states, and virtual reality control based on neuron cluster activity.

In the field of closed-loop real-time optogenetic neural regulation, research identifies the entire brain neuron cluster through functional clustering, and uses the spontaneous activity of the selected cluster as a trigger signal to implement real-time optogenetic stimulation on the target neuron cluster. Compared to open-loop stimulation, closed-loop stimulation effectively activates downstream brain regions.

In the real-time visual stimulation experiment of phase-locked, the study applied visual stimulation on the excitation phase of the locus coeruleus, which characterizes the awake state of animals, through real-time monitoring of the noradrenergic system activity of the locus coeruleus. It was observed that other neurons in the brain responded more strongly. Research has shown that the brain state can regulate the processing of visual information, and closed-loop sensory stimuli can help explore the interaction between the internal state of the brain and the external environment.

In terms of virtual reality implemented through the whole brain optical brain computer interface, research is conducted on real-time dimensionality reduction of all neural activity in the high-dimensional whole brain to the activity of multiple neural clusters, and the activity of any cluster is closed-loop connected to the visual environment. A virtual reality system based on optical imaging is established, which directly connects the activity of brain neurons to the visual environment. In this virtual reality, the gain of coupling neural activity with the environment can be adjusted arbitrarily, so that the neural cluster controlling the environment can adaptively adjust its activity output according to the change in gain. In the future, relying on real-time analysis of big data streams and high-throughput whole brain imaging technology, researchers will be able to screen neural population activity characteristics suitable for optical brain computer interfaces and reveal their mechanisms, developing more efficient optical brain computer interface technologies.

The research work was supported by the Ministry of Science and Technology, the National Natural Science Foundation of China, the Chinese Academy of Sciences, Shanghai and Shenzhen.

Paper link:

论文链接

The left figure shows how this innovative system connects the neural activity of the inner world with the external real world; The right figure shows the optical brain computer interface technology implemented using this system, which achieves virtual reality control through the activity of neural clusters within the brain.