WO2023123628A1 - Multi-dimensional signal control system and method - Google Patents

Abstract

A multi-dimensional signal control system and a method. The system comprises: a three-dimensional visual stimulation device (20), configured to generate a three-dimensional visual stimulation signal on the basis of a pre-loaded three-dimensional visual stimulation perception task, and display the three-dimensional visual stimulation signal to the eyes of a subject, so that the eyes of the subject move with the displayed three-dimensional visual stimulation signal; an eye movement detection device (30), configured to detect eye detection information of the subject and send the eye detection information to a multi-dimensional signal control device (10); a multi-dimensional signal control device (10), configured to generate eye movement information according to the eye detection information, and send the eye movement information to an eye movement signal processing device (40); an eye movement signal processing device (40), configured to generate a feedback signal according to the eye movement information, and send the feedback signal to the multi-dimensional signal control device (10), the multi-dimensional signal control device (10) also being configured to execute an operation corresponding to the feedback signal.

Description

Multi-dimensional signal control system and method

This disclosure claims priority to a Chinese patent application with application number 202111644254.3 filed with the China Patent Office on December 29, 2021, the entire contents of which are incorporated herein by reference.

technical field

The embodiments of the present application relate to the technical field of visual cognition detection and control, for example, to a multi-dimensional signal control system and method.

Background technique

Non-human primates are ideal model animals for studying brain cognitive function and exploring the neural network of brain diseases. Among various primates, macaques have always been the main model animals for vision-based brain cognition research due to their visual system very similar to that of humans and their high intelligence. Related technologies Based on the visual cognition research of macaques, a large amount of histological, behavioral and electrophysiological experimental data have been accumulated, which has laid a good foundation for future research on the analysis of brain functional circuits. Vision-based attention, learning and memory, decision-making, emotional perception and regulation are important components of higher brain cognitive functions, and abnormal visual cognitive functions are also closely related to a variety of brain diseases.

In the natural environment, the visual system mainly receives visual stimuli. In order to meet the needs of the natural environment, the structure and function of the visual system are also gradually evolved in the process of adapting to the complex natural environment. Therefore, visual stimuli that can be precisely controlled and that are similar to natural environments are crucial for resolving visual cognitive brain functions.

Contents of the invention

The embodiment of the present application provides a multi-dimensional signal control system and method to realize the cognitive function test in the three-dimensional environment and improve the test signal dimension of the cognitive function test.

In the first aspect, the embodiment of the present application provides a multi-dimensional signal control system, including: a multi-dimensional signal control device, a three-dimensional visual stimulation device, an eye movement detection device and an eye movement signal processing device, wherein:

The three-dimensional visual stimulation device is configured to generate a three-dimensional visual stimulation signal based on a preloaded three-dimensional visual stimulation perception task, and display the three-dimensional visual stimulation signal to the eyes of the examinee, so that the eyes of the examinee follow the displayed three-dimensional Visual stimuli signal movement;

The eye movement detection device is configured to detect the eye detection information of the inspected party, and send the eye detection information to the multi-dimensional signal control device;

The multi-dimensional signal control device is configured to generate eye movement information according to the eye detection information, and send the eye movement information to the eye movement signal processing device;

The eye movement signal processing device is configured to generate a feedback signal according to the eye movement information, and send the feedback signal to the multi-dimensional signal control device;

The multi-dimensional signal control device is also configured to perform the operation corresponding to the feedback signal.

In the second aspect, the embodiment of the present application also provides a multi-dimensional signal control method, which is executed by the multi-dimensional signal control system provided in any embodiment of the present application, including:

The 3D visual stimulation device generates 3D visual stimulation signals according to the preloaded 3D visual stimulation perception tasks, and displays the 3D visual stimulation signals to the eyes of the inspected party, so that the inspected party's eyes follow the displayed 3D visual stimulation signals. sports;

The eye movement detection device detects the eye detection information of the inspected party, and sends the eye detection information to the multi-dimensional signal control device;

The multi-dimensional signal control device generates eye movement information according to the eye detection information, and sends the eye movement information to the eye movement signal processing device;

The eye movement signal processing device generates a feedback signal according to the eye movement information, and sends the feedback signal to the multi-dimensional signal control device;

The multi-dimensional signal control device executes the operation corresponding to the feedback signal.

Description of drawings

FIG. 1 is a schematic structural diagram of a multi-dimensional signal control system provided in Embodiment 1 of the present application;

FIG. 2 is a schematic structural diagram of a computer device provided in Embodiment 1 of the present application;

FIG. 3 is a flow chart of a multi-dimensional signal control method provided in Embodiment 2 of the present application.

Detailed ways

The application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only some structures related to the present application are shown in the drawings but not all structures.

The cognitive tasks of related technologies are realized based on two-dimensional visual stimuli, which are somewhat different from the three-dimensional visual environment in the natural environment. Therefore, how to realize the cognitive function test in the three-dimensional environment is a technical problem to be solved urgently.

Embodiment one

FIG. 1 is a schematic structural diagram of a multi-dimensional signal control system provided in Embodiment 1 of the present application. The multi-dimensional signal control system provided in this embodiment is applicable to situations where primates are controlled based on multi-dimensional control signals. As shown in Figure 1, the multi-dimensional signal control system includes a multi-dimensional signal control device 10, a three-dimensional visual stimulation device 20, an eye movement detection device 30 and an eye movement signal processing device 40, wherein:

The three-dimensional visual stimulation device 20 is configured to generate a three-dimensional visual stimulation signal based on a preloaded three-dimensional visual stimulation perception task, and present the three-dimensional visual stimulation signal to the subject's eyes, so that the subject's eyes follow the displayed Three-dimensional visual stimuli signal movement;

The eye movement detection device 30 is configured to detect the eye detection information of the inspected party, and send the eye detection information to the multi-dimensional signal control device 10;

The multi-dimensional signal control device 10 is configured to generate eye movement information according to the eye detection information, and send the eye movement information to the eye movement signal processing device 40;

The eye movement signal processing device 40 is configured to generate a feedback signal according to the eye movement information, and send the feedback signal to the multi-dimensional signal control device 10;

The multi-dimensional signal control device 10 is further configured to perform an operation corresponding to the feedback signal.

In one embodiment, the three-dimensional visual stimulation device, the eye movement detection device, and the eye movement signal processing device are all communicatively connected to the multi-dimensional signal control device.

In this embodiment, the multi-dimensional signal control device is the core of the multi-dimensional signal control system, and coordinates the interaction with the 3D visual stimulation device and the eye movement detection device to realize the input and interaction of the 3D visual stimulation device and the eye movement detection device. The output control can complete operations such as eye movement signal transmission, reward signal transmission, and event coding signal transmission.

In one embodiment, the three-dimensional visual stimulation sensory task is developed in the development tool in advance, and the developed three-dimensional visual stimulation perception task is imported into the three-dimensional visual stimulation device, and the three-dimensional visual stimulation device stores the three-dimensional visual stimulation perception task, and the three-dimensional visual stimulation can be performed. perception task. The three-dimensional visual stimulation equipment consists of two parts: a control system and a display device. Among them, the control system is used for operations such as storing, switching, and transmitting three-dimensional visual stimulation perception tasks, and the display device can be a head-mounted 3D display screen for displaying three-dimensional visual stimulation signals. Generally speaking, the 3D visual stimulation device generates 3D visual stimulation signals based on the preloaded 3D visual stimulation perception tasks for display, the eyes of the examinee move with the displayed 3D visual stimulation signals, and the eye movement detection equipment detects the motion of the examinee. Eye movement, get eye detection information, and send the eye detection information to the multi-dimensional signal control device, the multi-dimensional signal control device generates eye movement information according to the eye detection information, and sends the eye movement information to the eye movement signal The processing device, the eye movement signal processing device generates a feedback signal according to the eye movement information, and sends the feedback signal to the multi-dimensional signal control device, so that the multi-dimensional signal control device performs the operation corresponding to the feedback signal to complete the three-dimensional visual cognition task. Among them, the feedback signal can be task execution information, such as task execution completion or task execution incomplete, and it can also be a reward signal after task execution is completed. The display device in the three-dimensional visual stimulation device may adopt a related technology virtual reality (Virtual Reality, VR) and/or augmented reality (Augmented Reality, AR) device.

Optionally, a 3D visual stimulus perception task can be pre-developed based on Unity3D. Unity3D supports multiple mainstream platforms at the same time, which greatly reduces the difficulty for developers to release multiple platforms; it also supports scripting languages such as C#, JavaScript, and Boo; it also supports major art resource file formats, which can meet the needs of developers with no development experience. In addition, it also provides highly optimized graphics rendering, embedded Mecanim animation system, Shuriken particle system and other functions. The development logic of the three-dimensional visual stimulation perception task is to present the subject (such as a macaque) with a visual cognition task based on three-dimensional visual stimulation, and at the same time collect the subject's eye movement signals. On the basis of the above, a reward signal can be sent to reward the inspected party when the inspected party successfully completes the task; it can also be used during the completion of the three-dimensional visual stimulus perception task, such as the beginning of the event, the end of the event, the appearance of the visual stimulus, the visual Stimulus types and reward delivery are digitally coded, and task events are marked by digitally coded events.

Optionally, the multi-dimensional signal control device can be developed based on an embedded development board. The embedded development board has strong computing power, flexible development architecture, a sufficient number of digital and analog I/O ports, serial port interrupts, and functions such as timers and counters at the same time, meeting the multi-dimensional requirements of 3D visual cognitive function tests. Functional requirements for signal control equipment.

Optionally, the eye movement detection device may be an eye movement detector (eye tracker). The eye tracker is based on a fast image processing method, uses a high-speed camera to directly collect eye images, and calculates the position of the eyes based on the eye images. Based on existing algorithms and the computing power of electronic computers, eye trackers in related technologies can sample eye movements at a frequency above 1000 hertz (Hz), with an accuracy of <=0.5 degrees, and have gradually become the main method for tracking eye movements. The eye tracker may use an eye tracker of related technology. Take SensoMotoric Instrumens iView X High-speed Primate Eye Tracker (SMI Eye Tracker) as an example. Reliable tracking, high temporal and spatial resolution, and easy integration as part of an animal training system. The SMI eye tracker can track the dark pupils of one or both eyes, and output the analog voltage of the X and Y directions of the eye position, and the pupil size and other information at the same time. The eye movement signal of the subject is collected by the eye tracker, instead of directly implanting a tracking device into the subject's eye through the eye tracking device built in the 3D helmet or an invasive method. The method is simple to implement and can be directly and effectively observed The status of the party under prosecution.

When performing a three-dimensional visual cognitive function test, the eye tracker was placed in front of the macaque, and its position was adjusted so that it could completely track the pupil of the macaque, that is, the white coil wrapped around the dark pupil. Through the control software of the eye tracker, set the hardware parameters of sampling frequency, eye position voltage range, single eye or binocular output. Hardware parameters can be determined according to actual test requirements.

On the basis of the above solution, the eye detection information is analog voltage information, and the eye movement information is generated according to the eye detection information, including: performing analog-to-digital conversion on the analog voltage information to obtain eye movement information. Assume that the multi-dimensional signal control device is developed based on the STM32 development board, and the eye movement detection device is an eye tracker. During the test, two analog voltage output wires are drawn from the eye tracker port. The two signals are the position of the eye movement. X and Y coordinates, the range can generally be set to -5V ~ +5V. However, for the conversion range of the Analog-to-Digital Converter (ADC) of the STM32 development board: 0V to 3.6V, it does not meet the ADC requirements of the two output voltages of the eye tracker. Therefore, an ADC module with a true bipolar analog voltage range of ±5V, ±10V is required. Related technologies ADC modules that highly meet the requirements include the AD7606 multi-channel AD data acquisition module (AD7606 module) and the like. The eye detection information is collected through the eye tracker, and two analog voltage signals are output. The analog voltage signals are converted to analog to digital to obtain eye movement information, which can then be input to the 3D visual stimulation device. Optionally, the eye detection information in the form of an analog voltage signal first passes through the AD7606 module, and converts the analog voltage in the range of -5V to +5V with 16-bit precision for analog-to-digital conversion. Then, the STM32 development board timer interrupts the data sampling of the AD7606 module, packs the data to obtain eye movement information, and outputs the data to the 3D visual stimulation device through the USB serial port communication.

The multi-dimensional signal control system provided by the embodiment of the present application includes: multi-dimensional signal control equipment, three-dimensional visual stimulation equipment, eye movement detection equipment and eye movement signal processing equipment, wherein: the three-dimensional visual stimulation equipment is set based on preloaded three-dimensional The visual stimulation perception task generates a three-dimensional visual stimulation signal, and displays the three-dimensional visual stimulation signal to the eyes of the inspected party, so that the inspected party's eyes move with the displayed three-dimensional visual stimulation signal; the eye movement detection device is set In order to detect the eye detection information of the inspected party, and send the eye detection information to the multi-dimensional signal control device; the multi-dimensional signal control device is set to generate eye movement information according to the eye detection information, and send the eye movement information to to the eye movement signal processing device; the eye movement signal processing device is also set to generate a feedback signal according to the eye movement information, and sends the feedback signal to the multi-dimensional signal control device; the multi-dimensional signal control device is also set to execute the feedback signal corresponding operate. Realize stimulation in a three-dimensional environment by pre-loading the developed three-dimensional visual stimulation perception task in the three-dimensional visual stimulation device, and realize the information interaction between the three-dimensional visual stimulation device and the eye movement detection device through the multi-dimensional signal control device, and realize the three-dimensional environment. The cognitive function test improves the test signal dimension of the cognitive function test.

In one embodiment of the present application, generating the feedback signal according to the eye movement information includes: judging whether the three-dimensional visual stimulation perception task is completed according to the eye movement information; when the three-dimensional visual stimulation perception task is completed, generating a reward signal as the feedback signal . Optionally, the feedback signal generated by the 3D visual stimulation device may be a reward signal, which rewards the inspected party when completing the 3D visual stimulation perception task, and positively promotes the completion of the 3D visual stimulation perception task. Exemplarily, the three-dimensional visual stimulation signal can be a red sphere, and the displayed red sphere can be compared with the collected eye detection information to determine whether the subject is staring at the red sphere, and when the subject is staring at the red sphere, It is judged that the three-dimensional visual stimulation perception task is completed, if the subject does not stare at the red sphere, it is judged that the three-dimensional visual stimulation perception task is not completed.

On the basis of the above scheme, the system also includes a reward device; the multi-dimensional signal control device is also set to generate a set level signal according to the reward signal and output it to the reward device; the reward device is set to receive the set level signal Perform reward actions. Optionally, a reward device is set in the system to complete the reward operation for the inspected party. Taking macaques as the subject of the prosecution as an example, in order to show the macaques that the task is completed correctly or to appease the macaques, they will give grape juice, apple juice and other drinks as rewards. After receiving the reward signal sent by the three-dimensional visual stimulation device, the multi-dimensional signal control device generates a signal with a set level and outputs it to the reward device, so that the reward device performs a reward operation. Optionally, the set level signal can be set according to actual needs, such as the high level signal can be set as the reward level signal, the low level signal can be set as the stop reward level signal; the low level signal can also be set For reward level signal, set high level signal to stop reward level signal.

In one embodiment, generating a set level signal according to the reward signal and outputting it to the reward device includes: determining the reward duration according to the reward signal, using the reward duration as the counting period; starting timing, outputting the reward level signal, and When the output time reaches the reward duration, the stop reward level signal is output. Exemplarily, the output of the set level signal can be implemented based on a timer. First, determine the duration of the reward according to the set bit value in the reward signal, use the duration of the reward as the counting period of the timer, call the timer initialization, and configure the timer based on the counting period of the timer. When the multi-dimensional cognitive function test is executed, the timer function is turned on, and the timer starts counting; after the timer starts counting, the reward level signal is output by setting the output port, and the timer counting ends in response to the timer interrupt function , the timer is closed, the interrupt flag is cleared, and the timer interrupt function outputs the stop reward level signal by setting the output port to complete the reward program.

Optionally, the incentive device includes a liquid storage device, a solenoid valve pump and a liquid conduit, one end of the liquid conduit is connected to the liquid storage device, and the reward operation is performed according to the received level signal, including: when receiving the reward level signal, the electromagnetic The valve pump is turned on, and the liquid in the liquid storage device flows out through the liquid conduit; when receiving the stop reward level signal, the electromagnetic valve pump is closed, and the liquid in the liquid storage device stops flowing out. Exemplarily, the reward device may be a square box with a solenoid valve pump integrated inside, a liquid conduit arranged above the box, and a power line, conduction control line, and ground line below the box. One end of the liquid conduit is inserted into a measuring bottle (i.e., a liquid storage device) containing fruit juice, and the other end is connected to a small nozzle and put into the mouth of the inspected party. When the set reward voltage is applied to the conduction control line, the electromagnetic valve is opened, and the juice flows out along the conduit, and the inspected party is rewarded at this time. When the set stop reward voltage is applied to the conduction control line, the electromagnetic valve is closed, and the juice stops flowing out. At this time, the inspected party does not get a reward.

On the basis of the above scheme, the system also includes an electrophysiological signal acquisition device; a three-dimensional visual stimulation device, which is also configured to generate an electrophysiological event trigger signal and send it to the multi-dimensional signal control device when a preset electrophysiological event is detected; the multi-dimensional The signal control device is also configured to process the electrophysiological event trigger signal to obtain the electrophysiological event marker signal, and send the electrophysiological event marker signal to the electrophysiological signal acquisition device; the electrophysiological signal acquisition device is configured to monitor the electrophysiological event The labeled signal is processed to obtain the detection and analysis results. Optionally, the preset electrophysiological event includes at least one of appearance of visual stimulus, disappearance of visual stimulus, change of presentation position of visual stimulus, and change of presentation depth of visual stimulus.

Optionally, the preset electrophysiological events may also include a visual stimulus set position event and a visual stimulus set depth event.

In order to realize the recording of the test process and the statistical analysis of the test, an electrophysiological signal acquisition device is added to collect the electrophysiological event signals during the test. Taking the development of multi-dimensional signal control equipment based on the STM32 development board as an example, the principle of electrophysiological event marking is: when an electrophysiological event is detected, the Strobe bit is initialized as a high-level signal, and the electrophysiological event is transmitted to 16 digital signal input bits , and then give Strobe a low level, and then quickly give a high level. The multi-dimensional signal control device refreshes the receiving port at a set frequency and reads the value once. In this way, the encoded transmission of an electrophysiological event is successfully completed, and the marker signal of the electrophysiological event can be seen on the control host interface of the multi-dimensional signal control device.

In one embodiment, the electrophysiological event trigger signal is an electrophysiological event code, and processing the electrophysiological event trigger signal to obtain an electrophysiological event marker signal includes: converting the encoded value of the electrophysiological event code into a binary value to obtain an electrophysiological event Physiological event marker signals.

In order to realize the rapid transmission of electrophysiological events, the electrophysiological event trigger signals can be set in the form of encoding. Exemplarily, the transmitted electrophysiological event signal is packaged data, which is a series of arrays with different byte lengths. The data at each position in the array sets its corresponding meaning. After receiving the electrophysiological event code, the coded value of the electrophysiological event code is converted into a binary value to obtain an electrophysiological event marker signal, which is sequentially assigned to each output port for output. Always keep the output until the next output flush.

It should be noted that in this embodiment, the three-dimensional visual stimulation equipment, eye movement detection equipment, eye movement signal processing equipment, multi-dimensional signal control equipment, electrophysiological signal acquisition equipment and rewarding devices can all be in the form of general computer equipment Performance.

As shown in FIG. 2, FIG. 2 shows a schematic structural diagram of an exemplary computer device 212 suitable for implementing the embodiments of the present application. The computer device 212 provided by the present application may include: one or more processors or processing units 216 , the system memory 228, the bus 218 connecting different system components (including the system memory 228 and the processing unit 216).

Wherein, the bus 218 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure in various bus structures. These architectures may include, for example, an Industrial Standard Architecture (ISA) bus, a Micro Channel Architecture bus, an Enhanced ISA bus, a Video Electronics Standards Association local bus, and a Peripheral Component Interconnect bus.

Computer device 212 may include a variety of computer system readable media. These media can be any available media that can be accessed by computing device 212 and include both volatile and nonvolatile media, removable and non-removable media.

System memory 228 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 230 and/or cache memory 232 . Computer device 212 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 234 may be used to read from and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). System memory 228 may also provide disk drives for reading and writing to removable non-volatile disks (such as "floppy disks"), and for removable non-volatile optical disks (such as CD-ROM, DVD-ROM, or other optical media). ) read and write CD-ROM drive. In these cases, each drive may be connected to bus 218 through one or more data media interfaces. The system memory 228 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of the embodiments of the present application.

Programs/utilities 240 may be stored, for example, in system memory 228 as a set (at least one) of program modules 242, such program modules 242 including an operating system, one or more application programs, other program modules, and program data, which Each or some combination of the examples may include the implementation of a network environment. The program module 242 generally executes the functions and/or methods in the embodiments described in this application.

The computer device 212 may also communicate with one or more external devices 214 (e.g., a keyboard, pointing device, display 224, etc.), and with one or more devices that enable a user to interact with the computer device 212, and/or with Any device (eg, network card, modem, etc.) that enables the computing device 212 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 222 . Also, the computer device 212 can also communicate with one or more networks (eg, a local area network, a wide area network, and/or a public network, such as the Internet) through the network adapter 220 . As shown in FIG. 2 , network adapter 220 communicates with other modules of computer device 212 via bus 218 . It should be appreciated that other hardware and/or software modules may also be used in conjunction with computer device 212, including: microcode, device drivers, redundant processing units, external disk drive arrays, disk array systems, tape drives, and data backup storage systems, among others.

The processing unit 216 can execute various functional applications and data processing by running at least one of other programs among a plurality of programs stored in the system memory 228 .

In one embodiment, any of the three-dimensional visual stimulation equipment, eye movement detection equipment, eye movement signal processing equipment, multi-dimensional signal control equipment, electrophysiological signal acquisition equipment and rewarding device may include the general-purpose computer as described above equipment.

Embodiment two

FIG. 3 is a flow chart of a multi-dimensional signal control method provided in Embodiment 2 of the present application. This embodiment is applicable to the situation of controlling primates based on multi-dimensional control signals. The method is executed by the multi-dimensional signal control system provided in any embodiment of the present application. In this embodiment, explanations of terms that are the same as or corresponding to those in the foregoing embodiments are not repeated here.

As shown in Figure 3, the method includes:

S310. The three-dimensional visual stimulation device generates three-dimensional visual stimulation signals according to the preloaded three-dimensional visual stimulation perception tasks, and displays the three-dimensional visual stimulation signals to the eyes of the examinee, so that the eyes of the examinee follow the displayed three-dimensional vision. Stimulate signal movement.

S320. The eye movement detection device detects the eye detection information of the inspected party, and sends the eye detection information to the multi-dimensional signal control device.

S330. The multi-dimensional signal control device generates eye movement information according to the eye detection information, and sends the eye movement information to the eye movement signal processing device.

S340. The eye movement signal processing device generates a feedback signal according to the eye movement information, and sends the feedback signal to the multi-dimensional signal control device.

S350. The multi-dimensional signal control device executes an operation corresponding to the feedback signal.

In the embodiment of the present application, a 3D visual stimulation device is used to generate a 3D visual stimulation signal according to a preloaded 3D visual stimulation perception task, and the 3D visual stimulation signal is displayed to the examinee's eyes, so that the examinee's eyes follow the display. The three-dimensional visual stimulation signal movement; the eye movement detection device detects the eye detection information of the inspected party, and sends the eye detection information to the multi-dimensional signal control device; the multi-dimensional signal control device generates eye movement information according to the eye detection information , and send the eye movement information to the eye movement signal processing device; the eye movement signal processing device generates a feedback signal according to the eye movement information, and sends the feedback signal to the multi-dimensional signal control device; the multi-dimensional signal control device executes the feedback signal corresponding to Operation, through the multi-dimensional signal control equipment to realize the information interaction between the three-dimensional visual stimulation equipment and the eye movement detection equipment, realize the cognitive function test in the three-dimensional environment, and improve the test signal dimension of the cognitive function test.

Optionally, on the basis of the above scheme, generating a feedback signal according to eye movement information includes: judging whether the three-dimensional visual stimulation perception task is completed according to the eye movement information; when the three-dimensional visual stimulation perception task is completed, generating a reward signal as feedback Signal.

Optionally, on the basis of the above scheme, the system also includes a reward device, and the method also includes:

The multi-dimensional signal control device generates a set level signal according to the reward signal and outputs it to the reward device;

The reward means performs a reward operation upon receiving the set level signal.

Optionally, on the basis of the above scheme, a set level signal is generated according to the reward signal and output to the reward device, including:

Determine the reward duration according to the reward signal, and use the reward duration as the counting period;

Start the timing, output the reward level signal, and output the stop reward level signal when the reward voltage output time reaches the reward duration.

Optionally, on the basis of the above solution, the incentive device includes a liquid storage device, a solenoid valve pump, and a liquid conduit. One end of the liquid conduit is connected to the liquid storage device, and the reward operation is performed according to the received level signal, including:

When the reward level signal is received, the solenoid valve pump is turned on, and the liquid in the liquid storage device flows out through the liquid conduit;

When receiving the stop incentive level signal, the solenoid valve pump is turned off, and the liquid in the liquid storage device stops flowing out.

Optionally, on the basis of the above solution, the eye detection information is analog voltage information, and the eye movement information is generated according to the eye detection information, including:

Perform analog-to-digital conversion on the analog voltage information to obtain eye movement information.

Optionally, on the basis of the above solution, the system also includes an electrophysiological signal acquisition device, and the method also includes:

When the three-dimensional visual stimulation device detects a preset electrophysiological event, it generates an electrophysiological event trigger signal and sends it to the multi-dimensional signal control device;

The multi-dimensional signal control device processes the electrophysiological event trigger signal to obtain the electrophysiological event marker signal, and sends the electrophysiological event marker signal to the electrophysiological signal acquisition device;

The electrophysiological signal acquisition device processes the electrophysiological event marker signal to obtain detection and analysis results.

Optionally, on the basis of the above scheme, the electrophysiological event trigger signal is an electrophysiological event code, and the electrophysiological event trigger signal is processed to obtain an electrophysiological event marker signal, including:

The encoded value of the electrophysiological event code is converted into a binary value to obtain an electrophysiological event marker signal.

Optionally, based on the above solution, the preset electrophysiological events include at least one of appearance of visual stimulus, disappearance of visual stimulus, change of presentation position of visual stimulus, and change of presentation depth of visual stimulus.

Claims (10)

1. A multi-dimensional signal control system, including: multi-dimensional signal control equipment, three-dimensional visual stimulation equipment, eye movement detection equipment and eye movement signal processing equipment, wherein:

   The 3D visual stimulation device is configured to generate a 3D visual stimulation signal based on a preloaded 3D visual stimulation perception task, and present the 3D visual stimulation signal to the eyes of the examinee, so that the examinee's Eyes move with the three-dimensional visual stimulation signal displayed;

   The eye movement detection device is configured to detect eye detection information of the inspected party, and send the eye detection information to the multi-dimensional signal control device;

   The multi-dimensional signal control device is configured to generate eye movement information according to the eye detection information, and send the eye movement information to the eye movement signal processing device;

   The eye movement signal processing device is configured to generate a feedback signal according to the eye movement information, and send the feedback signal to the multi-dimensional signal control device;

   The multi-dimensional signal control device is further configured to perform an operation corresponding to the feedback signal.
2. The system according to claim 1, wherein said generating a feedback signal according to said eye movement information comprises:

   judging whether the three-dimensional visual stimulus perception task is completed according to the eye movement information;

   When the three-dimensional visual stimulus perception task is completed, a reward signal is generated as the feedback signal.
3. The system of claim 2, further comprising a reward device;

   The multi-dimensional signal control device is further configured to generate a set level signal according to the reward signal and output it to the reward device;

   The rewarding device is configured to perform a rewarding operation when receiving the set level signal.
4. The system according to claim 3, wherein said generating a set level signal according to said reward signal to output to said reward device comprises:

   determining a reward duration according to the reward signal, using the reward duration as a counting period;

   Start timing, output a reward level signal, and output a stop reward level signal when the reward voltage output time reaches the reward duration.
5. The system according to claim 4, wherein the incentive device comprises a liquid storage device, a solenoid valve pump, and a liquid conduit, one end of the liquid conduit is connected to the liquid storage device, and the execution according to the received level signal Reward actions, including:

   When receiving the reward level signal, the solenoid valve pump is turned on, and the liquid in the liquid storage device flows out through the liquid conduit;

   When receiving the stop incentive level signal, the solenoid valve pump is turned off, and the liquid in the liquid storage device stops flowing out.
6. The system according to claim 1, wherein the eye detection information is analog voltage information, and the generating eye movement information according to the eye detection information comprises:

   performing analog-to-digital conversion on the analog voltage information to obtain the eye movement information.
7. The system according to claim 1, further comprising an electrophysiological signal acquisition device;

   The three-dimensional visual stimulation device is further configured to generate an electrophysiological event trigger signal and send it to the multi-dimensional signal control device when a preset electrophysiological event is detected;

   The multi-dimensional signal control device is further configured to process the electrophysiological event trigger signal to obtain an electrophysiological event marker signal, and send the electrophysiological event marker signal to the electrophysiological signal acquisition device;

   The electrophysiological signal acquisition device is configured to process the electrophysiological event marker signal to obtain detection and analysis results.
8. The system according to claim 7, wherein the electrophysiological event trigger signal is an electrophysiological event code, and the processing of the electrophysiological event trigger signal to obtain an electrophysiological event marker signal includes:

   converting the encoded value of the electrophysiological event into a binary value to obtain the electrophysiological event marker signal.
9. The system according to claim 7, wherein the preset electrophysiological event includes at least one of appearance of visual stimulus, disappearance of visual stimulus, change of visual stimulus presentation position, and change of visual stimulus presentation depth.
10. A multi-dimensional signal control method, performed by the multi-dimensional signal control system according to any one of claims 1-9, comprising:

    The 3D visual stimulation device generates 3D visual stimulation signals according to the preloaded 3D visual stimulation perception tasks, and displays the 3D visual stimulation signals to the eyes of the examinee, so that the eyes of the examinee follow the displayed The three-dimensional visual stimulation signal movement;

    The eye movement detection device detects the eye detection information of the inspected party, and sends the eye detection information to the multi-dimensional signal control device;

    The multi-dimensional signal control device generates eye movement information according to the eye detection information, and sends the eye movement information to an eye movement signal processing device;

    The eye movement signal processing device generates a feedback signal according to the eye movement information, and sends the feedback signal to a multi-dimensional signal control device;

    The multi-dimensional signal control device executes an operation corresponding to the feedback signal.

Images