CN109542221B - Marking system for accurate acquisition of brain signals in near-infrared spectrum of upper limb movement state - Google Patents

Abstract

The invention relates to a marking system for accurately acquiring brain signals of an upper limb movement state by using near infrared spectrum, belonging to the field of brain movement intention identification. The marking system is combined with the near infrared spectrum brain signal collecting device, brain signals generated by four actions of pushing, pulling, lifting and releasing of the upper limb in a spontaneous state are accurately calibrated in time starting and stopping, accurate and quick signal marking is achieved, and therefore the accuracy of the intention identification of the motion state of the upper limb based on the near infrared spectrum is improved. The method can accurately record the brain signals of the tested person in the natural motion state, ensures the spontaneity of the tested person in the four motion paradigms, more truly restores the essential reaction of the brain signals, and accurately reflects the start-stop time of the four motions by the recorded marking points, thereby providing an important analysis basis for researching the upper limb motion intention identification based on the near infrared spectrum and having important significance for accelerating and promoting the application of the near infrared spectrum on wearable equipment.

Description

Marking system for accurate acquisition of brain signals in near-infrared spectrum of upper limb movement state

technical field

The invention relates to a marking system for accurate collection of brain signals of upper limb movement state in near-infrared spectrum, in particular to a marking system and method for accurate collection of brain signals of upper limb movement state in near-infrared spectrum, which belongs to brain movement intention recognition field.

Background technique

Since the 1960s, exoskeletons have been gradually applied as a concept of bionics to rehabilitation and military assistance, providing a reliable guarantee for improving the quality of human life and enhancing the combat capability of soldiers. Most of the existing power-assisted exoskeleton control methods are based on the human-machine force coupling method, which has the disadvantage that the response lags and cannot respond quickly. The above problems can be effectively solved by obtaining motion signals directly from the human brain through near-infrared spectroscopy equipment, and analyzing data from the human brain signals to predict limb movements.

Relevant studies have shown that human limb movements are mainly related to the motor area of the brain, and the preparation before exercise is related to some functional areas of the frontal lobe area of the brain. When pre-judging the upper limb movement, it is necessary to know the precise time when the testee starts and stops the movement, that is, when marking and recording the signal, the movement start and stop time must be accurately recorded. At present, near-infrared spectroscopy technology is mainly used in the field of medical neurorehabilitation, cognitive neuroscience, etc. These application scenarios are characterized by non-spontaneous stimulation of the brain to generate corresponding neural responses. Completed, there is no precise requirement for the time point of when the testee starts to move and when he stops moving, so these existing marking systems and methods cannot be used in the natural movement state (without any task guidance, completely by the subject. The brain signal of the tester's own decision) is marked. Secondly, the existing marking system does not meet the precise requirements for marking the brain signal, and there is a considerable error between the recorded start and stop marks and the actual start and stop times of the movement.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem that the existing marking system cannot realize the marking of the brain signal in the natural movement state, and the recording of the start and end time of the movement is inaccurate, so as to provide accurate brain signals for the near-infrared spectrum upper limb movement state. Collection of marking systems. The marking system is combined with the near-infrared spectral brain signal acquisition device to perform accurate time start-stop calibration on the brain signals generated by the four actions of pushing, pulling, lifting and releasing the upper limbs in a spontaneous state, so as to solve the problem of existing marking systems and The existing guiding factors of the method can realize accurate and fast signal marking, and then improve the accuracy of the upper limb motion state intention recognition based on near-infrared spectroscopy.

In order to achieve the above object, the technical scheme adopted in the present invention is:

A marking system for accurate acquisition of brain signals in near-infrared spectrum upper limb motion state includes a motion platform, a signal transceiver, marking information acquisition software and a signal acquisition device.

The motion platform can realize four actions of pushing, pulling, lifting and releasing; an acceleration sensor and a plurality of photoelectric switches are installed on the motion platform;

The signal transceiver device is used to collect the signals of the acceleration sensor and the photoelectric switch, and transmit the signals to the signal acquisition device after processing them according to the communication protocol. process signal;

The marking information acquisition system is used for serial port information transmission. Since the signal acquisition device cannot directly read the serial port of the signal transceiver device, the defect is caused by the signal acquisition device. It is forwarded to the serial port of the signal acquisition device to achieve the purpose of data marking;

The motion platform also includes: a frame, a horizontal motion guide rail and a horizontal motion slide block, a vertical motion guide rail and a vertical motion slide block, a horizontal beam, a handle, and a stopper. Horizontal motion guide rails are installed on the left and right sides of the rack, one up and one down, and four in total to move in the horizontal direction. The vertical rails are installed on each side of the rack, the vertical motion rails are installed on the horizontal sliders corresponding to the horizontal motion rails, the two vertical motion rails are connected together by the horizontal beam, and the handle is installed in the middle of the horizontal beam Location;

When the block is fixed, pushing the handle back and forth can realize the pushing and pulling movement of the upper limb in the horizontal direction, and pushing the handle up and down can realize the lifting and releasing movement of the upper limb in the vertical direction. Stops are used to limit movement in one direction, vertical movement when horizontal movement is performed, and horizontal movement when vertical movement is performed.

When the stopper is not fixed, the tilting motion of the upper limb at any angle can be realized, and is no longer limited to horizontal motion and vertical motion.

The sensor is installed on the horizontal beam, and the output line of the acceleration sensor is combined with the output line of the photoelectric switch at the upper and lower ends of the vertical direction, and is connected to the signal transceiver device through the transfer terminal and the FPC cable. The acceleration sensor collects the acceleration change at a return rate of 50Hz (20ms once) and sends it to the microcontroller. After acquiring the information, the microcontroller unpacks and reads the acceleration information using a prescribed protocol, and analyzes the movement of different groups of people. The speed is sampled and analyzed, and the appropriate threshold is selected for calibration, so as to make a judgment on the movement direction of the tested person.

The photoelectric switch includes a front photoelectric switch, a rear photoelectric switch, a horizontal photoelectric switch cover, an upper photoelectric switch, a lower photoelectric switch, a vertical photoelectric switch upper plate and a vertical photoelectric switch lower plate, the front end photoelectric switch and the rear end photoelectric switch. The photoelectric switch is installed on both ends of the lower horizontal motion guide rail, and the horizontal photoelectric switch baffle is installed on the lower horizontal motion slider on the right side. When the handle is pushed back and forth, the action of the front photoelectric switch and the rear photoelectric switch is the photoelectric switch state. Changes are recorded by the signal transceiver; in the same way, the upper photoelectric switch and the lower photoelectric switch act with the vertical photoelectric switch upper shutter and the vertical photoelectric switch lower shutter respectively to record the state when moving up and down;

Wherein, the signal transceiver device includes: a microcontroller, an adapter board, and a serial port module communicating with the upper computer. The microcontroller communicates with the acceleration sensor through its own serial port, and collects the state of the photoelectric switch through the IO port. The breakout board is used to connect the sensor unit and the microcontroller. One end of the serial port module communicating with the host computer is connected with the microcontroller, and the other end is connected with the host computer through USB.

Among them, the marking information acquisition software is developed based on QT software, including: serial port 1 information box, serial port 2 information box, and motion status indicator box. The serial port 1 information box displays the processing information sent by the signal transceiver device after scanning the serial port 1 port and configuring the baud rate. The serial port 2 information box is displayed and sent to the brain signal acquisition device after scanning to obtain the serial port 2 port and configuring the baud rate. The motion state indication box contains a model diagram of a motion platform. When the upper limb performs four kinds of motions, the corresponding motion direction indication will appear on the model diagram.

Further, the exercise platform further includes four feet, which are installed under the frame of the exercise platform and have a non-slip function to ensure that the frame does not slide relative to the desktop when performing upper limb movements.

The present invention also provides a marking method for accurate collection of brain signals in the motion state of the upper limbs in the near-infrared spectrum:

The test subject first wears the near-infrared brain signal acquisition device, and performs autonomous movement without the external prompt of the task. The signal transceiver device sends the accurate time information of the movement start and stop to the host computer for marking according to the set communication protocol. Information acquisition software, marking information acquisition software unpacks the data according to the communication protocol. On the one hand, the action start and stop information is sent to the brain signal acquisition device through the serial port module, and the start and stop information is recorded in the brain signal data stream, which is the later brain signal. It provides time reference for pattern classification and upper limb motion prediction based on brain signals. On the other hand, the motion direction information is reflected in the display interface in real time through the indication window of the marking information acquisition software.

Beneficial effects:

Compared with the existing marking system (depending on software), the marking system includes a micro-control system, an acceleration sensor and a number of photoelectric switches, which can accurately record the brain signals of the test subject in the natural movement state, ensuring that The spontaneity of the test subjects when performing the four exercise paradigms, that is, not exercising under the prescribed time and prompts, more truly restores the essential response of brain signals, and the recorded marking points can accurately reflect the start and stop times of the four types of exercise. , which provides an important analytical basis for the study of upper limb motion intention recognition based on near-infrared spectroscopy, and is of great significance to accelerate the application of near-infrared spectroscopy in wearable devices.

Description of drawings

1 is a schematic diagram of a marking system according to an embodiment of the present invention;

FIG. 2 is a schematic interface diagram of the marking information collection software of the present invention.

Among them, 1—motion platform, 2—signal transceiver, 3—marking information collection software, 4—frame, 5—horizontal motion guide rail, 6—horizontal motion slider, 7—vertical motion guide rail, 8—vertical motion Motion slider, 9—acceleration sensor, 10—front photoelectric switch, 11—rear photoelectric switch, 12—horizontal photoelectric switch shutter, 13—upper photoelectric switch, 14—lower photoelectric switch, 15—vertical photoelectric switch upper block Sheet, 16—vertical photoelectric switch lower cover, 17—horizontal beam, 18—handle, 19—foot, 20—microcontroller, 21—transfer board, 22—serial port module for communication with the host computer, 23—block Block, 24—serial port 1 information box, 25—serial port 2 information box, 26—motion status indication box.

Detailed ways

The technical solutions of the present invention will be further described below through examples and in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

A marking system based on near-infrared spectrum for accurate acquisition of brain signals of upper limb motion state, as shown in Figure 1, includes a motion platform 1, a signal transceiver 2 and a marking information acquisition software 3.

The motion platform 1 includes: a frame 4, a horizontal motion guide rail 5, a horizontal motion slider 6, a vertical motion guide rail 7, a vertical motion slider 8, an acceleration sensor 9, a front-end photoelectric switch 10, a rear-end photoelectric switch 11, and a horizontal photoelectric switch Switch shutter 12, upper photoelectric switch 13, lower photoelectric switch 14, vertical photoelectric switch upper shutter 15, vertical photoelectric switch lower shutter 16, horizontal beam 17, handle 18, foot 19, microcontroller 20, adapter The board 21 and the serial port module 22 that communicate with the upper computer.

The frame 4 is a cuboid or cube frame constructed of aluminum profiles; the bottom end of the frame 4 is provided with feet 19; four horizontal motion guide rails 5 are symmetrically arranged inside the frame 4; two vertical motion guide rails 7 pass through The horizontal movement slider 6 is fixed on the horizontal movement guide rail 5; the horizontal beam 17 is fixed between the two vertical movement guide rails 7, and can move along the vertical movement guide rail 7 through the vertical movement slider 8; the handle 18 is fixed on the horizontal beam 17; the front end photoelectric switch 10 and the rear photoelectric switch 11 are installed on both ends of the horizontal motion guide rail 5; the horizontal photoelectric switch block 12 is installed on the horizontal motion slider 6; The end photoelectric switch 11 is in the same horizontal position; both ends of the vertical motion guide rail 7 are installed with a vertical photoelectric switch upper baffle 15 and a vertical photoelectric switch lower baffle 16; the upper photoelectric switch 13 and the lower photoelectric switch 14 are fixed on the horizontal beam 17 The upper part corresponds to the upper block 15 of the vertical photoelectric switch and the lower block 16 of the vertical photoelectric switch; the block 23, the acceleration sensor 9, the microcontroller 20, the adapter board 21 and the serial port module 22 communicating with the host computer are installed on the On the frame 4; pushing the handle 18 back and forth can realize the pushing and pulling movement of the upper limb in the horizontal direction; pushing the handle 18 up and down can realize the lifting and lowering movement of the upper limb in the vertical direction. In order to ensure that the vertical movement is not disturbed by the horizontal movement, the stopper can be installed on the horizontal guide rail and the vertical guide rail to limit the movement of the slider.

The state changes of the acceleration sensor 9 and the four photoelectric switches are connected to the port of the microcontroller 20 by the signal transceiver 2 through the adapter board 21 . The serial port pin of the microcontroller 20 is read, and the state of the photoelectric switch is read through the IO port of the microcontroller 20 . After judging the state of the acceleration sensor 9 and the photoelectric switch, the collected data is stored in the general protocol shown in Table 1. First, the first frame data 0x55 of the communication protocol is used to judge whether the received data is correct, and the second If the frame data has data 0x01, it means that the upper limb is moving forward, the third frame data has 0x01, which means the upper limb is moving backward, the fourth frame data has 0x01, which means the upper limb is moving upward, and the fifth frame has data 0x01, which means the upper limb is moving downward. , indicating that the data is sent to the upper computer for processing through the serial port module 22 that communicates with the upper computer according to the specified communication protocol.

Table 1 Communication Protocol of Signal Transceiver 2

The signal acquisition device used in this embodiment does not support reading the serial port information of the signal transceiver device 2 directly, so the marking information acquisition software 3 is used to forward the data. Unpack and display the received data in the serial port 1 information box 24. As long as the action state of the upper limb changes, the serial port 2 will send the corresponding flag to the brain signal acquisition device, and display the marking information in the information box 25 of the serial port 2. The motion state indication box 26 will be displayed according to the upper limb motion direction information contained in the protocol, and the three-dimensional model diagram in the motion state indication box corresponds to the actual marking system.

A marking method for accurate acquisition of brain signals in near-infrared spectrum upper limb movement states, the specific method is as follows: The test subject wears a near-infrared brain signal acquisition device, and there is no external prompt, and the test subject decides the start and stop of the exercise independently. , the purpose is to get closer to the natural state of motion. In this embodiment, the push-pull, lift-and-release action of the test subject is in a horizontal or vertical direction. In other embodiments, the push-pull, lift-and-release action may be any linear motion. When the near-infrared brain signal starts to be collected, the subject holds the handle 18 of the exercise platform 1 and starts to move autonomously. At this time, the acceleration signal will increase from zero, and at the same time, the state of one of the photoelectric switches will change. , in the case of simultaneous occurrence of the two, the movement direction of the upper limb is judged, and the captured movement marking signal will be sent to the signal acquisition device through the signal transceiver device 2, and recorded in the brain signal data stream. After the experiment, according to the recording points of the marking information, targeted signal feature extraction and classification processing were performed on the brain signals at the moment and the adjacent moments, and a machine learning model corresponding to the four upper limb movement states was established. The model predicts the subsequent motion state. Further, the prediction of the motion state can be used as the control input of the power-assisted exoskeleton, so as to realize the real exoskeleton wearable device directly controlled by the human brain.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (7)

1. Marking system that upper limbs motion state brain signal of near infrared spectrum was gathered accurately, its characterized in that: the marking system comprises a motion platform, a signal receiving and transmitting device, a signal acquisition device and marking information acquisition software;

the motion platform can realize four actions of pushing, pulling, lifting and releasing; an acceleration sensor and a plurality of photoelectric switches are arranged on the motion platform;

the signal transceiver is used for acquiring signals of the acceleration sensor and the photoelectric switch, processing the signals according to a communication protocol and transmitting the processed signals to the signal acquisition device, wherein the transmitted signals are signals only when the movement starts and stops or signals including the movement process;

the marking information acquisition software is used for serial port information transmission, and because the signal acquisition device cannot directly read the serial port of the signal transceiver, the defect is caused by the signal acquisition device, and serial port data of the signal transceiver needs to be forwarded to the serial port of the signal acquisition device through the marking information acquisition software so as to achieve the purpose of marking data;

the method comprises the steps that a testee firstly wears a near-infrared brain signal acquisition device to perform autonomous movement under the condition of no external prompt, the signal transceiver sends accurate time information of movement starting and stopping to upper computer marking information acquisition software according to a set communication protocol, the marking information acquisition software unpacks data according to the communication protocol, on one hand, action starting and stopping information is sent to the brain signal acquisition device through a serial module, starting and stopping information is recorded in a brain signal data stream, time reference is provided for mode classification of later-stage brain signals and upper limb movement prediction based on the brain signals, and on the other hand, movement direction information is reflected on a display interface in real time through an indication window of the marking information acquisition software.

2. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of claim 1, wherein: the motion platform includes: the device comprises a rack, a horizontal movement guide rail, a horizontal movement sliding block, a vertical movement guide rail, a vertical movement sliding block, a horizontal cross beam, a handle and a stop block; the horizontal movement guide rails are respectively installed one above the other at the left side and the right side of the rack, and the total number of the horizontal movement guide rails is four; the two sides of the rack are respectively provided with one vertical rail, the vertical moving guide rails are arranged on horizontal sliding blocks corresponding to the horizontal moving guide rails, the two vertical moving guide rails are connected together through a horizontal cross beam, and the handle is arranged in the middle of the horizontal cross beam;

when the stop block is fixed, the handle is pushed forwards and backwards to realize the pushing and pulling movement of the upper limb in the horizontal direction, and the handle is pushed upwards and downwards to realize the lifting and releasing movement of the upper limb in the vertical direction; the stop block is used for limiting the movement in one direction, limiting the vertical movement when the stop block performs horizontal movement, and limiting the horizontal movement when the stop block performs vertical movement;

when the stop block is not fixed, the upper limb can tilt at any angle without being limited by horizontal movement and vertical movement.

3. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of claim 1, wherein: the acceleration sensor is arranged on the horizontal cross beam, and an output line of the acceleration sensor is converged with output lines of the photoelectric switches at the upper end and the lower end in the vertical direction and then connected with the signal transceiver through a switching terminal and an FPC (flexible printed circuit) flat cable; the acceleration sensor collects acceleration changes at a 50Hz return rate and sends the acceleration changes to the microcontroller, the microcontroller unpacks and reads acceleration information by using a specified protocol after acquiring the information, sampling and analyzing the movement speeds of different crowds, and selecting a proper threshold value for calibration so as to judge the movement direction of a testee at one time.

4. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of claim 1, wherein: the photoelectric switch comprises a front end photoelectric switch, a rear end photoelectric switch, a horizontal photoelectric switch baffle, an upper end photoelectric switch, a lower end photoelectric switch, an upper vertical photoelectric switch baffle and a lower vertical photoelectric switch baffle, wherein the front end photoelectric switch and the rear end photoelectric switch are arranged at two ends of a lower end horizontal movement guide rail, the horizontal photoelectric switch baffle is arranged on a right lower end horizontal movement sliding block, and when the handle is pushed forwards and backwards, the photoelectric switch state changes with the action of the front end photoelectric switch and the rear end photoelectric switch respectively so as to be recorded by the signal transceiving device; in the same way, the upper end photoelectric switch and the lower end photoelectric switch respectively act with the upper baffle plate of the vertical photoelectric switch and the lower baffle plate of the vertical photoelectric switch to record the state when the vertical photoelectric switch moves up and down.

5. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of claim 1, wherein: the signal transceiving apparatus includes: the system comprises a microcontroller, a patch panel and a serial port module communicated with an upper computer; the microcontroller is communicated with the acceleration sensor through a serial port and acquires the state of the photoelectric switch through an IO port; the adapter plate is used for connecting the sensor unit and the microcontroller; one end of the serial port module communicated with the upper computer is connected with the microcontroller, and the other end of the serial port module is connected with the upper computer through a USB.

6. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of the upper limb movement state of claim 2, wherein: marking information acquisition software is based on QT software development, include: a serial port 1 information frame, a serial port 2 information frame and a motion state indication frame; the serial port 1 information frame displays processing information sent by the signal receiving and sending device after scanning to obtain a serial port 1 and a configured baud rate; the serial port 2 information frame is displayed and sent to the brain signal acquisition device after scanning to obtain a serial port 2 and a configured baud rate; the motion state indication frame comprises a model diagram of a motion platform, and corresponding motion direction indications can appear in the model diagram when the upper limb performs four motions.

7. The marking system for the accurate acquisition of brain signals of the near infrared spectrum of any one of claims 1 to 6, wherein: the motion platform also comprises four foot pads.

Images