Marking system for accurately acquiring brain signals of near infrared spectrum upper limb movement state
Technical Field
The invention relates to a marking system for accurately acquiring near infrared spectrum upper limb movement state brain signals, in particular to a marking system and method for accurately acquiring near infrared spectrum upper limb movement state brain signals, and belongs to the field of brain movement intention identification.
Background
From the 60 s of the 20 th century to the present, the exoskeleton is gradually applied to a plurality of aspects such as rehabilitation walking aid, military assistance and the like as a bionics concept, and provides reliable guarantee for improving the life quality of human beings and enhancing the fighting capacity of soldiers. Most of the existing power-assisted exoskeleton control modes are based on a man-machine force coupling mode, and the defect that the response lag cannot be quickly responded occurs. The motion signal is directly obtained from the brain of the human body through the near infrared spectrum equipment, and the problem can be effectively solved by analyzing data from the brain signal and further prejudging the limb motion.
Relevant researches show that human limb movement is mainly related to a brain movement area, meanwhile, preparation work before movement is related to a part of functional areas of a frontal lobe area of a brain, and when brain signals of the movement area and the frontal lobe area acquired by near infrared spectrum equipment are used for upper limb movement prejudgment, the accurate time of starting movement and stopping movement of a tested person must be accurately known, namely the time of starting movement and stopping movement must be accurately recorded when marking and recording signals. The existing near infrared spectrum technology is mainly applied to the fields of medical nerve rehabilitation, cognitive neurology and the like, and the application scenes are characterized in that the brain is stimulated to generate corresponding nerve reaction in a non-spontaneous mode, the acquired brain signals are generally finished under the guidance of corresponding tasks, and no accurate requirement is provided for the time point when the testee starts to move and stops moving, so that the existing marking systems and methods cannot mark the brain signals in a natural movement state (without any task guidance and completely determined by the testee) firstly, the marking of the brain signals by the existing marking systems does not meet the accurate requirement, and the recorded start-stop marks have a great error with the start-stop time of actual movement.
Disclosure of Invention
The invention aims to solve the problems that the existing marking system cannot mark brain signals in a natural motion state and the recording of the starting time and the ending time of motion is not accurate, and provides a marking system for accurately acquiring the brain signals of an upper limb motion state in a near infrared spectrum. The marking system is combined with the near infrared spectrum brain signal collecting device, and brain signals generated by four actions of pushing, pulling, lifting and releasing of upper limbs in a spontaneous state are accurately calibrated in time starting and stopping mode, so that guiding factors existing in the existing marking system and method are solved, accurate and quick signal marking is achieved, and accuracy of upper limb movement state intention identification based on near infrared spectrum is improved.
In order to achieve the purpose, the invention adopts the technical scheme that:
a marking system for accurate collection of near infrared spectrum upper limb movement state brain signals comprises a movement platform, a signal receiving and transmitting device, marking information collection software and a signal collection device.
The motion platform can realize four actions of pushing, pulling, lifting and releasing; an acceleration sensor and a plurality of photoelectric switches are arranged on the motion platform;
the signal transceiver is used for acquiring signals of the acceleration sensor and the photoelectric switch, processing the signals according to a communication protocol and transmitting the processed signals to the signal acquisition device, wherein the transmitted signals are signals only when the movement starts and stops or signals including the movement process;
the marking information acquisition system is used for serial port information transmission, and because the signal acquisition device cannot directly read the serial port of the signal receiving and transmitting device, the defect is caused by the signal acquisition device, and serial port data of the signal receiving and transmitting device needs to be forwarded to the serial port of the signal acquisition device through marking information acquisition software so as to achieve the purpose of marking data;
the motion platform further comprises: the device comprises a rack, a horizontal movement guide rail, a horizontal movement sliding block, a vertical movement guide rail, a vertical movement sliding block, a horizontal cross beam, a handle and a stop block. The horizontal movement guide rails are respectively installed one above the other at the left side and the right side of the frame, and the total number of the horizontal movement guide rails is four. The two sides of the rack are respectively provided with one vertical rail, the vertical moving guide rails are arranged on horizontal sliding blocks corresponding to the horizontal moving guide rails, the two vertical moving guide rails are connected together through a horizontal cross beam, and the handle is arranged in the middle of the horizontal cross beam;
when the stop block is fixed, the handle is pushed forwards and backwards to realize the pushing and pulling movement of the upper limb in the horizontal direction, and the handle is pushed upwards and downwards to realize the lifting and releasing movement of the upper limb in the vertical direction. The stopper serves to restrict movement in one direction, to restrict vertical movement when performing horizontal movement, and to restrict horizontal movement when performing vertical movement.
When the stop block is not fixed, the upper limb can tilt at any angle without being limited by horizontal movement and vertical movement.
The sensor is installed on the horizontal beam, and the output line of the acceleration sensor is connected with the signal transceiver through the switching terminal and the FPC flat cable after being converged with the output line of the photoelectric switch at the upper end and the lower end in the vertical direction. The acceleration sensor collects acceleration changes at a 50Hz return rate (once in 20 ms) and sends the acceleration changes to the microcontroller, the microcontroller unpacks and reads acceleration information by using a specified protocol after acquiring the information, the movement speeds of different crowds are sampled and analyzed, and a proper threshold value is selected for calibration, so that the movement direction of a tested person is judged once.
The photoelectric switch comprises a front end photoelectric switch, a rear end photoelectric switch, a horizontal photoelectric switch baffle, an upper end photoelectric switch, a lower end photoelectric switch, an upper vertical photoelectric switch baffle and a lower vertical photoelectric switch baffle, wherein the front end photoelectric switch and the rear end photoelectric switch are arranged at two ends of a lower end horizontal movement guide rail, the horizontal photoelectric switch baffle is arranged on a right lower end horizontal movement sliding block, and when the handle is pushed forwards and backwards, the photoelectric switch state changes with the action of the front end photoelectric switch and the rear end photoelectric switch respectively so as to be recorded by the signal transceiving device; in the same way, the upper end photoelectric switch and the lower end photoelectric switch respectively act with the upper baffle plate of the vertical photoelectric switch and the lower baffle plate of the vertical photoelectric switch to record the state of the vertical photoelectric switch during up-and-down movement;
wherein the signal transceiving apparatus comprises: microcontroller, keysets, the serial port module with host computer communication. The microcontroller is communicated with the acceleration sensor through a serial port and collects the state of the photoelectric switch through an IO port. The adapter plate is used for connecting the sensor unit and the microcontroller. One end of the serial port module communicated with the upper computer is connected with the microcontroller, and the other end of the serial port module is connected with the upper computer through a USB.
Wherein, marking information acquisition software development based on QT software includes: serial port 1 information frame, serial port 2 information frame, motion state instruction frame. And the serial port 1 information frame displays the processing information sent by the signal receiving and sending device after scanning to obtain the serial port 1 and the configured baud rate. And the serial port 2 information frame is displayed and sent to the brain signal acquisition device after scanning to obtain the serial port 2 and the configured baud rate. The motion state indication frame comprises a model diagram of a motion platform, and corresponding motion direction indications can appear in the model diagram when the upper limb performs four motions.
Furthermore, the motion platform still includes four foot pads, the foot pad is installed in the below of motion platform's frame and has anti-skidding function, guarantees when carrying out the upper limbs motion that relative slip can not appear in frame and the desktop.
The invention also provides a marking method for accurately acquiring the brain signals of the near infrared spectrum upper limb movement state, which comprises the following steps:
the method comprises the steps that a testee firstly wears a near-infrared brain signal acquisition device to perform autonomous movement under the condition of no task external prompt, the signal transceiver sends accurate time information of movement starting and stopping to upper computer marking information acquisition software according to a set communication protocol, the marking information acquisition software unpacks data according to the communication protocol, on one hand, action starting and stopping information is sent to the brain signal acquisition device through a serial module, starting and stopping information is recorded in a brain signal data stream, time reference is provided for mode classification of later-stage brain signals and upper limb movement prediction based on the brain signals, and on the other hand, movement direction information is reflected on a display interface in real time through an indication window of the marking information acquisition software.
Has the advantages that:
compared with the existing marking system (depending on software), the marking system comprises a micro control system, an acceleration sensor and a plurality of photoelectric switches, brain signals of a tested person in a natural motion state can be accurately recorded, the spontaneity of the tested person in the four motion paradigms is guaranteed, namely, the tested person does not move under the specified time and prompt, the essential reaction of the brain signals is more truly restored, the recorded marking points can accurately reflect the starting and stopping time of the four motions, an important analysis basis is provided for researching the upper limb motion intention identification based on the near infrared spectrum, and the marking system has important significance for accelerating the application of the near infrared spectrum on wearable equipment.
Drawings
FIG. 1 is a schematic view of a marking system according to an embodiment of the present invention;
fig. 2 is a schematic interface diagram of marking information acquisition software according to the present invention.
The system comprises a motion platform 1, a signal transceiver 2, marking information acquisition software 3, a frame 4, a horizontal motion guide rail 5, a horizontal motion sliding block 6, a vertical motion guide rail 7, a vertical motion sliding block 8, an acceleration sensor 9, a front-end photoelectric switch 10, a rear-end photoelectric switch 11, a horizontal photoelectric switch baffle 12, an upper-end photoelectric switch 13, a lower-end photoelectric switch 14, a vertical photoelectric switch upper baffle 15, a vertical photoelectric switch lower baffle 16, a horizontal beam 17, a handle 18, a pad foot 19, a microcontroller 20, a transfer plate 21, a serial port module 22 communicating with an upper computer, a stop block 23, a serial port 1 information frame 24, a serial port 2 information frame 25, and a motion state indication frame 26.
Detailed Description
The technical solution of the present invention is further described below by way of examples with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
A marking system for accurately acquiring brain signals of an upper limb movement state based on near infrared spectrum is shown in figure 1 and comprises a movement platform 1, a signal transceiver 2 and marking information acquisition software 3.
The motion platform 1 includes: the device comprises a frame 4, a horizontal movement guide rail 5, a horizontal movement sliding block 6, a vertical movement guide rail 7, a vertical movement sliding block 8, an acceleration sensor 9, a front end photoelectric switch 10, a rear end photoelectric switch 11, a horizontal photoelectric switch baffle 12, an upper end photoelectric switch 13, a lower end photoelectric switch 14, a vertical photoelectric switch upper baffle 15, a vertical photoelectric switch lower baffle 16, a horizontal beam 17, a handle 18, a pad foot 19, a microcontroller 20, a switching board 21 and a serial port module 22 communicated with an upper computer.
The frame 4 is a cuboid or cube frame constructed by aluminum profiles; the bottom end of the frame 4 is provided with a foot pad 19; four horizontal moving guide rails 5 are symmetrically arranged on the inner side of the frame 4; the two vertical moving guide rails 7 are fixed on the horizontal moving guide rail 5 through the horizontal moving slide block 6; the horizontal beam 17 is fixed between the two vertical moving guide rails 7 and can move along the vertical moving guide rails 7 through the vertical moving slide block 8; the handle 18 is fixed on the horizontal beam 17; a front-end photoelectric switch 10 and a rear-end photoelectric switch 11 are arranged at two ends of the horizontal movement guide rail 5; a horizontal photoelectric switch baffle 12 is arranged on the horizontal moving slide block 6; the horizontal photoelectric switch baffle 12 is positioned at the same horizontal position with the front end photoelectric switch 10 and the rear end photoelectric switch 11; the two ends of the vertical moving guide rail 7 are provided with a vertical photoelectric switch upper baffle piece 15 and a vertical photoelectric switch lower baffle piece 16; the upper end photoelectric switch 13 and the lower end photoelectric switch 14 are fixed on the horizontal beam 17 and correspond to the upper baffle 15 and the lower baffle 16 of the vertical photoelectric switch; the stop block 23, the acceleration sensor 9, the microcontroller 20, the adapter plate 21 and the serial port module 22 communicated with the upper computer are arranged on the rack 4; the pushing and pulling movement of the upper limb in the horizontal direction can be realized by pushing the handle 18 forwards and backwards; the lifting and releasing movement of the upper limbs in the vertical direction can be realized by pushing the handle 18 up and down. To ensure that the horizontal movement is not disturbed when the vertical movement is horizontal, stops may be mounted on the horizontal and vertical rails to limit the movement of the slide.
The state changes of the acceleration sensor 9 and the four photoelectric switches are connected to the port of the microcontroller 20 through the signal transceiver 2 by the adapter plate 21, the acceleration sensor 9 inquires the acceleration change at a return rate of 50Hz (once in 20 ms) and reads the acceleration change through the serial port pin of the microcontroller 20, and the states of the photoelectric switches are read through the IO port of the microcontroller 20. After the acceleration sensor 9 and the photoelectric switch are subjected to state judgment processing, collected data are stored in a passing protocol shown in table 1, first frame data 0x55 of the communication protocol is used for judging whether the received data are correct, if the second frame data have data 0x01, the upper limb moves forwards, the third frame data have data 0x01, the fourth frame data have data 0x01, the upper limb moves upwards, the fifth frame data have data 0x01, the upper limb moves downwards, and the data are sent to an upper computer for processing through a serial port module 22 communicated with the upper computer according to a specified communication protocol.
Table 1 communication protocol of signal transmitting/receiving device 2
The signal acquisition device used in this embodiment does not support direct reading of the serial port information of the signal transceiver 2, so the marking information acquisition software 3 is used to forward data, unpack the data according to the communication protocol after receiving the data of the lower computer signal transceiver 2, and display the received data in the serial port 1 information frame 24. As long as the action state of the upper limb changes, the serial port 2 sends the corresponding mark bit to the brain signal acquisition device, and the marking information is displayed in the serial port 2 information frame 25. The motion state indication box 26 is displayed according to the upper limb motion direction information contained in the protocol, and the three-dimensional model diagram in the motion state indication box corresponds to the actual marking system.
A marking method for accurately acquiring brain signals of an upper limb movement state by using a near infrared spectrum comprises the following specific steps: the tested person wears the near-infrared brain signal acquisition device, and the external world does not have any prompt and is completely independently decided by the tested person to start and stop movement, so that the aim is to be closer to a natural movement state. In this embodiment, the push-pull and lifting actions of the testee are all in the horizontal or vertical direction, and in other embodiments, the push-pull and lifting actions can be any linear motion. After the near-infrared brain signals begin to be collected, a testee holds the handle 18 of the motion platform 1 to autonomously start to move, at the moment, the acceleration signals can increase from zero, meanwhile, the state of one of the photoelectric switches can change, the motion direction of the upper limb is judged under the condition that the acceleration signals and the state of the other photoelectric switch occur simultaneously, the captured motion marking signals can be sent to the signal collecting device through the signal receiving and sending device 2, and the motion marking signals are recorded in the brain signal data stream. After the experiment is finished, according to the recording points of the marking information, performing targeted signal feature extraction and classification processing on the brain signals at the moment and the adjacent moments, establishing machine learning models corresponding to four upper limb motion states, and then performing prejudgment on the subsequent motion states by using the trained models. Furthermore, the device can be used as the control input of the power-assisted exoskeleton by pre-judging the motion state, so that the real exoskeleton wearing device directly controlled by the human brain is realized.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.