CN114587579A

CN114587579A - Magnetic laser endoscope control method based on LSTM neural network predictive control

Info

Publication number: CN114587579A
Application number: CN202210347195.1A
Authority: CN
Inventors: 张琦; 张锡维; 李晓; 李育铭
Original assignee: Guilin University of Electronic Technology
Current assignee: Guilin University of Electronic Technology
Priority date: 2022-04-01
Filing date: 2022-04-01
Publication date: 2022-06-07
Anticipated expiration: 2042-04-01
Also published as: CN114587579B

Abstract

The invention belongs to the field of medical robot control and nonlinear system control, and particularly relates to a magnetic laser endoscope control method based on LSTM neural network predictive control, aiming at solving the problem of optimal control of a magnetic laser endoscope under nonlinear, uncertain links and condition constraints. The method comprises the following implementation steps: initializing a control sequence; carrying out model prediction by using the LSTM neural network after training; collecting the coordinates of laser points emitted by the magnetic laser endoscope in real time, and combining the coordinates with the result of model prediction to realize feedback correction; and the difference between the output of the feedback correction and the set track is input into a rolling optimization link and the optimal control quantity is calculated. The invention improves the robustness and stability of the control of the magnetic laser endoscope and solves the problem of difficult modeling caused by nonlinear and uncertain links of the magnetic laser endoscope.

Description

Magnetic laser endoscope control method based on LSTM neural network predictive control

Technical Field

The invention belongs to the field of medical robot control and nonlinear system control, and particularly relates to a magnetic laser endoscope control method based on LSTM neural network predictive control.

Background

The laser minimally invasive surgery is an excellent surgical method for treating throat cancer, has small damage to organs, small amount of bleeding and high surgical precision, is particularly suitable for operations such as vocal cord tumor excision and the like, and can reduce the recovery time of patients. The magnetic laser endoscope is a new device for performing laser minimally invasive throat surgery, and compared with the traditional surgical instruments, the magnetic laser endoscope is more ergonomic and more flexible, can reduce the operation difficulty of the surgery, can keep a more natural posture of a patient in the surgery, and effectively avoids muscle and ligament injuries caused by long-time stiff holding (Long Ficher, "growing the light inside the body to the patient surgery," Science Robotics, 6, 50, 2021). Compared with other types of laser endoscopes, the magnetic laser endoscope uses a magnetic field as a driving mechanism, has biocompatibility, has small influence on a human body, has enough scanning speed, ensures that the driving current and the driving voltage are within the safety range of the human body, and effectively eliminates potential safety hazards in the operation.

However, the magnetic field generated by the electromagnetic coil of the magnetic laser endoscope has high distribution nonlinearity in space, the stress analysis of the magnet in the magnetic field is complex, and part of link parameters of the magnetic laser endoscope are unknown and are not easy to measure (such as the elastic coefficient of a hard optical fiber, the magnetic moment of a magnetic thin shell and the like), so that the modeling and control of the magnetic laser endoscope are difficult. The existing solution is to use a linear second-order differential equation with unknown parameters as a model to describe the Magnetic Laser endoscope system, then to perform parameter identification on the unknown parameters of the linear second-order differential equation by using least square method according to experimental data (Alperen elementary, et al, ' Design and Control of a Magnetic Laser Scanner for Endoscopic microscopies, IEEE/ASME Transactions on mechanics, 24, 2, 2019), and finally to realize the open-Loop Control or PID Control of the inverse model by combining with the identified model (Hammed Mohammadbagherer, et al, ' Closed-Loop Control of a Magnetic acquired Fiber-Coupled Laser for Computer-Assisted Laser Scanner, ' ICAR, 2019). The method has simple modeling process and easy realization, but the established model can not well describe the nonlinear characteristics of the magnetic laser endoscope, and the used control method can not properly process the interference from the nonlinear link, so that the scanning range of the magnetic laser endoscope is narrow, and the application of the magnetic laser endoscope in practical clinic is limited.

The neural network has strong nonlinear fitting capability and simple use process, and is often used for describing a nonlinear system. The LSTM neural network is a dynamic neural network and can process the modeling problem of a long-time sequence, so that a model can be established for a system with both nonlinear links and dynamic characteristics. The model prediction control is a closed-loop effective optimal control method, comprises three links of rolling optimization, model prediction and feedback correction, can realize stable and accurate optimal control on a nonlinear system by combining a prediction model, and has better containment on errors and mismatch conditions of the prediction model. The control method combining the LSTM neural network and the model prediction is suitable for a system with a nonlinear link and unknown parameters.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a magnetic laser endoscope control method based on LSTM neural network prediction control by combining the advantages of the LSTM neural network and model prediction.

In order to achieve the above object, the present invention provides a magnetic laser endoscope, comprising: the device comprises a shell, hard optical fibers, an electromagnetic coil, a focusing lens and a miniature camera.

The shell is cylindrical and comprises four mounting grooves and a micro camera slot; the mounting groove is cylindrical, is located the end of magnetic laser endoscope, the axis coincidence of two just right mounting grooves, and two adjacent mounting groove axes are perpendicular, and the axis of four mounting grooves all is located the coplanar, and the overall arrangement does benefit to like this the produced magnetic field of solenoid is symmetrical each other in the space. The miniature camera slot is cylindrical and is embedded in the wall of the shell, the opening of the miniature camera slot is positioned at the tail end of the magnetic laser endoscope, and the axis of the miniature camera slot is parallel to the axis of the shell.

The hard optical fiber is used for conducting laser, can support the focusing lens, has enough elasticity and resilience, and ensures that the focusing lens can return to the central axis in the shell under the condition of no magnetic field or magnetic field removal.

Solenoid is cylindrical, contains the magnetic core, can just inlay in the mounting groove, their axis respectively with the axis one-to-one coincidence of mounting groove, the magnetic core is used for increasing the intensity that the coil produced the magnetic field, promotes the scanning ability of magnetism laser endoscope.

The focusing lens comprises a magnetic thin shell and a focusing lens, the magnetic thin shell is used for connecting the focusing lens and the hard optical fiber and increasing the stress of the focusing lens in a magnetic field, and the focusing lens is used for gathering laser and preventing the laser from excessively diverging.

The miniature camera is used for collecting the position of a laser point in real time, is placed in the miniature camera slot and ensures that a visual field capable of covering the scanning range of the magnetic laser endoscope can exist.

The invention relates to a magnetic laser endoscope control method based on LSTM neural network prediction control, which comprises the following steps.

And 0, before prediction control is carried out by using the LSTM neural network, training the LSTM neural network by using a training sample acquired from the magnetic laser endoscope, wherein an input data set B of the training sample consists of continuously-changed magnetic induction intensity signals, and an output data set Y is the laser point projection position coordinate of the magnetic laser endoscope corresponding to the input data set.

In some preferred embodiments, the training algorithm of the LSTM neural network is Adam algorithm, and the structure of the LSTM neural network is: the number of LSTM neurons of each layer corresponding to the structure is as follows: 2-36-20-2.

Step 1, when the control of the magnetic laser endoscope is started, a control quantity sequence U ═ U (1), U (2), U (3).... U (i) ], wherein i is a control time domain length, is initialized, and the current position coordinate O of the laser spot emitted by the magnetic laser endoscope is acquired through the miniature camera.

And 2, sequentially sending the control quantity sequence U into a trained LSTM neural network prediction model for operation to obtain a prediction sequence Om ═ Om (1), Om (2), Om (3).... Om (k) corresponding to the position coordinates of the output laser points of the magnetic laser endoscope, wherein k is the length of a prediction time domain.

And 3, inputting the prediction sequence Om and the position coordinate O into a feedback correction link, and obtaining a corrected feedback quantity sequence Op [ Op (1), Op (2), Op (3).... Op (k) ] through calculation in the feedback correction link.

And 4, at the current time t, sequentially comparing a reference position coordinate sequence Or [ Or (t), Or (t +1).. Or (t + k) ] extracted from the expected track with the corrected feedback quantity sequence Op, sending the result after operation to a rolling optimization link for primary rolling optimization to obtain an optimized control quantity sequence U', writing the expected track as [ Or (1), Or (2), Or (3). once.. Or (n) ], and collecting all laser point reference position coordinate sequences, wherein n is larger than k.

And 5, substituting the once optimized control quantity sequence U 'into [ U' (1), U '(2) and U' (3).... U '(k) ] into the set loss function, judging whether the set loss function meets the set index, if the set loss function does not meet the index, giving the element of U' to U, returning to the step 2 to be executed again, and if the set loss function meets the index, taking the first element U '(1) of U' as the optimal control quantity to control the magnetic laser endoscope.

And 6, updating the reference position coordinate sequence Or of the laser points according to the time sequence, wherein the specific method comprises the following steps: and (2) extracting backwards in the expected track, and if the current time is Or [ Or (t), Or (t +1).. Or (t + k) ], then the next time is Or [ Or (t +1), Or (t +2).. Or (t + k +1) ], and then returning to the step 1 to execute again until the scanning of the expected track is completed.

The invention has the beneficial effects that: aiming at the magnetic laser endoscope, the data modeling method which adopts an LSTM neural network as a frame has dynamic characteristics, can process a signal sequence which depends on for a long time, has better approaching capability for a nonlinear and strongly coupled system, does not need to clear the mechanism of the magnetic laser endoscope, is more suitable for the magnetic laser endoscope system of which part of link parameters are difficult to measure, and can more perfectly describe the system characteristics of the magnetic laser endoscope compared with the existing method. In addition, different from the existing control method, the method adopts model prediction control as a control algorithm of the magnetic laser endoscope, has compatibility on errors and mismatch of models, can give real-time optimal control to a system through rolling optimization, and can realize better stability, stronger robustness and larger scanning range for the magnetic laser endoscope by combining the LSTM neural network prediction model.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural view of a magnetic laser endoscope according to the present invention.

FIG. 2 is a schematic diagram of the LSTM neural network prediction model structure according to the present invention.

FIG. 3 is a schematic diagram of the structure of a single neuron of the LSTM neural network according to the present invention.

FIG. 4 is a schematic diagram of the LSTM neural network predictive model training process of the present invention.

FIG. 5 is a control block diagram of the control method of the magnetic laser endoscope based on the prediction control of the LSTM neural network.

The device comprises a shell 1, a mounting groove 2, an electromagnetic coil 3, a magnetic core 4, a micro camera slot 5, a hard optical fiber 6, a focusing lens 7, a focusing lens 8, a magnetic thin shell 9 and a micro camera 10.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.

As shown in fig. 1, the magnetic laser endoscope of the present invention includes: the device comprises a shell 1, a hard optical fiber 6, an electromagnetic coil 3, a focusing lens 7 and a micro camera 10.

The shell 1 is cylindrical and comprises four mounting grooves 2 and a slot 5 of a miniature camera 10; the mounting groove 2 is cylindrical and is positioned at the tail end of the magnetic laser endoscope, the axes of the two opposite mounting grooves 2 are overlapped, the axes of the two adjacent mounting grooves 2 are vertical, and the axes of the four mounting grooves 2 are positioned on the same plane, so that the arrangement is favorable for mutual symmetry of magnetic fields generated by the electromagnetic coils 3 in space. The slot 5 of the miniature camera 10 is cylindrical and is embedded in the wall of the shell 1, the opening of the slot is positioned at the tail end of the magnetic laser endoscope, and the axis of the slot is parallel to the axis of the shell 1. All the materials of the housing 1 are made of non-magnetic materials, and do not interfere with the magnetic field generated by the electromagnetic coil 3.

The hard optical fiber 6 is used for conducting laser, can support the focusing lens 7, has enough elasticity and resilience, and ensures that the focusing lens 7 can be restored to the central axis inside the shell 1 under the condition of no magnetic field or magnetic field removal.

Solenoid 3 is cylindrical, contains magnetic core 4, can just inlay in mounting groove 2, their axis respectively with the axis one-to-one coincidence of mounting groove 2, magnetic core 4 is used for increasing the intensity that the coil produced magnetic field, promotes the scanning ability of magnetism laser endoscope.

The focusing lens 7 comprises a magnetic thin shell 9 and a focusing lens 8, the magnetic thin shell 9 is used for connecting the focusing lens 8 and the hard optical fiber 6 and increasing the stress of the focusing lens 7 in a magnetic field, and the focusing lens 8 is used for gathering laser and preventing the laser from excessively diverging.

The miniature camera 10 is used for collecting the position of a laser point in real time, is placed in the slot 5 of the miniature camera 10, and ensures that a visual field capable of covering the scanning range of the magnetic laser endoscope can be provided.

And step 0, before the control of the magnetic laser endoscope is executed, the LSTM neural network needs to be trained so as to describe the system characteristics of the magnetic laser endoscope and be used as a prediction model for control.

In this embodiment, the structure of the LSTM neural network prediction model is shown in fig. 2, and is represented as: the number of LSTM neurons of each layer corresponding to the structure is as follows: 2-36-20-2, in fig. 2, the input layers Hx, Hy represent two magnetic induction signal components at the center of the movement plane of the focusing lens 7, the output layers Omx, Omy represent laser point projection position coordinate components predicted by the LSTM neural network, the structure of a single LSTM neuron is shown in fig. 3, wherein the relationship of the variables can be expressed as formula (1):

wherein c (t) is the current time state of the neuron, c (t-1) is the last time state of the neuron, h (t) is the current time output of the neuron, h (t-1) is the last time output of the neuron, x (t) is the current time input of the neuron, f (t), i (t), c' (t) and o (t) are intermediate calculation results in the neuron, Wa, Wb, Wc, Wd, We, Wf, Wg and Wh are weight values connected between the states, output and input, and b1, b2, b3 and b4 are biases of the states, output and input.

As shown in fig. 4, in the present embodiment, the actual operation data set of the magnetic laser endoscope is used as the training sample of the LSTM neural network, the input data set B of the training sample is composed of continuously varying magnetic induction intensity signals, the output data set Y is the laser point projection position coordinates of the magnetic laser endoscope corresponding to the input data set, and the training algorithm is Adam algorithm. The specific training process is as follows: and sequentially inputting the training sample input data set B into the LSTM neural network to obtain an LSTM neural network output data set Ym, comparing the LSTM neural network output data set Ym with an output data set Y of the training sample to obtain an error E, adjusting the weight of the LSTM neural network by an Adam algorithm, and repeating the processes until the error E is small enough. The calculation method of the error E is shown as the formula (2):

in the formula, g is the total number of elements in the output data set of the LSTM neural network and the output data set of the training sample, Y (j) represents the jth element in the output data set of the training sample, Ym (j) represents the jth element in the output data set of the LSTM neural network, Y (j-1) represents the jth element in the output data set of the training sample, and Ym (j-1) represents the jth element in the output data set of the LSTM neural network.

The trained LSTM neural network can be used as a prediction model to implement predictive control, and the control block diagram of the predictive control is shown in fig. 5, and the flow of the control includes the following steps.

Step 1, when the control of the magnetic laser endoscope is started, firstly initializing a control quantity sequence U ═ U (1), U (2), U (3).. U (i), wherein i is a control time domain length, and acquiring a current position coordinate O of a laser spot emitted by the magnetic laser endoscope through the micro camera.

Step 3, inputting the prediction sequence Om and the position coordinate O into a feedback correction link, and obtaining a corrected feedback quantity sequence Op [ Op (1), Op (2), Op (3).... Op (k) ] through calculation in the feedback correction link, wherein the calculation in the feedback correction link can be expressed as formula (3):

Op(l)＝Om(l)+λ(l)(Om(l)-O),l＝1,2......k (3)

in the formula, λ (l) is a scale factor sequence.

Step 4, at the current time t, sequentially comparing a reference position coordinate sequence Or [ Or (t), Or (t +1).. Or (t + k) ] extracted from the expected track with the corrected feedback quantity sequence Op, and sending the result after operation to a rolling optimization link to perform rolling optimization for one time to obtain a control quantity sequence U' after optimization for one time, wherein the expected track is written as [ Or (1), Or (2), Or (3).. Or.. n (n), and is a set of all laser point reference position coordinate sequences, and n > k, and the rolling optimization can be a solution for the following quadratic programming problem with conditional constraints:

in the formula, the matrices A, B are all positive definite matrices, θ is a proportionality coefficient, olast' is the last optimal controlled variable sequence, Ub is the lower limit of the controlled variable U, and Ul is the upper limit of the controlled variable U. The quadratic programming problem solving the equation (4) may use a dual fast algorithm (DFGM).

And 5, substituting the once optimized control quantity sequence U ' ([ U ' ═ U ' (1) (1), U ' (2), U ' (3).... U ' (k)) into the set loss function, judging whether the set loss function meets the set index, if the set loss function does not meet the index, giving the element of U ' to U, returning to the step 2 to be executed again, and if the set loss function meets the index, taking the first element U ' (1) of U ' as the optimal control quantity to control the magnetic laser endoscope. The loss function is:

F＝(Op-Or)A(Op-Or)^T+θ(U'-Ulast')B(U'-Ulast')^T(5) the satisfaction index may be the number of iterations reached or a loss function sufficiently small.

While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.

Claims

1. The control method of the magnetic laser endoscope based on the prediction control of the LSTM neural network comprises the following steps: the device comprises a shell, a hard optical fiber, an electromagnetic coil, a focusing lens and a miniature camera; the shell is cylindrical and comprises four mounting grooves and a micro camera slot; the mounting grooves are cylindrical and are positioned at the tail end of the magnetic laser endoscope, the axes of the two opposite mounting grooves are overlapped, the axes of the two adjacent mounting grooves are vertical, and the axes of the four mounting grooves are positioned on the same plane; the miniature camera slot is cylindrical and is embedded in the wall of the shell, the opening of the miniature camera slot is positioned at the tail end of the magnetic laser endoscope, and the axis of the miniature camera slot is parallel to the axis of the shell; the hard optical fiber is used for conducting laser, can support the focusing lens and has enough elasticity; the electromagnetic coils are cylindrical and comprise magnetic cores, and are placed in the mounting grooves, and the axes of the electromagnetic coils are respectively superposed with the axes of the mounting grooves in a one-to-one correspondence manner; the focusing lens comprises a magnetic thin shell and a focusing lens, and is positioned at the central axis inside the shell when the focusing lens is static; the miniature camera is used for collecting the position of a laser point in real time and is placed in the miniature camera slot; the method is characterized by comprising the following steps:

step 1, when the control of the magnetic laser endoscope is started, firstly initializing a control quantity sequence U ═ U (1), U (2), U (3).. U (i), wherein i is a control time domain length, and acquiring a position coordinate O of a laser spot emitted by the magnetic laser endoscope through the micro camera;

step 2, sequentially sending the control quantity sequence U into a trained LSTM neural network prediction model for operation to obtain a prediction sequence Om ═ Om (1), Om (2), Om (3) · Om (k) corresponding to the position coordinates of the output laser points of the magnetic laser endoscope, wherein k is the length of a prediction time domain;

step 3, inputting the prediction sequence Om and the position coordinate O into a feedback correction link, and obtaining a corrected feedback quantity sequence Op [ Op (1), Op (2), Op (3).... Op (k) ] through calculation in the feedback correction link;

step 4, comparing the laser point reference position coordinate sequence Or at the current time t [ Or (t), Or (t +1).. Or (t + k) ] with the corrected feedback quantity sequence Op in sequence, and then sending the calculated result to a rolling optimization link for primary rolling optimization to obtain an optimized control quantity sequence U';

step 5, substituting the optimized control quantity sequence U 'into [ U' (1), U '(2) and U' (3).... U '(k) ] into the set loss function, judging whether the loss function meets the set index, if the loss function does not meet the index, giving an element of U' to U, returning to the step 2 to execute again, and if the loss function meets the index, taking the first element U '(1) of U' as an optimal control quantity to control the magnetic laser endoscope;

and 6, updating the coordinate sequence Or of the reference position of the laser point, returning to the step 1 and executing again until the scanning of the expected track is finished.

2. The LSTM neural network predictive control-based magnetic laser endoscope control method of claim 1, wherein the training sample of the LSTM neural network predictive model in step 2 is from an actual operation data set of the magnetic laser endoscope, an input data set B of the training sample is composed of continuously varying magnetic induction intensity signals, and an output data set Y is a laser spot projection position coordinate of the magnetic laser endoscope corresponding to the input data set.

3. The LSTM neural network predictive control-based magnetic laser endoscope control method of claim 1, wherein the desired trajectory in step 6 contains all laser point reference position coordinate sequences, Or is a subset of the desired trajectory for any one laser point reference position coordinate sequence, the desired trajectory is represented as [ Or (1), Or (2), Or (3).. Or. (n) ], where n is the total number of elements contained in the whole set, and n > k.

4. The LSTM neural network prediction control-based magnetic laser endoscope control method according to claim 1, wherein the updating of the laser point reference position coordinate sequence Or in step 6 is extracted backward from the desired trajectory in time sequence on the premise of ensuring the sequence length to be constant, such as time t Or [ Or (t), Or (t +1) ·.