CN113647893B - Autonomous navigation system, device and medium for wireless capsule endoscope - Google Patents

Autonomous navigation system, device and medium for wireless capsule endoscope Download PDF

Info

Publication number
CN113647893B
CN113647893B CN202110931078.5A CN202110931078A CN113647893B CN 113647893 B CN113647893 B CN 113647893B CN 202110931078 A CN202110931078 A CN 202110931078A CN 113647893 B CN113647893 B CN 113647893B
Authority
CN
China
Prior art keywords
capsule endoscope
wireless capsule
driver magnet
wireless
magnetic force
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110931078.5A
Other languages
Chinese (zh)
Other versions
CN113647893A (en
Inventor
孟李艾俐
许杨昕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yuanhua Intelligent Technology Shenzhen Co ltd
Original Assignee
Yuanhua Intelligent Technology Shenzhen Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yuanhua Intelligent Technology Shenzhen Co ltd filed Critical Yuanhua Intelligent Technology Shenzhen Co ltd
Priority to CN202110931078.5A priority Critical patent/CN113647893B/en
Publication of CN113647893A publication Critical patent/CN113647893A/en
Application granted granted Critical
Publication of CN113647893B publication Critical patent/CN113647893B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/041Capsule endoscopes for imaging
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147Holding or positioning arrangements
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147Holding or positioning arrangements
    • A61B1/00156Holding or positioning arrangements using self propulsion

Abstract

The embodiment of the application is applicable to the technical field of medical equipment, and provides an autonomous navigation system, equipment and a medium of a wireless capsule endoscope, wherein the system is used for executing the following operations: driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity; automatically advancing the wireless capsule endoscope along the abdominal cavity; when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area; driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area. By adopting the system, autonomous navigation and driving of the movement of the wireless capsule endoscope in the human body can be realized.

Description

Autonomous navigation system, device and medium for wireless capsule endoscope
Technical Field
The embodiment of the application belongs to the technical field of medical equipment, and particularly relates to an autonomous navigation system, equipment and a medium of a wireless capsule endoscope.
Background
The wireless capsule endoscope is a technology for complete digestive tract examination, and has the characteristics of no pain, non-invasiveness and the like. The wireless capsule endoscope can not only inspect abdominal organs such as esophagus, stomach, duodenum, large intestine and the like, but also can completely inspect small intestine, which is an area that can not be inspected by the traditional gastroscope or the traditional enteroscope. The wireless capsule endoscope is a capsule-shaped micro-robot, and is provided with an illumination module, a camera module, an image processing module, a wireless transmission module, and the like. After the wireless capsule endoscope enters the digestive tract of a human body through swallowing of a patient, images can be shot in vivo and transmitted to the outside of the body in real time. A doctor or a computer can make a disease diagnosis based on the received image.
Generally, wireless capsule endoscopes rely primarily on natural peristalsis of the gastrointestinal tract to advance within the body, which takes 10-12 hours to complete a complete examination. Recently, systems have emerged that use an external magnetic field to control a wireless capsule endoscope to perform an examination in the gastrointestinal tract, greatly reducing examination time. However, the wireless capsule endoscope controlled by the external magnetic field needs to manually control the external magnetic field by an operator to examine the gastrointestinal tract, and the whole process is very complicated.
Disclosure of Invention
In view of the above, embodiments of the present application provide an autonomous navigation system, an apparatus, and a medium for a wireless capsule endoscope, so as to implement autonomous navigation and driving of movement of the wireless capsule endoscope in a human body.
A first aspect of an embodiment of the present application provides an autonomous navigation system of a wireless capsule endoscope, the system being configured to perform the following operations:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
A second aspect of an embodiment of the present application provides an autonomous navigation device of a wireless capsule endoscope, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the following method when executing the computer program:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
A third aspect of embodiments of the present application provides an autonomous navigation device of a wireless capsule endoscope, comprising:
the reciprocating rotary motion driving module is used for driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
the capsule automatic propelling module is used for automatically propelling the wireless capsule endoscope to advance along the abdominal cavity;
the expected examination path acquisition module is used for acquiring an expected examination path corresponding to a suspected lesion area when an examination instruction for the suspected lesion area is received;
a desired examination path tracking module for driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements a method of:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
A fifth aspect of embodiments of the present application provides a computer program product, which when run on a terminal device, causes the terminal device to perform the following method:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
Compared with the prior art, the embodiment of the application has the following advantages:
the autonomous navigation system provided by the embodiment of the application can be used for navigation tasks of the wireless capsule endoscope in complex environments such as a human abdominal cavity channel and the like, and comprises automatic propelling examination in an unchecked cavity channel and controlled repeated examination in a preliminarily checked cavity channel. After a patient takes the wireless capsule endoscope, the system can automatically reciprocate to rotate and drive the wireless capsule endoscope and automatically propel the wireless capsule endoscope to advance in an abdominal cavity. After the doctor finds the suspected lesion area, the system can drive the wireless capsule endoscope to track the expected examination path set by the doctor according to the examination path designated by the doctor and reach the designated suspected lesion area. By adopting the autonomous navigation system provided by the embodiment of the application, the diagnosis efficiency can be improved, and the manual operation burden and the mental stress of doctors are greatly reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic view of an application scenario of an autonomous navigation system of a wireless capsule endoscope provided in an embodiment of the present application;
fig. 2 is a schematic diagram of a driver provided in an embodiment of the present application;
FIG. 3 is a schematic view of a wireless capsule endoscope provided by embodiments of the present application;
FIG. 4 is a schematic diagram illustrating an autonomous navigation flow of a wireless capsule endoscope provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of trajectory tracking for a wireless capsule endoscope provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of a driver magnet interacting with a wireless capsule endoscope as provided by an embodiment of the present application;
FIG. 7 is a schematic view of a magnetic field experienced by a wireless capsule endoscope provided by an embodiment of the present application;
FIG. 8 is a schematic diagram illustrating magnetic forces acting on a wireless capsule endoscope according to an embodiment of the present application;
FIG. 9 is a schematic diagram illustrating a variation in magnetic force experienced by a wireless capsule endoscope during a cycle of reciprocal rotation of a driver magnet according to an embodiment of the present application;
FIG. 10 is a schematic diagram of an optimal sensor sub-array provided by an embodiment of the present application;
FIG. 11 is a schematic illustration of the positioning of a wireless capsule endoscope based on an optimized sub-array of sensors as provided by an embodiment of the present application;
FIG. 12 is a schematic flow chart illustrating a trajectory tracking algorithm of a wireless capsule endoscope according to an embodiment of the present disclosure;
FIG. 13 is a schematic view of a wireless capsule endoscope tracking within the abdominal cavity provided by an embodiment of the present application;
FIG. 14 is a schematic diagram illustrating an autonomous navigation flow of another wireless capsule endoscope provided by an embodiment of the present application;
FIG. 15 is a schematic view of an autonomous navigation device of a wireless capsule endoscope provided by an embodiment of the present application;
fig. 16 is a schematic diagram of an autonomous navigation device of a wireless capsule endoscope provided by an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
Referring to fig. 1, a schematic view of an application scenario of an autonomous navigation system of a wireless capsule endoscope provided in an embodiment of the present application is shown. Before the examination begins, the patient 101 is asked to swallow a wireless capsule endoscope 102 and lie on an examination table 103. The examination table 103 is covered with a large magnetic sensor array 104, and the whole magnetic sensor array 104 comprises a plurality of magnetic sensors which can be arranged in a rectangular shape. For example, the magnetic sensor array 104 may include 80 magnetic sensors, and the 80 magnetic sensors may be arranged in an 8 × 10 manner. As shown in fig. 1, a robot arm 105 is mounted beside the examination table 103, and a drive 106 is mounted on an end effector of the robot arm 105. The actuator 106 may be configured as shown in fig. 2 to include an actuator motor 1061 and an actuator magnet 1062, where the actuator magnet 1062 may be a spherical permanent magnet. The wireless capsule endoscope 102 swallowed by the patient 101 may be configured as shown in fig. 3, and includes a magnetic ring 1021, with the magnetic ring 1021 being enclosed by a capsule housing 1022.
In fig. 1, the wireless capsule endoscope 102, the examination couch 103, the magnetic sensor array 104, the robot arm 105, the driver 106, and the like collectively constitute an autonomous navigation system of the wireless capsule endoscope according to the embodiment of the present application. Further, the above-described autonomous navigation system may further include a data processing computer (not shown in fig. 1) connected to the examination couch 103, the sensor array 104, the robot arm 105, the drive 106, and the like. The data processing computer can store an autonomous navigation frame program which comprises a wireless capsule endoscope positioning algorithm, a reciprocating rotation driving algorithm, an automatic propelling algorithm, a track tracking algorithm and essential basic program codes such as display, storage and the like. During the examination process, the autonomous navigation and driving of the wireless capsule endoscope can be realized by executing the various algorithms.
Specifically, as shown in fig. 4, it is a schematic diagram of an autonomous navigation flow of a wireless capsule endoscope provided in an embodiment of the present application. According to the process shown in fig. 4, after the system is started, the system can call an automatic propulsion algorithm, and the wireless capsule endoscope is driven by the driver magnet to automatically propel and perform a preliminary examination in an unknown abdominal cavity by using the visual information and the positioning information. Meanwhile, the system can call a wireless capsule endoscope positioning algorithm to position the wireless capsule endoscope and generate a motion track in the area which is preliminarily checked. During the automatic propelling process of the wireless capsule endoscope, if a doctor or a computer automatic diagnosis system does not find a suspected lesion area, the wireless capsule endoscope is continuously propelled and a motion trail diagram of the wireless capsule endoscope is established. After the doctor or the computer automatic diagnosis system finds the suspected lesion area, the doctor or the computer automatic diagnosis system may issue an examination instruction for the suspected lesion area, and set a desired examination path or a desired examination point of the wireless capsule endoscope. Fig. 5 is a schematic diagram illustrating trajectory tracking of a wireless capsule endoscope according to an embodiment of the present application. The abdominal cavity 501 shown in fig. 5 includes a suspected lesion area 503. At this point, the system may begin executing a trajectory tracking algorithm to move the wireless capsule endoscope 102 to follow the desired examination path 502. During the track-following examination, if the doctor wants to end the track-following examination, the doctor can give an instruction to end the track-following examination in the system, and the system will enter the self-propelled mode again and drive the wireless capsule endoscope to automatically propel. If the physician does not stop tracking the examination, the system will track the wireless capsule endoscope along the desired examination path and keep the wireless capsule endoscope relatively stationary after reaching the desired examination point (suspected diseased region 502). Thereafter, if the physician updates the desired checkpoint or desired examination path again, the system will continue to drive the wireless capsule endoscope to track the physician's updated desired examination path.
It should be noted that, in addition to the automatic propulsion algorithm and the trajectory tracking algorithm, there is no chronological order relationship between the wireless capsule endoscope positioning algorithm and the reciprocating rotation driving algorithm. After the system is started, both the wireless capsule endoscope positioning algorithm and the reciprocating rotation driving algorithm are in operation all the time.
First, various algorithms related to the embodiments of the present application will be described below. It is to be noted that, in the respective embodiments described in the present application, all the belts are provided unless otherwise specified
Figure BDA0003211289970000061
Expressions of symbols each represent a direction of a physical quantity corresponding to the expression. For example,
Figure BDA0003211289970000062
representing the axis of rotation omega of the driver magnetaIn the direction of (a).
(1) Reciprocating rotary drive algorithm
The autonomous navigation system provided by the embodiment of the application can drive the movement of the wireless capsule endoscope in a Reciprocating Rotating Magnetic Actuation (RRMA) mode. In a reciprocating rotary magnetic driving mode, the driver magnet can rotate in a reciprocating rotary interval to generate magnetic force with periodically changed direction. The magnetic force acts on the wireless capsule endoscope, so that the wireless capsule endoscope rotates in different directions in sequence in one period of reciprocating rotation of the driver magnet. For example, during the first half of the reciprocal rotation of the driver magnet, the magnetic force generated by the driver magnet may cause the wireless capsule endoscope to rotate in a clockwise direction; during the second half of the reciprocal rotation of the driver magnet, the magnetic force generated by the driver magnet will cause the wireless capsule endoscope to rotate in a counter-clockwise direction. By driving the wireless capsule endoscope to rotate in a reciprocating manner, the risk of cavity twisting possibly caused in the process of the wireless capsule endoscope moving in the abdominal cavity can be effectively reduced.
FIG. 6 is a schematic diagram of a driver magnet interacting with a wireless capsule endoscope as provided by an embodiment of the present application. As shown in FIG. 6, the wireless capsule endoscope can be driven by an extracorporeal driver magnet, which is a spherical permanent magnet. The desired axis of rotation of the wireless capsule endoscope and the axis of rotation of the driver magnet may be used separately
Figure BDA0003211289970000063
And
Figure BDA0003211289970000064
and (4) showing. When in the initial state
Figure BDA0003211289970000065
And
Figure BDA0003211289970000066
are all aligned with the positive x-axis direction, and can be represented using two angles since the unit vector has only two degrees of freedom
Figure BDA0003211289970000067
And
Figure BDA0003211289970000068
illustratively, θ may be employedczAnd thetacyTo represent
Figure BDA0003211289970000069
Using thetaazAnd thetaayTo represent
Figure BDA00032112899700000610
Namely that
Figure BDA00032112899700000612
Figure BDA00032112899700000613
Wherein, thetaczRepresent
Figure BDA0003211289970000071
Angle of rotation about z-axis, thetacyTo represent
Figure BDA0003211289970000072
Angle of rotation about the y-axis, thetaazWatch holder
Figure BDA0003211289970000073
Angle of rotation about z-axis, thetaayRepresent
Figure BDA0003211289970000074
Angle of rotation about the y-axis.
Order to
Figure BDA0003211289970000075
Aligned in the initial state with the positive z-axis direction, thetaaxRepresenting magnetic moment of driver magnet
Figure BDA0003211289970000076
About the axis of rotation of the driver magnet
Figure BDA0003211289970000077
The angle of rotation. In combination with the above-mentioned angle thetaazAnd thetaayThe magnetic moment of the driver magnet can be calculated
Figure BDA0003211289970000078
In the direction of (a).
Desired axis of rotation for a given wireless capsule endoscope
Figure BDA0003211289970000079
In order to be able to rotate the wireless capsule endoscope at a desired position thereof
Figure BDA00032112899700000710
For generating a rotating magnetic field for the shaft, the axis of rotation of the driver magnet
Figure BDA00032112899700000711
The following formula can be adopted to calculate:
Figure BDA00032112899700000712
wherein r is Pc-PaIndicating the center P of a Wireless Capsule endoscopecWith the centre P of the driver magnetaRelative positional relationship therebetween. As shown in FIG. 7, H is PaIn that
Figure BDA00032112899700000713
Projection of (2); alpha represents r and PaH, the angle between H, i.e., the drive angle of the driver magnet; beta represents Pa、PcAnd H, and an included angle between a plane U formed by the H and the vertical line. Assume center P of wireless capsule endoscopecWith the centre P of the driver magnetaD, the center P of the wireless capsule endoscope can be calculatedcWith the centre P of the driver magnetaRelative position therebetween
Figure BDA00032112899700000714
On the basis, the magnetic field of the wireless capsule endoscope can be calculated according to a magnetic field calculation formula under a magnetic dipole model, wherein the magnetic field is as follows:
Figure BDA00032112899700000715
the embodiment of the application does not assume the real magnetic moment of the wireless capsule endoscope
Figure BDA00032112899700000716
Always in the direction of the applied magnetic field
Figure BDA00032112899700000717
Alignment, which is generally only true in an enlarged container filled with liquid. In the context of the application of the embodiments of the present application, as shown in FIG. 7, the motion of a wireless capsule endoscope is often limited and affected by the walls of the elongated tubular lumen. Thus, the present embodiments assume the true magnetic moment direction of a wireless capsule endoscope
Figure BDA00032112899700000718
By a magnetic field bcAnd the current rotation axis of the wireless capsule endoscope
Figure BDA00032112899700000719
Joint determination. Namely, the real magnetic moment direction of the wireless capsule endoscope is the magnetic field and the magnetic field at the current rotating shaft
Figure BDA00032112899700000720
Unit vector of the difference of the projections on.
In the embodiment of the present application, the magnetic force f applied to the wireless capsule endoscope can be regarded as a resultant force of a plurality of components. For example, the magnetic force f experienced by a wireless capsule endoscope can be viewed as three components f as shown in FIG. 8ρ、flAnd frThe resultant force of (a). I.e. the first component fρIs along the desired axis of rotation of the wireless capsule endoscope
Figure BDA00032112899700000721
A second component f of the propulsionlIs a lateral force perpendicular to the U-plane, the third component frIs the remaining residual force i.e. fr=f-fρ-fl
FIG. 9 shows the variation in magnetic force experienced by the wireless capsule endoscope as the driver magnet rotates one cycle. As can be seen from FIG. 9, the propulsive force fρThe magnitude of (c) is hardly changed with the change of the rotation angle, and the value is very stable. When the rotation angle is about 180 degrees, the lateral force flAlmost 0, and the remaining force frA maximum value is reached. The change in the magnetic force f is also the slowest when the angle of rotation is around 180 °. Therefore, the reciprocating rotation interval can be designed by taking 180 degrees as the center. Let thetaarFor the reciprocating rotation angle of the driver magnet, the reciprocating rotation interval of the driver magnet can be expressed as: thetaax∈[180°-θar,180°+θar]。
Generally, θarSmall and the frequency of the reciprocating rotation of the driver magnet is relatively fast, and the propulsive force fρIt is the focus of the embodiments of the present application, so the embodiments of the present application may use the magnetic force at the rotation angle of 180 ° to represent the magnetic force in one reciprocating rotation period.
Through simulation, different reciprocating rotation angles can cause different cavity twisting risks and propelling efficiencies. With the increase of the reciprocating rotation angle, the amplitude of the cavity channel torsion or deformation is larger, but the strength of the cavity channel opening towards the side direction is larger, so that the forward propulsion of the wireless capsule endoscope is facilitated. Therefore, the embodiment of the application can select a balance point between the cavity torsion risk and the propulsion efficiency, and determine the optimal reciprocating rotation angle to be 90 degrees.
(2) Wireless capsule endoscope positioning algorithm
The embodiments of the present application provide a large scale sensor array, sufficient to cover the entire abdominal area of a patient, comprising a plurality of sensors. For example, the sensor array may consist of 80 sensors arranged in an 8 × 10 matrix. Since too many sensors are involved in the positioning calculation, the sensor reading takes a lot of time, resulting in a very low positioning frequency and a significant reduction in positioning accuracy. To solve this problem, as shown in fig. 10, the embodiment of the present application provides an adaptive sensor sub-array positioning algorithm. The algorithm only needs to activate a plurality of sensors which are relatively close to the wireless capsule endoscope at each time, and the sensors are combined into a sensor sub-array in an optimal arrangement mode. Next, as shown in fig. 11, when the wireless capsule endoscope is positioned, the activated best sensor sub-array may be determined according to the positioning result of the last wireless capsule endoscope, the superimposed magnetic field is measured by using the sensors, the 6-dimensional pose of the wireless capsule endoscope is solved from the magnetic field data by using an improved multi-magnetic object tracking algorithm (IMOT) which integrates a multi-magnetic object tracking algorithm (MOT), normal vector fitting, bezz curve fitting, and a spherical linear interpolation algorithm, and the movement speed of the wireless capsule endoscope is estimated. And after the latest wireless capsule endoscope pose is obtained, determining the sensor subarray required to be activated for next positioning.
It should be noted that the 6-dimensional pose of the wireless capsule endoscope can be divided into 3-dimensional position information and 3-dimensional rotation information. In the embodiment of the application, the magnetic field data can be used for solving the 3-dimensional position information of the wireless capsule endoscope, and the 3-dimensional rotation information can be read by an inertial sensor built in the wireless capsule endoscope.
Therefore, the wireless capsule endoscope has higher positioning precision and higher positioning frequency while the working space is expanded.
(3) Self-propelled algorithm
It has been found through extensive experimentation that lumens in different shapes (e.g., straight and curved intestines) and different environmental resistances (e.g., differences in intestinal secretions and intestinal surface structure resulting in different resistances) can greatly affect the propulsion efficiency of a wireless capsule endoscope. Therefore, the embodiment of the application provides a method for adaptively adjusting the driving propelling force according to environmental conditions, so that the wireless capsule endoscope can move in different environments more efficiently.
It is clear that the shape of the lumen channel can be relatively easily determined from the visual information. In a straight tract, the far center of the tract is close to the image center. In curved channels, the far center of the channel is far from the center of the image. Therefore, the orientation of the magnetic force applied to the wireless capsule endoscope can be changed according to the relationship between the center of the deep and far position of the cavity and the image center, so that the head of the wireless capsule endoscope always faces the deep and far position of the cavity, and the automatic propelling efficiency is improved.
On the other hand, the change of the environmental resistance in the cavity is difficult to obtain through visual information, and the movement speed of the wireless capsule endoscope is greatly reduced due to the larger resistance. Therefore, the embodiment of the application can judge the resistance applied to the wireless capsule endoscope by means of the speed change of the wireless capsule endoscope, and further adaptively adjust the magnetic force applied to the wireless capsule endoscope according to the movement speed of the wireless capsule endoscope, so that the wireless capsule endoscope can be driven efficiently under different environmental resistances.
In the embodiment of the application, the adjustment of the magnetic force applied to the wireless capsule endoscope can be realized by reducing the distance between the driver magnet and the wireless capsule endoscope, and can also be realized by changing the driving angle (namely the angle alpha) in the configuration information of the driver pose.
In the process of driving the wireless capsule endoscope to move, the positioning data of the wireless capsule endoscope can be recorded and stored. Then, a Gaussian Mixture Model (GMM) is combined with an expectation-maximization (EM) algorithm to cluster each position point in the motion process into a plurality of points, and a cubic spline interpolation algorithm (cubic spline interpolation) is used for generating a smooth curve for representing the track of the preliminarily checked cavity area for use in track tracking.
(4) Trajectory tracking algorithm
The autonomous navigation system provided by the embodiment of the application can realize 5D control on the wireless capsule endoscope, and comprises 3D magnetic control and 2D wireless capsule endoscope orientation control. In a tubular environment, the current axis of rotation of a wireless capsule endoscope may not always be aligned with the desired axis of rotation. If the included angle between the current rotating shaft of the wireless capsule endoscope and the expected rotating shaft is too large, the wireless capsule endoscope cannot normally rotate under the action of the rotating magnetic field. To address this problem, embodiments of the present application use a spherical linear interpolation algorithm (spherical linear interpolation) to determine the next desired rotational direction of the wireless capsule endoscope
Figure BDA0003211289970000101
In general, the desired magnetic force f experienced on a given wireless capsule endoscopedAnd desired direction of rotation of a wireless capsule endoscope
Figure BDA0003211289970000102
The configuration information (d, α, β) of the driver magnets can be obtained by solving the following equations:
Figure BDA0003211289970000103
provided that the amount of friction added to the wireless capsule endoscope is constant and always opposite to the speed direction of the wireless capsule endoscope, the friction on the wireless capsule endoscope can be expressed as:
Figure BDA0003211289970000104
wherein f isfricIs intestinal wall resistance, rho, suffered by the wireless capsule endoscopefricThe coefficient of resistance is expressed as a function of,
Figure BDA0003211289970000105
indicating the current speed of the wireless capsule endoscope,
Figure BDA0003211289970000106
indicating the magnitude of the current speed of the wireless capsule endoscope. Therefore, the temperature of the molten metal is controlled,
Figure BDA0003211289970000107
indicating the direction of the current speed of the wireless capsule endoscope.
On the other hand, the coefficient R can be used for representing the change of friction force caused by different cavity positions and cavity peristalsis, and f is useddistThe noise which is smaller in the cavity and uncertain in direction is shown, the total acting force f of the cavity to the wireless capsule endoscope is shownenvCan be expressed as:
fenv=R·ffric+fdist
according to the information, a dynamic model in the movement process of the wireless capsule endoscope can be constructed:
Figure BDA0003211289970000108
wherein m iscRepresenting the mass of the wireless capsule endoscope, fgIn order to be the gravity force,
Figure BDA0003211289970000109
representing the acceleration in the motion of the wireless capsule endoscope.
Fig. 12 is a schematic flowchart illustrating a trajectory tracking algorithm of a wireless capsule endoscope according to an embodiment of the present application. Before trajectory tracking, the trajectory generator may generate the desired trajectory from the user (physician) -specified keypoints. Such asFig. 13 is a schematic view illustrating a track tracing of a wireless capsule endoscope in an abdominal cavity according to an embodiment of the present application. The curve generated by connecting the key points in fig. 13 is the desired trajectory. In the trajectory tracking process, assume that the control frequency of the controller is fcAt the current position P of a given wireless capsule endoscopecIn the case of (1), the wireless capsule endoscope is combined with the kinetic model to obtain the desired position P at the next N timesd,n+iAnd desired speed
Figure BDA0003211289970000111
This can be obtained by solving the following equation:
Figure BDA0003211289970000112
setting the current position and current speed of the wireless capsule endoscope to a state:
Figure BDA0003211289970000113
and sets the future expected state within the entire prediction window to Xd,n,...,Xd,n+NWherein:
Figure BDA0003211289970000114
the function for predicting motion of the wireless capsule endoscope within the prediction window may be defined as:
Figure BDA0003211289970000115
in the model predictive controller, an uncertainty parameter can be set, i.e. the coefficient R in the function phi is assumed to be varied. For example, the coefficient R may be a piecewise function, with M possibilities. Taking into account R in the most critical first prediction, within a prediction window of length NUncertainty, M kinds of probability of R can be put into the optimization function for calculation, that is, R in the function phi is set as a variable parameter, and finally the magnetic force sequence of the optimized output will adapt to the situation of different R values. Expected sequence of magnetic forces f within a prediction windowd,n,...,fd,n+N-1This can be obtained by solving the following equation:
Figure BDA0003211289970000116
wherein, WN,WxAnd WfAnd respectively representing the end point state error of the prediction window, the process error in the prediction window and the weight of the output magnetic force change in the prediction window.
As shown in FIG. 12, after the controller outputs the desired magnetic force sequence, a first desired magnetic force f may be selectedd,nThe system can enable the wireless capsule endoscope to automatically track the desired trajectory and reach the target location by solving the configuration information (d, α, β) of the driver magnets by the magnetic force generator and then starting a new round of optimization calculation.
It should be noted that any type of controller may be used in the trajectory tracking algorithm, such as a robust multi-stage model predictive controller (robust multi-stage MPC) or the like, which is not limited in this embodiment of the present invention.
The reciprocating rotation driving algorithm, the wireless capsule endoscope positioning algorithm, the automatic propelling algorithm and the trajectory tracking algorithm jointly form a framework of the autonomous navigation system in the embodiment of the application. In the process of applying the autonomous navigation system, the reciprocating rotation driving algorithm and the wireless capsule endoscope positioning algorithm are always in a running state. In the process, the system can continuously update the latest pose of the wireless capsule endoscope, and the driver magnet can also accurately drive the wireless capsule endoscope. The self-propulsion algorithm is used for a self-propulsion mode of the wireless capsule endoscope, and the trajectory tracking algorithm is used for a trajectory tracking mode of the wireless capsule endoscope. The user (physician) can switch between the self-propulsion and trajectory tracking modes by commanding the control system. By adopting the autonomous navigation system provided by the embodiment of the application, the accuracy of disease diagnosis is improved, and the burden and the mental stress of manual operation of doctors are reduced.
The embodiment of the application imitates the concepts of simultaneous localization and mapping (SLAM), Path Planning (Path Planning) and Navigation (Navigation) in the field of mobile robots, and summarizes the related concepts corresponding to the active wireless capsule endoscope. The process of automatically advancing the wireless capsule endoscope in an unknown cavity or organ by the system and generating the track of the motion path can be summarized as "Automatic Propulsion" (SLAM stage) corresponding to the mobile robot. The process of performing an iterative motion check of the suspected lesion area Following the desired Trajectory/path in the tract or organ on which the Trajectory has been established can be summarized as "Trajectory tracking", which corresponds to the path planning phase of the mobile robot in a known map. The whole framework process of combining "self-propulsion" and "trajectory tracking" to automatically propel and establish a trajectory in an unknown lumen or organ and then examine it on command in a known trajectory is called "Autonomous Navigation" (Autonomous Navigation), and also corresponds to an overall Navigation framework of exploring, mapping, and then planning a path in a known environment in a mobile robot. The concept of autonomous navigation for magnetically controlled wireless capsule endoscopes in abdominal cavity is proposed for the first time in the industry.
The technical solution of the present application is described below by specific examples.
Referring to fig. 14, a schematic diagram illustrating an autonomous navigation flow of another wireless capsule endoscope provided in the embodiment of the present application is shown, which may specifically include the following steps:
s1401, driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity.
It should be noted that, when the autonomous navigation system is applied to execute the step S1401, the reciprocating rotation driving algorithm, the wireless capsule endoscope positioning algorithm, and the automatic propulsion algorithm in the system are in an operating state. Wherein, the reciprocating rotation driving algorithm is used for controlling the wireless capsule endoscope to do reciprocating rotation motion in the abdominal cavity; the wireless capsule endoscope positioning algorithm is used for positioning the wireless capsule endoscope in real time and determining the pose of the wireless capsule endoscope; the self-propelling algorithm is used for driving the wireless capsule endoscope to automatically propel in the abdominal cavity.
When the wireless capsule endoscope is controlled to do reciprocating rotation motion in the abdominal cavity, the reciprocating rotation section of the driver magnet can be determined firstly, and the reciprocating rotation section is the angle range of the reciprocating rotation of the driver magnet. Then, the wireless capsule endoscope can be driven to move in a reciprocating rotation mode in the abdominal cavity channel by controlling the driver magnet to rotate in a reciprocating manner in the reciprocating rotation section. The wireless capsule endoscope moves in the abdominal cavity in a reciprocating rotation mode, which means that the rotation directions of the wireless capsule endoscope in the first half period and the second half period are opposite in one reciprocating rotation period of the driver magnet. For example, the wireless capsule endoscope rotates clockwise about the rotation axis in the first half cycle and counterclockwise about the rotation axis in the second half cycle.
In the embodiment of the present application, in order to determine the reciprocating rotation interval of the driver magnet, the reciprocating rotation angle of the driver magnet may be first determined. As can be seen from the description of the reciprocating rotation driving algorithm in conjunction with the foregoing embodiment, the optimal reciprocating rotation angle that can effectively balance the risk of lumen twisting and the propulsion efficiency is 90 °. Of course, according to actual needs, other angle values may be selected as the reciprocating rotation angle, which is not limited in the embodiment of the present application.
Then, a reciprocating rotation section of the driver magnet, that is, an angle section formed by reciprocating and rotating the determined reciprocating rotation angle around the target rotation angle, may be determined according to the reciprocating rotation angle.
The change in magnetic force is slowest as the driver magnet rotates around 180 deg., and at this angle the lateral force is almost 0, but the remaining force reaches a maximum. Therefore, the target rotation angle may be 180 °. That is, the reciprocating rotation interval of the driver magnet can be expressed as: thetaax∈[180°-θar,180°+θar]Wherein, thetaarThe angle of rotation of the driver magnet to and fro.
S1402, the wireless capsule endoscope is automatically pushed to advance along the abdominal cavity.
It is noted that the advancement of the self-propelled wireless capsule endoscope within the abdominal cavity may be accomplished by executing a self-propelled algorithm. The reciprocating rotary drive algorithm and the wireless capsule endoscope positioning algorithm are still in operation while the self-propulsion algorithm is executed.
In the embodiment of the application, the propelling efficiency of the wireless capsule endoscope is greatly influenced by the difference of the cavity channel shape and the cavity channel environment. Therefore, the propulsive magnetic force exerted on the wireless capsule endoscope can be adjusted according to the environmental conditions in which the wireless capsule endoscope is located.
In a specific implementation, the cavity environment where the wireless capsule endoscope is currently located can be determined firstly; the propulsion magnetic force exerted by the driver magnet on the wireless capsule endoscope is adjusted according to the lumen environment. The channel environment may include a straight channel or a curved channel, etc. Illustratively, when the wireless capsule endoscope is in a curved lumen, the propelling magnetic force exerted on the wireless capsule endoscope can be increased to enable better propulsion.
In general, the presence of a wireless capsule endoscope in a straight or curved lumen can be relatively easily determined from visual information. Therefore, when the cavity environment where the wireless capsule endoscope is currently located is determined, the wireless capsule endoscope can be controlled to acquire the images of the abdominal cavity, and then the cavity environment where the wireless capsule endoscope is currently located is determined according to the images.
Generally, in a straight lumen, the center of the deep distal portion of the lumen is close to the center of the image; in curved channels, the far center of the channel is far from the center of the image. Therefore, the direction of the propelling magnetic force can be adjusted according to the cavity environment, so that the head of the wireless capsule endoscope faces to the deep and far part of the abdominal cavity. Namely, the direction of the propelling magnetic force is adjusted to make the head of the wireless capsule endoscope approach to the center of the image in the acquired image as much as possible.
On the other hand, the change of the environmental resistance in the cavity is difficult to obtain through visual information, and the movement speed of the wireless capsule endoscope is greatly reduced due to the larger resistance. Therefore, the resistance to the wireless capsule endoscope can be judged by the change of the movement speed of the wireless capsule endoscope.
In particular implementations, a velocity of movement of a wireless capsule endoscope in a lumen environment can be determined. If the movement speed of the wireless capsule endoscope in the cavity environment is smaller than the preset speed threshold, the propelling magnetic force can be increased, so that the movement speed of the wireless capsule endoscope can reach the preset speed threshold under the action of the increased propelling magnetic force, and the propelling efficiency is ensured.
In the embodiment of the application, the increase of the propelling magnetic force can be realized by controlling and reducing the distance between the driver magnet and the wireless capsule endoscope and also can be realized by adjusting the driving angle of the driver magnet.
In the embodiment of the application, during the process of automatically propelling the wireless capsule endoscope, the system can also determine a plurality of position points which are passed by the wireless capsule endoscope during the process of moving in the abdominal cavity, and generate the motion track of the wireless capsule endoscope according to the plurality of position points.
When a plurality of position points passed by the wireless capsule endoscope in the movement process are determined, a wireless capsule endoscope positioning algorithm needs to be called to position the wireless capsule endoscope.
In the embodiment of the application, a plurality of magnetic sensors can be activated from the magnetic sensor array at any position where the wireless capsule endoscope passes in the process of moving in the abdominal cavity, and the plurality of magnetic sensors form an optimal magnetic sensor sub-array. The optimal magnetic sensor sub-array may be a magnetic sensor sub-array with optimal positioning accuracy, which is composed of a fixed number of magnetic sensors arranged in a preset arrangement manner. Then, the wireless capsule endoscope can be positioned based on the optimal magnetic sensor sub-arrays, and a position point corresponding to the current position of the wireless capsule endoscope can be obtained.
In a specific implementation, the optimal magnetic sensor sub-array can be used for measuring the magnetic field generated by the driver magnet to obtain magnetic field data; then determining the pose of the wireless capsule endoscope at the current position based on the measured magnetic field data; and determining a position point corresponding to the current position of the wireless capsule endoscope according to the pose.
According to the plurality of position points, the motion trail of the wireless capsule endoscope can be generated.
In the embodiment of the application, when the motion track of the wireless capsule endoscope is generated according to the plurality of position points, the plurality of position points can be clustered to obtain a plurality of positioning points of the wireless capsule endoscope; and then generating a motion track of the wireless capsule endoscope according to the positioning points. For example, a smooth curve, namely the motion track of the wireless capsule endoscope in the abdominal cavity can be drawn based on a plurality of positioning points by adopting a cubic spline interpolation algorithm and the like.
S1403, when an inspection instruction for a suspected lesion area is received, a desired inspection path corresponding to the suspected lesion area is acquired.
In the embodiment of the present application, the examination instruction for the suspected lesion area may be issued by a doctor or a computer automatic diagnosis system. For example, after the doctor finds a suspected lesion area, the doctor may issue an examination instruction to instruct the wireless capsule endoscope to perform repeated examinations on the suspected lesion area.
After receiving an inspection instruction for a suspected lesion area, the system may acquire a desired inspection path corresponding to the suspected lesion area. The desired examination path may be a portion of a motion trajectory generated by execution of a wireless capsule endoscopic positioning algorithm during the self-propulsion process.
S1404, driving the wireless capsule endoscope to move to the suspected lesion area along the expected examination path.
In an embodiment of the present application, when the autonomous navigation system receives an inspection instruction for a suspected lesion area, the autonomous navigation system may switch from the self-propelled mode to the trajectory tracking mode. After the mode switch is completed, the autopilot algorithm stops running and the trajectory tracking algorithm begins running.
It should be noted that, in the trajectory tracking mode of the autonomous navigation system, the system not only runs the trajectory tracking algorithm, but also continues to run the reciprocating rotation driving algorithm and the wireless capsule endoscope positioning algorithm.
In an embodiment of the present application, when the trajectory tracking algorithm is run to drive the wireless capsule endoscope along the desired examination path to the suspected lesion area, the desired magnetic force generated by the driver magnet corresponding to the desired examination path may be first determined. The desired magnetic force is the propulsion force required to act on and drive the wireless capsule endoscope along the desired examination path.
In a particular implementation, to determine the desired magnetic force, a kinetic model of the wireless capsule endoscope during motion may first be constructed. Then, a current location of the wireless capsule endoscope may be determined, and a plurality of desired locations of the wireless capsule endoscope and a desired velocity at each desired location may be determined based on the current location of the wireless capsule endoscope. On the basis of the dynamic model, the system can calculate the magnetic force sequence of the wireless endoscope at a plurality of expected positions according to the plurality of expected positions and the expected speed at each expected position. The system may take the first magnetic force in the sequence of magnetic forces as the desired magnetic force.
As described above with respect to the trajectory tracking algorithm, given a desired magnetic force and a desired rotational direction of the wireless capsule endoscope during motion, configuration information for the driver magnets may be determined by solving equations, which may include: the distance between the center of the wireless capsule endoscope and the center of the driver magnet, the driving angle of the driver magnet, and the included angle between the specific plane and the vertical line. Wherein, the specific plane is a plane formed by the center of the wireless capsule endoscope, the center of the driver magnet and the projection point of the center of the driver magnet on the expected rotation axis of the wireless capsule endoscope.
After determining the configuration information, the driver magnet may be configured according to the configuration information and controlled to generate a corresponding desired magnetic force to drive the wireless capsule endoscope along the desired examination path.
It should be noted that when the wireless capsule endoscope is moved to the next desired position, the above steps may be repeated to recalculate the desired magnetic force of the driver magnet. By cycling through the above steps, the system can automatically enable the wireless capsule endoscope to track the expected examination path and reach the suspected lesion area.
When the wireless capsule endoscope moves to the suspected lesion area, the system can control the wireless capsule endoscope to check the suspected lesion area.
Since the execution processes in the steps S1401 to S1404 are implemented on the basis of running the corresponding algorithms, the running processes of the algorithms are similar to those described in the foregoing embodiments for each algorithm, and the description is simple, and the relevant points can be referred to each other.
It should be noted that, the sequence numbers of the steps in the foregoing embodiments do not mean the execution sequence, and the execution sequence of each process should be determined by the function and the inherent logic of the process, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Referring to fig. 15, a schematic diagram of an autonomous navigation apparatus of a wireless capsule endoscope provided in an embodiment of the present application is shown, and specifically may include a reciprocating rotary motion driving module 1501, a capsule automatic propulsion module 1502, a desired examination path acquisition module 1503, and a desired examination path tracking module 1504, where:
the reciprocating rotary motion driving module 1501 is used for driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
a capsule self-propulsion module 1502 for automatically propelling the wireless capsule endoscope along the abdominal cavity;
a desired examination path obtaining module 1503, configured to, when receiving an examination instruction for a suspected lesion area, obtain a desired examination path corresponding to the suspected lesion area;
a desired examination path tracking module 1504 for driving the wireless capsule endoscope along the desired examination path to the suspected diseased region.
In the embodiment of the present application, the reciprocating rotary motion driving module 1501 is specifically configured to: determining a reciprocating rotation interval of the driver magnet; and controlling the driver magnet to rotate in a reciprocating manner in the reciprocating rotation interval so as to drive the wireless capsule endoscope to move in a reciprocating manner in the abdominal cavity.
In the present embodiment, the reciprocating rotary motion driver module 1501 is further configured to: determining a reciprocating rotation angle of the driver magnet; and determining a reciprocating rotation interval of the driver magnet according to the reciprocating rotation angle, wherein the reciprocating rotation interval is an angle interval formed by rotating the reciprocating rotation angle in a reciprocating manner by taking the target rotation angle as a center.
In the embodiment of the present application, the target rotation angle may be 180 degrees.
In the present embodiment, the capsule automatic propelling module 1502 is specifically configured to: determining the current cavity environment of the wireless capsule endoscope; adjusting a propulsion magnetic force exerted by a driver magnet on the wireless capsule endoscope according to the lumen environment.
In the present embodiment, the capsule self-propelling module 1502 is further configured to: controlling the wireless capsule endoscope to acquire images of the abdominal cavity; and determining the current cavity environment of the wireless capsule endoscope according to the image.
In the present embodiment, the capsule self-propelling module 1502 is further configured to: according to the cavity environment, the direction of the propelling magnetic force is adjusted, so that the head of the wireless capsule endoscope faces to the deep and far part of the abdominal cavity.
In the present embodiment, the capsule self-propelling module 1502 is further configured to: determining a speed of movement of the wireless capsule endoscope in the lumen environment.
In the present embodiment, the capsule self-propelling module 1502 is further configured to: and if the movement speed of the wireless capsule endoscope in the cavity environment is smaller than a preset speed threshold, increasing the propelling magnetic force so that the movement speed of the wireless capsule endoscope reaches the preset speed threshold under the action of the propelling magnetic force.
In the present embodiment, the capsule self-propelling module 1502 is further configured to: controlling to decrease a distance between the driver magnet and the wireless capsule endoscope; alternatively, the drive angle of the driver magnet is adjusted.
In an embodiment of the present application, the apparatus further includes a positioning module, where the positioning module is configured to: determining a plurality of position points passed by the wireless capsule endoscope in the process of moving in the abdominal cavity; and generating a motion track of the wireless capsule endoscope according to the plurality of position points.
In an embodiment of the present application, the positioning module is specifically configured to: the wireless capsule endoscope comprises a wireless capsule endoscope body, a plurality of magnetic sensors, a plurality of sensor sub-arrays and a plurality of sensor sub-arrays, wherein the wireless capsule endoscope body is provided with a plurality of magnetic sensors; and positioning the wireless capsule endoscope based on the optimal magnetic sensor subarray to obtain a position point corresponding to the current position of the wireless capsule endoscope.
In an embodiment of the present application, the positioning module is further configured to: measuring the magnetic field generated by the driver magnet by adopting the optimal magnetic sensor sub-array to obtain magnetic field data; determining a pose of the wireless capsule endoscope at a current location based on the magnetic field data; and determining a position point corresponding to the current position of the wireless capsule endoscope according to the pose.
In an embodiment of the present application, the positioning module is further configured to: clustering the plurality of position points to obtain a plurality of positioning points of the wireless capsule endoscope; and generating a motion track of the wireless capsule endoscope according to the positioning points.
In an embodiment of the present application, the expected inspection path tracing module 1504 is specifically configured to: determining a desired magnetic force produced by a driver magnet corresponding to the desired inspection path; determining configuration information for the driver magnet based on the desired magnetic force and a desired rotational direction of the wireless capsule endoscope during movement, the configuration information comprising: the distance between the center of the wireless capsule endoscope and the center of the driver magnet, the driving angle of the driver magnet, and the included angle between the specific plane and the vertical line; wherein the specific plane is a plane formed by the center of the wireless capsule endoscope, the center of the driver magnet and the projection point of the center of the driver magnet on the expected rotation axis of the wireless capsule endoscope; and configuring the driver magnet according to the configuration information and controlling the driver magnet to generate the expected magnetic force so as to drive the wireless capsule endoscope to move to the suspected lesion area along the expected examination path.
In an embodiment of the present application, the expected inspection path tracing module 1504 is further configured to: constructing a dynamic model of the wireless capsule endoscope in the motion process; determining a current position of the wireless capsule endoscope; determining a plurality of desired positions of the wireless capsule endoscope and a desired velocity at each desired position according to a current position of the wireless capsule endoscope; calculating a sequence of magnetic forces of the wireless endoscope at the plurality of desired locations based on the kinetic model according to the plurality of desired locations and a desired velocity at each desired location; taking a first magnetic force in the sequence of magnetic forces as the desired magnetic force.
In an embodiment of the present application, the expected inspection path tracing module 1504 is further configured to: recalculating the desired magnetic force of the driver magnet when the wireless capsule endoscope moves to the next desired position.
In an embodiment of the present application, the apparatus further includes an examination module configured to control the wireless capsule endoscope to examine the suspected lesion area.
For the apparatus embodiment, since it is substantially similar to the method embodiment, it is described relatively simply, and reference may be made to the description of the method embodiment section for relevant points.
Referring to fig. 16, a schematic diagram of an autonomous navigation device of a wireless capsule endoscope provided by an embodiment of the present application is shown. As shown in fig. 16, the autonomous navigation apparatus 1600 of the wireless capsule endoscope of the present embodiment includes: a processor 1610, a memory 1620, and computer programs 1621 stored in the memory 1620 and executable on the processor 1610. The processor 1610, when executing the computer program 1621, implements the steps in the embodiments of the autonomous navigation flow of the wireless capsule endoscope described above, such as steps S1401 to S1403 shown in fig. 14. Alternatively, the processor 1610, when executing the computer program 1621, implements the functions of each module/unit in each device embodiment described above, for example, the functions of the modules 1501 to 1503 shown in fig. 15.
Illustratively, the computer program 1621 may be partitioned into one or more modules/units that are stored in the memory 1620 and executed by the processor 1610 to implement the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions that may be used to describe the execution of the computer program 1621 in the autonomous navigation device 1600 of the wireless capsule endoscope. For example, the computer program 1621 may be divided into a reciprocating rotary motion drive module, a capsule automatic advancement module, a desired examination path acquisition module, and a desired examination path tracking module, each of which functions as follows:
the reciprocating rotary motion driving module is used for driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
the capsule automatic propelling module is used for automatically propelling the wireless capsule endoscope to advance along the abdominal cavity;
the expected examination path acquisition module is used for acquiring an expected examination path corresponding to a suspected lesion area when an examination instruction for the suspected lesion area is received;
a desired examination path tracking module for driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
The autonomous navigation device 1600 of the wireless capsule endoscope may include, but is not limited to, a processor 1610, a memory 1620. Those skilled in the art will appreciate that fig. 16 is merely an example of the autonomous navigation device 1600 of the wireless capsule endoscope and does not constitute a limitation of the autonomous navigation device 1600 of the wireless capsule endoscope, and may include more or fewer components than illustrated, or combine certain components, or different components, e.g., the autonomous navigation device 1600 of the wireless capsule endoscope may also include input-output devices, network access devices, buses, etc.
The Processor 1610 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 1620 may be an internal storage unit of the wireless capsule endoscope autonomous navigation device 1600, such as a hard disk or a memory of the wireless capsule endoscope autonomous navigation device 1600. The memory 1620 may also be an external storage device of the autonomous navigation device 1600 of the wireless capsule endoscope, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), and the like, provided on the autonomous navigation device 1600 of the wireless capsule endoscope. Further, the memory 1620 may also include both internal and external memory units of the autonomous navigation device 1600 of the wireless capsule endoscope. The memory 1620 is used to store the computer program 1621 and other programs and data required by the autonomous navigation device 1600 of the wireless capsule endoscope. The memory 1620 may also be used to temporarily store data that has been output or is to be output.
The embodiment of the application also discloses an autonomous navigation device of the wireless capsule endoscope, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the following method:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
The embodiment of the application also discloses a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program realizes the following method:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
The embodiment of the present application also discloses a computer program product, when the computer program product runs on a terminal device, the terminal device is enabled to execute the following method:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
driving the wireless capsule endoscope to move along the desired examination path to the suspected lesion area.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (19)

1. An autonomous navigation system for a wireless capsule endoscope, the system being configured to perform the operations of:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
determining a desired magnetic force produced by a driver magnet corresponding to the desired inspection path;
determining configuration information for the driver magnet based on the desired magnetic force and a desired rotational direction of the wireless capsule endoscope during movement, the configuration information comprising: the distance between the center of the wireless capsule endoscope and the center of the driver magnet, the driving angle of the driver magnet, and the included angle between the specific plane and the vertical line; wherein the specific plane is a plane formed by the center of the wireless capsule endoscope, the center of the driver magnet and the projection point of the center of the driver magnet on the expected rotation axis of the wireless capsule endoscope;
and configuring the driver magnet according to the configuration information and controlling the driver magnet to generate the expected magnetic force so as to drive the wireless capsule endoscope to move to the suspected lesion area along the expected examination path.
2. The system of claim 1, wherein the driving wireless capsule endoscope reciprocates in a rotational motion within the abdominal cavity, comprising:
determining a reciprocating rotation interval of the driver magnet;
and controlling the driver magnet to rotate in a reciprocating manner in the reciprocating rotation interval so as to drive the wireless capsule endoscope to move in a reciprocating manner in the abdominal cavity.
3. The system of claim 2, wherein the determining a reciprocating rotation interval of the driver magnet comprises:
determining a reciprocating rotation angle of the driver magnet;
and determining a reciprocating rotation interval of the driver magnet according to the reciprocating rotation angle, wherein the reciprocating rotation interval is an angle interval formed by rotating the reciprocating rotation angle in a reciprocating manner by taking the target rotation angle as a center.
4. The system of claim 3, wherein the target angle of rotation is 180 degrees.
5. The system of any one of claims 1-4, wherein the self-propelled wireless capsule endoscope is advanced along the abdominal cavity, comprising:
determining the current cavity environment of the wireless capsule endoscope;
adjusting a propelling magnetic force exerted by a driver magnet on the wireless capsule endoscope according to the lumen environment to propel the wireless capsule endoscope along the abdominal cavity.
6. The system of claim 5, wherein the determining the channel environment in which the wireless capsule endoscope is currently located comprises:
controlling the wireless capsule endoscope to acquire images of the abdominal cavity;
and determining the current cavity environment of the wireless capsule endoscope according to the image.
7. The system of claim 5, wherein the adjusting the propelling magnetic force exerted by the driver magnet on the wireless capsule endoscope according to the lumen environment comprises:
according to the cavity environment, the direction of the propelling magnetic force is adjusted, so that the head of the wireless capsule endoscope faces to the deep and far part of the abdominal cavity.
8. The system of claim 5, further comprising, prior to adjusting the propelling magnetic force exerted by the driver magnet on the wireless capsule endoscope according to the lumen environment:
determining a speed of movement of the wireless capsule endoscope in the lumen environment.
9. The system of claim 8, wherein the adjusting the propulsive magnetic force exerted by the driver magnet on the wireless capsule endoscope as a function of the lumen environment further comprises:
and if the movement speed of the wireless capsule endoscope in the cavity environment is smaller than a preset speed threshold, increasing the propelling magnetic force so that the movement speed of the wireless capsule endoscope reaches the preset speed threshold under the action of the propelling magnetic force.
10. The system of claim 9, wherein said increasing said propulsive magnetic force comprises:
controlling to decrease a distance between the driver magnet and the wireless capsule endoscope; alternatively, the first and second electrodes may be,
adjusting a drive angle of the driver magnet.
11. The system of any of claims 1-4 or 6-10, wherein the system is further configured to:
determining a plurality of position points passed by the wireless capsule endoscope in the process of moving in the abdominal cavity;
and generating a motion track of the wireless capsule endoscope according to the plurality of position points.
12. The system of claim 11, wherein the determining a plurality of location points through which the wireless capsule endoscope passes during movement within the abdominal cavity comprises:
the wireless capsule endoscope comprises a wireless capsule endoscope body, a plurality of magnetic sensors, a plurality of sensor sub-arrays and a plurality of sensor sub-arrays, wherein the wireless capsule endoscope body is provided with a plurality of magnetic sensors;
and positioning the wireless capsule endoscope based on the optimal magnetic sensor subarray to obtain a position point corresponding to the current position of the wireless capsule endoscope.
13. The system of claim 12, wherein the locating the wireless capsule endoscope based on the optimal magnetic sensor sub-array to obtain a location point corresponding to a location at which the wireless capsule endoscope is currently located comprises:
measuring a magnetic field generated by a driver magnet by adopting the optimal magnetic sensor subarray to obtain magnetic field data;
determining a pose of the wireless capsule endoscope at a current location based on the magnetic field data;
and determining a position point corresponding to the current position of the wireless capsule endoscope according to the pose.
14. The system of claim 13, wherein the generating a motion trajectory of the wireless capsule endoscope from the plurality of location points comprises:
clustering the plurality of position points to obtain a plurality of positioning points of the wireless capsule endoscope;
and generating a motion track of the wireless capsule endoscope according to the positioning points.
15. The system of claim 1, wherein the determining a desired magnetic force produced by a driver magnet corresponding to the desired inspection path comprises:
constructing a dynamic model of the wireless capsule endoscope in the motion process;
determining a current position of the wireless capsule endoscope;
determining a plurality of desired positions of the wireless capsule endoscope and a desired velocity at each desired position according to a current position of the wireless capsule endoscope;
calculating a sequence of magnetic forces of the wireless endoscope at the plurality of desired locations based on the kinetic model according to the plurality of desired locations and a desired velocity at each desired location;
taking a first magnetic force in the sequence of magnetic forces as the desired magnetic force.
16. The system of claim 15, wherein the system is further configured to perform the following:
recalculating the desired magnetic force of the driver magnet when the wireless capsule endoscope moves to the next desired position.
17. The system of any of claims 1-4, 6-10, or 12-16, wherein the system is further configured to:
and controlling the wireless capsule endoscope to inspect the suspected lesion area.
18. An autonomous navigation device of a wireless capsule endoscope comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the following method when executing the computer program:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
determining a desired magnetic force produced by a driver magnet corresponding to the desired inspection path;
determining configuration information for the driver magnet based on the desired magnetic force and a desired direction of rotation of the wireless capsule endoscope during motion, the configuration information comprising: the distance between the center of the wireless capsule endoscope and the center of the driver magnet, the driving angle of the driver magnet, and the included angle between the specific plane and the vertical line; wherein the specific plane is a plane formed by the center of the wireless capsule endoscope, the center of the driver magnet and the projection point of the center of the driver magnet on the expected rotation axis of the wireless capsule endoscope;
and configuring the driver magnet according to the configuration information and controlling the driver magnet to generate the expected magnetic force so as to drive the wireless capsule endoscope to move to the suspected lesion area along the expected examination path.
19. A computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, carries out the method of:
driving the wireless capsule endoscope to do reciprocating rotary motion in the abdominal cavity;
automatically advancing the wireless capsule endoscope along the abdominal cavity;
when an inspection instruction for a suspected lesion area is received, acquiring an expected inspection path corresponding to the suspected lesion area;
determining a desired magnetic force produced by a driver magnet corresponding to the desired inspection path;
determining configuration information for the driver magnet based on the desired magnetic force and a desired rotational direction of the wireless capsule endoscope during movement, the configuration information comprising: the distance between the center of the wireless capsule endoscope and the center of the driver magnet, the driving angle of the driver magnet, and the included angle between the specific plane and the vertical line; wherein the specific plane is a plane formed by the center of the wireless capsule endoscope, the center of the driver magnet and the projection point of the center of the driver magnet on the expected rotation axis of the wireless capsule endoscope;
and configuring the driver magnet according to the configuration information and controlling the driver magnet to generate the expected magnetic force so as to drive the wireless capsule endoscope to move to the suspected lesion area along the expected examination path.
CN202110931078.5A 2021-08-13 2021-08-13 Autonomous navigation system, device and medium for wireless capsule endoscope Active CN113647893B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110931078.5A CN113647893B (en) 2021-08-13 2021-08-13 Autonomous navigation system, device and medium for wireless capsule endoscope

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110931078.5A CN113647893B (en) 2021-08-13 2021-08-13 Autonomous navigation system, device and medium for wireless capsule endoscope

Publications (2)

Publication Number Publication Date
CN113647893A CN113647893A (en) 2021-11-16
CN113647893B true CN113647893B (en) 2022-06-10

Family

ID=78479869

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110931078.5A Active CN113647893B (en) 2021-08-13 2021-08-13 Autonomous navigation system, device and medium for wireless capsule endoscope

Country Status (1)

Country Link
CN (1) CN113647893B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114916898A (en) * 2022-07-20 2022-08-19 广州华友明康光电科技有限公司 Automatic control inspection method, system, equipment and medium for magnetic control capsule

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826780A (en) * 2009-03-07 2010-09-08 深圳先进技术研究院 Method for driving magnet and method and system for driving capsule endoscope
CN102256531A (en) * 2008-10-21 2011-11-23 奥林巴斯医疗株式会社 Capsule guidance system
CN102793526A (en) * 2005-12-28 2012-11-28 奥林巴斯医疗株式会社 Operation method of magnetic field generating device
CN103281949A (en) * 2011-01-28 2013-09-04 奥林巴斯医疗株式会社 Guidance system for capsule medical devices and magnetic field-enerating apparatus
CN105411505A (en) * 2014-09-15 2016-03-23 上海安翰医疗技术有限公司 Apparatus and method for controlling the movement of a capsule endoscope in the digestive tract of a human body
CN105559739A (en) * 2015-12-22 2016-05-11 苏州向东智造医疗科技有限公司 Capsule endoscope motion control method based on magnetic field intensity change
CN107049212A (en) * 2017-03-22 2017-08-18 重庆金山医疗器械有限公司 Camera lens tilting capsule endoscope
CN108175368A (en) * 2017-11-28 2018-06-19 重庆金山医疗器械有限公司 A kind of control system of capsule endoscopic
CN110693445A (en) * 2019-10-15 2020-01-17 联博智能科技有限公司 Wireless capsule endoscope control system
CN110913743A (en) * 2017-05-29 2020-03-24 马克思-普朗克科学促进协会 Magnetically actuated capsule endoscope, magnetic field generating and sensing device and method of actuating magnetically actuated capsule endoscope
CN110969603A (en) * 2019-11-26 2020-04-07 联博智能科技有限公司 Relative positioning method and device for lesion position and terminal equipment
CN112493970A (en) * 2020-11-30 2021-03-16 元化智能科技(深圳)有限公司 Tracking and positioning method and system of wireless capsule endoscope
CN112515611A (en) * 2020-11-30 2021-03-19 元化智能科技(深圳)有限公司 Positioning method and device of wireless capsule endoscope and terminal equipment
CN112515610A (en) * 2020-11-30 2021-03-19 元化智能科技(深圳)有限公司 Driving method, device and system of wireless capsule endoscope

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10346276B4 (en) * 2003-10-06 2007-01-18 Siemens Ag System for the automated localization of lesions in the gastrointestinal tract for therapy with laser light of an endorobot
US20050192478A1 (en) * 2004-02-27 2005-09-01 Williams James P. System and method for endoscopic optical constrast imaging using an endo-robot
US20080039688A1 (en) * 2004-08-30 2008-02-14 Olympus Corporation Body-insertable apparatus system
WO2009060460A2 (en) * 2007-11-09 2009-05-14 Given Imaging Ltd. Apparatus and methods for capsule endoscopy of the esophagus
JP5314913B2 (en) * 2008-04-03 2013-10-16 オリンパスメディカルシステムズ株式会社 Capsule medical system
DE102008018723B3 (en) * 2008-04-14 2009-07-16 Siemens Aktiengesellschaft Method for controlling the movement of an endoscopy capsule
JP5475208B1 (en) * 2012-05-07 2014-04-16 オリンパスメディカルシステムズ株式会社 Magnetic field generator and capsule medical device guidance system
CN103222842B (en) * 2013-04-18 2015-09-09 安翰光电技术(武汉)有限公司 A kind of device and method controlling capsule endoscope and move at human body alimentary canal
WO2020111539A1 (en) * 2018-11-28 2020-06-04 한양대학교 산학협력단 Magnetic field drive system

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102793526A (en) * 2005-12-28 2012-11-28 奥林巴斯医疗株式会社 Operation method of magnetic field generating device
CN102256531A (en) * 2008-10-21 2011-11-23 奥林巴斯医疗株式会社 Capsule guidance system
CN101826780A (en) * 2009-03-07 2010-09-08 深圳先进技术研究院 Method for driving magnet and method and system for driving capsule endoscope
CN103281949A (en) * 2011-01-28 2013-09-04 奥林巴斯医疗株式会社 Guidance system for capsule medical devices and magnetic field-enerating apparatus
CN105411505A (en) * 2014-09-15 2016-03-23 上海安翰医疗技术有限公司 Apparatus and method for controlling the movement of a capsule endoscope in the digestive tract of a human body
CN105559739A (en) * 2015-12-22 2016-05-11 苏州向东智造医疗科技有限公司 Capsule endoscope motion control method based on magnetic field intensity change
CN107049212A (en) * 2017-03-22 2017-08-18 重庆金山医疗器械有限公司 Camera lens tilting capsule endoscope
CN110913743A (en) * 2017-05-29 2020-03-24 马克思-普朗克科学促进协会 Magnetically actuated capsule endoscope, magnetic field generating and sensing device and method of actuating magnetically actuated capsule endoscope
CN108175368A (en) * 2017-11-28 2018-06-19 重庆金山医疗器械有限公司 A kind of control system of capsule endoscopic
CN110693445A (en) * 2019-10-15 2020-01-17 联博智能科技有限公司 Wireless capsule endoscope control system
CN110969603A (en) * 2019-11-26 2020-04-07 联博智能科技有限公司 Relative positioning method and device for lesion position and terminal equipment
CN112493970A (en) * 2020-11-30 2021-03-16 元化智能科技(深圳)有限公司 Tracking and positioning method and system of wireless capsule endoscope
CN112515611A (en) * 2020-11-30 2021-03-19 元化智能科技(深圳)有限公司 Positioning method and device of wireless capsule endoscope and terminal equipment
CN112515610A (en) * 2020-11-30 2021-03-19 元化智能科技(深圳)有限公司 Driving method, device and system of wireless capsule endoscope

Also Published As

Publication number Publication date
CN113647893A (en) 2021-11-16

Similar Documents

Publication Publication Date Title
CN103370001B (en) The system and method for image stream self-navigation capsule based on internal seizure
US7578788B2 (en) Capsule-type medical device
US7381183B2 (en) Method for capturing and displaying endoscopic maps
US7343036B2 (en) Imaging method for a capsule-type endoscope unit
US9155450B2 (en) Guiding apparatus and capsule medical device guiding system
Xu et al. A novel system for closed-loop simultaneous magnetic actuation and localization of WCE based on external sensors and rotating actuation
CN112493970B (en) Tracking and positioning method and system of wireless capsule endoscope
CN112515610B (en) Driving method, device and system of wireless capsule endoscope
EP1922995A1 (en) System and method for modelling a tracking curve of an in vivo device
Iakovidis et al. Deep endoscopic visual measurements
CN112515611B (en) Positioning method and device of wireless capsule endoscope and terminal equipment
CN113647893B (en) Autonomous navigation system, device and medium for wireless capsule endoscope
Xu et al. Adaptive simultaneous magnetic actuation and localization for WCE in a tubular environment
CN113520275A (en) Trajectory tracking system, apparatus and medium for wireless capsule endoscope
Alsunaydih et al. Next-generation ingestible devices: sensing, locomotion and navigation
US11950868B2 (en) Systems and methods for self-alignment and adjustment of robotic endoscope
CN113475997B (en) Reciprocating rotary drive system, device and medium for wireless capsule endoscope
CN115886692B (en) Real-time positioning method, system and storage medium for magnetically-controlled suspension capsule robot
Zhang et al. Deep reinforcement learning-based control for stomach coverage scanning of wireless capsule endoscopy
CN108460820B (en) Micro mobile device control device and method based on image feedback
CN115067863B (en) Wireless capsule endoscope driving system based on spherical driver
Liu et al. Hybrid magnetic and vision localization technique of capsule endoscope for 3D recovery of pathological tissues
WO2016157596A1 (en) Capsule endoscope guidance system and capsule endoscope guidance apparatus
Kukuk A model-based approach to intraoperative guidance of flexible endoscopy
Zhang et al. Design and implementation of a highly integrated dual hemisphere capsule robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant