CN110811489A

CN110811489A - Capsule endoscope with 3D measuring function and corresponding imaging method

Info

Publication number: CN110811489A
Application number: CN201911267384.2A
Authority: CN
Inventors: 章逸舟; 李光元; 鲁远甫; 焦国华; 陈良培; 刘鹏
Original assignee: Shenzhen Institute of Advanced Technology of CAS
Current assignee: Shenzhen Institute of Advanced Technology of CAS
Priority date: 2019-12-11
Filing date: 2019-12-11
Publication date: 2020-02-21
Also published as: WO2021115068A1

Abstract

The invention relates to a capsule endoscope, which is mainly a binocular imaging system and adopts the design of a double-objective lens single-image sensor. Specifically, the binocular imaging system comprises a prism unit and an image sensor used in cooperation with the prism unit. The prism unit comprises two same 45-degree-angle rhombic prisms which are arranged side by side in a staggered 180-degree azimuth relation, and the emergent surfaces of the light paths are in a plane and respectively correspond to two equal division areas on the target surface of the image sensor. The capsule endoscope has compact optical structure, can realize color imaging of a lesion area, can also obtain the function of 3D measurement, and simultaneously reduces the whole manufacturing cost.

Description

Capsule endoscope with 3D measuring function and corresponding imaging method

Technical Field

The invention belongs to the technical field of medical instruments, and particularly relates to a capsule endoscope.

Background

The capsule endoscope is a visual medical gastrointestinal examination device in the shape of a capsule. The capsule endoscope enters a human body, and can observe the health conditions of the intestines, the stomach and the esophagus of the human body, thereby helping doctors to diagnose patients. The capsule endoscope has the advantages of no wound, accuracy, reusability and the like which are beyond the traditional flexible gastrointestinal endoscope. At present, the commercialized capsule endoscope mainly integrates a single camera of 2D imaging, the observation endoscope usually has high distortion in order to obtain a larger visual field, and the capsule endoscope is only used for observation rather than measurement for a long time, and the defects make it difficult for doctors to judge the severity of lesion by using the capsule endoscope. In order to improve the accuracy and authenticity of the physician's observation of the lesion, it is highly advantageous to obtain three-dimensional information in the intestines and stomach by capsule endoscopy.

The patent CN107317954A acquires a light field signal of a position scene to be detected through a light field technology, and a light field camera records four-dimensional information of light in a propagation process, so that a 3D state of an observed object can be reconstructed. However, because the light field camera is used for imaging, and the light field camera is composed of the main lens group, the micro lens array and the photosensitive element, the micro lens array is expensive, and the whole set of equipment has very fine manufacturing requirements, so the cost is very high.

Patent CN105996961A discloses a structured light based 3D stereoscopic imaging capsule endoscope system, which generates structured light by a structured light generating module and acquires three-dimensional information by cooperating with an illumination device. However, the 3D imaging band for imaging is formed by filtering white light and using the structured light generating module, and therefore, the color information of the imaging target cannot be accurately reflected, and although 3D measurement can be performed, the important information of the dimension of lacking the color of the lesion area has a great influence on the diagnosis of the doctor.

The virtual reality capsule endoscope of patent CN104720735A simulates the left and right eyes of a human being with two cameras, each including a CCD image sensor and a camera lens. However, since the capsule endoscope has a high requirement on the size, the imaging technology using the dual objective lens and the dual image sensor is very unfavorable for the capsule endoscope to meet the size requirement in application, and is also unfavorable for reducing the cost.

Therefore, the defects of the existing three-dimensional imaging capsule endoscope are high in cost, complex in manufacturing process, difficult to meet size requirements, insufficient in imaging information and the like, and if the defects can be improved, more convenience is brought to doctors for diagnosing pathological changes through the capsule endoscope.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides a capsule endoscope with a 3D measurement function for gastrointestinal examination, and the capsule endoscope system realizes the purpose of 3D measurement by using a prism module with special design and a single image sensor with double objective lenses, thereby not only reducing the size space of the system, but also having the 3D measurement function while keeping spectral information.

The capsule endoscope comprises a capsule shell, and a binocular imaging system, an illumination module, an image acquisition and processing module, a battery module, a magnetic cover positioning module and a wireless transmitting module which are arranged in the capsule shell; the binocular imaging system comprises a prism unit and an image sensor matched with the prism unit; the prism unit comprises two same 45-degree-angle rhombic prisms which are arranged side by side in a direction relationship of 180 degrees in a staggered manner; the light path emergent surfaces of the two rhombic prisms are in the same plane and respectively correspond to the two equal division areas on the target surface of the image sensor.

Furthermore, the binocular imaging system further comprises two sets of imaging objective lens systems, and the two sets of imaging objective lens systems respectively correspond to the incident surfaces of the two rhombic prisms. The imaging objective system comprises, for example, a lens unit and a diaphragm. Preferably, the lens unit comprises a first lens, a second lens, a third lens and a diaphragm; the lens is a lens with negative diopter, and the curved surface on one side of the object space is concave to the image space, and the curved surface on one side of the image space is concave to the object space; the second lens is a lens with negative diopter, the curved surface at one side of the object space is concave to the image space, and the curved surface at one side of the image space is concave to the image space; the third lens is a lens with positive diopter, and the curved surface on the object side is concave to the image side, and the curved surface on the image side is concave to the image side.

Further, an optical filter may be further disposed between the prism unit and the image sensor.

Preferably, the illumination module has one or more illumination units, for example using LEDs, arranged to the side of the binocular imaging system.

Meanwhile, the invention also provides a method for performing three-dimensional imaging by using the capsule endoscope, which comprises the following operation steps:

the external magnetic control equipment is used for controlling the magnetic shield positioning module in the capsule to enter the gastrointestinal tract and continuously move forwards;

the gastrointestinal tract is illuminated through the illumination module, and a lesion area in the intestinal tract is imaged through the binocular imaging system; in the imaging process, image signals entering the two rhombic prisms have parallax errors and are imaged to two equally divided areas on the target surface of one image sensor; the image formed by the image sensor is collected by the image collecting and processing module.

Further, the method also comprises the step that the image acquisition and processing module transmits the processed image to the wireless transmitting module, and the wireless transmitting module encodes the processed image and transmits the encoded image to an external receiving device.

Preferably, the receiving means performs three-dimensional image reconstruction by means of computer software.

Compared with the prior art, the capsule endoscope adopting the double-objective single-image sensor has a compact optical structure, can realize color imaging of a lesion area, can also obtain a 3D measurement function, and simultaneously reduces the overall manufacturing cost.

The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical solutions of the present application more clear and clear, and to implement the technical solutions according to the content of the description, the following detailed description is made with reference to the preferred embodiments of the present application and the accompanying drawings.

Drawings

The following describes embodiments of the present invention in further detail with reference to the accompanying drawings;

FIG. 1 is a binocular stereo imaging principle;

FIG. 2 is a schematic view of a depth calculation;

FIG. 3 is a schematic view of a dual objective single image sensor capsule endoscope of the present invention;

FIG. 4 is a three-dimensional schematic diagram of a cemented prism in a binocular imaging system;

fig. 5 is a diagram of an example dual-objective single-image sensor binocular imaging system.

Reference numerals:

the system comprises a capsule shell 1, a binocular imaging system 2, an illumination module 3, an image acquisition and processing module 4, a battery module 5, a magnetic cover positioning module 6, a wireless transmission module 7, a lens I21, a lens II 22, a lens III 23, a diaphragm 24, a cemented prism 25, an

oblique square prism

251 and 252, an

exit surface

251a and 252a oblique square prism, an

incident surface

251b and 252b oblique square prism, a light filter 26 and an image sensor 27.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.

In order to realize stereo measurement, a basic imaging principle is that two optical systems are arranged in a three-dimensional imaging capsule endoscope system and are respectively used for shooting separated images corresponding to a left eye and a right eye, and then three-dimensional reconstruction is carried out through software.

The basic principle of binocular stereo imaging is shown in the attached figure 1 of the specification, wherein two identical cameras are arranged below the figure, and corresponding parameters of the two cameras are respectively marked by subscripts 1 and r. Wherein O is₁And O_rThe centers of projection of the two cameras, respectively, and the distance between them is called the base line distance, and is denoted by B. O is₁n₁And O_rn_rRespectively left and right camera image planes C₁And C_rA point A (X, Y, Z) in the world coordinate system is on the left and right camera image planes C₁And C_rThe image points on are respectively a₁(u₁,v₁) And a_r(u_r,v_r) These two points are also referred to as "conjugate points". Knowing these two points, connecting them to the projection centers of the corresponding cameras, respectively, to obtain the projection line a₁O₁And a_rO_r。

Assuming that the two camera image planes are on the same plane, the Y coordinates of the image points of the object point A on the left and right camera image planes are the same, i.e. v₁＝v_rFrom the trigonometric relationship, one can obtain:

in equation (1), f is the focal length of the two cameras, so the parallax is:

the formula (2) is a basic principle of binocular stereo imaging, and after parallax information is obtained, depth information and three-dimensional information of an image can be obtained according to a projection model.

Meanwhile, after the parallax is acquired, accurate calculation of three-dimensional information can be performed according to parameters obtained after stereo correction, image coordinates of points to be measured and left and right image detectors, such as CCD parallax. Referring to the specification, as shown in FIG. 2, the

Is the center of the left CCD and is,

is the center of the right CCD, and the distance between the centers of the cameras is T, x_lIs a point to be matched on the left CCD, x_rThe matching point on the right CCD. From similar triangles

Can be solved to obtain Z ═ -Tf/x_l-x_r。

Therefore, under the condition that the T and the f are calibrated and the aberration xl-xr of one point p on the main image is known, the depth information Z can be calculated, and (x, y) can be calculated through calibration information, so that the three-dimensional information (x, y, Z) of the point to be measured can be recovered.

Based on the above three-dimensional imaging working principle, as mentioned above, the conventional capsule endoscope currently acquires two images by using the imaging technology of dual objective lenses and dual image sensors, which is not suitable for the capsule endoscope that originally has a high requirement on size.

The capsule endoscope is structurally shown in the attached figure 3 in the specification, and comprises a capsule shell 1, a binocular imaging system 2, an illumination module 3, an image acquisition and processing module 4, a battery module 5, a magnetic cover positioning module 6 and a wireless transmitting module 7. The binocular imaging system 2, the illumination module 3, the image acquisition and processing module 4, the battery module 5, the magnetic shield positioning module 6 and the wireless transmitting module 7 are all arranged and fixed in the capsule shell 1. The capsule housing 1 may be formed by two part housings pressed together, the part of the capsule housing 1 that the binocular imaging system 2 and the illumination module 3 face being transparent. A typical capsule shell 1 may be sized to about 12mmx30mm mm. In the present embodiment, a binocular imaging system 2 is provided at one end within the capsule housing 1, which is a key component for acquiring three-dimensional information, as will be described in detail later. An illumination module 3 is provided on the side of the binocular imaging system 2 for providing illumination light to the imaging target area. The lighting module 3 may be one lighting unit, or may be two or more lighting units. As shown in fig. 3, in the present embodiment, the illumination module 3 has two illumination units respectively disposed at two sides of the binocular imaging system 2, so as to provide a similar illumination environment for the two signals. In order to reduce the cost, increase the service life, and achieve a compact structure, it is preferable to use LEDs for each unit of the lighting module 3. An image acquisition and processing module 4, a battery module 5, a magnetic cover positioning module 6 and a wireless transmitting module 7 are installed behind the optical path of the binocular imaging system 2, image signals acquired by the binocular imaging system 2 are transmitted to the image acquisition and processing module 4, the image acquisition and processing module 4 processes the signals to acquire three-dimensional image information, and the three-dimensional image information can be transmitted to external equipment through the wireless transmitting module 7 to be displayed to doctors. The battery module 5 supplies power to other parts. The magnetic shield positioning module 6 is used for controlling the whole capsule endoscope by a doctor.

The greatest difference between the present invention and the prior art is the binocular imaging system 2. As shown in fig. 4 and 5 of the specification, the core of the binocular imaging system 2 is a cemented prism 25 and an image sensor 27 used in cooperation with the cemented prism 25. The cemented prism 25 specifically adopts two identical 45-degree-angle

rhombic prisms

251 and 252, the two rhombic prisms are placed side by side in a staggered 180-degree azimuth relationship, that is, the rhombic prism 251 rotates 180 degrees around a dotted line rotating shaft L shown in FIG. 5, which is the direction of the other rhombic prism 252, and the two rhombic prisms are placed in a staggered and parallel manner, so that the bottom surface 251a of the rhombic prism 251 and the bottom surface 252a of the rhombic prism 252 are coplanar, and the two surfaces are also the emergent surfaces of the imaging optical path. And a surface 251b of the rhombic prism 251 and a surface 252b of the rhombic prism 252 are incident surfaces for imaging. The overlapped parts of the two

rhombic prisms

251 and 252 can be bonded together by gluing, so that the images of the two objective lenses are respectively imaged to two equally divided areas on the left and right sides of the target surface of the same image sensor 27. The two rhombus prisms and components of the binocular imaging system such as the image sensor 27 may be further fixed within the capsule housing via a circuit board. Because the prism is close to the gap and small enough, the two image boundaries are clear enough, and the left and right optical information can not interfere with each other.

The binocular imaging system 2 of the present invention adopts a dual-objective optical axis translation zoom design, as shown in fig. 5 in the specification, and two imaging objective systems with the same optical parameters are used for two incident surfaces 251b and 252b of the cemented prism 25. In the present embodiment, the binocular imaging system 2 is composed of two lenses one 21, two lenses two 22, two lenses three 23, two diaphragms 24, a cemented prism 25 with a filter 26, and an image sensor 27. Each set of imaging objective system consists of 3 lenses and a diaphragm 24: the first lens 21 is a lens with negative diopter, and the curved surface at the object side is concave to the image side, and the curved surface at the image side is concave to the object side; the second lens 22 is a lens with negative diopter, and the curved surface on the object side is concave to the image side, and the curved surface on the image side is concave to the image side; the third lens 23 is a lens with positive diopter, and the curved surface at the object side is concave to the image side, and the curved surface at the image side is concave to the image side; the diaphragm 24 is located between the second lens 22 and the third lens 23. The typical field angle of the objective lens is not less than 90 degrees, and the caliber is not more than 2 mm.

When the capsule endoscope is used, the target area is illuminated by the illumination module 3, the light of the lesion area can be captured by two sets of imaging objective lens systems, and the image signals of the same lesion area entering the rhombic prism 251 and the rhombic prism 252 have parallax. Specifically, the light entering the first set of imaging objective system, that is, the light entering the upper imaging objective system in fig. 5 sequentially passes through the first lens 21, the second lens 22, the diaphragm 24, and the third lens 23 in the upper imaging objective system, then enters the cemented prism 25 through the incident surface 251b of the rhombic prism 251, and after two 90 ° reflections in the rhombic prism 251, the light enters the optical filter 26 through the emergent surface 251a of the rhombic prism 251, and at this time, the principal ray of the central field of view is parallel to the original optical axis, and after being filtered by the optical filter 26, the light is imaged on one side of the image sensor 27. The light entering the second set of imaging objective system, that is, the light of the lower imaging objective system in fig. 5 passes through the first lens 21, the second lens 22, the diaphragm 24 and the third lens 23 in the lower imaging objective system in sequence, then enters the cemented prism 25 through the incident surface 252b of the rhombic prism 252, the light is reflected twice by 90 ° in the rhombic prism 252, and then enters the optical filter 26 through the emergent surface 252a of the rhombic prism 252, at this time, the principal ray of the central field of view is also parallel to the original optical axis, and is filtered by the optical filter 26 and then imaged on the other side of the image sensor 27. Therefore, the images captured by the two imaging objective systems are imaged to two equally divided areas on the left and right sides of the target surface of the same image sensor 27 through different optical paths, and the prism is close to the gap and small enough, so that the boundaries of the two images are clear enough, and the left and right optical information are not mutually interfered.

In the specific application, after a patient swallows the capsule endoscope, the external magnetic control equipment enters the gastrointestinal tract by controlling the magnetic cover positioning module 6 in the capsule and continuously moves forwards, when a diseased region is found, the gastrointestinal tract is illuminated by the illumination module 3, the diseased region in the intestinal tract is imaged by the binocular imaging system 2, a common imaging region is formed in the same scene region corresponding to the imaging range of the two objective lenses, an image formed by the image sensor is collected and processed by the image collecting and processing module 4 and then is transmitted to the wireless transmitting module 7, the wireless transmitting module 7 codes the processed image and transmits the coded image to an external receiving device, the received image is divided into a left screenshot and a right screenshot by software technology, and then three-dimensional reconstruction and measurement can be carried out by computer software. The power for all the devices inside the capsule endoscope is powered by the battery module 5.

The key point of the capsule endoscope is that the 3D measurement is realized by adopting a double-objective lens single-image sensor. Two imaging objective systems with the same optical parameters are used, and two parallel rhombic prisms with 180-degree phase error are arranged behind the imaging objective systems respectively, so that images captured by the two imaging objective systems are imaged to two equally divided areas on the left and right of the target surface of the same image sensor respectively, and are identified by a software algorithm and a three-dimensional image is reconstructed. Compared with the prior art, the capsule endoscope has the advantages that the reflection in the prism is utilized in the imaging light path of the capsule endoscope, and the two images with parallax are obtained by using the image sensor, so that the whole imaging light path is more compact in structure and smaller in caliber, the size of the capsule endoscope is smaller, and the cost is lower. Meanwhile, pictures shot by the binocular imaging system can be color images or monochrome images, and abundant image information is helpful for doctors to distinguish lesion areas.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A capsule endoscope comprises a capsule shell (1), and a binocular imaging system (2), an illumination module (3), an image acquisition and processing module (4), a battery module (5), a magnetic shield positioning module (6) and a wireless transmitting module (7) which are arranged in the capsule shell (1);

wherein the binocular imaging system (2) comprises a prism unit and one image sensor (27) used in cooperation with the prism unit;

the prism unit comprises two same 45-degree-angle rhombic prisms which are arranged side by side in a direction relationship of 180 degrees in a staggered manner; the light path emergent surfaces of the two rhombic prisms are in one plane and respectively correspond to two equal-dividing areas on the target surface of the image sensor (27).

2. The capsule endoscope of claim 1, wherein: the binocular imaging system (2) further comprises two sets of imaging objective systems, and the two sets of imaging objective systems respectively correspond to the incident surfaces of the two rhombic prisms.

3. The capsule endoscope of claim 2, wherein: the imaging objective system comprises a lens unit and a diaphragm (24).

4. The capsule endoscope of claim 3, wherein: the lens unit comprises a first lens (21), a second lens (22), a third lens (23) and a diaphragm (24); the first lens (21) is a lens with negative diopter, and the curved surface on the object side is concave to the image side, and the curved surface on the image side is concave to the object side; the second lens (22) is a lens with negative diopter, and the curved surface on the object side is concave to the image side, and the curved surface on the image side is concave to the image side; the third lens (23) is a lens with positive diopter, and the curved surface on the object side is concave to the image side, and the curved surface on the image side is concave to the image side.

5. The capsule endoscope of claim 1, wherein: an optical filter (26) is also provided between the prism unit and the image sensor (27).

6. The capsule endoscope of claim 1, wherein: the lighting module (3) has one or more lighting units arranged on the sides of the binocular imaging system (2).

7. A method for three-dimensional imaging using the capsule endoscope of any of claims 1-6, comprising the following operative steps:

the external magnetic control equipment is used for controlling the magnetic shield positioning module (6) in the capsule to enter the gastrointestinal tract and continuously move forwards;

the gastrointestinal tract is illuminated through the illumination module (3), and a lesion area in the intestinal tract is imaged through the binocular imaging system (2); in the imaging process, image signals entering the two rhombic prisms have parallax errors and are imaged to two equally-divided areas on the target surface of one image sensor (27);

the image formed by the image sensor (27) is acquired by the image acquisition and processing module (4).

8. The method of claim 7, wherein: the system is characterized by further comprising an image acquisition and processing module (4) for transmitting the processed image to the wireless transmitting module (7), wherein the wireless transmitting module (7) encodes the processed image and transmits the encoded image to an external receiving device.

9. The method of claim 8, wherein: the receiving device carries out three-dimensional image reconstruction through computer software.