KR101923404B1 - Autonomous driving method of externally powered wireless endoscope system using image process - Google Patents

Abstract

The present invention relates to an autonomous navigation method using an image processing of a wireless endoscope in which external manipulation is performed, and more particularly, to a capsule endoscope including a camera module for obtaining image information and magnetized in an arbitrary direction, (S110) of extracting a center point (Xc, Yc) of an input image received from the camera module, and a step (S110) of extracting a center point ; (b) extracting a target point (X _T , Y _T ) using the slope descent search algorithm for the input image (S 131) (S 132); (c) calculating an Euclidean distance D between the center point and the target point (S141), comparing the Euclidean distance D with a preset threshold value T (S142) Or outputting a driving signal for driving the steering wheel (S150) (S160).

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to an autonomous driving method for an externally controlled wireless endoscope,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to an autonomous driving method using image processing of a wireless endoscope to which external manipulation is performed.

The endoscope is a medical instrument that allows the internal organs or body cavity to be seen directly, and the conventional streamlined endoscope is inserted through the mouth or anus, which makes it difficult to perform the procedure and involves patient suffering. To solve these problems, a capsule endoscope has recently been developed. In the capsule endoscope, a capsule-shaped endoscope is swallowed with a mouth, and diagnosis is performed while moving inside the digestive organs by the peristaltic motion of the digestive organs.

Currently, the capsule endoscope, which is currently being commercialized, has no self-driving function, so movement through the peristaltic motion of the digestive organ is carried out. Therefore, only passive observation is possible and thus there are many problems in observing the lesion.

In order to solve such a problem, there has been proposed a micro robot system composed of a capsule endoscope having a magnetization direction and an electromagnetic field generating means for forming an electromagnetic field in the capsule endoscope to drive the capsule endoscope for steering.

For example, Japanese Patent Application Laid-Open No. 10-2013-0022547 (Publication Date: Mar. 03, 2013), Laid-Open Patent Publication No. 10-2013-0024246 (Open Date: Mar. 03, 2013) -0024401 (published on March 31, 2013) is composed of a capsule endoscope having a magnetization direction and a coil unit for generating a magnetic field for driving the capsule endoscope, and through control of the current applied to the coil unit The capsule endoscope is steered or controlled in any direction on the three-dimensional plane of the coil unit, or the translational motion is controlled.

In such a microrobot system, the capsule endoscope can be remotely operated by controlling the current applied to the coil unit while monitoring the position of the capsule endoscope.

On the other hand, there is an inconvenience that such a wireless endoscope that is controlled from the outside requires manual control while continuously monitoring the operator.

Japanese Patent Application Laid-Open No. 10-2013-0022547 (Publication date: March 3, 2013) Japanese Patent Application Laid-Open No. 10-2013-0024246 (Publication date: March 3, 2013) Japanese Patent Application Laid-Open No. 10-2013-0024401 (publication date: 2013.03.08)

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a method capable of autonomous travel by using image information photographed by a wireless endoscope which is externally controlled.

To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described herein, there is provided an autonomous navigation method for an externally controlled wireless endoscope, including: a capsule endoscope including a camera module for acquiring image information and magnetized in an arbitrary direction; (A) a step of extracting a center point (Xc, Yc) of an input image received from the camera module; and a step of extracting a center point (Xc, Yc) of an input image received from the camera module; (b) extracting a target point (X _T , Y _T ) using the slope descent search algorithm for the input image; (c) calculating a Euclidean distance (D) between the center point and the target point, and comparing the Euclidean distance (D) with a set threshold value (T) to drive the capsule endoscope And outputting a signal.

Preferably, the step (c) may output a driving signal for translational driving of the capsule endoscope when the Euclidean distance D is smaller than a preset threshold value T.

More preferably, in the step (c), the distance information is received using the distance detection sensor unit built in the capsule endoscope to extract a three-dimensional target point (X _T , Y _T , Zt) ? Y,? Z) and outputting a drive signal in accordance with the motion vector.

Preferably, the step (c) further comprises the step of outputting a driving signal for steering driving of the capsule endoscope when the Euclidean distance D is larger than the set threshold value T, And a preprocessing step for the input image.

The autonomous running method using the image processing of the wireless endoscope according to the present invention can extract the target point of the capsule endoscope using the image information obtained through the camera module provided in the capsule endoscope, It is possible to shorten the overall intervention time of physicians and users who have a large amount of labor, and to provide convenience of operation.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall configuration diagram of a system for a wireless endoscope self-running method according to the present invention;
2 is a view showing a preferred embodiment of a wireless capsule endoscope,
FIG. 3 is a flowchart showing a method for autonomous navigation of a wireless endoscope according to the first embodiment of the present invention;
FIG. 4 is a flow chart showing the steps of driving the capsule endoscope in FIG. 3,
5 is a diagram for explaining the principle of calculating the route line of the wireless endoscope self-running method according to the present embodiment,
FIG. 6 is a graph showing a cross-section including a minimum value among three-dimensional gradient down-point distributions that can be derived on the basis of brightness values per pixel obtained in a two-
FIG. 7 is a flowchart showing a wireless endoscopic autonomous navigation method according to a second embodiment of the present invention; FIG.

The specific structure or functional description presented in the embodiment of the present invention is merely illustrative for the purpose of illustrating an embodiment according to the concept of the present invention, and embodiments according to the concept of the present invention can be implemented in various forms. And should not be construed as limited to the embodiments described herein, but should be understood to include all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Meanwhile, in the present invention, the terms first and / or second etc. may be used to describe various components, but the components are not limited to the terms. The terms may be referred to as a second element only for the purpose of distinguishing one element from another, for example, to the extent that it does not depart from the scope of the invention in accordance with the concept of the present invention, Similarly, the second component may also be referred to as the first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but it should be understood that there may be other elements in between something to do. On the other hand, when it is mentioned that an element is " directly connected " or " directly contacted " to another element, it should be understood that there are no other elements in between. Other expressions for describing the relationship between components, such as "between" and "between" or "adjacent to" and "directly adjacent to" should also be interpreted.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It will be further understood that the terms " comprises ", or " having ", and the like in the specification are intended to specify the presence of stated features, integers, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall configuration diagram of a system for a wireless endoscopic autonomous navigation method according to the present invention.

1, the system according to the present embodiment includes a capsule endoscope 100, a coil unit 200 for generating an electromagnetic field for driving the capsule endoscope 100, a capsule endoscope 100, And a control unit 300 for receiving the image information transmitted from the coil unit 200 and controlling the power (current) applied to the coil unit 200.

Referring to FIG. 2, the capsule endoscope 100 includes a housing that constitutes an outer appearance and accommodates a plurality of components. For example, the capsule endoscope 100 includes a permanent magnet 110, a camera module 120 for obtaining image information, A data transmission module 130 for transmitting image information acquired through the camera module 120 to the outside and a power source such as a battery for supplying power necessary for driving the camera module 120 and the data transmission module 130. [ And a supply module 140.

The permanent magnet 110 is for wireless adjustment by the electromagnetic drive of the capsule endoscope 100, and can be steered or propelled by a magnetic field externally magnetized in an arbitrary direction.

The camera module 120 may be disposed in front of the housing to perform image capturing, and may be provided with a lighting device for lighting in the photographing direction.

The capsule endoscope 100 may further include a distance detection sensor unit 150 capable of detecting distance information, and a well-known ultrasonic sensor may be used as the distance detection sensor unit 150.

Referring again to FIG. 1, the coil unit 200 includes uniform magnetic field generating modules 210, 220 and 230 capable of generating a uniform magnetic field for steering (rotating) the capsule endoscope 100 in three dimensions .

In the present embodiment, the uniform magnetic field generating modules 210, 220 and 230 illustrate a pair of saddle coil parts 210 and 220 and a Helmholtz coil part 230 arranged at right angles to each other, But various known coils can be used in combination.

The coil unit 200 may further include a gradient coil capable of generating a gradient magnetic field to propel the capsule endoscope 100 in a predetermined direction.

The control unit 300 may include a receiving unit 310, a driving control unit 320, and a power supply unit 330.

The receiving unit 310 receives image information transmitted from the capsule endoscope 100.

The driving control unit 320 may determine the position of the capsule endoscope 100 based on the image information received through the receiving unit 310 and may determine the position of the capsule endoscope 100 to be applied to the coil unit 200 for driving the capsule endoscope 100 The power (current) can be controlled.

The power supply unit 330 serves to supply a power (current) to be applied to the coil unit 200 according to a driving signal transmitted from the driving control unit 320.

Although not shown, the control unit 300 includes well-known input means for manipulating the direction or position of the capsule endoscope directly by the operator, and well-known output means capable of displaying the image information received through the receiving unit 310 .

In the system configured as described above, the driving control unit 320 can perform autonomous driving without manual operation by an external operator using the image information acquired by the camera module 120 provided on the front surface of the capsule endoscope 100 have.

The driving control unit 320 may determine the direction in which the camera module 120 should continue to determine the received image information and control the power supply unit 330 so that the self-driving of the capsule endoscope 100 may be performed.

For example, the center area of the image received by the driving controller 320 may be a time point at which the camera module 120 is looking at, and the target point of the forward direction may be set as a darker and deeper area due to the characteristics of the gastrointestinal tract . Accordingly, the driving control unit 320 searches the image area for the target area in the image information, and can appropriately extract the slope descent points in the two-dimensional image.

Since the three-dimensional gradient descent map expresses extreme values similar to the target region, the image processing technique applied in the present invention can extract only the extreme values by domain reduction and pseudo-extreme value filtering processing. The image region in which the extreme value appears can be represented by coordinates in a two-dimensional image.

3 is a flowchart illustrating a method for autonomous navigation of a wireless endoscope according to a first embodiment of the present invention.

Referring to FIG. 3, the method according to the first embodiment of the present invention includes: a capsule endoscope including a camera module for obtaining image information and magnetized in an arbitrary direction; An autonomous running method of a wireless endoscope including a control unit for wirelessly controlling a capsule endoscope, comprising the steps of: (a) extracting a center point (Xc, Yc) of an input image received from the camera module (S110); (b) extracting a target point (X _T , Y _T ) using the slope descent search algorithm for the input image (S 131) (S 132); (c) calculating the Euclidean distance D between the center point and the target point (S141), and comparing the Euclidean distance D with the set threshold value T (S142) to drive the capsule endoscope (S150) (S160).

Preferably, the step (b) further includes a preprocessing step (S120) such as filtering on the input image.

Specifically, the step (c) calculates the Euclidean distance D between the center point Xc, Yc and the target point X _T , Y _T , 1].

[Equation 1]

The Euclidean distance D thus calculated is compared with a predetermined threshold value T. The Euclidean distance D is compared with the predetermined threshold value T and the translation or steering of the capsule endoscope is performed according to the size between the Euclidean distance D and the set threshold value D. [ Driving is performed. In the present invention, the threshold value T used for the Euclidean distance D may be any value that can be obtained through optimization.

Preferably, when the Euclidean distance D is smaller than the set threshold value T, a driving signal for translational driving of the capsule endoscope is outputted (S150), wherein the translational driving may be a forward movement.

On the other hand, if the Euclidean distance D is greater than the threshold value T, a driving signal for steering (rotation) driving of the capsule endoscope is outputted (S160).

Specifically, referring to FIG. 4, it is possible to know which direction the target point belongs to from the center point for steering drive by obtaining the direction vector (u _x , u _y ) between the center point and the target point, and using the following equation And extracts a direction vector (S161).

&Quot; (2) "

)의 두 가지 파라미터를 가질 수 있다. 구해진 방향벡터로 초기값(θ,

)으로 회전한 뒤 현재의 방향벡터를 저장한다. 이는 조향 이후 방향벡터를 다시 구하는 과정에서 조향 이전과 가리키는 방향이 같은지를 확인하기 위함이다. 만일 다른 경우, 초기값에서 증가된 값으로 같은 방향으로 조향을 유지함으로써 목표점과 중앙점의 일치가 효과적으로 이루어질 수 있다.The parameters of the direction vector are the left-right direction (?) And the up-down direction (

). ≪ / RTI > The obtained initial vector (?,?

) And stores the current direction vector. This is to confirm whether the steering direction and the pointing direction are the same in the process of obtaining the direction vector again after the steering. In other cases, by maintaining the steering in the same direction with an increased value from the initial value, the matching of the target point and the center point can be effected effectively.

The initial value update method is as follows.

An update can be made by comparing the direction vector of the previous steering and the newly obtained direction vector, and (1) only the left and right directions are different; (2) only the up and down directions are different; (3) when the directions are different from each other; (4) When the directions of left, right, up and down coincide with each other; Here, an update unit for left and right steering is defined as?, And an update unit for up and down steering is defined as?.

(1), update is performed in the opposite direction (-α) to the left and right direction indicated by the current direction vector. At this time, since the upper and lower directions are matched, the update is made in the same direction (+?) As the previous angle (S162).

(2), update is performed in the direction opposite to the up-and-down direction indicated by the current direction vector (-β). At this time, since the left and right directions coincide with each other, update is performed in the same direction (+ alpha) as the previous angle (S163).

(3), updating is performed in the opposite direction (-.alpha., -.Beta.) In the opposite direction to the current direction (S164).

(4), weights are added (+ α, + β) to each direction because the steering is in progress.

After updating the steering parameters in each case, the current direction vector (u _x , u _y ) is updated to the next unit vector u _px , u _py (S166) In order to prevent excessive translational motion when the translational motion is performed after the parameters are updated in a somewhat large number of steering motions. However, in an extreme case, when the steering motion is infinitely limited, the translational motion step (Z) should not become '0', so it is gradually reduced while maintaining the previous value as much as possible. I decide.

)을 곱해 조향을 마무리한다.When the updating of the parameters for the steering motion is no longer required (CHECKs == false), the left and right up and down values (?,?

) To finish the steering.

Referring again to FIG. 3, when the Euclidean distance D is smaller than the threshold value T by the steering drive, that is, when it is determined that the steering motion is sufficiently performed, the translational motion proceeds. In the case of translational motion, it is impossible to measure the depth when there is no distance detection sensor. Therefore, when translational movement is performed with initial value (Z), when translation is necessary in the next image, Value (+?), The translation speed can be effectively controlled.

5 is a view for explaining the principle of calculating the route line of the wireless endoscope self-running method according to the present embodiment.

5, the viewpoint of the camera module installed in front of the capsule endoscope is indicated as 'Point B' as the center of the image, and the deep part of the digestive tract to be processed by the capsule endoscope, that is, And is indicated by "Point A". Accordingly, vectors, distances and angles can be secured as coordinates between the two points A and B obtained from the two-dimensional image, and this information is transmitted as a drive signal to the power supply unit 330 for driving the capsule endoscope, As shown in FIG.

FIG. 6 is a graph showing a cross section including a minimum value among three-dimensional gradient down-point distributions that can be derived based on a brightness value per pixel obtained from a two-dimensional image. In the gastrointestinal tract, the point to be the target point of the capsule endoscope is consistently dark. In order to search only this part, the inclination descending extremum may be slowed down by filtering. In FIG. 6, it is necessary to perform the slope descending algorithm for the pole value of the target point except for the blurred pole values. In addition, the slope descending algorithm can be performed with other algorithms such as the Newton method used in the numerical analysis theory Do.

FIG. 7 is a flowchart illustrating a method for autonomous navigation of a wireless endoscope according to a second embodiment of the present invention.

Referring to FIG. 7, the method according to the second embodiment of the present invention includes: a capsule endoscope including a camera module that can acquire image information and a distance detection sensor unit that can detect a distance, the capsule endoscope being magnetized in an arbitrary direction; A method for autonomous navigation of a wireless endoscope, comprising the steps of: generating a magnetic field in an arbitrary direction on a three-dimensional plane to wirelessly control the capsule endoscope, the method comprising the steps of: (a) calculating a center point (Xc, Yc) Extracting (S210); (b) extracting a target point (X _T , Y _T ) on a two-dimensional plane using an oblique descent search algorithm for the input image (S231) (S232); (c) The steps (S241) and (S242) for calculating the Euclidean distance D between the center point and the target point and comparing the Euclidean distance D with the set threshold value T are the same as the first embodiment .

Particularly, in the second embodiment, in the case where the Euclidean distance D is smaller than the set threshold value T in the step (c), the distance detecting sensor unit (X _T , Y _T , Zt), calculates motion vectors (X, Y, and Z), and outputs a drive signal in accordance with the motion vector .

Preferably, the distance (? Z) from the current position of the capsule endoscope to the target point on the three-dimensional image is extracted through the distance detection sensor, and then translational movement is performed by half of the actual distance? Z / 2 to prevent collision with the internal wall surface . Then, in the next image, a distance measurement is performed again through the distance detection sensor, and the translational motion is performed by half of the distance. In this process, the distance D from the center point of the image to the target point, To the point where it exceeds.

On the other hand, in the present embodiment, when the Euclidean distance D is larger than the set threshold value T, the steering operation of the capsule endoscope is substantially the same as that described above (see FIG. 4) .

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the inventions. It will be apparent to those of ordinary skill in the art.

100: capsule endoscope 110: permanent magnet
120: camera module 130: data transfer module
140: power supply module 150: distance detection sensor
200: coil unit 210, 220: saddle coil part
230: Helmholtz coil part 300: Control unit
310: Receiving unit 320:
330: Power supply

Claims (5)

A capsule endoscope including a camera module for obtaining image information and magnetized in an arbitrary direction, and a control unit for generating a magnetic field in an arbitrary direction on a three-dimensional image in an arbitrary direction to wirelessly control the capsule endoscope, As a method,
(a) extracting a center point (Xc, Yc) of an input image received from the camera module;
(b) extracting a target point (X _T , Y _T ) using the slope descent search algorithm for the input image;
(c) calculating a Euclidean distance (D) between the center point and the target point, and comparing the Euclidean distance (D) with a set threshold value (T) to drive the capsule endoscope And outputting a signal to the autonomous mobile endoscope. 2. The capsule endoscope according to claim 1, wherein step (c) comprises: outputting a driving signal for translational driving of the capsule endoscope when the Euclidean distance (D) is smaller than a preset threshold value (T) Autonomous method of wireless endoscopy. The method according to claim 2, wherein the step (c) comprises the steps of: receiving distance information using a distance detection sensor unit built in the capsule endoscope, extracting a three-dimensional target point (X _T , Y _T , Zt) ,? Y,? Z), and outputting a drive signal in accordance with the movement vector. The method according to claim 1, wherein step (c) includes the step of: outputting a drive signal for steering drive of the capsule endoscope when the Euclidean distance (D) is greater than a set threshold value (T) Autonomous method of wireless endoscopy. The method of claim 1, further comprising a preprocessing step of the input image before step (b).

Images