KR102010000B1 - Method and system for shooting control of capsule endoscope - Google Patents

Abstract

A system for controlling imaging of a capsule endoscope is disclosed.
The capturing control system of the capsule endoscope according to an embodiment of the present invention receives a control signal from an intelligent terminal, selects one of the plurality of photographing modes according to the control signal, performs a photographing, and photographs an image. Capsule endoscope for transmitting to the intelligent terminal; And estimating the location of the site where the image is captured, and determining whether or not the lesion is suspected in the location where the image is captured, and based on at least one of the location estimation result and the result of determining whether the lesion is suspected. It may include an intelligent terminal for transmitting a control signal for controlling the capsule endoscope.

Description

Method and system for capturing the capsule endoscope {Method and system for shooting control of capsule endoscope}

The present invention relates to a method and a system for controlling imaging of a capsule endoscope. More particularly, the present invention relates to a method for analyzing the position of a capsule endoscope and a site where a lesion is suspected, to perform a more precise imaging, and to control a clear view (CV), and a system for performing the method.

The capsule endoscope is 11mm thick, 26mm long, and 4g in weight, about the size of a vitamin tablet that we commonly see. When the receiver is worn on the waist and the capsule is eaten on an empty stomach, the receiver receives and stores image information from the capsule.

The receiver receives and stores video information (2-3 fps) taken by the capsule 2-3 times per second. The test is conducted for 8 to 12 hours, during which the person who is being tested can naturally have daily life. You can drink water from 2 hours after swallowing capsules, and you can eat from 4 hours later. After the test, the doctor connects the receiver to the computer and checks the recorded information.

As such, the capsule endoscope takes 6-120,000 images of the large intestine through the digestive tract to the small intestine during the time when the subject swallows through the mouth. The vast images transmitted and stored in the receiver by wireless RF communication or human body communication are downloaded to the computer and read by the examiner (doctor, radiologist, etc.).

Such a conventional capsule endoscopy system has difficulty in obtaining a clear view (CV) of a high suspicion of a lesion of high importance because it is not possible to directly check an image and control the capsule in real time. In addition, it takes considerable time for medical staff to diagnose a large amount of images.

In order to solve this problem, the conventional capsule endoscope reading program includes a map view for viewing the entire captured capsule endoscope image at a time, a range view for observing a specific range of the entire endoscope image in detail, and Provide the same technology. However, in order to perform accurate diagnosis using these techniques, it is necessary to secure sufficient clear view of the suspected lesion site.

Accordingly, the necessity for a capsule endoscopy control method and a system for performing the method is increasingly increased in the area of suspected lesions.

The technical problem to be solved by the present invention is to provide a method for controlling the imaging of the capsule endoscope and a system for performing the method.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present disclosure, a capsule control endoscope imaging control system according to an embodiment of the present invention receives a control signal from an intelligent terminal, and photographs by selecting any one of a plurality of photographing modes according to the control signal. Performing capsules and transmitting the captured images to the intelligent terminal; And estimating the location of the site where the image is captured, and determining whether or not the lesion is suspected in the location where the image is captured, and based on at least one of the location estimation result and the result of determining whether the lesion is suspected. It may include an intelligent terminal for transmitting a control signal for controlling the capsule endoscope.

Preferably, the intelligent terminal, when the position estimation result is a predetermined digestive organ or the determination result of the suspected lesion is determined to suspect the lesion, the capsule endoscope transmits a control signal to perform the shooting in the fine photographing mode It is.

Preferably, the intelligent terminal is to transmit a control signal to the capsule endoscope to shoot in the fine photographing mode by increasing at least one of the frame, the resolution and the intensity of the light source.

Preferably, the intelligent terminal is to estimate the position of the region where the image is photographed based on a learning model for the position of the capsule endoscope according to the time elapsed after the capsule endoscope is injected into the examinee.

Preferably, the intelligent terminal further receives a first measurement value of the acceleration sensor embedded in the capsule endoscope, and estimates the position of the portion where the image is photographed based on the first measurement value.

Preferably, the intelligent terminal corrects the first measured value by using the second measured value of the acceleration sensor embedded in the intelligent terminal, and a part of the image captured based on the corrected first measured value. To estimate the position of.

Preferably, the intelligent terminal further receives a measurement value of the speed sensor embedded in the capsule endoscope, and compares the measured value with a range of speeds previously measured for each digestive organ, whereby the image is captured. To estimate the location.

Preferably, the intelligent terminal is to estimate the position of the site where the image is captured based on a learning model for classifying which digestive organ image is captured by the capsule endoscope.

Preferably, the intelligent terminal determines that the lesion is suspected when the first color is distributed over the preset value in the captured image of the capsule endoscope.

Preferably, the intelligent terminal, based on a learning model for classifying whether the image captured by the capsule endoscope is the image of the normal region or the image of the lesion region to determine whether the lesion suspected in the region where the image was taken will be.

Preferably, the intelligent terminal adds and stores at least one of the location estimation result, the suspicion of whether the lesion is suspected, and the mode in which the image is captured as meta information to the received image.

Preferably, the learning model receives and stores images from a plurality of intelligent terminals, and machine-learns the plurality of images to classify whether the captured images of the capsule endoscope are images of a normal region or images of a lesion region. The server may further include.

According to another aspect of the present invention, there is provided a method for controlling capturing an capsule endoscope, wherein an intelligent terminal receives an image captured by the capsule endoscope from a capsule endoscope; Estimating, by the intelligent terminal, the position of the portion where the image is captured; Determining, by the intelligent terminal, whether a lesion is suspected in a region where the image is captured; And transmitting, by the intelligent terminal, to the capsule endoscope a control signal for controlling a capturing mode of the capsule endoscope based on at least one of the position estimation result and the lesion suspicion determination result.

Preferably, the step of transmitting the control signal to the capsule endoscope, when the position estimation result is a predetermined digestive organ or the result of the suspicion of the suspected lesion is determined to be the suspected lesion, the capsule endoscope in the fine imaging mode And transmitting a control signal to perform imaging, wherein the precision imaging mode is a mode in which the capsule endoscope performs imaging by increasing at least one of a frame, a resolution, and an intensity of a light source.

Preferably, the step of estimating the position of the region where the image is taken, using at least one of the time elapsed after the capsule endoscope is injected into the subject or the measured value of the acceleration sensor built in the capsule endoscope or the image The method may include estimating the position of the portion where the image is captured.

Effects according to the present invention are as follows.

Using the method of controlling the capturing of the capsule endoscope proposed in the present invention, the conventional capsule endoscope cannot secure enough images of the suspected lesion because only the imaging is performed until it is naturally discharged and cannot be controlled in the body. Can be solved. That is, by using the present invention, the capsule endoscope photographs sufficient images at the suspected lesion, thereby obtaining a clear view of the suspected lesion.

In addition, by using the method of controlling the capturing the capsule endoscope proposed in the present invention, it is possible to reduce the time required to analyze the vast image data of the capsule endoscope and the cost incurred by the input of a large number of experts. This will ultimately provide quality care to patients who do not receive health care at a cost.

Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a view for explaining the capsule endoscope that can be used in one embodiment of the present invention.
2 is a view for explaining a system for controlling the capturing the capsule endoscope according to an embodiment of the present invention.
3 is a flow chart illustrating a method of controlling the capturing the capsule endoscope according to an embodiment of the present invention in terms of the endoscope capsule.
4 is a flowchart illustrating a method of controlling capturing the capsule endoscope according to an embodiment of the present invention from an intelligent terminal side.
5A to 5B are diagrams for explaining an algorithm for estimating the position of the capsule endoscope that can be used in one embodiment of the present invention.
6A to 6C are diagrams for describing an algorithm for determining whether a lesion is suspected by analyzing an image captured by a capsule endoscope which may be used in an embodiment of the present invention.
7 is a signal flow diagram illustrating a system for controlling imaging of the capsule endoscope according to an embodiment of the present invention.
8 is a view for explaining a system for controlling the capturing the capsule endoscope according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view for explaining the capsule endoscope that can be used in one embodiment of the present invention.

Capsule endoscope 200 is an electronic device of the vitamin pill size we eat everyday. When the examinee swallows the capsule endoscope 200, the capsule endoscope 200 moves by the peristalsis and gravity of the digestive system, photographs the human digestive system, and transmits the captured information to the outside through communication.

As such, the capsule endoscope 200 transmits a bio technology (BT) having stability enough to be swallowed by a person, nano technology (NT) in which various small electronic components are combined in a small capsule, and photographed images. An electronic device in which Information Technology (IT) is incorporated.

Referring to FIG. 1, the capsule endoscope 200 that may be used in an embodiment of the present invention may include an optical dome, an optical system including an LED, a micro lens, a battery, a telemetry, an antenna, and the like.

The tiny lens captures organs inside the body that are reflected by the LED's illuminated light through an optical dome. At this time, the photographed image information is compressed to JPEG (Joint Photographic Coding Experts Group) or a similar compression standard. In this case, the lens of the capsule endoscope 200 preferably has a wide field of view (FOV) so that a wider area can be photographed.

After that, add a header including information such as the acquisition time of the capsule endoscope image, location information in the human body, capsule unique number, capsule endoscope receiver number, test unique number, patient number, patient name, sex, test location, etc. The standard is converted to DICOM (Digital Imaging and Communications in Medicine) or a standard format and transmitted to the receiver.

Since the capsule endoscope 200 is very small, about 11 mm thick, 26 mm long, and 4 g in weight, it is very important to efficiently use a limited battery. However, conventionally, the internal organs of the body were constantly photographed at about 3 fps for an average of about 12 hours until the capsule endoscope 200 was discharged.

It takes about 3 pictures per second, so for 12 hours, it takes about 130,000 pictures, as shown by the formula 12 * 60 * 60 * 3 = 129,600. As such, since the capsule endoscope 200 has a large number of image information photographed, it is almost impossible for a person to inspect them one by one.

Therefore, the program that provides the medical staff with the image information captured by the capsule endoscope 200 removes the repeated image information, or the map view or the entire scope of the endoscope image, which can view the image information at once, in detail. It provides a mode like range view that can be observed.

However, in order to perform accurate diagnosis using these techniques, it is necessary to secure sufficient clear view of the suspected lesion. Therefore, while using the battery efficiently, the lesion suspected areas need a method and system for controlling the capture of the capsule endoscope 200 to secure a clearer image.

2 is a view for explaining a system for controlling the capturing the capsule endoscope according to an embodiment of the present invention.

The system for controlling the capturing of the capsule endoscope 200 according to an embodiment of the present invention may include a capsule endoscope 200 and an intelligent terminal 300. It may also optionally further include a hospital server (400).

Capsule endoscope 200 is difficult to incorporate a pharmaceutical storage device of size. The image captured by the capsule endoscope 200 may vary depending on the shooting speed and image quality, but requires a large capacity ranging from a few gigabytes to a few tens of gigabytes. Most of the capsule endoscope 200 transmits the image taken through a variety of communication methods to the receiver (Receiver).

The capsule endoscope 200 and the receiver may perform communication using an RF method (Radio Frequency) or an HBC method (Human Body Communication). RF frequency bands are typical of 402-405 MHz and 433.92 MHz, an Industrial Scientific and Medical (ISM) band. That is, the RF method wirelessly transmits the still image data photographed from the image sensor to the outside of the human body through the RF module and the transmitting antenna, and the receiver stores the data by RF receiving and demodulating it with the receiving antenna.

The wireless communication method called the HBC method or the human body communication method converts the image digital information obtained from the capsule endoscope 200 into a micro voltage / current signal using a single conductor, and then transmits the human body to a channel. The attached patch detects and detects the electric field signal and demodulates it. This technology is discussed in the TG BAN (Body Area Network) under the IEEE 802.15.6 WPAN and corresponds to CM (Channel Model) 2, which is a communication model from inside the body to the outside of the body.

In the present invention, the intelligent terminal 300 replaces the role of the receiver. However, if the conventional receiver merely receives and stores the image information captured through the wireless communication with the capsule endoscope 200, the intelligent terminal 300 of the present invention analyzes the received image and the capsule endoscope as a feedback The control signal may be transmitted to 200.

The intelligent terminal 300 may transmit the image information transmitted by the capsule endoscope 200 to the hospital server 400 through wired / wireless communication. The medical staff in the hospital accesses the hospital server 400 and reads whether there is a lesion through the image captured by the capsule endoscope 200.

At this time, convenience functions such as video and video are provided, and forward and backward navigation is also possible. In order to shorten the reading time, a technology for automatically notifying a point where a lesion is suspected is being developed, and a technology for more precisely knowing at which point a lesion occurs is being developed.

In the system for controlling the capturing the capsule endoscope according to an embodiment of the present invention, the biggest difference between the conventional receiver and the intelligent terminal 300 is that the capsule endoscope 200 simply stores or receives the captured image. Whether the control signal can be transmitted to the capsule endoscope 200 by analyzing the image. The intelligent terminal 300 of the present invention is a device capable of processing complex calculations even though it is a small device that the test subject 100 can carry, such as a smartphone.

Of course, the intelligent terminal 300 is inevitably inferior in computing power compared to the hospital server 400. However, the hospital server 400 is a device that requires a high computational capacity for analyzing 100,000 image information, the intelligent terminal 300 analyzes the image captured by the capsule endoscope 200 with a simplified algorithm and the site of the lesion suspected It is a sufficient device with low computational power to judge cognition.

The intelligent terminal 300 of the present invention performs a function as a computer-aided diagnosis (CAD) for determining in real time a suspected lesion site using an image captured by the capsule endoscope 200, and when the lesion is detected as a result of the capsule, the capsule Instructs the endoscope 200 to shoot high quality / high frame. In addition, the image capturing position of the capsule is estimated to classify the images by organs such as the stomach, duodenum and colon.

After completing the capsule endoscopy, the clear view and the photographing position information of the high frame focused on the suspected lesion site that is classified and transmitted to the hospital server 400 are used as an aid for precise diagnosis, thereby further increasing the diagnosis rate compared to the conventional capsule endoscopy system. Can be effective. In addition, by reducing the time required to analyze and classify massive images, it is possible to reduce the burden on the input of professional personnel and provide high-quality medical services to those who cannot receive medical benefits at cost.

3 is a flow chart illustrating a method of controlling the capturing the capsule endoscope according to an embodiment of the present invention in terms of the endoscope capsule.

Referring to FIG. 3, a photographing control method viewed from the side of the capsule endoscope 200 can be seen. Capsule endoscope 200 of the present invention receives and confirms the control signal while periodically communicating with the intelligent terminal 300 (S1100). The control signal is a control signal associated with capturing the capsule endoscope 200.

For example, the control signal is related to the intensity of the LED of the capsule endoscope 200 or the photographing frame or resolution of the camera. After receiving the control signal, the capsule endoscope 200 receives the control signal in a high quality or high frame (S1200), and changes the camera setting of the capsule endoscope 200 to perform high quality or high frame (S1300). .

On the other hand, if it is not a control signal to shoot in high quality or high frame, shooting in the normal shooting mode (S1350). Shoots with lower quality and lower frame quality than high quality or high frame rate. That is, the capsule endoscope 200 according to the embodiment of the present invention has at least two or more shooting modes.

One is a shooting mode called a normal shooting mode, which minimizes battery consumption but provides a relatively unclear image. The other is a high-definition shooting mode, which consumes more battery than normal shooting mode but allows clear clear view.

According to the present invention, at least two or more shooting modes may be used, depending on the situation, sometimes in a general shooting mode and sometimes in a high quality shooting mode. In particular, the suspected lesion may be photographed in a high-definition photographing mode, and a clear view may be secured to more effectively utilize the image of the capsule endoscope 200 for diagnosis of the examinee 100.

That is, while the conventional capsule endoscope 200 consumes a battery constantly with the same setting, the capsule endoscope 200 of the present invention is appropriately distributed between the normal shooting mode and the high-definition shooting mode by capturing the conventional capsule endoscope 200 and The same or less battery consumption, but the suspected lesions have the advantage of ensuring a clear view.

This is summarized in Table 1 below.

division Normal area Suspected lesion Conventional method 3 fps / 320x240 3 fps / 320x240 Proposal method 1 fps / 160x120 6 fps / 640x480

That is, in the conventional method, the imaging was performed in the same manner as the normal region or the suspected lesion, but when the capturing control method of the capsule endoscope 200 of the present invention is used, the normal region is photographed at a lower quality or frame, You can shoot with higher image quality or frame.

Through this, the capsule endoscope 200 may reduce the amount of image data captured by the capsule, which means that the absolute amount of image information that the hospital server 400 needs to analyze is reduced. When the absolute amount of the image information decreases, the medical staff performing the examination by viewing the image taken by the capsule endoscope 200 through the hospital server 400 may perform the treatment more efficiently.

However, the setting of the general photographing mode for photographing the normal region and the setting of the high-quality photographing mode for photographing the suspected lesion, as shown in Table 1, is an example to help the understanding of the invention and is not necessarily limited to the corresponding setting. In some cases, only the frame can be adjusted, only the resolution can be adjusted, or only the light quantity of the LED can be adjusted.

3 again, the capsule endoscope 200 transmits the image photographed in the normal photographing mode or the image photographed in the high definition photographing mode to the intelligent terminal 300 (S1400). As such, the step of capturing the capsule endoscope 200 in various modes according to the control signal of the intelligent terminal 300 is repeated until the capsule endoscope 200 is discharged out of the body (S1500).

As described above, the capsule endoscope 200 of the present invention continuously photographs the internal organs during the natural discharge and communicates with the intelligent terminal 300, thereby requiring a low power. Therefore, power management unit and ultra low power digital / baseband modem technology for low power operation are needed.

4 is a flowchart illustrating a method of controlling capturing the capsule endoscope according to an embodiment of the present invention from an intelligent terminal side.

Referring to FIG. 4, a photographing control method viewed from the side of the intelligent terminal 300 can be seen. The intelligent terminal 300 receives an image while periodically communicating with the capsule endoscope 200 (S2100). And analyzes the position of the capsule endoscope 200 (S2200). In addition, it is determined whether the lesion is suspected on the basis of the received image information (S2300).

If the lesion is detected in the image of the current capsule endoscope 200 (S2400), the control signal is transmitted to the capsule endoscope 200 to perform high-definition / high frame (S2500). On the contrary, when the image currently being captured is the image of the normal portion, the control signal is transmitted to perform the shooting in the normal shooting mode (S2550).

When the capsule endoscope 200 receives the control signal for the photographing mode from the intelligent terminal 300, as shown in FIG. 3, the capsule endoscope 200 performs the photographing according to the control signal and transmits the captured image back to the intelligent terminal 300. The intelligent terminal 300 receiving the image is classified and stored as a lesion image according to a mode in which the image is photographed (S2600) or divided into a general image and stored (S2650).

When the capsule endoscope 200 is discharged to the outside of the body, or photographed most of the organs inside the body or the battery is discharged, and transmits the shooting end signal to the intelligent terminal 300. The intelligent terminal 300 continuously analyzes the image while communicating with the capsule endoscope 200 until receiving the end signal for photographing, and determines whether it is a normal region or a suspected lesion, and controls the photographing mode according to the determination result. Send the signal.

The hospital server 400 receiving image information from the intelligent terminal 300 through wired / wireless communication may perform analysis by prioritizing only images classified as lesion images. That is, the intelligent terminal 300 primarily determines whether the lesion is present, and the hospital server 400 determines whether the lesion is secondary, so that the image captured by the capsule endoscope 200 may be analyzed more efficiently.

In summary, the intelligent terminal 300 analyzes the image data in real time to detect the position estimation and the presence of a lesion, and instructs the capsule endoscope 200 to capture high quality / high frame when detecting the lesion. To this end, a position estimation algorithm of a lightweight image-based capsule endoscope 200 using medical big data and a lesion detection and image processing algorithm using deep learning are required.

5A to 5B are diagrams for explaining an algorithm for estimating the position of the capsule endoscope that can be used in one embodiment of the present invention.

Referring to Figure 5a is shown the digestive system from the esophagus to the rectum. When the examinee 100 swallows the capsule endoscope 200, the stomach is discharged through the stomach, duodenum, small intestine, large intestine, and rectum. It is possible to freely shoot from the esophagus to the stomach and to the large intestine through a conventional endoscope inserted into the mouth or a colonoscopy inserted into the anus.

However, the small intestine is a digestive organ that is difficult to shoot with an endoscope because of its small diameter and long length compared to other digestive organs. The capsule endoscope 200 can be used to efficiently photograph the lesion of the small intestine. That is, most capsule endoscopes so far have been developed with the main purpose of taking small intestine.

In this regard, the capsule endoscope 200 proposed in the present invention may be photographed in high quality according to the position of the capsule endoscope 200 as well as photographed in high quality in the case of a lesion site. For example, the esophagus to the stomach to shoot in the normal shooting mode, the small intestine in the high-quality shooting mode, the large intestine can shoot in the normal shooting mode again.

In addition, depending on the location and lesion of the capsule endoscope 200 may be different shooting mode. For example, the first photographing mode for photographing the normal region of the stomach, the second photographing mode for photographing the suspected region of the stomach, the third photographing mode for photographing the normal region of the small intestine, and the fourth photographing mode for suspected lesions of the small intestine Assuming that there is, the priority of the image quality can be determined in the same order as the first shooting mode <third shooting mode <second shooting mode <fourth shooting mode.

Since the priority of the image quality is immediately related to the power consumption of the battery, the capsule endoscope 200 may be set in advance in a shooting mode or the like in which part of which position to take more priority. This is to use a limited battery more efficiently.

5A again, the time required for the capsule endoscope 200 to be inserted by the examinee 100 and discharged to the large intestine through the esophagus is not constant. This is because the capsule endoscope 200 moves only by the peristalsis of gravity and the digestive system. Of course, a plurality of pads may be attached to the body of the examinee 100, and the position of the capsule endoscope 200 may be estimated using the magnitude of the electrical signal, but this does not match the utilization of the capsule endoscope 200.

Accordingly, the present invention is to propose a simple method for estimating the position of the capsule endoscope 200. Referring to FIG. 5B, it can be seen that a plurality of users have graphed the time until the capsule endoscope 200 is discharged after being swallowed. Although there is a difference for each user, the shooting time is displayed around 10 hours.

Referring to FIG. 5B, the user 1 may have a capsule endoscope 200 about 2 hours for passing through the esophagus, about 3 hours for passing through the stomach and duodenum, about 4 hours for passing through the small intestine, and about 3 hours for passing through the large intestine. Capsule endoscope 200 over a total of about 12 hours can be seen that the digestive organs taken.

Likewise, analyzing the image of the capsule endoscope 200 photographing various test subjects 100 from user 2 to user n may obtain a kind of statistics. That is, the image captured by the capsule endoscope 200 may be analyzed through deep learning, and the correlation between the body information such as gender, age, height, and weight and the position of the capsule endoscope 200 may be analyzed.

As such, when a learning model is generated based on the physical information of the examinee 100 through big data analysis, the position of the capsule endoscope 200 may be estimated according to time without any other device. The big data analysis is performed in the hospital server 400 or a separate high performance computing device, and the resulting learning model may be mounted on the intelligent terminal 300 and used to estimate the position of the capsule endoscope 200.

In addition to estimating the position of the capsule endoscope 200 based on the time, the capsule endoscope 200 may also include a built-in three-axis acceleration sensor to estimate the distance moved by the capsule endoscope 200. However, in this case, since the subject 200 may move after swallowing the capsule endoscope 200, it is necessary to correct this.

That is, the capsule endoscope is compared with the value measured using the 3-axis acceleration sensor of the capsule endoscope 200 and the value measured using the 3-axis acceleration sensor of the intelligent terminal 300 that the user can attach and move freely. The exact location of the capsule endoscope 200 may be estimated by extracting only the result of the movement of the 200.

In addition, the position of the capsule endoscope 200 may be estimated by measuring a moving speed of the capsule endoscope 200 using a speed sensor. Each digestive organ has a different digestion rate. That is, the speed of the capsule endoscope passes through the esophagus, the rate of passage through the stomach, the rate of passage through the small intestine, the rate of passage through the large intestine is different depending on the characteristics of each digestive system. By using this method, the distribution of the velocity through which the capsule endoscope 200 passes for each digestive organ is expressed and expressed in the range, so that it may be determined which digestive organ is passed according to the speed of the capsule endoscope 200.

In addition, the position of the capsule endoscope 200 may be estimated by analyzing the image information captured by the capsule endoscope 200 and transmitted to the intelligent terminal 300 in addition to a method using an acceleration sensor or a speed sensor. In FIG. 5B, a process of analyzing an image captured by the capsule endoscope 200 for a plurality of users may be seen.

By generating a training model for estimating the position of the capsule endoscope 200 by analyzing the image captured by the capsule endoscope 200, and using the generated training model, the capsule endoscope ( The image photographed by 200 may be classified as an image of which part. That is, when the classification model is set based on the hue, saturation, and brightness (HSV; Hue, Saturation, Value) of the image information and machine learning is performed, the body of the subject 100 is classified through the corresponding learning model. Is possible.

6A to 6C are diagrams for describing an algorithm for determining whether a lesion is suspected by analyzing an image captured by a capsule endoscope which may be used in an embodiment of the present invention.

6A is image data of the small intestine taken by the capsule endoscope 200, and FIG. 6B is image data of the large intestine taken by the capsule endoscope 200. As shown in FIGS. 6A and 6B, the images of the small intestine and the large intestine taken by the capsule endoscope 200 show similar color, brightness, and saturation when the photographing part is a normal part.

However, if bleeding occurs in certain areas of the small intestine or tumors are found, the color, lightness, and saturation may vary. Therefore, the similarity between the images taken by the capsule endoscope 200 inside the body of the examinee 100 is determined, and a portion showing similarity difference between the existing image and the preset value may be determined as a suspected lesion and the photographing mode may be changed. have.

That is, the examinee obtains the similarity between the first image photographed after swallowing the capsule endoscope 200 and the second image photographed after a predetermined time elapses from the time point at which the first image was captured, and whether the similarity of the two images exceeds a threshold. As a guideline, one can simply determine whether a lesion is suspected. If you suspect a lesion, change the camera's shooting mode to get a clear view.

Alternatively, the current examinee 100 is compared with a classification model made based on an image captured by the capsule endoscope 200 in the body of the examinee 100 and an image previously captured by the capsule endoscope 200 in the body of another user. It is possible to determine whether the photographed area inside the body of the suspected lesion.

Referring to FIG. 6C, the capsule endoscope 200 performs recording at 3 fps until it detects a suspected lesion, and then detects a suspected tumor 110 at 10 fps. After you leave the area, you can see that the image is taken at 3 fps.

Then, when bleeding occurs again and finds a suspected site (120), the imaging is performed at 10 fps. As such, using the method of controlling the capturing of the capsule endoscope 200 proposed in the present invention, the capturing mode of the capsule endoscope 200 may be variously selected depending on whether the capsule endoscope 200 is a normal region or a suspected lesion.

As such, when bleeding occurs inside the small intestine, the specific gravity of the red in the image taken by the capsule endoscope 200 is rapidly increased. Or if a tumor is found inside the small intestine, the dark color rate will increase rapidly. Therefore, when the ratio of a specific color in the image captured by the capsule endoscope 200 increases rapidly, it may be determined as a suspected lesion and change the mode of the camera.

This reduces battery consumption, reduces the absolute amount of image data captured, and allows the suspected lesions to be used for diagnosis by securing a clear view with clearer image quality. Through this, the examinee 100 can be examined more efficiently.

In the present invention, the machine learning may be used in the step of estimating the location of the location photographed by the capsule endoscope 200 by analyzing the image taken by the capsule endoscope 200 or in determining the suspected lesion of the photographed area. . For example, the digestive organs may be classified by K-Means clustering the distribution of the images in three dimensions based on the color, brightness, and saturation of the image.

Alternatively, the classifier may be generated by applying a support vector machine (SVM). Through this, the capsule endoscope 200 may classify the location of the photographed part or whether the lesion is suspected. However, this is to help the understanding of the invention, it is possible to determine the location or lesion based on the image by applying a variety of algorithms of machine learning.

7 is a signal flow diagram illustrating a system for controlling imaging of the capsule endoscope according to an embodiment of the present invention.

Referring to FIG. 7, the capsule endoscope 200 transmits image data to the intelligent terminal 300 and receives a control signal. According to the control signal, the user may shoot in the normal shooting mode or shoot in the high quality shooting mode or the precise shooting mode. In the Fine Shooting mode, you can increase the frame, increase the resolution, or increase the intensity of the LED light source.

The intelligent terminal 300 receives the image data from the capsule endoscope 200 and analyzes the shape data to determine the location of the capsule endoscope 200 and whether the lesion is suspected. If the position of the capsule endoscope 200 is a place requiring more precise shooting, such as the small intestine, or if a lesion suspected site is found, the control signal for precise imaging is transmitted to the capsule endoscope 200.

The hospital server 400 receives image data through wired / wireless communication with the intelligent terminal 300 and provides analysis data. Intelligent terminal 300 by receiving the image data of a plurality of examinee and applying the image processing and machine learning to generate a learning model for estimating the position of the capsule endoscope 200 and a learning model for determining whether the lesion suspected To provide.

8 is a view for explaining a system for controlling the capturing the capsule endoscope according to an embodiment of the present invention.

Referring to FIG. 8, the capsule endoscope 200 uses the communication module 210 for transmitting / receiving image data and control signals to and from the intelligent terminal 300, and photographing module 230 according to a control signal received from the intelligent terminal 300. A control module 220 for controlling and transmitting the image captured by the photographing module 230 to the intelligent terminal 300 and a power module 240 for supplying power to the capsule endoscope 200 may be included.

The intelligent terminal 300 transmits and receives image data and control signals to and from the capsule endoscope 200, and receives the communication module 310 and the capsule endoscope 200 to transmit and receive image data and analysis data with the hospital server 400. A location estimation module 330 for estimating the location of the capsule endoscope by analyzing the image, a lesion analysis module 340 for determining whether the lesion is a suspected site, and a control module 320 for controlling the image may be included.

The hospital server 400 may include a communication module 410 for communicating with the intelligent terminal 300, a big data analysis module 440 for performing machine learning to track the location of the capsule endoscope 200 through big data analysis, It may include an image analysis module 430 for providing a model that can determine whether the lesion is suspected based on the image processing and the control module 420 for controlling it.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims (15)

A capsule endoscope configured to receive a control signal from an intelligent terminal, perform photographing by selecting any one of a plurality of photographing modes according to the control signal, and transmit a photographed image to the intelligent terminal; And
Estimating the location of the site where the image was taken, determining in real time whether a lesion was suspected in the site where the image was captured, using the computer-aided detectionf (CAD); An intelligent terminal for automatically generating a control signal for controlling a capturing mode of the capsule endoscope based on at least one and transmitting the control signal to the capsule endoscope;
The intelligent terminal,
Estimating the position of the image-captured part based on a learning model of the position of the capsule endoscope according to the time elapsed after the capsule endoscope was injected into the examinee,
The intelligent terminal,
Obtaining a similarity between the first image taken by the subject after swallowing the capsule endoscope and the second image taken after a predetermined time elapses from the time point at which the first image was taken, and the first image and the second image An imaging control system for a capsule endoscope that determines whether a lesion is suspected based on whether the similarity exceeds a threshold. The method of claim 1,
The intelligent terminal,
When the location estimation result is a predetermined digestive organ or the result of determining whether the lesion is suspected to be a lesion, the capsule endoscope transmits a control signal to perform shooting in the fine photographing mode.
Capture control system of capsule endoscope. The method of claim 2,
The intelligent terminal,
The capsule endoscope transmits a control signal to perform shooting in the fine shooting mode by increasing at least one of the frame, the resolution and the intensity of the light source,
Capture control system of capsule endoscope. delete delete delete delete delete delete delete The method of claim 1,
The intelligent terminal,
And adding at least one of the location estimation result, the lesion susceptibility determination result, and the mode in which the image is photographed to the received image as meta information.
Capture control system of capsule endoscope. The method of claim 1,
Receiving and storing images from a plurality of intelligent terminals, and machine learning a plurality of images to generate a learning model for classifying whether the image taken by the capsule endoscope is a normal image or a lesion image Included,
Capture control system of capsule endoscope. Receiving, by the intelligent terminal, an image captured by the capsule endoscope from the capsule endoscope;
Estimating, by the intelligent terminal, the position of the portion where the image is captured;
Determining, by the intelligent terminal, whether a lesion is suspected in a region where the image is captured by using a computer-aided detectionf (CAD) in real time; And
And automatically generating, by the intelligent terminal, a control signal for controlling a capturing mode of the capsule endoscope based on at least one of the position estimation result and the result of determining whether the lesion is suspected, and transmitting the control signal to the capsule endoscope.
The intelligent terminal estimates the position of the part where the image is photographed based on a learning model of the position of the capsule endoscope according to the time elapsed after the capsule endoscope is input to the examinee,
The intelligent terminal obtains a similarity between the first image photographed by the examinee after swallowing the capsule endoscope and the second image photographed after a predetermined time elapses from the time point at which the first image was photographed. And a capsule endoscope photographing control method for determining whether a lesion is suspected based on whether the similarity of the second image exceeds a threshold. The method of claim 13,
Transmitting the control signal to the capsule endoscope,
If the position estimation result is a predetermined digestive organ or the result of the suspicion of the lesion is determined to be suspected, the capsule endoscope includes the step of transmitting a control signal to perform the shooting in the fine photographing mode,
The precision shooting mode,
The capsule endoscope is a mode in which photographing is performed by increasing at least one of a frame, a resolution, and an intensity of a light source.
Image capturing method of capsule endoscope. delete

Images