CN110600108A

CN110600108A - Redundant image processing method of capsule endoscope

Info

Publication number: CN110600108A
Application number: CN201910820199.5A
Authority: CN
Inventors: 胡延兴
Original assignee: Xiamen Yinnuo Medical Technology Co Ltd
Current assignee: Xiamen Yinnuo Medical Technology Co Ltd
Priority date: 2019-09-01
Filing date: 2019-09-01
Publication date: 2019-12-20

Abstract

The invention provides a redundant image processing method of a capsule endoscope, which comprises the following steps: unpacking original data to an arithmetic unit, firstly extracting the mapping in the original memory of the data according to different data formats, hashing the data, and sending the hashed data to the arithmetic unit to start an arithmetic flow; and (3) perception hash processing: in the algorithm stage, performing algorithm processing on the data according to the hash information to calculate similar object information; and (3) an abstract outputting process: and outputting the processed data in the corresponding format according to the object information obtained in the algorithm process and the abstract information format, organizing and packaging the data, and outputting the data to finish the processing process. The method can effectively solve the problem of redundancy removal of the capsule endoscopy image, belongs to the generalized image processing problem, and meets the technical requirements of rapidly processing similar images and even completely removing the similar images.

Description

Redundant image processing method of capsule endoscope

Technical Field

The invention relates to an image processing method, in particular to a redundant image processing method of a capsule endoscope.

Background

The gastrointestinal endoscopy is an important diagnosis and treatment means for gastrointestinal diseases and is also the key for early screening of gastrointestinal tumors. However, the painful experience of gastrointestinal endoscopy makes it prohibitive for many patients. The capsule endoscope can enable a patient to complete the examination of the whole digestive tract in a noninvasive and painless way at a lower cost. However, due to the problems of the working mode, the capsule endoscope can generate 6 to 12 ten thousand continuous images of the digestive tract during the examination process. The endoscopy doctor needs to read all capsule endoscopy image sequences in sequence to make diagnosis, the average diagnosis time of a single patient is 3-6 hours, and huge medical labor cost and painful film reading experience are generated. Thereby resulting in lack of prescription will of doctors and slow progress of capsule endoscopy diagnosis projects in various hospitals.

The redundancy mainly comprises (1) blurred images, (2) similar images and (3) non-human body images, wherein the blurred images mainly refer to images which are taken by a capsule endoscope during rapid movement or when the capsule endoscope is attached to the wall of the alimentary canal in a very close manner, the images do not have clear images of organs of the alimentary canal and cannot provide information with diagnostic value to a doctor, the similar images refer to images which are suspended for some time period in the alimentary canal but are not stopped, so that the contents of continuous images of dozens of frames or even hundreds of frames are very similar, and only one image is provided from the perspective of the doctor, the doctor needs to read images of dozens of frames or hundreds of frames, the working efficiency is greatly reduced, the non-human body images refer to images which the visual field of vision is influenced by food residues before and after the capsule endoscope enters the entrance, or after the capsule endoscope is discharged, the redundant images of the capsule endoscope enters the end of the alimentary canal, the redundant images of the food residues influence the visual field of the images, if the three types of redundant images can be removed, the time cost for reading of the capsule endoscope is effectively reduced, the doctor increases the doctor, the doctor needs to process the redundant image processing, and the redundant image processing cost of the redundant image processing is increased by the additional high-based on the clinical image processing data of the additional high-based on the technical problem that the redundant image processing cost of the redundant image processing personnel, such as the redundant image processing cost of the redundant image processing personnel is increased.

In 2013, in journal Computers in biology and media, Hyun-Gyu Lee et al proposed a machine learning algorithm based on optical flow information as a feature in reduction in wireless capsule endoscopy video, which can remove more than 53% of repeated capsule endoscopy images of small intestine. However, the main drawbacks of this method are the following three points:

(1) redundant images are removed by using optical flow information as characteristics in a machine learning mode, a large amount of time and computer resources are consumed, a set of images processed often exceeds dozens of minutes or even one hour, and the diagnosis timeliness requirement required by a hospital cannot be met;

(2) the optical flow information is used as the characteristic to process the capsule endoscopy image, about 53 percent of repeated capsule endoscopy images in small intestine sections can be removed, on one hand, the redundancy removal rate is not high enough, and on the other hand, the processing of fuzzy images and non-human body images is lacked;

(3) the lack of analysis on the leak rate of positive images in the literature has great influence on the diagnosis accuracy of doctors if the algorithm processes the images with lesions as redundant images.

There are currently available practical algorithms that can be used to quickly screen similar images, but in the case of images of an endoscopic capsule sequence, there are some specificities: firstly, the capsule endoscope may have the situation that the lens rotates in the process of shooting the same position image, and some search algorithms which cannot adapt to the rotating lens are excluded; secondly, the images of the capsule endoscopy have sequence, and the relation between front and rear frames needs to be considered; thirdly, as shown in fig. 2, the data size of the images of the capsule endoscope is too large, which has a high requirement on the robustness of the algorithm.

Disclosure of Invention

Therefore, the technical problems to be solved in the image processing process of the capsule endoscope are as follows:

(1) the problem of redundancy removal of images of a capsule endoscope belongs to the problem of generalized image processing, and the method needs to achieve the technical requirement of rapidly processing similar images and even completely removing the similar images.

(2) Based on the practical application scene of the hospital, the technology for removing similar data can reduce the economic cost and improve the availability and stability of the redundancy removing technology.

(3) Under the conditions of large computation amount and insufficient system availability and stability in an actual application scene, a technical scheme capable of rapidly compressing a large amount of data is needed on the premise that the data amount needs to be compressed for processing.

(4) A need has arisen in practical application scenarios to be able to process a large amount of image data in a short time.

(5) In an image sequence, frame data associated in front and back are likely to be similar, so that repetitive processing is required, and a technique for compressing video is required in order to cope with such a scene. The information meaning of the same object appearing on each previous and subsequent frame may be different when the previous and subsequent frames are associated, and therefore, the process of compressing the video is performed continuously.

In order to solve the technical problem, the invention provides a redundant image processing method of a capsule endoscope, which comprises the following steps of

And (3) data unpacking: unpacking original data to an arithmetic unit, firstly extracting the mapping in the original memory of the data according to different data formats, hashing the data, and sending the hashed data to the arithmetic unit to start an arithmetic flow;

and (3) perception hash processing: in the algorithm stage, performing algorithm processing on the data according to the hash information to calculate similar object information;

and (3) an abstract outputting process: and outputting the processed data in the corresponding format according to the object information obtained in the algorithm process and the abstract information format, organizing and packaging the data, and outputting the data to finish the processing process.

Further, the processing method further comprises an arithmetic unit process, the arithmetic unit process selects the GPU or the CPU as an arithmetic unit according to the running hardware of the processing method, and the fastest feedback speed is obtained by selecting a proper arithmetic unit.

Further, the perceptual hashing process is a zero compression process.

Still further, the perceptual hashing process includes a reduced color process for converting image information into a gray scale map.

Further, the perceptual hashing process further includes a DCT process, which is a discrete cosine transform process for converting the image information into a set of frequencies and scalars.

Still further, the DCT process is a modified discrete cosine transform process.

The invention also discloses a capsule endoscope redundant image processing system operated by applying the method and a processing end loaded with the system.

Furthermore, the processing end is a computer host.

Compared with the prior art, the invention provides a new method, which can effectively remove the fuzzy, repeated or redundant images of non-human organs in the capsule endoscope, can complete the redundancy removal work of 10W ~ 13W capsule endoscope images within about 40 seconds, removes more than 70 percent of the images, and simultaneously ensures that all positive images are kept.

Drawings

FIG. 1 is a conventional image processing flow in the background art;

FIG. 2 is a process flow of fast image filtering by algorithm in the background art;

FIG. 3 is a basic operating logic diagram of an embodiment of the present invention;

FIG. 4 is a logic diagram of the bottom level of the method of the present invention;

FIG. 5 is a logic diagram illustrating the operation of the CPU and the GPU according to the embodiment of the present invention;

FIG. 6 is an interactive interface diagram according to an embodiment of the present invention;

FIG. 7 is a diagram of an original image signal with a length of 4 according to an embodiment of the present invention;

FIG. 8 is a diagram of the extended time domain period of 2N-2 according to an embodiment of the present invention;

FIG. 9 is a diagram of the result X [ k ] of DCT-1 according to the embodiment of the present invention and the time domain signal after prolongation.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

The hardware pre-configuration used in this embodiment is as follows:

(1) under Linux or Window system

A: having a python3.5 environment

B: memory space above 4GB

C: available hard disk space with more than twice the data size

D: program occupation ensuring 10MB hard disk space

(2) When the operator is selected as GPU:

a: hardware need to provide a video card (NVIDIA video card) supporting CUDA (8.0)

B: the system is provided with a CUDA call library pytorch of a GPU, and the version is according to the CUDA version

C: the video memory of the video card may need a certain degree of extra space according to the size of the data single item

(3) The working environment is as follows:

a: without special heat dissipation structure, if GPU is adopted, the system can not work in high temperature environment for a long time (12 hours and more)

B: generally cannot work under hardware conditions of too low temperature and too high temperature

C: environment requiring relatively stable voltage

The method of the embodiment of the invention is essentially a basic software middleware, the provided framework can be borne by a single hardware, and the basic operation logic is as shown in figure 3:

(1) subroutine-select operator: and under the default condition, the software selects a default optimal processing method, and if the GPU meets the condition, the GPU is adopted for operation.

(2) Sub-process-data unpacking: and unpacking the data to an arithmetic unit, firstly extracting the mapping in the original memory of the data according to different data formats, then hashing the data, and entering the arithmetic unit to start an arithmetic flow after hashing.

(3) Sub-process-aware hash processing: in the algorithm stage, algorithm processing is carried out according to the hash information, and similar object information is calculated.

(4) Subflow-output summary: and outputting the processed data in the corresponding format according to the required abstract information format, and organizing and packaging the data.

(5) The arithmetic unit can be a GPU or a CPU, and a more proper arithmetic unit is selected to obtain better feedback speed. In the test sample, the computational efficiency of the GPU is about 200 times that of the CPU. The method bottom logic is shown in FIG. 4:

specifically, the method provided by the embodiment of the invention realizes screening and removal of redundant pictures in a capsule endoscopy sequence on the basis of a Perceptual Hash Algorithm (PHA). The embodiment of the invention utilizes a perceptual hash algorithm to process the image as follows:

A. and (4) zero compression. The traditional image processing method needs to compress an image to a certain extent before image processing, but because the capsule endoscope image has a small format and some tiny in-situ meaningful lesions are possibly removed in the compression process from the practical application, zero compression operation is carried out on the image in the capsule endoscope image processing process. Taking PillCam capsule data of Given corporation in Israel as an example, the single-frame image size is 256 × 256 pixels, and the single-frame image size of the national Jinshan OMOM capsule is 480 × 480 pixels, which can be directly processed.

B. The color is simplified. And converting the pictures in the capsule endoscope image sequence into 256-step gray-scale images.

C. The DCT is calculated. DCT (discrete cosine transform) transforms the image into a collection of frequencies and scalars.

Among them, Discrete Cosine Transform (DCT for Discrete Cosine Transform) is a Transform related to Fourier Transform, which is similar to Discrete Fourier Transform (DFT for Discrete Fourier Transform), but uses only real numbers. The discrete cosine transform corresponds to a discrete fourier transform of approximately twice its length, which is performed on a real even function (since the fourier transform of a real even function is still a real even function), and within some variants it is necessary to shift the position of the input or output by half a unit. The specific explanation is as follows:

the DFT transforms the signal into a linear combination of complex exponential signals and if the time domain signal is even symmetric, the frequency domain will have only the real part (cosine part of the complex exponential). So, if a finite long signal is extended to even symmetry, it can be transformed into a linear combination of cosine signals. As shown in fig. 7:

if the original signal length is N, the extended time domain period is 2N-2, as shown in fig. 8.

Since the time domain period is 2N 2, the fundamental frequency is cos (2PI/(2N 2)). The summation interval should be [0, 2N-2-1], considering that the cosine function is an even function, the summation interval may be defined as [0, N-1], while weighting x [ N ], and multiplying the summation result by 2. At frequency doubling, k =1, the basis function is cos (π N/(N-1)); at 2N-3 doubling frequency, k =2N-3, the basis function is cos (pi (2N-3) N/(N-1)) = cos (pi (2N-2-1) N/(N-1)) = cos (pi (2N-2) N/(N-1) -pi N/(N-1)) = cos (pi N/(N-1)). I.e. the 1-fold frequency and the 2N-3-fold frequency have the same basis functions. The concrete formula is as follows:

thus, the result X [ k ] of DCT-1 has the same symmetric form as the extended time domain signal, as shown in FIG. 9:

in a practical approach, embodiments of the present invention use modified discrete cosine transforms. Modified Discrete Cosine Transform (MDCT), which is a linear orthogonal lapped Transform, is used for short. The method uses a time domain aliasing cancellation Technology (TDAC), comprises a 50% time domain overlapping window, effectively overcomes the edge effect in the processing operation of a windowed Discrete Cosine Transform (DCT) block under the condition of not reducing the coding performance, thereby effectively removing the periodic noise generated by the edge effect, and under the condition of the same coding rate, the MDCT performance is superior to the DCT, and the method is widely applied to the transform coding of voice, broadband audio and image signals.

(1) In order to be convenient to apply, all capsule endoscope image information is regarded as general data of a 'text', so that a 'abstract' concept corresponding to the capsule endoscope image information is provided, and both useful and non-redundant capsule endoscope information and a Hash algorithm are combined to realize the abstract algorithm basis of perceptual Hash.

(2) Because the text data has a context relationship, namely the information of the previous frame and the next frame of the same image is considered, the abstract aims at the work in the context environment, the same data is not eliminated in different contexts, and the relevance of the contexts is ensured.

(3) Under the existing hardware condition, the GPU is more suitable for batch high-speed operation than the CPU, as shown in fig. 5, and simultaneously, the GPU supports concurrent processing, so that the operation speed can meet the application scene requirement by entering the algorithm into the GPU for operation. The software can also be simulated with a high performance CPU for experimental and basic testing purposes.

In the implementation process of the method, the embodiment of the invention provides an interactive interface for a processing system, as shown in fig. 6:

in the using process, the redundant images in the whole set of capsule endoscope image sequence can be completely deleted only by clicking the abstract button.

In this embodiment, a Given example of a capsule of the pilcam corporation is Given to Given, and two times of capsule endoscopy image acquisition are performed to obtain one piece of image information with an original video length of more than 7 hours and one piece of image information with an original video length of more than 2 hours.

Source video length	Video length of summary	CPU processing duration	GPU processing duration
				7h34min31s	1h57min1s	670s	32s
2h14min57s	24min10s	160s	0.7s

Based on the experimental result of the method, the method can be confirmed to utilize the GPU to compress the original video length of more than 7 hours to 2 hours within 40 seconds, namely, 100,000 frames of redundant capsule endoscopy images can be compressed to be within 20,000 frames, and 80% of redundant data can be removed.

And (3) clinical verification:

the method utilizes 100 capsule endoscope actual cases in Xiamen city to carry out verification, aiming at the data of 4 real patients with different capsule endoscope brands, the method can remove more than 70 percent of redundant data in the whole alimentary tract capsule endoscope image sequence and ensure that 0 omission of the image containing the focus is avoided.

Compared with the prior method, the processing method provided by the embodiment of the invention has the following advantages:

(1) the speed is high, and the redundancy removing function of more than 10 ten thousand pictures can be completed within 40 seconds. The real-time requirement of the hospital is met;

(2) the relation of the front frame and the rear frame in the capsule endoscopy image is considered, the image containing the focus is ensured not to be omitted, and the workload of a capsule endoscopy doctor is reduced.

In summary, Capsule Endoscopy (CE) is a high and new technology product developed and produced in recent years, and can non-invasively detect small lesions of the whole digestive tract mucosa. However, the method has the defects that the pictures shot in the digestive tract are various, and a large number of redundant pictures are included, so that the time and labor are consumed for a doctor to read the pictures, on one hand, a large amount of labor and time cost are consumed, and on the other hand, the diagnosis accuracy is influenced. Based on the method, more than 70% of redundant data shot in the capsule endoscopy examination process can be removed, the capsule endoscopy diagnosis time is greatly reduced, the diagnosis accuracy of doctors is improved, and the medical homogenization construction is promoted.

Finally, it should be noted that the above-mentioned embodiments are only used for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the modifications and equivalents of the specific embodiments of the present invention can be made by those skilled in the art after reading the present specification, but these modifications and variations do not depart from the scope of the claims of the present application.

Claims

1. A redundant image processing method of a capsule endoscope is characterized by comprising the following steps

2. The redundant image processing method according to claim 1, wherein the processing method further comprises an operator process, and the operator process selects a GPU or a CPU as an operator according to running hardware of the processing method, and obtains a fastest feedback speed by selecting a suitable operator.

3. A method of redundant image processing according to claim 1 wherein the perceptual hashing process is a zero compression process.

4. A redundant image processing method according to claim 3 wherein said perceptual hashing process comprises a reduced color process for converting image information into a gray scale map.

5. A redundant image processing method according to claim 4 wherein said perceptual hashing process further comprises a DCT process, said DCT process being a discrete cosine transform process for converting image information into a collection of frequencies and scalars.

6. A redundant image processing method according to claim 5, wherein said DCT process is a modified discrete cosine transform process.

7. A redundant image processing system for an endoscopic capsule operated by the method of any of claims 1 to 6.

8. A processing terminal loaded with the system of claim 7.

9. The processing terminal of claim 8, wherein the processing terminal is a host computer.